PixelNotes Blog

The print page feature produces really attractive pages when it works and really awful ones when it doesn’t. Formulas failed consistently for dumb reasons and in subtle ways. I blamed it on LaTeX vs. AMS-LaTeX incompatibilities and ignored it. It was bugging me today so I looked into getting all the compatibility worked out and had the amsmath package included and everything. My isolated tests showed rendering as spot on, but the integrated tests failed miserably. I isolated the problem to the RedClothForTex conversion code and managed to get a nice hack in.

In summation, PixelNotes printing should work much better now. If you find any errors in documents, send them to me along with the note that caused the problem. I think that a very nice printing solution is very possible.

This week testing began with Pixelnotes. Test participants are gradually coming online. I have been receiving good feedback and feature suggestions. I am trying to implement these things fairly quickly. I am basically deploying a new version every night or so (the server is down for updates at around 1AM generally).

If you notice anything funny while using the system, have any suggestions for features, have any usage issues at all, or just think something could be changed to improve the experience, please let me know. You can either email me or post a comment on this blog post. I will try and incorporate your feedback as quickly as possible.

Earlier this week I transferred Pixelnotes to a dedicated server. This move will allow the use linguistic daemons that will add features to the program. The move went fairly smoothly for a 1 A.M. migration and the data transferred without a hitch. There was a little downtime due to me screwing up my DNS configuration a bit, but, overall, things went positively swimmingly.

In preparation for the enabling of the advanced processing system that shall automagically link pages together and possibly solve world hunger, I am have been parsing and processing gigabytes upon gigabytes of Wikipedia data. My initial samplings show some pretty consistent trends (the top hundred or so terms from any sampling I do are nearly identical with the next couple thousand showing great similarity). I have prepared a graph (see below) that shows the frequency of words found in this corpus. It highlights the probability trends pretty well. The results so far are very optimistic for the usefulness and accuracy of this process.

In other news, I have discovered a truly wonderful Ruby on Rauls podcast. If you have ever even thought of “Riding the Rails,” you should check out Railscasts.

I am proud to announce that the core feature of PixelNotes, automatic wikipedia-style intertextual linking, is hot off my favorite text editor’s buffer. The system is now capable of randomly choosing one of the experimental text algorithms and running it on the note. I currently have two control groups (Null control and Random control) that help to characterize the performance of my Naive-Bayesian classifier. I am working on adding a basic heuristic classifier and an advanced Support Vector Machine algorithm.

The point here is that through occult black magick, the program is able to automatically discern noun or noun phrases (including the boundaries thereof) and then link the important ones together at blazing speeds (thousands of words per second). The system is not complete, but it is working pretty well. I expect it to work even better once I parse and generate a proper training corpus.

Due to testing and infrastructure needs, this feature will probably not go online for at least two weeks.

A large part of this project is finding premade components so that the wheel is not reinvented. The time frame simply does not allow much tangential custom development. Unfortunately, combining components can create some extensibility issues. Overcoming these is a core part of the design process.

One of the key functions of this application is text formatting. I have been using the Redcloth library for handling Textile formatted input, but application specific custom syntax has been a requirement from the start. My initial approach was to modify the Redcloth library to add my syntax rules into the core (an approach that quickly devolved into Monkey patching ), but this proved to be a death by a thousand edge-cases. Earlier this week I realized that I could just write a preprocessor to translate my custom syntax into Textile. No side-effects, no ambiguity, no fuss. Really, why can’t all things just be this easy.

The practical upshot of this is that PixelNotes now supports mathematical typesetting through TeX. The text formatting code is also a lot cleaner now. Ultimately, this preprocessor will allow for all sorts of custom filters. We have formula support now, but in the future we could have chemical structure support or some other killer feature for notes. Really, the sky’s the limit. I love it when a design decision made out of necessity actually opens up possibilities.

Developing PixelNotes has been a lesson in agile development with ruby on rails. Rails really streamlines the process and makes web application development some degree of fun again. Ruby is a wonderful, powerful, and delightfully idiomatic language that really works well for many things. The entire system is just a joy to use. The agility of the framework has really become evident in the ease with which I can completely overhaul the database schema when I gain insight into a better design.

This framework allowed me to get an application from my whiteboard to my beta server in a few weeks of development. Even without actually having any time to work on this application over the summer, I have managed to stay slightly ahead of schedule. I successfully deployed the initial beta on Thursday.

That said, some things just never change. Within a few hours of the initial deployment I was already receiving complaints about problems in Internet Explorer. The problems seemed to stem from IE’s lack of properly padding the <H1> box element. This was fixed with a bit of browser specific CSS wizardry. Looking into it further I discovered that IE also has a host of undocumented “features” relating to JavaScript. The fun part is trying to debug why $(el_id).innerHTML = "Foo" works and $(el_id).innerHTML = "<p>Foo</p>" stops execution and throws the helpful exception Unknown Runtime Error.

Apparently, it has some problem with using an inline element to cache AJAX data that contains block-level elements. I love it when IE does something completely differently from every other browser on the market. I suppose it keeps life interesting

Anyway, IE should now be fully supported. If you stumble upon any bugs email me (remove the NOSPAM of course).