MIT developing sketch-recognition software

With tablet computing hitting a new high, tablet-specific methods of data entry, such as handwriting recognition, are receiving more attention than ever. However, researchers at MIT are working on going a step further and developing sketch-interpreting software, carrying data entry into realms where keyboards aren’t enough.

From MIT news:


…while a drawing can be rich in information, it’s information that’s usually inaccessible to computers. If you draw a diagram on the screen of a tablet computer, like the new Apple iPad, the computer can of course store the drawing as an image. But it can’t tell what the image means.

MIT researchers intend to change that, with a new system that can interpret sketches.

In the video demo, the computer is able to recognize a molecule of Acetaminophen drawn on the screen, returning data as if its name was entered in a search engine. In addition to recognizing chemical structure, the system also promises to identify circuit diagrams or simply clean up rough sketches. This opens tremendous possibilities for serious scientific work, but I imagine the ability to uses sketches for data entry will be a boon to casual tablet users as well. I personally have had opportunity to sketch molecules, draw maps and write out math problems that could have benefited from interpretation software. Anyone have any particular hopes for this software?

Via jkOnTheRun