MIT developing sketch-recognition software

Posted by | 02/19/2010 | 4 Comments

With tablet computing hitting a new high, tablet-specific methods of data entry, such as handwriting recognition, are receiving more attention than ever. However, researchers at MIT are working on going a step further and developing sketch-interpreting software, carrying data entry into realms where keyboards aren’t enough.

From MIT news:

…while a drawing can be rich in information, it’s information that’s usually inaccessible to computers. If you draw a diagram on the screen of a tablet computer, like the new Apple iPad, the computer can of course store the drawing as an image. But it can’t tell what the image means.

MIT researchers intend to change that, with a new system that can interpret sketches.

In the video demo, the computer is able to recognize a molecule of Acetaminophen drawn on the screen, returning data as if its name was entered in a search engine. In addition to recognizing chemical structure, the system also promises to identify circuit diagrams or simply clean up rough sketches. This opens tremendous possibilities for serious scientific work, but I imagine the ability to uses sketches for data entry will be a boon to casual tablet users as well. I personally have had opportunity to sketch molecules, draw maps and write out math problems that could have benefited from interpretation software. Anyone have any particular hopes for this software?

Via jkOnTheRun

Tags: , ,

Category: Software

About the Author (Author Profile)

My name is Mark Sumimoto; I am Sumocat. I dabble in all areas of mobile computing, but my focus is Windows-based Tablet PCs and pen input. They’ve been part of my arsenal since 2004, and I’m proud to have pioneered the field of ink blogging, earning a spot as a Microsoft MVP for Touch and Tablets in the process. My current tools include a Fujitsu Lifebook T900, TEGA v2, and iPhone 4. Email me: sumocat [at] notebooks.com
  • rainman

    As an architect, I would like to be able to sketch something and have it search for similar drawings or photographs. There are many times I can picture what I want in my head, but I can’t remember the building name, the client, the magazine I saw the picture in, etc.

  • Wevenhuis

    I feel “star-treky” already ;)

    For me sketch recognition would be nice to do a visual search for medical netter sketches to explain to patients the anatomy of their problem. I could also help simplify making diagrams and decision trees into neat formats. It beats using the mouse for making all the boxes, textsboxes and arrows.
    It could also perhaps help translate sketches of builings roads or maps to google maps for quicker orientation. It could certainly be easier and fun for children, making us perhaps understand better earlier what their symbolic drawings are interpreted to. Great for learning and communication.

    It could also help in foreign countries with communication barriers due to not understanding or speaking the language. Small sketches could then be recognized to real world object possiblities. It is then only a question of pointing to the right illustration in the search results.

    …But let MIT first focus on improving inking and ink recognition. Once we have managed that, it will be a whole different world.

  • SAM

    If they can develope a programme to recognize my chicken scratch, then they’ll have something…LOL

  • Dave-in-Mi

    I would like to see music notation recognition. Just draw your notes on the digital staff, let the program translate it, and click “Play”. I think it would be popular with both composers and music students. I’ve already seen people using tablets & laptops in place of sheet music. Imagine a composer, working with a symphony, all using tablets in place of sheet music. She’s struck with an inpiration and scribbles some changes to her music on her tablet and sends the changes immediately to the networked devices of the musicians. “Let’s take it from the top…”

«
»