Microsoft Research presents paper on Simultaneous Pen + Touch

Mary-Jo Foley at ZDNet reports that the Microsoft Research team behind Manual Deskterity is presenting the white paper on their work on Simultaneous Pen + Touch at the ACM Symposium on User Interface Software and Technology.

As you may recall but likely have forgotten, the team released a video preview of their work back in April. The basic concept is that the current dual digitizer technology allows pen input OR touch input, but not both at the same time. Their project allows both to work in conjunction for effects that can be clumsy for one or the other. Or as they put it in their paper:

Based on our explorations we advocate a division of labor between pen and touch: the pen writes, touch manipulates, and the combination of pen + touch yields new tools.

The paper provides very useful background info to what you see in the video (re-posted below), as well as descriptions of the simultaneous pen + touch tools they show. I am particularly impressed by how they examined the usage of a paper notebook not simply for writing notes but also for scrapbooking and manipulation.

The paper makes passing mention of Courier, which MJF fawns over. What I found more interesting was the reference to Codex, which I did not realize was presented in a paper by Ken Hinckley and his team last year. So while the work was done on a Microsoft Surface, they are definitely looking at mobile implementations. Just hope we can see it in consumer form some time.

Alleged Apple fanboi, accused Android apologist, and confirmed Microsoft MVP for touch and tablet Mark Sumimoto a.k.a. Sumocat dabbles in all areas of mobile computing with a focus on Windows-based Tablet PCs and pen input. A mobile computing enthusiast since 2004, he pioneered the field of ink blogging via his personal blog, Sumocat's Scribbles. His current tools include a Fujitsu Lifebook T900, TEGA v2, and iPhone 4. Email: sumocat [at]


  1. Well that looks way better than the usual MS Surface stuff which isn’t that useful. Some of those features could be used for artists. They could be zooming and panning with their left hand and painting with their right. It can be good as long as they pair it with a powerful tool like Photoshop.

    I have used pen + touch before with a Lenovo X201, but I have to move the stylus away enough from the screen and then use my fingers. Like I do here –

    Thanks for sharing this Mark!

Leave a Reply