Digital Clay Combines Gesture, Touch, Click Interfaces
CREDIT: Michelle Higa
VANCOUVER, BRITISH COLUMBIA -- The mouse was always a metaphor; a physical avatar of the digital pointer. And gestures, while rich in variation, don't produce the tactile feedback humans need from their tools. "Recompose," a prototype input device on display here at the SIGGRAPH interactive technology conference, corrects those shortcomings by presenting computerized data in a 3-D tiled surface that users can manipulate like a keyboard via touch or gestures.
Originally conceived at the MIT Media Lab as a kind of digital clay that would allow users to "sculpt" data by shaping a motorized surface, Recompose serves as an interactive model for any data with topography, from maps to stock portfolios. A hacked Microsoft Kinect provides gesture support, letting the user manipulate Recompose's 144 motorized buttons using either sweeping gestures or fine touches. Through Recompose, users can actually feel their data, not just click on it.
"When I work on computers, I work in a much more goal-oriented way. When people work with their hands, they experiment more. A lot of that is lost with digital interaction," said Daniel Leithinger, a researcher at the MIT Media Lab who worked on the Recompose project. "Here, you can influence the properties of the material, and that's very inspirational. You can move things around like a magician, and people like playing with the invisible string."
The entire Recompose system involves the combination of a number of off-the-shelf interface technologies. Each of the 144 tabs, arranged in a 12 by 12 square, contain a servomotor and a sensor, effectively turning each one into a keyboard button that can dynamically rise or fall several inches in either direction. A Kinect suspended above the table monitors the user's hands, so that they can pull the buttons up with virtual string, or select entire sections of buttons to raise or lower.
[Read More: Human Hand Controlled by Cyborg Attachment]
To demonstrate the power of the system, the researchers draped a nylon sheet over the naked grid of buttons. They then projected an image of the Alps onto the sheet, causing Recompose to simulate its topography. As the users scrolls across the map, Recompose transforms into a digital sand table, modeling the topography of the region in real time.
In future applications, Recompose could serve as a 3-D, interactive stock ticker that allows its user to buy and sell as their holdings rise and fall. Similarly, a doctor could use Recompose to monitor a patient's vital signs, and adjust a drug drip accordingly. On the more creative side, almost two-thirds of conference attendees asked if the researchers had used Recompose to make music, Leithinger told InnovationNewsDaily.
To help enable those more complex tasks, the researchers are currently working to expand the resolution of Recompose from 144 buttons to 900 buttons.
"The mouse does not represent content. Recompose makes the content real. The main problem with gesture is giving feedback. By connecting data to a physical object, Recompose adds context," Leithinger said. "It's the interface that makes things power, and people just get this."
Follow InnovationNewsDaily on Twitter @News_Innovation, or on Facebook.
