Sound Waves Allow Computer to Detect and Obey Gestures
A tester plays Tetris with hand movements using a new program that can make most laptops and desktop computers recognize gestures.
CREDIT: Screenshot from "SoundWave: Using the Doppler Effect to Sense Gestures" by ubicomplab on Youtube
Researchers with Microsoft and the University of Washington in Seattle have created a program that lets people turn almost any computer, including laptops, into a gesture-controlled machine. Though many gesture-controlled programs now use Microsoft's Kinect system to detect motions, this new program, called SoundWave, has a more unusual method of figuring out what's going on around it. It works a little bit like a dolphin.
The program sends out sound waves through the computer's speakers at very high frequencies that are inaudible to humans, usually between 18 Hertz and 22 Hertz. The sound waves bounce off anything within about a meter of the computer's speakers and return to the computer's microphone. If anybody is making movements in front of the computer, the sound waves that return are shifted as a result of the Doppler effect — yes, the same effect that makes ambulance sirens shift in tone as they zoom past. The program uses the Doppler shift to calculate what movements are happening in front of it.
SoundWave recognizes scrolling, fast and slow single and double taps, and the see-saw action that rotates pieces during games of Tetris, according to a paper published for the Association for Computing Machinery conference that starts May 5.
SoundWave's creators tested the program with six people, on 11 laptops and desktop computers. They found it keeps working when people play music on their computers. It works in noisy places as well as quiet ones. They also found that people make several involuntary movements while using computers, so they wrote the program to ignore brief movements and to turn off temporarily when users are typing.
The program also makes the computer "sleep" when someone walks away from it and awaken when someone walks toward it.
Watch SoundWave at work in a video from the University of Washington's UbiComp Lab:
In their paper, SoundWave's creators pointed out the program is especially convenient because it uses the speakers and microphone already embedded in computers. It works "across a wide range of existing hardware," the researchers wrote, "to facilitate immediate application development and adoption."
One main drawback is that it cannot recognize still gestures; because it depends on the Doppler effect, it must work with movements. The researchers also aren't sure if the high frequencies might perturb pets or kids.