Software Lets Drones Recognize Flight Crew Arm Signals
|
|
CREDIT: Melanie Gonick |
New software could allow future aircraft-carrier crew to guide drones to safe landings with the same hand gestures and arm motions that are used to signal human pilots.
In a new study, researchers at MIT recorded a series of videos in which several people performed a set of 24 gestures commonly used by aircraft-carrier deck personnel. They then ran the gestures through a gesture-identification software program they developed which works by analyzing the body pose and also the shape of the hands of each subject in each frame of the video. (The team's software is similar to Microsoft's Kinect gesture recognition system, which unfortunately wasn't available when the research was being conducted.)
[Rise of the Drones: Photos of Unmanned Aircraft]
To simplify the task, the researchers reduced each frame of video down to only a few variables: three-dimensional data about the positions of the elbows and wrists, whether the hands were open or closed, and whether the thumbs are pointed up or down.
The software is designed to look at each motion and calculate the probability that it belongs to one of 24 gestures stored in its database. In tests, the program correctly identified the gestures collected in the training database with 76 percent accuracy. That's not a high enough percentage for use with drones in real world situations, but the team thinks their system will improve.
The research was conducted by MIT PhD student Yale Song, computer science professor Randall Davis, and David Demirdjian, a research scientist at MIT’s Computer Science and Artificial Intelligence Laboratory, and is published in the March issue of the journal ACM Transactions.
New software could allow future aircraft-carrier crew to guide drones to safe landings with the same hand gestures and arm motions that are used to signal human pilots.
In a new study, researchers at MIT recorded a series of videos in which several people performed a set of 24 gestures commonly used by aircraft-carrier deck personnel. They then ran the gestures through a gesture-identification software program they developed which works by analyzing the body pose and also the shape of the hands of each subject in each frame of the video. (The team's software is similar to Microsoft's Kinect gesture recognition system, which unfortunately wasn't available when the research was being conducted.)
[Rise of the Drones: Photos of Unmanned Aircraft]
To simplify the task, the researchers reduced each frame of video down to only a few variables: three-dimensional data about the positions of the elbows and wrists, whether the hands were open or closed, and whether the thumbs are pointed up or down.
The software is designed to look at each motion and calculate the probability that it belongs to one of 24 gestures stored in its database. In tests, the program correctly identified the gestures collected in the training database with 76 percent accuracy. That's not a high enough percentage for use with drones in real world situations, but the team thinks their system will improve.
The research was conducted by MIT PhD student Yale Song, computer science professor Randall Davis, and David Demirdjian, a research scientist at MIT’s Computer Science and Artificial Intelligence Laboratory, and is published in the March issue of the journal ACM Transactions.
