Handheld Scanner Reads and Plays Handwritten Music
|
|
A new device, still under development, lets people scan handwritten music to control an electric piano.
CREDIT: From "Gocen: a Staff-Based Interface for Musical Performance and Learning Music" by youbaba on YouTube |
A new device that resembles a fat, square pen allows users to play a simplified musical score they've written simply by tracing the score with the pen.
"This is an innovative device that can read any handwritten score and play that music," wrote Preston Smith in the SIGGRAPH 2012 Official Media Blog. Smith is the emerging technologies chair at SIGGRAPH, a computer graphics conference where researchers will present the music-playing device in August. "This will change the way children and adults learn, create and interact with music."
Users drag the pen, named Gocen, over a handwritten piece of music while Gocen sends the images it sees to a computer. The computer analyzes the images using a program Gocen's creators, design researchers at the Tokyo Metropolitan University, wrote.
The computer then plays the notes it sees through either an electronic instrument or speakers. In a video demonstration researchers created, a person stands over a MIDI piano, scanning notes with his Gocen pen while the piano keys beneath him depress in ghostly succession.
Gocen has a few other functions, too. Users can change the pitch of a note slightly by rocking the Gocen pen up or down over a note. They can change instruments by hovering the pen over handwritten instructions, such as "epf" for electronic piano, "bs" for acoustic bass or "dr" for drums. They can record a series of notes they want repeated, like a recorded background beat. In the video, a researcher records a simple bass line for the MIDI piano to play, then sits down to play a melody on the piano alongside his program.
Gocen's creators are now working on creating other devices that can play together in a duet, quartet or other ensemble, according to the paper they wrote for SIGGRAPH. They're also working on making Gocen into a user-friendly system for teaching music.
In general, devices in SIGGRAPH's emerging technologies showcase won't be ready to sell to consumers for another three to five years, Smith wrote.
Watch Gocen at work below:
Follow InnovationNewsDaily on Twitter @News_Innovation, or on Facebook.
