Google Street View Reveals Surprising City Features
|
|
Besides the French writing, what makes the street corner on the left look distinctively Parisian? A new computer program automatically extracted the features on the right as things that give Paris its characteristic look.
CREDIT: From "What Makes Paris Look like Paris?" by Carl Doersch and colleagues |
To get just the right feel, the animators who created Pixar's feature film "Ratatouille" flew to Paris for a week and immersed themselves in the city, according to a book one of the directors wrote. Now, a group of researchers has created a computer program that automatically finds the features — doorways, balconies, streetlamps — that characterize Paris, Prague, Boston and other cities and neighborhoods around the world. It's a program that searches photos the way most search engines search text, then mathematically replicates the artistic process of getting a feel for a place.
"We figured out what are the things that make a city be itself," said Alexei Efros, a computer scientist at Carnegie Mellon University who led the research. "This is why this is Paris, this is why this is Barcelona, this is why this is San Francisco."
The program could create a database for artists and moviemakers, saving cash-strapped producers a trip, Efros and his colleagues wrote in a paper they plan to present at SIGGRAPH, a computer graphics conference that will be held in Los Angeles in August.
The same technology might later identify the locations of photos based on the scenery inside, blogged a SIGGRAPH reviewer who was not involved in the research. That wasn't the main goal of the program, however, Efros told InnovationNewsDaily. He previously wrote a program that identified regions in photos, but he wasn't sure what the program was looking for to make its identifications. This time, Efros wanted to find cities' visual markers.
To find a town's characteristic style, he and other computer scientists at Carnegie Mellon and at the Ecole Normale Supérieure in Paris gathered about 10,000 Google Street View photos from each of 12 cities worldwide. The photos came digitally tagged with their locations. Efros called the process "data mining for visual information."
"Data mining has been a big thing," he said, "but it hasn't really been done in the visual domain because it's just a much harder problem."
He and his colleagues wrote a program that compared the photos, searching for small features that are common in a city, yet distinctive to that city. For example, sidewalks are a very common city feature, but hardly distinctive. On the other hand, the Eiffel Tower is unique to Paris, but it doesn't show up in every photo of Paris. [10 Building Materials from the Future]
What the program ultimately found were the little things a local might know: wrought-iron balustrades, green-and-blue street signs and a particular type of streetlamp for Paris; double-headed street lamps, black-shuttered windows and a particular type of tile for Boston. When the researchers checked a well-known book about 19th-century Parisian architecture, they found their program had identified several of the same distinctive features.
Some of the features the program found were things most people might not have consciously noticed, but are familiar once someone points them out. The program found the obvious bay windows and steep stairs for San Francisco, for example, but it also noticed that the city has many garage doors painted in pastel colors with a white grid. "It's something I wouldn't have thought of, but once you see them, you say, 'Oh, yeah, that makes sense,'" Efros said.
The program also found other features that historians and architects might miss. It sometimes identified certain makes of cars as distinctive for American cities, for example, and when researchers tried the program on neighborhoods within Paris, the program found that one wealthy neighborhood was characterized by more closed shutters than other areas. Efros guessed it might be because the neighborhood is popular with foreign investors who buy property there, but don't live there year-round.
Besides artists and companies interested in identifying the locations in photos, the program might help urban planners learn more regarding which features appear in a city and where, Efros and his colleagues wrote. Historians could use the program to track how architectural styles spread around the world. With the right tweaks, the program might even identify what makes products look like they're made by certain companies, such as Apple.
Searching photos for these little visual markers is much more difficult in computer programming than searching text. Computers easily recognize numbers and letters, but have more difficulty with visual data. Yet searching photos and videos will become increasingly important in the future, Efros thinks.
"If you look at the Internet, what is the biggest source of data?" he said. "It's not text. Text is dying."
