Geo-referenced 3D model of Zurich city created by crowdsourced photogrammetry ‘lite’
You know the millions of pictures you take on vacations and flood the Internet with? The ones that after a while become nostalgia triggers, only to resurface annually in Facebook’s ‘On this Day’ posts? Well, researchers at Swiss Federal Institute of Technology (ETH), Zurich, have found a pretty cool use for them.
They have developed a computer vision platform called VarCity which can create 3D models of cities using imagery data alone. This data is being fueled by geotagged public images found on photo-sharing sites and social media networks. It also uses video material from YouTube, public surveillance cameras, and mobile mapping.
The project, which took 5 years in the making, leverages machine learning algorithms to differentiate between different components of a city, such as, buildings, roads, vegetation and water bodies. The shapes of the buildings can be analyzed to identify their architectural uniqueness. The key points across all images are tracked to extract the 3D geometry of an area. These points are then used to create street-level point clouds through multi-view stereo techniques.
Now, this reminded us of Mapillary, a service for crowdsourcing map photos. Mapillary improves positioning quality by computing the relative camera positions and a 3D reconstruction of the environment based on images of the same area. A side effect this process allows Mapillary to generate 3D point cloud models based on images.
What differentiates VarCity is its automatic analysis of the dynamics within the city, such as, traffic. For example, the technology can study car trajectories over time and store car density statistics for each one of them. Such an analysis of traffic data allows VarCity to fuel applications like real-time parking monitoring solutions. That’s not all. The applications of VarCity are aplenty. The automatic detection of landmarks through 3D shape analysis can be used by cities to optimize routes for architecture enthusiasts. It can also be used to build voice-guided navigation systems or to determine how much sunlight a building receives.
Talk about pushing the boundaries of crowdsourced photogrammetry, the ‘lite’ version style!