AI Assistance in Creating 3D City Visualizations
In recent days the news is full of information about disasters occuring on our globe. Scientists are wondering how to reduce the impact of such events, coming up with disaster management. However, to create a disaster plan some 3D data is required. Unfortunately, possible solutions for 3D reconstruction have their faults. The lidar method may be too expensive, whereas satellite optical data struggles with cloud cover issues. Satellite radar data instead may be a good alternative for other techniques, especially when it’s combined with AI tools. Thanks to this unique combination outstanding products can be designed, for example 3D City Maps.
Presenting SAR2Height
A true revolution was developed by Aerospace engineers at the University of the Bundeswehr Munich, combining radar data and AI. Professor Michael Schmitt and Michael Reclastarted acquired SAR images from German TerraSAR-X and high resolution height maps, based on airborne laser scanning, to carry out a training of deep neural network.
As Schmitt says: “The results were amazing. We trained our model purely on TerraSAR-X imagery, but out of the box it works quite well on imagery from other commercial satellites.” What about the accuracy? This model is able to predict the height component of the building with an accuracy of several meters.
The image below shows a difference between the 2D radar image and 2.5D Digital Surface Model with 0.5m resolution. Combined together, they show a 3D visualization of the city, which looks truly impressive.
From radar image to 3D vizualization. Source: Research Gate
AI solution taming earthquakes
The SAR2Height model is hoped to be a tool for predicting building damages, particularly in case of an earthquake. The vivid example, how useful SAR2Height can be at the present time, is the recent earthquake at an extremely short distance from New York City on 5th of April 2024. It’s a matter of time before we probably see a 3D visualization of the NY’s earthquake aftermath.
Earthquake near New York City, 05.04.2024. Source: Daily Overview
In a chase of perfection
This solution, however, is not without drawbacks. There are some issues like Radar Image Distortions. One of them is a radar layover. It’s a situation where a radar signal reaches high objects. Difference between return signals from the top point (B) and the bottom point (A) causes a deformation of the final result and 3D relations.
Radar layover. Source: Natural Resources Canada
Additionally, at the moment not every European or North American city possesses periodic lidar datasets needed for neural network training. It is often said that governments flying missions have too slow revisit cycles and it’s regrettable that projects like SAR2Height are not supported.
SAR2Height is constantly learning. As Mr. Schmitt highlights, his method is not as accurate as classical methods, but still, it’s better than nothing. Especially when it comes to developing an urgent plan for disaster response.
In conclusion, this research opens new possibilities for topographic reconstruction and enhances the potential for remote sensing applications that require rapid, weather-independent observations. The approach benefits established radar techniques by offering an understanding of terrain, which is valuable for SAR data tasks like phase unwrapping or cross-modal image registration.
Source: Research Gate