Taking inspiration from real-world settings is nothing new for video game developers. Fallout 3’s ruined version of Georgetown is a loving recreation of the real thing, down to the bricks themselves. It is lauded as one of the finest sci-fi adaptations of a real-world space in all of gaming. For a more historical effort in a detective based action/drama setting, L.A. Noire’s version of 1940’s Los Angeles using archived maps and Robert Spence’s photographs was a triumph. Players were awed by the look and the feel of a city seemingly torn out of the past.

Of course, these were top studios with massive budgets, which meant that they could dedicate a decent chunk of time and money to researching the landscape in great detail. Not all studios are so lucky.

The problem with using real space as an inspiration for video game terrain can be summed up with one word: Fidelity. In the past, even the best interpretations of real-world locations have fallen short in the eyes of some fans. That’s because the level designers and artists were usually working off of things like 2D photos, hand cam video, and the like. Particularly well off studios might fly the team out for a brief field trip, but trying to capture everything about a location from three hours of exposure is still a daunting task.

Finding a solution to the challenge of creating open worlds in games might have always been just across the corner: geospatial technologies. Why not map the real world instead of virtually creating an artificial one? The latest release of Cesium for Unreal Engine is bridging the gap between geospatial tech and gaming. The plugin has been under development since June 2020, when Cesium got Epic MegaGrant to roll out the solution.

Photogrammetry and LIDAR in games

The solution takes the best features of Cesium engines and enables it for the world of games.

It includes a full-scale, high-accuracy WGS84 globe that can be integrated directly with Unreal Engine. On top of it, users can add their custom datasets e.g. drone-based photogrammetry or LiDAR scans. Everything will work smoothly with game-like human movement, vehicle physics, as well as shadows, clouds etc. adding an extra layer of realism to the real-world terrain.

Moreover, the geospatial datasets can be “copied” to the user’s own world without enforcing the correct geolocation, which gives users the possibility to play around with the tool and the data. Sounds awesome.

With or without a real world anchor, new data can be fed into the Unreal Engine (version 4.26 or later) using a number of industry standard protocols, many of which should be familiar to you: Wavefront Object (.obj), Filmbox (.fbx), Digital Asset Exchange (.dae), glTF (.gltf), Binary glTF (.glb), USGS ASCII DEM (.dem), GeoTIFF (.tiff, .tif), Cesium Terrain Database (.terraindb), Floating Point Raster (.flt), JPEG (.jpg, .jpeg), PNG (.png), CityGML (.citygml, .xml, .gml), KML/COLLADA (.kml, .kmz), and LASer (.las, .laz).

Knowing the imperfections of photogrammetry and LiDAR data there is always a need for manually fine-tuning. A full example of adding photogrammetry as the terrain can be found in the Cesium documentation.

The Results

In the past, programmers relied on a personal sense of depth perception, old public records that were often confusing or simply wrong, and human memory. This is why a lot of older games based on real-world locations are inconsistent with things like scale. They’ll compress a few city blocks into a single piece of terrain, and then scatter the landmarks contained in that area all over the place. Walking ‘half way’ across an island might take ten minutes while walking across the other half might take twenty. Features and distance simply didn’t hold fidelity.

With Cesium for Unreal Engine 5, those issues disappear. True open worlds can be created, translated directly from the input that the game creator desires. Fidelity of distance and fidelity of relative landmark location are always accurate. Height and depth are reflected accurately, conveying the vertigo of a tall skyscraper and the eerie isolation of a dark, abandoned sub-basement. This makes complete, realistic immersion possible, even for players who have been to those locations.

And the ‘tight space’ translation issue disappears when every feature is faithfully represented in the digital world. The need to weave around odd obstacles, subtly change levels, and cope with claustrophobic environments is perfectly preserved.

In Conclusion

Cesium for Unreal Engine 5 gives game creators so many new tools in the toolbox, it’s difficult to fully comprehend the enormity of its future applications. A single developer using photogrammetry or LIDAR for mapping video game terrain is just the tip of the iceberg. Thinking about not only the assets that creators can clone from real-world settings but the inevitable outpouring of high-quality community-created content as well as creating a hybrid world…

That’s not just exciting and inspiring. It’s an opportunity. Imagine games that have an ‘explore’ mode where you can walk through and examine actual settings somewhere else in the world. Possibly even in VR. Moreover, the gaming engines can be also used for simulation purposes, which then could be used for military or lower enforcement purposes among others. There is a twinned entertainment, tactical and educational opportunity here, in addition to the obvious gaming boons that such technology brings.

We follow in the footsteps of some amazingly ambitious projects, like Assassin’s Creed and Watch Dogs, that delivered the texture and flavour of real-world settings like nobody before have ever dreamed. But with Cesium for Unreal Engine 5, we have a leg up on our predecessors. All that’s left is to pair their level of love and attention to detail with the amazing technology that is available today.