The Technology Behind Cesium’s Integration with NVIDIA Omniverse
Cesium for Omniverse, Cesium’s extension for NVIDIA Omniverse, a platform for developing Universal Scene Description (OpenUSD)-based applications for industrial digitalization, creates a bridge between the Cesium, 3D geospatial, and Omniverse OpenUSD ecosystems. The extension makes it possible to create full-scale 3D geospatial simulations and digital twins in Omniverse built on open standards such as 3D Tiles. As we engineered and optimized Cesium’s extension for streaming 3D content over the web, we learned the best practices for perfecting the interoperability between glTF, OpenUSD, and Fabric. This will be the first of three blog posts that cover the best practices Cesium has established for integrating with the Omniverse ecosystem using extensions and Omniverse Connectors.
Extensions for Omniverse can be of different types. The majority currently available are integrated using Omniverse’s Python API. For cases where Python alone is insufficient, Omniverse C++ API is available, and extensions implemented in C++ may generate Python bindings via pybind11. When designing Cesium for Omniverse, we decided early on that the architecture for Cesium’s goals was to use Python for the functional and UI layer and C++ for efficient streaming of 3D Tiles by connecting Cesium Native to Omniverse's Fabric scene delegate. Cesium Native is a set of C++ libraries for 3D Tiles runtime streaming, glTF, and high-precision geospatial math functions, and is the core library used for Cesium for Omniverse, as well as Cesium for Unreal and Cesium for Unity.
Cesium for Omniverse High-Level Architecture
Omniverse is built atop OpenUSD as the primary data model for describing and composing scenes. OpenUSD is an extensible platform maintained by the Alliance for OpenUSD for collaboratively constructing and aggregating 3D scenes. OpenUSD was originally designed by Pixar to enable many artists to collaborate simultaneously on a single scene, often referred to in OpenUSD nomenclature as a Stage, through the use of layer stacks. The layer stacks can then be combined and resolved through a strength ordering to create the final scene. Omniverse leverages Hydra, OpenUSD’s extensible rendering pipeline architecture, to add a new scene delegate for Fabric, a data layout optimized for large-scale scene updates in real time, populated at load time from source USD.
Cesium for Omniverse defines several different OpenUSD schemas to represent different elements of geospatial functionality. Every Stage has a Cesium and Cesium Georeference prim, and can have a mix of Cesium Tileset Prims and Imagery Prims for Raster Overlays. A recent addition has been the creation of a Cesium Globe Anchoring schema, which can be applied to any Xformable Prim in the OpenUSD Stage to anchor on a global coordinate.
Cesium for Omniverse Rendering Architecture
To get the best possible performance, several strategies are used, including render pooling and Omniverse's new Fabric scene delegate, which is currently released in an early experimental state. Fabric is a data layout representing the post-composed OpenUSD Stage, allowing for more direct access to the GPU, among other runtime optimizations. We eventually want to move to interfacing with USDRT instead of Fabric directly, as USDRT provides an OpenUSD-like abstraction on top of Fabric. This will provide easier compatibility with mainline OpenUSD allowing reuse of our code beyond Omniverse and eventually will allow easier interoperability throughout the entire OpenUSD ecosystem.
Geometry is populated at runtime into Fabric specific data structures but the parameters describing various attributes of the geometry generally correspond to OpenUSD attributes. While OpenUSD geometry generally uses subdivision surface algorithms with every mesh, the source data always represents the unrefined mesh. Cesium for Omniverse’s geometry is configured to use no subdivision surface algorithms ensuring that the geometry between the 3D Tile and OpenUSD is one-to-one.
glTF PBR materials from the glTF model payload are translated to custom MDL materials based on the glTF PBR MDL material defined within Omniverse. The custom materials used by Cesium extension enable features such as imagery blending with multiple raster overlays, and are fully compatible with the Omniverse Material Graph editor. Custom material nodes exist for a variety of world-mapped functions including texture and color lookups. We’re actively working on adding more features like 3D Tiles styling for further visualization and analytics use cases and provides highly optimized streaming capabilities and rapid 3D geospatial context.
Cesium’s integration with Omniverse is rapidly being developed and adding new features for our users. Omniverse powered by OpenUSD provides a promising foundation for building industrial digitalization solutions and digital twins, and we are proud to be a part of the Omniverse OpenUSD ecosystem.
In the next Cesium for Omniverse Technical Deep-Dive, we’ll go into more detail about USDRT and Fabric, and how we utilize these technologies to provide fast 3D Tiles streaming in Omniverse.
Ready to get started with Cesium for Omniverse? Visit the Cesium Learning Center for step-by-step tutorials.