Cesium Roadmap for Bridging the Built and Natural Environment

It’s been just over nine months since Cesium joined Bentley Systems. We’ve used that time to learn more about the technologies built by Bentley and opportunities to bring them to the community. As we look forward to the first Cesium Developer Conference, we are sharing our roadmap to seek your guidance as we bring iTwin digital twin capabilities to Cesium.

Our mission is to empower the community to build 3D geospatial experiences at the convergence of the built and natural environment. 3D Tiles has seen tremendous adoption, including by Google, Esri, Ansys, FOX Sports, The Weather Company, GE Aerospace, MathWorks, CAE, Trimble, and DroneDeploy, to name a few. Cesium ion receives more than 300 million monthly requests from over 30,000 monthly active users. Cesium’s runtime engines—CesiumJS, Cesium for Unreal, Cesium for Unity, and Cesium for Omniverse—continue to enable rapid development of new 3D geospatial applications by an ever-growing community as we continue to deeply invest in open source. There is bottomless demand for more curated data, high-performance tiling pipelines that support more inputs, tools for more efficient end-to-end workflows, and improved end-to-end performance.

As part of Bentley, we have the opportunity to radically empower and expand the 3D geospatial community by leveraging best-in-class technology from Bentley to benefit all the broader ecosystem. Cesium has always been guided by community input, and we’d like your feedback on these themes on our roadmap:

Advance 3D Tiles and open standards;
Integrate reality modeling and AI-powered analysis services with Cesium ion;
Expose iTwin Platform APIs and services through Cesium;
Introduce composable tiling pipelines to optimize your data as 3D Tiles;
Advance Cesium runtime engines to capitalize on new 3D Tiles extensions and platform integrations;
Expand curated data offerings;
Provide reusable and configurable user experience tools and widgets; and
Improve learning material and ongoing maintenance.

Advance 3D Tiles and open standards

3D Tiles is the core of the platform. It is the thread that weaves through the entire platform and community to be a cohesive end-to-end workflow, empowering developers to build extraordinary applications. We’re adding several capabilities to 3D Tiles that have been long requested by the community.

Time-dynamic 3D Tiles

Users across industries are capturing the environment at ever greater resolutions, more frequently, and with more types of devices. This exponential scaling creates opportunities to extend the 3D Tiles specification for more efficient storage, streaming, visualization, and analysis of 3D geospatial data that is spatially coherent and updated over time.

Time-dynamic 3D Tiles is an extension that efficiently introduces the time dimension to 3D Tiles. Using time-dynamic tiles allows a single tileset to be updated over time, with each new capture updating only affected tiles. No longer will an entirely new tileset need to be created for each update; a single tileset can be updated incrementally, preserving a history of changes and allowing for more efficient and faster generation of updated tiles. These can be used for analytical workflows, too, such as changeset analysis over time and comparing tilesets over time to design models. Visit the GitHub issue to share your input.

*Time-dynamic tiles allow a single tileset to be updated over time, with each new capture updating only affected tiles.*

We’re also exploring connecting Bentley’s SYNCHRO 4D Schedule API to generate time-dynamic 3D tilesets. The SYNCHRO 4D Schedule API allows time-dynamic project data to be accessed and used over the web. This connection will converge time-dynamic geometry with business intelligence.

Volumetric data as voxels

Volumetric data has become an important way to visualize real-world content in numerous domains where continuous representation of data is required, such as geology, oceanography, and climate science. The 3D Tiles specification is intentionally flexible in supporting a variety of geometry types to best represent massive heterogenous data. By using voxels (volumetric elements) as a 3D payload with 3D Tiles, you will be able to build applications with global-scale volumetric data. We’re working on ellipsoidal, cuboid, and cylindrical voxels as a starting point in the 3D Tiles specification with glTF as the voxel payload. This will enable opportunities for volumetric data for subsurface, land, air, and space. Take a look at the draft Voxels extension.

*Water temperatures in the Gulf of Mexico represented using Voxels in CesiumJS.*

3D Gaussian splats

One of the most popular topics recently in computer graphics is 3D Gaussian splats. High-resolution reality meshes and point clouds are fantastic for digitalizing our natural environment, but they fall short in capturing a few types of information, such as cables, trees, bars, telecom towers, etc. 3D Gaussian splats have proven to be a fantastic solution for preserving visual detail while maintaining very high fidelity.

We are excited to add 3D Gaussian splats as a payload type in 3D Tiles, including adding the technology behind iTwin Capture to Cesium ion to enable modeling and tiling of 3D Gaussian splats. We’re collaborating with the glTF community and introducing a 3D Gaussian splat extension for glTF.

Comparing 3D Gaussian splats using 3D Tiles (left) with reality meshes (right), with particular significance to the thin-and-long geometry such as wires, dishes, poles, and scaffolding. — *Comparing 3D Gaussian splats using 3D Tiles (left) with Reality Meshes (right), with particular significance to the thin-and-long geometry such as wires, dishes, poles, and scaffolding.*

Geometry highlights and customizations

Bentley Systems was an early adopter of 3D Tiles and has developed extensions to enhance the 3D visualization capabilities required for detailed infrastructure models. Cesium joining Bentley presents the opportunity to converge these extensions and runtime implementations to make them accessible to the entire community. We are working on adding edges and wireframes, line rendering and materials (width, patterns, colors), customizable point sizes, billboards for annotations, and more.

*A 3D Tileset visualized in iTwin.js with glTF EXT_mesh_primitive_edge_visibility extension for highlighting edges.*

Implicit surfaces

Implicit surfaces enable representations of parametric shapes—like pipes, columns, beams, boxes, spheres, and cylinders—when used with other implicit surfaces for the creation of complex geometries that are hard to represent with triangle meshes. The benefits of this are in the storage and accuracy for analytics. Because the surfaces are not approximated as triangles meshes, the surfaces are rendered without geometric error. We are building on the glTF implicit shapes extension, introduced by Microsoft.

BIS semantics in 3D Tiles

The 3D Tiles specification supports rich metadata that adds properties and classifications to geometry. The metadata specification is written with the intent to add semantics at different granularities and for a variety of industries and use-cases. There is a need to define semantic vocabularies in the architecture, engineering, and construction (AEC) industry that also offer a taxonomy applicable to classes of objects. Capturing the meaning of these classes in a standard vocabulary enables downstream repurposing of this information in AI-driven data validation, data–interoperability, analytics, or interactivity. For example, downstream understanding that a class in a custom user-schema is modeling “doors” could be captured in terms of an applicable standard vocabulary—from IFC, Uniclass, or BIS—which other data consumers could operate on.

The Base Infrastructure Schema (BIS) is a family of modular schemas that express taxonomy, data structure, and relationships for modeling federated data from infrastructure engineering. BIS schemas are organized into layers, most of them defining the rules and patterns considered as standard in the BIS ecosystem. The semantics embedded in standard BIS schemas have proven to be valuable in contexts beyond iModels.

We’re working on a 3D Tiles extension for metadata semantics to cover classes, as well as ensuring encoding properly enables BIS and other similar semantic vocabularies.

Integrate reality modeling and AI-powered analysis services with Cesium ion

One of the most exciting outcomes of Cesium joining Bentley is the opportunity to integrate reality modeling and AI-powered analysis services with Cesium ion. You have been vocal about wanting to be able to upload photos to generate 3D models, and we’re thrilled to be able to deliver on that by integrating the industry-leading technology used in iTwin Capture with Cesium ion. By using an integrated end-to-end pipeline from photos to 3D Tiles and from 3D Tiles to rendering and analytics, we aim to provide the best-in-class experience for using reality data in your 3D geospatial applications. With the advancements in 3D Gaussian splats, this will also support the pipeline to go from photos to 3D tilesets of splats.

*Adding Reality Modeling to Cesium ion enables users to create 3D Tiles from photos as input using the reality modeling technology used in iTwin Capture.*

Reality analysis using AI feature detectors

Bentley has also built the technology to identify objects, regions of interest, or defects in reality data using AI. By integrating reality analysis into Cesium ion, we want to enable users to apply 3D segmentation and classification to reality. This analysis will be enabled in two forms:

Embedded metadata in the 3D Tileset, allowing for runtime querying, styling, and filtering.
Supplemental vector data as features extracted from the reality data, which can be used for overlays, annotations, and styling.

We are adding vector data support in Cesium’s native plugins to enable all runtimes to use the results of the reality analysis. The result: real-world insights into reality data, supporting data-driven decisions.

*A 3D Tileset of a point cloud with embedded classification generated using AI-enabled reality analysis service.*

For an initial release, we’ll enable analysis on reality meshes and point clouds, providing you with options through a growing list of curated analysis detectors. You will be able to analyze models already uploaded to ion, or generated with ion’s new reality modeling capabilities, creating an efficient, end-to-end workflow. The curated detectors include analysis for segmentation of trees, railway objects, and city elements, as well as others.

In future releases, you can create your own custom detectors to run in ion. We are planning for 3D analysis of Gaussian splats, 2D segmentation of photo collections, and offering analysis in Cesium ion Self-Hosted.

Expose iTwin Platform APIs and services through Cesium

iTwin Platform powers digital twin solutions that are used in AEC applications to design, build, and operate infrastructure around the world. By connecting iTwin and Cesium platforms, our goal is to provide a complete platform for digital twin experiences for the built and natural environment, with a strong commitment to open source, open standards, and open APIs.

Exposing iTwin Platform APIs and services through Cesium channels will enable you to leverage capabilities like clash detection for design reviews; change elements to monitor design updates; perform analysis, such as carbon calculations; and apply AI-powered federated search to search and filter across 3D objects, engineering information, and business documents. Using iTwin Platform’s synchronization services, you will be able to bring in many more inputs for design models, which can then be added to Cesium apps as 3D Tiles.

*Clash detection and review shown in iTwin.js.*

Introduce composable tiling pipelines to optimize your data as 3D Tiles

Tiling pipelines to create 3D Tiles enable you to bring your own data into your 3D geospatial applications, a core requirement for most use cases. Over the years, Cesium has developed and introduced many tiling pipelines for different types of data: reality data, point clouds, design models, 3D buildings, terrain, imagery, and more. All of these tiling pipelines have their technological merits, but they are siloed based on the type of input data. Adding Bentley’s tiling pipelines for reality data and AEC design models results in even more tiling and optimization algorithms—that are also siloed.

The vision for composable tiling pipelines is to have a unified, modularized, plugin-based pipeline that allows users to configure the stages for data conversion, tiling, and optimization. Example stages can include input file format conversions, coordinate system operations, mesh decimation, texture simplification, choice of tiling schemes, texture atlasing, mesh and texture compression, metadata processing, incremental updates, 3D Tiles-to-3D Tiles optimization, and output validation. The stages will be connected using a graph structure allowing the mix and match of stages, with a set of standard ready-to-use pipelines available to all of our users.

Improvements to AEC-to-3D Tiles workflows

Community reception of Cesium’s Design Tiler and Cesium ion for Autodesk Revit Add-In has been overwhelmingly positive, with lots of input on more design applications and features to be supported. With additions from Bentley software, we aim to level up the workflows for AEC data in key areas:

Support for other CAD formats to 3D Tiles, such as Navisworks, MicroStation, and SketchUp.
Ability to do incremental near real-time tiling. As you are creating and updating your designs, we want to add the ability to tile only the changed elements, rather than the entire model. This will shorten the life cycle by having a ready-to-go 3D Tileset alongside your design models at all times. This will be complemented by lightweight apps to synchronize your designs into a geospatial experience.
Enhancements to the LOD generation in the Design Tiler to better support even larger and high-detail projects.
Specialized 3D Tiles selection algorithms using metadata-driven rendering, filtering, and search.

*Triangle mesh of an AEC model with 10 million triangles created in Autodesk Revit and visualized in CesiumJS using 3D Tiles.*

If you’d like to see specific software or features supported, reach out to us on the community forum.

Advance Cesium runtime engines to capitalize on new 3D Tiles extensions and product integrations

Cesium’s open source runtime engines—CesiumJS for the browser and plugins for Unreal Engine, Unity, and NVIDIA Omniverse—are the face of the platform as the software users experience and interact with the most. The roadmap for runtime engines will advance the convergence of built and natural environment, AEC and geospatial, and of iTwin Platform and Cesium, providing developers and users with extended capabilities.

The biggest impact for runtime engines will be the implementation and deep integration of the aforementioned updates to the 3D Tiles specification including 3D Gaussian splats, voxels, time-dynamic 3D Tiles, geometry highlights, implicit surfaces, and more. The addition of these advancements in every runtime will be a giant leap forward for many industries and applications.

With the aim to lower the barrier to entry for building applications that integrate iTwin Platform APIs and services, we’ll add developer APIs and user interfaces for easy integration with iTwins. This means you will be able to build 3D geospatial applications integrating engineering and business information from iTwin into Cesium applications, similar to services like Cesium ion.

In addition to the cross-cutting updates we covered above, we’re also making engine-specific improvements:

CesiumJS

Add configurable preset camera options to allow other navigation types, such as first-person walkthroughs in indoor spaces, underground, and undersea, toward more immersive applications.
Improve vector data visualization, including rendering improvements to lines rendering, billboards, and overall scalability of these systems.
Support multiple viewports for different views in the same runtime with optimized streaming and selection algorithms. This benefits a broad spectrum of applications, including AEC, simulation, operational environments, infrastructure inspections, and many more.
Develop a new Sandcastle to make onboarding, prototyping, and learning easier.
Longer term, we’ll explore using WebGPU as a generational advancement in Graphics API. We’re also interested in unifying the 3D Tiles selection and traversal algorithms using Cesium Native via WebAssembly, creating a single reusable library for all Cesium runtimes, resulting in faster updates and efficiency across the board. As we explore these forward-looking upgrades, we’ll continue to invest in the broad compatibility of CesiumJS across browsers, platforms, and devices.

Cesium Native, Cesium for Unreal, Cesium for Unity, and Cesium for Omniverse

Introduce a decoration system for runtime enrichment of 3D geospatial scenes with 3D assets. The system empowers creators building rich 3D geospatial environments to add capabilities like runtime editing of 3D Tiles materials, adding 3D objects to the environment, such as trees, cars, and city furniture, and synchronizing with the cloud. This will allow you to dynamically augment captured reality data with high-fidelity models at runtime, such as from Unreal Engine’s Fab marketplace.
Performance is an evergreen pursuit with all runtime engines, but especially so with the native runtimes, where the benchmarks are real-time games. We’ll continue to invest in performance optimizations that reduce latency, improve game thread usage, lower memory overhead, and overall create a seamless experience.
With the introduction of time-dynamic 3D Tiles for use cases like project scheduling and simulation, we’ll also add capabilities that take advantage of game engine features like animations, physics, controllers, and more. This will help create a full digital twin of not just the environment but also the vehicles, heavy machinery, traffic simulation, equipment, tools, and other IoT devices.

*iModel of a soccer stadium in Singapore visualized in Cesium for Unreal as 3D Tiles using iTwin’s Mesh Export service.*

Expand curated data offerings

Building geospatial applications needs data, which is usually fragmented, unprocessed, and not streaming or runtime ready. We aim to lower the barrier to entry by making global datasets accessible, especially where data may be difficult to collect. Toward that goal, we are:

Upgrading Cesium World Terrain—adding new and higher resolution content across the world.
Adding high-resolution insets on the Moon.
Introducing a planetary dataset for Mars.
Providing analytically derived datasets, such as land-use classification.

In addition to curated data, we’ll continue to invest in collaborations with Microsoft, Google, and other ecosystem partners to add more datasets for the community via Cesium ion.

Provide reusable and configurable user experience tools and widgets

In line with our cross-cutting product and open standards investments for more powerful 3D geospatial applications, we will publish a library of reusable and configurable user experience tools and widgets that leverage existing libraries used by iTwin. These will allow you to build even more powerful applications faster and make them configurable, so that each application can refine the widgets and workflows.

*A representative mockup of the new Sandcastle built using the Strata library for components.*

Improve learning material and ongoing maintenance

Authentic education is part of Cesium’s mission. Over the years, we have written many tutorials, presented dozens of conference talks, provided hundreds of code examples in Sandcastle, and joined the community in answering over 18,000 topics on the forum, and now we are on the cusp of hosting our first developer conference.

As the field of 3D geospatial continues to advance, there is also an increase in demand for accessible, up-to-date learning content. To serve our community, we plan to roll out tutorials on reality modeling, AI feature detectors, 3D Gaussian splats, configuring runtime engines, performance optimization, and using and extending metadata in 3D Tiles. We are also creating a new Sandcastle for the example code repository that has improved modern developer tools like IntelliSense, code validation, and better search and organization so you can easily find what you are looking for.

Finally, ongoing maintenance should not be underrated. It is part of our pact with the community, the heartbeat of the platform, and it generates fresh ideas for our collective roadmap. Pull requests need to be reviewed. Bugs (or fixes!) in browsers or game engines need corresponding updates. Better workflows for engaging with and creating transparency for the community on GitHub and the community forum need to be explored, especially as we introduce iTwin Platform and iTwin Capture. We will increase our essential activity here.

Your input

Our aim is to strengthen and expand the open platform to serve the software developer community building digital twins for the built and natural environment. To do this, we will explore avenues to bring Bentley's capabilities to you, along with increasing our investments in R&D, the open-source community, and industry collaborations.

Throughout Cesium’s history, the community has guided us, and we want to continue to hear from you. What are the products, features, integrations, and bug fixes that you would like to see? Are there capabilities within iTwin or other Bentley products we should prioritize for integration? Are there open standards we should integrate or partnerships we should invest in?

We want to hear all requests—big or small—for what you would like to see from Cesium and for the ecosystem in general. Share your thoughts with us on the community forum.