Leveling Up Lighting in CesiumJS: IBL, Dynamic Environment Maps, and Ambient Occlusion
Author’s note: Many thanks to my colleague Jeshurun Hembd for his significant contributions to these results. This is part 2 in a series about visual quality improvements in CesiumJS; read part 1 here.
With the growing adoption of Cesium for Unreal and Cesium for Unity, developers of native applications have raised the bar for visual quality in Cesium apps. Specialized game engine features—like dynamic sun and sky lighting components, light probes, and global illumination systems—now can be applied to geospatial use cases. The CesiumJS squad has been working to bring browser-based geospatial apps to a new level with dynamic approaches to lighting.
glTF model of a polished metal sphere reflecting the environment, rendered in CesiumJS using improved image-based lighting and an HDR-image environment map.
Good lighting is essential to make the most out of physically based rendering, or PBR, a rendering technique common to both models and 3D Tiles. Think of a polished metal sphere: It’d appear flat and dull without anything interesting to reflect. To complement recent rendering improvements we’ve made to the glTF material model in CesiumJS, we’ve shipped several enhancements to our lighting system over the last few releases, including:
- Refinements to the image-based lighting pipeline
- Dynamically generated environment maps
- Scale-independent ambient occlusion
New York Buildings 3D Tileset, rendered in CesiumJS with ambient occlusion before these lighting updates (left, version 1.121) and after (right, version 1.125). The overall effect of the changes is more realistic lighting and more easily understood model form.
Image-based lighting
There are many strategies for lighting, but one tried-and-true method that’s been a part of CesiumJS for several years is image-based lighting, or IBL. Lighting for glTF PBR models and 3D Tiles combines a single directional light source (i.e., the sun) and IBL to simulate a variety of dynamic lighting conditions, such as the precise position of the sun based on time and geospatial location or scattering of light through the atmosphere.
glTF model, courtesy of noe-3d.at on Sketchfab, rendered in CesiumJS with directional lighting (left), image-based lighting (middle), and a combination of directional and image-based lighting (right). The combination results in the most apparent dimension and sense of place when viewing the model.
IBL is an appealing approach because it simulates a complex lighting environment—like hemispheric lighting from the sky, diffuse reflections from the ground, and any other light sources in a scene—with a single environment map. This map encodes a high dynamic range of lighting info in an omnidirectional texture, typically with an HDR image supplied by the user or a procedural texture generated at runtime.
Dynamic environment maps
While users have been able to import their own environment maps, this required offline use of external tooling like Khronos’s glTF IBL Sampler for preprocessing, making it less accessible to those without a knowledge of lighting pipelines. With our recent work, we can now use dynamic environment maps in CesiumJS that adapt to changes in time of day, atmospheric conditions, or model positioning. This means CesiumJS developers get the full benefits of IBL without any additional configuration.
Runtime generation on the GPU
Created on the GPU, dynamic environment maps can be updated in near-real time to reflect the current sun position and model location. When time of day, atmosphere settings, or model position change significantly, the environment map can be updated in the background over a few frames.
The Damaged Helmet glTF sample model rendered in CesiumJS with default procedural lighting in version 1.121 (left) and with dynamic environment maps in version 1.125 (right).
For apps with more specific lighting needs, there are options that can be adjusted, like the overall intensity of the light or the color reflected by the ground.
Atmospheric scattering
Our existing shaders for simulating the sky atmosphere based on time of day were flexible enough to be reused by adapting the viewing angle to the model origin. This means we can simulate lighting conditions at sunrise or sunset but also high up in the atmosphere for aircraft and satellites.
A glTF model of a polished metal sphere reflecting the environment at morning (left), noon (middle), and evening (right), as rendered with dynamic environment maps in CesiumJS.
Performance considerations
From the environment map, we can determine both the diffuse lighting as well as the specular reflections on any model or 3D tileset. For low-roughness materials like polished metals, the environment map can be used to look up the reflection coming from any particular direction. Rougher objects scatter light and therefore reflect light in a more diffuse way. To represent this, we need to compute the average light contribution across a range of angles representative of the degree of scattering, an operation which involves sampling the environment map hundreds, if not thousands, of times.
To optimize runtime performance, we precompute values on the GPU when a model is first loaded and then store them in data structures for lookup at runtime. The environment map is stored as a cubemap for quick lookup based on the direction of the reflected ray. Specular maps corresponding to various material roughness levels are stored in texture mipmaps that are used to sample the specular radiance. Finally, diffuse irradiance—that is the sum of all incoming light for any particular direction—is represented with third-order spherical harmonic coefficients, a total of nine sets of RBG values.
Data structures used for IBL precomputed on the GPU in CesiumJS: From the environment map, a cubemap is created with each increasing mipmap level representing the specular reflections of materials with increasing roughness. The most diffuse lighting is stored as an array of spherical harmonic coefficients.
Cubemaps
Another key improvement to our existing IBL implementation came from revisiting our cubemapping. We switched our environment map pipeline from octahedral projection mapping, a method which worked great as a stopgap solution in WebGL 1, to native cubemaps supported by WebGL 2. The simplified approach removed redundant work on the GPU and made it significantly easier to fix bugs with LOD selection for specular maps.
Ambient occlusion
Ambient occlusion, or AO, is key for understanding the form and depth of untextured models, such as those seen in OSM Buildings data or architectural design models. This technique darkens areas which would receive less ambient or indirect light, like corners or creases, creating a look of soft shadows. We use a post-processing screenspace-based approach, which is applied to a scene after it has been rendered.
glTF model, courtesy of noe-3d.at on Sketchfab, rendered in CesiumJS with combined lighting effects. Ambient occlusion alone highlights detail in the model’s form. When combined with image-based and directional lighting, we can approximate effects similar to a global lighting system without the need for baking lighting or other preprocessing.
As part of this effort, we fixed existing bugs with our screenspace implementation of AO and enabled a new scalable solution for use across a variety of scenarios.
Horizon-based ambient occlusion (HBAO)
Horizon-based ambient occlusion, or HBAO, uses sampling to determine the minimum angle at which the sky hemisphere is no longer visible from a position. When this is repeated across multiple directions, we have an estimate about the percentage of light in the scene that reaches each pixel. This technique helps visually communicate depth, form, and spatial proximity between objects in the scene, and it can be performed in screenspace for better runtime performance without the need to precompute lighting.
Global scalability
The key improvement made in our implementation was enhanced scalability. While typical AO use cases, like games or architectural visualization, are constrained to one scale, CesiumJS users can travel from outer space to underground in a single app. Previously, the AO parameters had to be configured manually based on trial and error to balance visual appearance with runtime performance. This was inconvenient for anything other than a specific view.
The limitation was the step size used to march along each sampling ray. Rather than using a fixed distance measured in meters, we implemented a normalized Gaussian distribution to scale the sample distance based on the distance from the camera, removing the need for any manual configuration. Occlusion now scales effectively from detailed interiors like pipes in a power plant to large cityscapes like buildings in New York City. Additionally, the falloff is now much smoother: As distance from the camera increases, the shadowed areas become wider, but less dark. In effect, it’s a much closer approximation of how light is occluded in reality.
glTF model, courtesy of noe-3d.at on Sketchfab, rendered in CesiumJS with ambient occlusion only. Scaling the sampling distance ensures views of the model at any scale, by default and without adjusting any settings.
glTF model showcasing diffuse lighting, courtesy of noe-3d.at on Sketchfab, rendered in CesiumJS before these lighting updates (left, version 1.121) and after (right, version 1.125). The overall effect of the changes is more realistic lighting and more apparent fine details.
Our efforts to enhance the visual quality of CesiumJS are always ongoing. We generated many ideas during this process—like an easier-to-use API for using HDR images and enhancements to how we render shadows—but your feedback is invaluable as we strive to further improve CesiumJS. Check out the Sandcastle example used throughout this blog, and please share your input on visual quality in this community thread.