Physically-Based Rendering in Cesium

by

With the arrival of glTF 2.0 support in Cesium 1.36, Cesium unlocked a new range of new features. The most groundbreaking feature added with this wave is the introduction of Physically-Based Rendering (PBR) as the approach for how Ceisum shades glTF models.

As a master’s student at the University of Pennsylvania studying Computer Graphics and Game Technology, a lot of my coursework focused on physically-based rendering techniques. During this time, I started a WebGL implementation for PBR using glTF 2.0, which is now the official reference implementation for glTF. As an intern this summer, I was given the opportunity to bring this content to Cesium, too!

Before we can discuss the implementation details and the value gained from PBR in Cesium, it is important to first mention what PBR is. The term itself doesn’t have any specific implementation tied to its definition, but rather refers to any set of techniques used to simulate how light interacts with material surfaces in a physically- and mathematically-cognizant manner. By doing this, we can create highly realistic, game-quality graphics that resemble the wide array of materials we interact with everyday.

What is Physically-Based Rendering?

Physically-based rendering is not a new term in computer graphics; it has been seen predominantly in offline productions such as in animated films and advertisements. However, with the increasing improvements to modern GPUs and an active research field, PBR has recently become a standard in real-time rendering and now Cesium.

Since the field is so grounded in the physics behind how light interacts with surfaces, things can get mathy very quickly. Luckily, it is up to the engine to decide how to implement this math while those who create the models can use a small, yet highly expressive parameterization for materials.

In glTF, a physically-based material is defined by three main properties: base color (albedo), metallicness, and roughness. With these three properties, the content creator can create materials that encompass many of the typical reflective behaviors we commonly see. As seen in the image below, we are able to simulate perfect mirrors (top-left), shiny plastics (bottom-left), dull metals (top-right), matte surfaces (bottom-right), and everything in between.

Parameterization of glTF materials visualized as an array of spheres.

By using textures for these parameters along with the aid of normal maps, emissive maps, and occlusion maps, even more detail can be added to the models to further emphasize realism, as seen in the truck model at the beginning of this post.

From left to right: Base color texture; occlusion, roughness, metallicness (in r, g, b, respectively) texture; normal map.

PBR that is unique to Cesium

With the introduction of glTF 2.0, independent shaders are no longer attached to the models themselves. Instead, it is completely on the engine to decide what physically-based techniques are used, and these techniques are what get applied to all models loaded in. For Cesium, this gave us the opportunity to use the sun’s motion and the atmospheric environment as the main light sources in the scene.

For the lighting of these models, we simulate the incoming light from the sun as a directional light, which means we neglect the distance and shape of the sun and model the incoming light as a single direction based on its position with respect to the earth. Although we have the necessary information to take into account the distance and the spherical shape of the sun, it is sufficiently far away that doing the extra computation is wasteful and doesn’t affect lighting at all in practice. With this, we have the main light source implemented. However, this isn’t enough to shade a model and we actually end up with a very dark scene.

A model rendered with only the sun as a light source.

This might not make sense at first since when we look outside in daylight, nothing ever seems as dark when facing the sun (or even in the shade). The reason for this is because light bounces and scatters around once it reaches the earth. This means we often get indirect lighting from the objects surrounding us and it’s unrealistic to suggest light only ever comes from a single direction.

Real-time rendering hasn’t gotten to the point that we can effectively simulate all of these bouncing light paths at runtime, as is done in a path tracer, but a key observation can be made here. The sky itself can be thought of as a consistent and strong light source, especially in Cesium where there are no clouds to block the sun.

With this in mind, Cesium emulates the incoming light from the environment as a procedurally-implemented reflection of the atmosphere and the globe at runtime. This means that we take into account the altitude of the model and can calculate how far the horizon of the earth is. Once we have this information, we can add in a reflection of the globe (seen as a dark blue reflection) and the atmosphere (seen as a light blue reflection). We can even make sure that the atmosphere sticks to the globe once the model is close enough to space, allowing us to reflect the stars alongside a smaller reflection of the earth.

A model rendered inside the earth's atmosphere.

A model rendered outside of the earth's atmosphere.

By synthesizing these two light sources, Cesium introduces one of the first physically-based rendering implementations that provides astronomically correct reflections!

Present and future gains from PBR

Besides the added benefit of being able to create captivating images, there are many reasons why PBR adds substantial value to Cesium.

For example, since we are dealing with such a compact parameterization for materials, a future addition to Cesium might be to procedurally generate a material description for models captured using techniques like photogrametry. With this, massive models of cities can be effectively lit such that key features like windows and streets accurately reflect the sun based on the generated material descriptions.

Another future use case could involve a user ignoring the “realistic” aspect of PBR and playing with the intuitive metallic and roughness parameters to draw a user’s attention to certain 3D models. For example, a key attribute of smooth, non-metallic materials is that they maintain their color while being very shiny. Cesium applications might want to use this as a way to highlight selected models by pairing it with a subtle base color change.

Finally, being able to render a wider range of physically-plausible materials can simply add to the immersion of applications. Cesium has highly accurate and captivating terrain that pair well with equally realistic 3D models.

To showcase this, we provide a Sandcastle demo in Cesium that demonstrates how faithful a glTF model within Cesium can be to the surrounding context and passing of time.

A timelapse Sandcastle demo within Cesium.

We encourage you to go forth and make cool things with glTF 2.0 PBR! Feel free to show us what you come up with on twitter, @CesiumJS.