Cesium ion

Getting started

This step-by-step guide will help get you up and running with Cesium ion for Kubernetes. While we expect you to already be experienced with running Kubernetes workloads, this guide is written to take you from zero to 3D Tiles with no prior knowledge.

What’s included

  • startHere.html - Cesium ion documentation (this file)

  • images/cesium-ion-asset-server.tar - Container image for the Cesium ion Asset Server

  • images/cesium-ion-tiling.tar - Container image for the Cesium ion 3D Tiling Pipeline

  • cesium-ion/ - Helm Charts for installing Cesium ion onto a Kubernetes cluster

  • sampleData/ - Sample data used throughout the documentation

  • thirdParty/ - Supporting code and data used by startHere.html

System requirements

Any x86-64 compatible system capable of running Kubernetes can also run Cesium ion. Requirements for production workloads will depend directly on your use case and Cesium ion performance will scale with additional CPU and RAM. For local development, we recommend a cluster with the following properties:

  • An x86-64 compatible processor

  • 4 or more CPU cores.

  • 32GB of RAM or greater.

  • At least a 32GB of volume storage

Additionally, to follow this guide you will need sudo, admin, or equivalent permissions.

Installing microk8s

This guide uses microk8s, a lightweight and easy to configure Kubernetes implementation. If you have an existing Kubernetes cluster you would like to use, you will need to change the supplied commands to those available with your Kubernetes installation. Skip to the Importing Images section if you are not installing microk8s.

  • Linux

  • Windows

  • MacOS

Run the below command to install microk8s using snap.

sudo snap install microk8s --classic

Update permissions and configuration

Once installed, run the below command to add yourself to the microk8s user group

sudo usermod -a -G microk8s $USER

You also need to create a .kube directory in your home folder to store microk8s configuration. Run the following commands:

mkdir ~/.kube
sudo chown -f -R $USER ~/.kube
sudo microk8s config > ~/.kube/config

Since we made changes to your group settings, you must log out and log back in before continuing.

We recommend using WSL2 to set up a microk8s environment. Use the official provided WSL2 microk8s installation instructions from: https://microk8s.io/docs/install-wsl2

Update permissions and configuration

Once installed, run the below command to add yourself to the microk8s user group

sudo usermod -a -G microk8s $USER

You also need to create a .kube directory in your home folder to store microk8s configuration. Run the following commands:

mkdir ~/.kube
sudo chown -f -R $USER ~/.kube
sudo microk8s config > ~/.kube/config

Since we made changes to your group settings, you must close out of WSL2 and run it again for the changes to take place.

When working with WSL2, you can easily access files on your root drive but you must use Unix style paths. For example, if your root drive is C:\ you can access files at \mnt\c

Microk8s uses HomeBrew on MacOS. Run the following commands to install microk8s.

brew install ubuntu/microk8s/microk8s

Now run the microk8s installer with the following command. This ensures that the node created by the VM has enough resources for the tiling jobs to get scheduled and function properly. It will also prompt and install multipass for you:

microk8s install --cpu 4 --mem 12 --disk 128
The next step moves your extracted files into your home directory (~/). This is needed because certain folders like Downloads and Documents have built-in ACLs (extra security configurations). This might give you permissioning errors in your microk8s VM if your mounted folder is within an ACL directory.

Move the extracted folder into your home directory:

mv /path/to/extracted/folder/ ~/cesium-ion-self-hosted

Change directories into the copied directory from above and mount the cesium ion self hosted directory with the following:

multipass mount . microk8s-vm:/cesium-ion

Verify installation

Once installed, you can confirm everything is operating as normal by running the following command:

microk8s status --wait-ready

This should produce output similar to the following. The important part is "microk8s is running". If you receive an error, review the Installing microk8s section.

microk8s is running
high-availability: no
  datastore master nodes: 127.0.0.1:19001
  datastore standby nodes: none
addons:
  enabled:
    ha-cluster           # (core) Configure high availability on the current node
    helm                 # (core) Helm - the package manager for Kubernetes
    helm3                # (core) Helm 3 - the package manager for Kubernetes
    metrics-server       # (core) K8s Metrics Server for API access to service metrics
  disabled:
    cert-manager         # (core) Cloud native certificate management
    community            # (core) The community addons repository
    dashboard            # (core) The Kubernetes dashboard
    dns                  # (core) CoreDNS
    gpu                  # (core) Automatic enablement of Nvidia CUDA
    host-access          # (core) Allow Pods connecting to Host services smoothly
    hostpath-storage     # (core) Storage class; allocates storage from host directory
    ingress              # (core) Ingress controller for external access
    kube-ovn             # (core) An advanced network fabric for Kubernetes
    mayastor             # (core) OpenEBS MayaStor
    metallb              # (core) Load balancer for your Kubernetes cluster
    minio                # (core) MinIO object storage
    observability        # (core) A lightweight observability stack for logs, traces and metrics
    prometheus           # (core) Prometheus operator for monitoring and logging
    rbac                 # (core) Role-Based Access Control for authorization
    registry             # (core) Private image registry exposed on localhost:32000
    storage              # (core) Alias to hostpath-storage add-on, deprecated

Verify required features

helm3 should be enabled by default and nothing else is required to install and run the application. If it is not shown as enabled, you should run the following commands:

microk8s enable helm3
microk8s config > $HOME/.kube/config

Microk8s installs kubectl and helm. You can run them as microk8s kubectl and kubectl helm to administer the cluster.

Importing images

Run the below commands to import the images into microk8s. Importing these images may take a few minutes for each step.

microk8s ctr image import images/cesium-ion-asset-server.tar
microk8s ctr image import images/cesium-ion-tiling.tar

Installing the Helm chart

While cesium-ion/values.yaml contains a number of configuration variables that can be modified to suit your exact architecture and application needs, for the sake of this tutorial we will only perform the minimum required configuration.

Volume configuration

Cesium ion tiling needs access to both an input volume and an output volume. We will also configure the asset server to use the same output volume for hosting. Follow these steps to configure them:

When editing cesium-ion/values.yaml on Mac and Windows, care must be taken.
On Windows: File paths should be from inside the WSL2 shell (/mnt/c/…​).
On MacOS: The file paths should represent the file paths from inside the microk8s VM. Remember to also mount the file paths you use here onto your microk8s VM with the necessary permissions.
  1. Run microk8s kubectl get nodes and copy the name of the node. You will need it below.

    • A node in Kubernetes refers to the hardware where the pod is run. In this case your machine will be the only node.

  2. Open cesium-ion/values.yaml in a text editor.

  3. Find the localPersistentVolumes section

  4. Under cesium-ion-tileset-volume, replace # REQUIRED: Name returned by "kubectl get nodes" with the node name from step 1.

  5. Under cesium-ion-sources-volume, replace # REQUIRED: Name returned by "kubectl get nodes" with the same node name from step 1.

  6. Create a directory for your tileset data.

  7. Under cesium-ion-tileset-volume, replace # REQUIRED: Path to output data on your filesystem. with the absolute path to the directory you created in step 6.

  8. Create a directory for your source data. This must be different from the tileset data directory in Step 6.

  9. Under cesium-ion-sources-volume, replace # REQUIRED: Path to input data on your filesystem. with the absolute path to the directory you created in step 8.

When you are done the localPersistentVolumes section should contain the correct information based on your install.

localPersistentVolumes:
  - name: cesium-ion-tileset-volume
    node: "[NODE NAME FROM STEP 1]"
    path: "[DIRECTORY FROM STEP 6]"
    capacity: 32Gi
  - name: cesium-ion-sources-volume
    node: "[NODE NAME FROM STEP 1]"
    path: "[DIRECTORY FROM STEP 8]"
    capacity: 32Gi

We’ve provided several types of sample data for you to play with. But you need to copy the data to a place where the tiling pipeline can access it. For this tutorial, that’s the directory you created in step 8 above. Copy the included sample data to that directory now by running the following command:

cp -r sampleData/* [DIRECTORY FROM STEP 8]

License configuration

Cesium ion requires a 3D Tiling Pipeline license, which is configured at the top of cesium-ion/values.yaml. By default it will be an empty string:

license: ""

Install your license by performing the following steps:

  1. Downloaded your license file from https://cesium.com/downloads

  2. Open the license file in a text editor and copy the entire contents into the clipboard

  3. Paste the contents into the license string at top of cesium-ion/values.yaml

Running the install command

Make sure you saved the changes to cesium-ion/values.yaml in the previous section and then install the chart by running the following command:

microk8s helm install cesium-ion cesium-ion/ --values cesium-ion/values.yaml

You should see output similar to the below.

NAME: cesium-ion
LAST DEPLOYED: Fri May 19 16:47:56 2023
NAMESPACE: default
STATUS: deployed
REVISION: 1
NOTES:
1. Get the application URL by running these commands:
  export CLUSTER_PORT=$(kubectl get --namespace default -o jsonpath="{.spec.ports[0].port}" services cesium-ion)
  export CLUSTER_IP=$(kubectl get --namespace default -o jsonpath="{.spec.clusterIP}" services cesium-ion)
  echo http://$CLUSTER_IP:$CLUSTER_PORT

  If DNS is configured, it can be accessed via
    http://assets.cesium-ion.local/

2. In order to tile your asset, you need to create a Job. You can generate the template by running the following command:

helm template cesium-ion/ --values cesium-ion/values.yaml -s templates/job.yaml

The above indicates success. You can ignore the output for now as we’ll provide explicit next steps below.

If you received an error instead, run microk8s helm uninstall cesium-ion to ensure any partially installed components are removed. Then review this section to ensure you didn’t miss a step and try again.

After the initial install, additional changes to cesium-ion/values.yaml will not take effect unless you run the helm upgrade command after modifying the file:

microk8s helm upgrade cesium-ion cesium-ion/ --values cesium-ion/values.yaml

Accessing the asset server

Run the below command to get the url of the asset server within the kubernetes cluster:

echo http://$(microk8s kubectl get --namespace default -o jsonpath="{.spec.clusterIP}" services cesium-ion):$(microk8s kubectl get --namespace default -o jsonpath="{.spec.ports[0].port}" services cesium-ion)
  • Linux

  • Windows

  • MacOS

Visit this url in the browser of your choice and you should be rewarded with a successful loading of the landing page. Be sure to read the landing page before continuing as it contains additional important information.

Currently WSL2 on Windows does not support accessing IP addresses other than local IP (0.0.0.0) out of the box. The easiest way to expose our application is to use Kubernetes port forwarding. With port forwarding, you can expose the ClusterIP IP address from the above step to your Windows host. Open a new WSL2 window and use the following command:

microk8s kubectl port-forward service/cesium-ion --address=0.0.0.0 8070

Visit http://localhost:8070 in the browser of your choice and you will be rewarded with a successful loading. Be sure to read the landing page before continuing as it contains additional important information.

To expose the asset server from the multipass VM to the host, we recommend using Kubernetes port forwarding. With port forwarding, you can expose the ClusterIP IP address from the above step to your MacOS host. Open a new terminal tab and use the following command:

microk8s kubectl port-forward service/cesium-ion --address=0.0.0.0 8070

Visit http://localhost:8070 in the browser of your choice and you will be rewarded with a successful loading. Be sure to read the landing page before continuing as it contains additional important information.

Tiling and viewing data

Now that the asset server is running, we can tile and view the sample data.

The Cesium ion tiling pipeline runs as a Kubernetes Job. You create one job for each tileset you wish to create. You can retrieve the job template at any time and store it in a file by running the below command. Do this now:

microk8s helm template cesium-ion/ --values cesium-ion/values.yaml -s templates/job.yaml > my_first_job.yaml

Open this file in your favorite text editor and look over the contents. In addition to standard Kubernetes job properties you might be familiar with, you will also see comments for configurable options marked as either REQUIRED or OPTIONAL. The only two required items are:

  1. A unique name for the job in metadata.name. A unique name was already generated for you when you ran the above command

  2. The containers.command which is the command line that configures and runs the tiling pipeline.

Reality models

For our first job, let us tile the reality capture OBJ model in /data/sources/Office_Park/Office_Park.obj. Cesium ion supports reality models in glTF, DAE, FBX, or OBJ format.

First, change the name: property at the top of my_first_job.yaml to cesium-ion-tiling-reality-model.

Next, find the command: section of my_first_job.yaml and replace the entire entry with the below. Remember that indentation is very important in yaml files. We’ve pre-indented the text for you, but command: should line up with the imagePullPolicy: Never above it.

          command:
            - "pipeline"
            - "--input"
            - "/data/sources/Office_Park/Office_Park.obj"
            - "--input-type"
            - "3D_CAPTURE"
            - "--output"
            - "/data/tilesets/Office_Park"
            - "--output-type"
            - "3DTILES"

While the 3D Tiling Pipeline reference guide has a full explanation of all options. Here is a brief summary of the above command:

  • pipeline - The Cesium ion pipeline executable. This is identical for all jobs.

  • --input /data/sources/Office_Park/Office_Park.obj - Specifies the absolute path to the input data in a directory that was mounted as part of the job template. Multiple input files are supported and are merged into a single tileset. Zip files are also supported and will be automatically decompressed before processing.

  • --input-type 3D_CAPTURE - The type of data we are tiling, in this case a 3D Capture, mesh, or reality model.

  • --output /data/tilesets/Office_Park - Specifies the absolute path to the output file in a directory that was mounted as part of the job template.

  • --output-type 3DTILES - The type of tileset to produce, in this case 3D Tiles.

We’re ready to create our first tileset. Save and close my_first_job.yaml and then launch the job by running the below command:

microk8s kubectl create -f my_first_job.yaml

You should see a message similar to the below, indicating success:

job.batch/cesium-ion-tiling-reality-model created

This small model only takes a few seconds to complete, but you can check the status by running the below command:

microk8s kubectl get pods

Which will show output similar to the following:

NAME                                    READY   STATUS    RESTARTS   AGE
cesium-ion-75c8576f6f-clx7g             1/1     Running   0          13h
cesium-ion-tiling-reality-model-sbxk6   1/1     Running   0          4s

The pod running your job will be named starting with cesium-ion-tiling-reality-model- and a unique suffix which will be different for you. The other pod is the asset server we created above and can be ignored.

If the status is Complete, the job already succeeded, Error indicates failure, and Running or anything else is considered in progress.

Once the job completes it will automatically clean itself up after 120 seconds unless you configure it with a different value. Before that time, you can run the below command to see the job log:

microk8s kubectl logs POD_NAME_FROM_ABOVE

Normally there is no need to look at the logs unless you are debugging a problem or working with Cesium support.

Since we’ve started the asset server in development mode, you can now go back to the landing page in your browser and click on View all tilesets. The list should now contain a single tileset with an id of Office_Park. The id was determined by the --output parameter in the above command and you can configure it to be whatever you like. For example, it could be a database identifier to data in the application you are building.

Click on Office_Park and you should see the data running in a simple CesiumJS-based viewer:

reality model sample

You can also load this 3D Tiles tileset into your Cesium application or any client that supports 3D Tiles via the YOUR_SERVER_ADDRESS/v1/3dtiles/Office_Park/tileset.json url. See the Cesium Learning center for the exact details of loading 3D Tiles into your Cesium client.

Congratulations, you’ve tiled your first tileset! Let’s do a few more.

Imagery

Cesium ion supports tiling of one or more raster imagery files into a tileset. Create a new job template named imagery.yaml by running the following command:

microk8s helm template cesium-ion/ --values cesium-ion/values.yaml -s templates/job.yaml > imagery.yaml

Open imagery.yaml in a text editor and change the command: section to the below:

          command:
            - pipeline
            - "--input"
            - "/data/sources/Courtyard.tif"
            - "--input-type"
            - "RASTER_IMAGERY"
            - "--output"
            - "/data/tilesets/Courtyard"
            - "--output-type"
            - "IMAGERY"

We’ll rely on the auto-generated name this time. Launch the job by running:

microk8s kubectl create -f imagery.yaml

After a few moments, go back to the View all tilesets page and refresh it. You should now see Courtyard listed in the assets. Click on it to view the data.

It should look similar to the below image:

imagery sample

This is a TileMapService (TMS) imagery tileset that can be loaded into any application that supports TMS layout via the YOUR_SERVER_ADDRESS/v1/imagery/Courtyard/ url.

Terrain

Cesium ion supports tiling of one or more raster terrains into a single tileset. Create a new job template named terrain.yaml by running the following command:

microk8s helm template cesium-ion/ --values cesium-ion/values.yaml -s templates/job.yaml > terrain.yaml

Open terrain.yaml in a text editor and change the command: section to the below:

          command:
            - pipeline
            - "--input"
            - "/data/sources/ZionNationalPark.tif"
            - "--input-type"
            - "RASTER_TERRAIN"
            - "--output"
            - "/data/tilesets/ZionNationalPark"
            - "--output-type"
            - "TERRAIN"

We’ll rely on the auto-generated name again. Launch the job by running:

microk8s kubectl create -f terrain.yaml

After a minute, go back to the View all tilesets page and refresh it. You should now see ZionNationalPark listed in the assets. Click on it to view the data.

It should look similar to the below image:

terrain sample

This is a quantized-mesh terrain tileset that can be loaded into any application that supports quantized-mesh via the YOUR_SERVER_ADDRESS/v1/terrain/ZionNationalPark/ url.

Point clouds

Cesium ion supports tiling of one or more LAS or LAZ point clouds into a single tileset. Create a new job template named points.yaml by running the following command:

microk8s helm template cesium-ion/ --values cesium-ion/values.yaml -s templates/job.yaml > points.yaml

Open points.yaml in a text editor and change the command: section to the below:

          command:
            - pipeline
            - "--input"
            - "/data/sources/House.laz"
            - "--input-type"
            - "POINT_CLOUD"
            - "--output"
            - "/data/tilesets/House"
            - "--output-type"
            - "3DTILES"

We’ll rely on the auto-generated name again. Launch the job by running:

microk8s kubectl create -f points.yaml

After a few moments, go back to the View all tilesets page and refresh it. You should now see House listed in the assets. Click on it to view the data.

It should look similar to the below image:

points sample

This is a 3D Tiles tileset that can be loaded into any application that supports 3D Tiles via the YOUR_SERVER_ADDRESS/v1/3dtiles/House/tileset.json url.

Arbitrary 3D Models

Cesium ion supports tiling of one or more glTF, DAE, FBX, or OBJ models into a single tileset. This option should be used whenever the model is not a reality mesh or similar large mesh. One example is BIM and CAD models.

Create a new job template named cad.yaml by running the following command:

microk8s helm template cesium-ion/ --values cesium-ion/values.yaml -s templates/job.yaml > cad.yaml

Open cad.yaml in a text editor and change the command: section to the below:

          command:
            - pipeline
            - "--input"
            - "/data/sources/OfficePlan/OfficePlan.obj"
            - "--input-type"
            - "3D_MODEL"
            - "--output"
            - "/data/tilesets/OfficePlan"
            - "--output-type"
            - "3DTILES"

We’ll rely on the auto-generated name again. Launch the job by running:

microk8s kubectl create -f cad.yaml

After a few moments, go back to the View all tilesets page and refresh it. You should now see OfficePlan listed in the assets. Click on it to view the data.

It should look similar to the below image: