Build Tooling Updates Coming to CesiumJS
Lately, we’ve been prioritizing developer-oriented improvements with the aim of modernizing CesiumJS and lowering the barrier to entry for new developers. These latest updates let maintainers take advantage of faster build times during development, while end users get the benefit of smaller distributions (16% smaller minified, and 33% smaller unminified) for faster load times.
New tooling has greatly shortened build times, which in turn shortens the iteration loop for developers.
The build changes we’ve made have resulted in much faster build times. The development flow has also been streamlined by revising existing build tasks and moving to a “just-in-time” method of bundling at runtime during development.
Due to streamlined bundling and removing extraneous whitespace and comments, we’re happy to ship smaller releases, both minified and unminified. Besides saving space, this will benefit the file load times in browsers.
As an additional benefit of the latest changes, since we now always bundle instead of loading every Cesium module in a browser at once, we work around a long running Chrome on Linux development issue.
When the CesiumJS project first got started, it was the first of its kind in many ways. At that time RequireJS was the most prominent tool for dependency management, and we wrote our code as Asynchronous Module Definitions (AMD). As we moved to ES6 JavaScript modules, we were able to level up our builds using Rollup.
CesiumJS has, up until now, erred on the side of lightweight, minimal build processes. The most substantial build process involved combining all of our source code files into one main file for distribution. We did not majorly transform any code. What we wrote in our source code needed to work in all environments.
Since those early days, the JS environment has changed drastically. Tools have gotten faster, and developers are more used to preprocessing their source code. It's time to re-evaluate.
We’ve chosen to use esbuild, replacing Rollup for our main bundling tasks. First and foremost, esbuild is fast. This is essential if we want our build to complete in a timeframe appropriate for a fast-moving developer. Second, while rollup plugins provide a lot of utility, esbuild provides much of the functionality we need out of the box. This includes automatically bundling third party code from the node_modules directory, code transformation, and minification. Finally, this allowed us to decrease our bundle sizes, both minified and unminified, for faster loading in the browser.
One major limitation was building the workers. Web Workers are standalone JavaScript scripts which run as background processes. In CesiumJS, they are used extensively for tasks such as creating geometry and decoding data.
Firefox is the last holdout among browsers for supporting ESM modules in workers, meaning we must use IIFE or AMD worker. However, esbuild only supports code splitting for ESM, and without code splitting, the worker files significantly increased in size. To avoid this increase, we’ve chosen to stick with our existing approach using Rollup for bundling web workers until the Firefox support is there, at which time we can revisit our approach.
Beyond tooling updates, we took the opportunity to re-evaluate the build process holistically, including the granularity of the build scripts.
The biggest change is a new approach to development builds. Thanks to the accelerated build speed esbuild allows (particularly the lightning fast incremental rebuilds), we’re now using a “just-in-time” approach. Instead of requiring ahead-of-time builds (npm run build) to run locally during development, developers now only need to install the necessary dependencies once and start the server with:
npm install
npm start
Over time, build scripts have been added piecemeal as tools were added and configurations changed. And as a result there was a hodgepodge of scripts defined in the gulpfile, many with different naming conventions. We re-evaluated what processes are still relevant and combined tasks where we could.
The build tasks in particular have been pared down to:
- build: Builds Cesium.js and related files
- build-watch: Starts a watch task that incrementally re-builds when a file changes
- build-ts: Generates TypeScript definitions
- build-apps: Creates the built version of development apps like Sandcastle and Cesium Viewer
- build-docs: Generates documentation pages
release: Runs the optimized build for production use cases and runs all other build tasks needed to create a CesiumJS release.
The build task has several options, but namely we break our CesiumJS build down into two categories: one targeted for development, and one targeted for production use.
The development version strives to be easily debuggable. While we can perform code transformations, we make sure that they can be easily mapped back to the original source in the browser tools with a sourcemap. It's also critical that it re-builds quickly for local development.
On the other hand, the production use case is optimized by minified code and stripping out any developer-oriented warnings.
Changes to testing configuration were fairly minimal. The biggest change was that we now want to test the bundled code rather than the source code directly. We already had ways of testing the bundled code for both unminified and minfied releases. So this was mostly a matter of making sure the correct options were passed through.
We now bundle the spec code by default, making sure to keep CesiumJS as an external file so we can swap in the minified or unminified bundles at runtime.
Coverage was the final stumbling block. We previously used karma plugins to instrument our code so we could ensure tests covered all code paths. However, introducing bundling into the mix made it difficult to control the order of operations. We wanted to run coverage tests on the bundled code, just like the other tests.
The solution was to pull out the plugin and instrument the source directly before bundling so we can have finer control over how the files get instrumented. Esbuild plugins come in handy to intercept the Source files as they are loaded and instrument the contents.
Pre-processing lets us make some needed short-term improvements, and will allow us to make other changes in the future like migrating to TypeScript if/when we choose to commit to it.
Once modules can be loaded as workers in Firefox, we will assess our overall worker strategy. Ideally, we could remove RequireJS and Rollup entirely, further speeding up the build process and reducing the size of releases.
Cesium.js is no longer an AMD module. This will affect you differently depending on how you’re using CesiumJS:
- If you are using the CesiumJS
Sourcefiles, a bundler is now required and all third party node modules must now be resolved from node_modules - If you were ingesting individual modules from the combined file
Build/Cesium/Cesium.jsorBuild/CesiumUnminified/Cesium.js, instead useBuild/Cesium/index.jsorBuild/CesiumUnminified/index.js, respectively - If you are loading the combined file (such as by a script tag) and using a global
Cesiumvariable, no changes are required - If you are using CesiumJS through Node.js, no code changes are required
For a production app, we recommend using the Source modules directly, which will allow your build tool of choice to reduce the final build size using tree shaking to eliminate unused code.
Lastly, CESIUM_BASE_URL should be set to either Build/Cesium or Build/CesiumUnminfied, not Source.
Try it out in main and let us know if you have any feedback on the community forum.