Skip to main content

Website Continuous Integration with Travis CI, Jekyll, gulp, and GitHub

Gabby Getz

We built the Cesium website using Jekyll, a static site generator. Jekyll allows both technical and non-technical content creators to write blog posts and other pages in Markdown format, which Jekyll converts into an HTML page. Some of the other tools we use are Bootstrap for layouts, Less for styling, and gulp for building the site.

Using Travis CI, we automate the build, validation, and review process so that we can have the confidence our changes aren’t breaking anything and we can focus on what matters: the content of the site.

After a contributor pushes content to GitHub or opens a pull request, Travis is triggered and it builds the site, runs validation on the content, and deploys the built site to a temporary URL for review.

After this runs, we get a status at the bottom of our pull requests letting us know that this branch builds and passes our tests.

And if any of our checks fail, we check the Travis logs to see what we have to fix.

Configuration

We configure the following in our .travis.yml file, which includes installing the necessary dependencies to build the site and run checks.

Since Jekyll is a Ruby tool, we specify our language as Ruby. Node.js and Python are already installed in the Travis build environment. We run npm install to install the Node dependencies specified in our package.json.

language: ruby
rvm:
  - 2.2
before_install:
  - npm install
  - pip install --user pillow
install: gem install jekyll html-proofer
script: npm run travis
env:
  global:
  - NOKOGIRI_USE_SYSTEM_LIBRARIES=true # speeds up installation of html-proofer

npm run travis is the meat of the Travis build and runs the build steps and the validation steps by executing gulp tasks.

Build

First we have a gulp task that compiles the .less files and outputs them to the proper directory.

// compiles less files and outputs them to css
gulp.task('compile-less', function() {
    return gulp.src(path.join(cssPath, '*.less'))
        .pipe(less())
        .pipe(gulp.dest(cssPath));
});

Then we have a another gulp task that depends on the abovecompile-lesstask. It executes the Jekyll command to build the website.

// Builds the static website with Jekyll
gulp.task('jekyll', ['compile-less'], function(done) {
    execute('jekyll build --future --destination '
        + path.join('..', websiteOutputDirectory), {
        cwd: websiteInputDirectory
    }, done);
});

function execute (cmd, opts, done) {
    util.log(util.colors.cyan(cmd));
    exec(cmd, opts,
        function (error, stdout, stderr) {
            util.log(util.colors.cyan(stdout));
            util.log(util.colors.red(stderr));
            done(error);
        }
    );
}

Validation

We run several validations on the output of the build to have better confidence in what we are deploying to our site. Each of the checks is an individual gulp task that we run in sequence as part of the travis script.

HTML Proofer

HTML Proofer validates the HTML of the output site and checks references to links, images, and scripts as well.

// Validates html and links
gulp.task('html-proofer', function(done) {
    execute('htmlproof ' +
        buildOutputDirectory +
        // html-proofer options
    , done());
});

function execute (cmd, opts, done) {
    util.log(util.colors.cyan(cmd));
    exec(cmd, opts,
        function (error, stdout, stderr) {
            util.log(util.colors.cyan(stdout));
            util.log(util.colors.red(stderr));
            done(error);
        }
    );
}

Bootlint

We use Bootstrap for our website layouts to provide a responsive website for both large and small screens. In order to validate our Bootstrap classes, we use the Bootlint tool, which scans the HTML output and lists any errors in use.

// Validates bootstrap
gulp.task('bootlint', function() {
    return gulp.src(htmlFiles)
        .pipe(bootlint({
            stoponerror: true,
            // bootlint options
        }));
});

Image Check

We also wrote a custom script to verify that the images we are including in the site are a reasonable size. It’s a common error to include monstrously large images, which both load slowly on the website and bloat the size of of GitHub repository.

The script uses a Python library called Pillow to check the file size, image dimensions, and image format to make sure the image file is as small as possible.

##Deploy

Travis allows us to automatically deploy our code once a build finishes. This allows us to host each branch of code so the result can be easily reviewed by both technical and non-technical contributors. There is no longer a need to check out a branch and build the site locally in order to review changes.

We deploy to S3 using this configuration in our .travis.yml:

deploy:
  provider: s3
  access_key_id: $AWS_ACCESS_KEY
  secret_access_key: $AWS_SECRET_KEY
  bucket: $BUCKET_NAME
  skip_cleanup: true
  local_dir: build
  upload-dir: cesium-website/$TRAVIS_BRANCH
  on:
    all_branches: true

Travis provides abefore_deploystep and anafter_deploystep, which we use to do the following. Before the deploy, we have a Node script that makes a request to the GitHub API to set a status named ‘deploy’ to ‘pending.’ After the site is successfully deployed, we run another Node script that sets this status to ‘success’ and provides the url to the built and deployed site.

function setStatus (state, targetUrl, description, context) {
    request.post({
        url:  'https://api.github.com/repos/' + process.env.TRAVIS_REPO_SLUG + 
              '/statuses/' + process.env.TRAVIS_COMMIT,
        json: true,
        headers : {
            'Authorization' : 'token ' + process.env.TOKEN,
            'User-Agent' : 'Cesium-Website'
        },
        body : {
            state : state,
            target_url : targetUrl,
            description : description,
            context : context
        }
    }, function (error, response, body) {
        if (error) {
            return console.error(error);
        }
    });
}

This results in an easy-to-use way to know the deploy has completed and to navigate to the build url.

Details

This continuous integration process set up with Travis allows a smoother development process and allows our contributors to be productive by spending time developing and creating content instead of building the site and searching for errors. There is also plenty of room to grow this process, as there are a multitude of tools available for validating content, such as spell checkers, YAML validators, and more!