Tag Archives: software

How to get started with continuous integration.

I’ve put off learning to use continuous integration tools for a few years now. There’s never a good time to complicate my development process. But today I finally did it. It works differently than I thought, and was easier than I expected, so it’s worth documenting the process for others.

I have a non-trivial number of GitHub repositories that are primarily data-based—that is, there’s no software to be tested, but instead the validity of data. I’m forever lazily checking in YAML, JSON, and XML that’s malformed, or doesn’t comply with the schema. I try to remember to test locally, but sometimes I forget. And sometimes others contribute code to my projects, and I have no way of knowing how they’ve tested those modifications.

So today I set up a Travis CI account. I started with their “Travis CI for Complete Beginners” guide, which immediately proved to be poorly named and generally confusing. Instead, I mucked around a bit until I figured things out.

Here are the salient points about Travis CI:

  • Once you link your GitHub account to Travis CI, every commit that you make is reported to their server, to (potentially) be tested.
  • By creating a .travis.yml file in a repo, you are telling Travis CI “please test this repository with each commit.”
  • The .travis.yml file tells Travis CI exactly what to test and how.
  • Travis CI runs tests by launching a little Docker container (I assume), per the specs established in your .travis.yml config file, and executing the program that you instruct it to run. That program might be a shell script or it might be something you write in any language that you want. You keep it in your repo (perhaps in a /tests/ directory) with everything else.
  • A test fails or succeeds based on your test program’s return status. If it returns 0, it succeeded, otherwise it failed. If the build fails, Travis CI will email you with your program’s output.

tl;dr: With every commit that you make, Travis CI runs one or more commands of your choosing (e.g., a linter) and, if that throws an error, you’ll be emailed a notification.

For example, here’s the .travis.yml file that I wrote:

language: node_js
node_js:
  - "stable"
script: tests/validate.sh
before_install:
  - npm install jsonlint -g

This tells Travis CI to launch a Docker instance that supports Node.js—using the most recent stable version—and install the jsonlint Node module (a JSON validator). Then it should run the script at tests/validate.sh within the repository.

validate.sh looks like this:

#!/bin/sh
find schemas -type f -exec jsonlint -q '{}' +

It’s simply making sure that every file in the /schemas/ directory is valid JSON. I could run all kinds of tests within this script, of course, but this is all that I’m interested in right now.

And that’s it! Every time that I make a commit to this repository, Travis CI is notified, it loads the config file, runs that test program, and tells me if it returns an error.

Of course, there’s a lot more that I need to learn about continuous integration. I should be writing tests for all of my software and running those tests on each commit. I imagine that continuous integration gets a lot more highfalutin’ than what I’ve done so far. But this new practice, if I’m good about making a habit of it, will improve the quality of my work and help make me a better member of the open source software world.

Links for December 25th

  • New York Times: Climate Scientists Hampered in Study of 2011 Extremes
    “'I’ve been a meteorologist 30 years and never seen a year that comes close to matching 2011 for the number of astounding, extreme weather events,' Jeffrey Masters, a co-founder of the popular Web site Weather Underground, said last month. 'Looking back in the historical record, which goes back to the late 1800s, I can’t find anything that compares, either.'”
  • Brent Simmons: ‘Gamification’ sucks
    Yes, yes it does.
  • Wikipedia: J. W. Westcott II
    This little boat, sailing out of Detroit, is the world's only post office boat that delivers mail to other ships under sail. It's got its own ZIP code: 48222. There can't be very many post offices that actually come to you.

Links for May 5th

  • Wikipedia: Timeline of Web Browsers
    A family tree for web browsers. There are a lot of browsers here I hadn't thought of for years. HotJava, ViolaWWW, and Cello, in particular.
  • Voice of America: Historian Recounts Role of Chinese Americans Who Fought in US Civil War
    In 1861, there were only 200 Asians living in the Eastern U.S. Fifty-eight of them fought in the Civil War, at least five of whom fought for the CSA. Two of the Confederates were Christopher and Stephen Bunker, the sons of famed Siamese twins Chang and Eng, who owned slaves on their North Carolina farm.
  • New York Times: The Ultimate Kentucky Derby
    A simulation pitting the last twenty Kentucky Derby winners against each other. (Barbaro repeats his 2006 win.) Without any context as to how the simulation calculates the winner, this is just an amusement for horse racing fans.
  • The First State of the Union Message
    President George Washington to Congress, 1790: "[T]here is nothing which can better deserve your patronage, than the promotion of Science and Literature."