Are you sure your CI setup is doing anything useful?

I’ve been thinking a lot lately about how Continuous Integration (CI) is used, especially within large distributed teams and kept coming back to a lightning talk I delivered at the Scottish Ruby Conference in 2010.

It talked about the common strategy of running your CI tool off of your ‘master’ or ‘gold’ branch – which is probably the one that ends up going into production – once someone has pushed their code up to that branch of your repository. I’ll call that branch ‘deployable’.

The problem with that is that if someone on your team hasn’t, or couldn’t run your tests / build your application before pushing up – you’ve got broken code right there in the branch that should always be deployable and your CI tool going crazy.

If you’re relying on your CI telling you that your ‘deployable’ branch is broken then you’ve got these problems:

  1. You will end up with something that isn’t deployable
  2. You aren’t making best use of CI as a tool to maintain quality control
  3. You’re stuffed if the CI fails, and might be deploying something that’s broken

The simple solution I see (and explain a little in the presentation slides) is that you probably have an intermediate branch, one where everyone’s changes are pushed to and the CI tool is run against, verifying that the build is successful / the tests pass.

Once that’s successful, and only if all the tests past, the CI tool (or some other mechanism) moves the correct code over to ‘deployable’ branch ready to go out whenever you want to.

The problem of coverage

The elephant in the room is that this I in ‘CI’ stands for Integration, and really there’s not much of that going on.

What would be useful is taking this one step further by checking the new code against the current ‘deployable’ revision – making sure that coverage of tests has increased by a configureable threshold where code has been added or changed.

You could even have your CI tool physically merging in new feature branches that are ready to do, but only after checking their tests pass individually, coverage (and maybe other code-quality metrics) are suitable and that the resulting merge into the ‘deployable’ branch continues to pass.

This is something that, in my opinion, a good team should be doing anyway as part of a solid peer-review driven quality control process. Doing this entirely manually will take time, however – so why not make your CI tool actually ‘integrate’ for you and act like the last bastion of defence for bad code making it into your ‘deployable’ branch?

The presentation