Automatic deployment with git and gitolite

I read a post recently by Sean Coates about deploy on push. The concept is nothing new: you set up a hook that listens for commits on specific branches or tags, and it then deploys your site from that revision.

Except I'd not done it myself. This is how I got there.

Sean's approach uses Github webhooks, which are a fantastic concept. Basically, once your commit completes, Github will send a JSON-encoded payload to a specific URI. Sean uses this to trigger an API call to a specific page in his website, which will then trigger a deployment activity.

Awesome, this should be easy; I already have a deploy script written that I trigger manually.

One small problem: my site, while in Git, is not on Github. I maintain it on my own Gitolite repository. Which means I needed to write my own hooks.

I originally went down the route of using a post-receive hook. However, I had problems determining what branch the given commit was on, despite a variety of advice I found on the subject on StackOverflow and git mailing lists. I ended up finding a great example using post-update, which was actually perfect for my needs.

In order to keep the post-update script non-blocking when I commit, I made it do very little: It simply determines what branch the commit was on, and if it was the master branch, it touches a specific file on the filesystem and finishes. The entire hook looks like this:

#!/bin/bash
branch=$(git rev-parse --symbolic --abbrev-ref $1)
echo "Commit was for branch $branch"
if [[ "$branch" == "master" ]];then
    echo "Preparing to deploy"
    echo "1" > /var/local/mwop.net.update
fi

Now I needed something to detect such a push, and act on it.

I considered using cron for this; it'd be relatively easy to have it fire up once a minute, and simply act on it. But I decided instead to write a simple little daemon using perl. Perl daemons are trivially easy to write, and if you use module such as Proc::Daemon and follow a few trivial defensive coding practices, you can keep memory leaks contained (or at least minimal). Besides, it gave me a chance to dust off my perl chops.

I decided I'd have it check for the file in 30 second intervals, simply sleeping if no changes were detected. If the file was found, however, it should attempt to deploy. Additionally, I wanted it to quit if it was unable to remove the file (as this could lead to multiple deploy attempts), and log success and failure status of the deploy. The full script looks like this:

#!/usr/bin/perl
use strict;
use warnings;
use Proc::Daemon;

Proc::Daemon::Init;

my $continue = 1;
$SIG{TERM} = sub { $continue = 0 };

my $updateFile   = "/var/local/mwop.net.update";
my $updateScript = "/home/matthew/bin/deploy-mwop";
my $logFile      = "/var/local/mwop.net-deploy.log";
while ($continue) {
    # 30s intervals between iterations
    sleep 30;

    # Check for update file, and restart loop if not found
    unless (-e $updateFile) {
        next;
    }

    # Remove update file
    if (!unlink($updateFile)) {
        # If unable to unlink, we need to quit
        system('echo "' . time() . ': Failed to REMOVE ' . $updateFile . '" >> ' . $logFile);
        $continue = 0;
        next;
    }

    # Deploy
    system($updateScript);
    if ( $? == -1 ) {
        system('echo "' . time() . ': FAILED to deploy: ' . $! . '" >> ' .  $logFile);
    } else {
        system('echo "' . time() . ': Successfully DEPLOYED" >> ' . $logFile);
    }
}

The system() calls for logging could have been done using Perl, but I didn't want to deal with additional error handling and file pointers; simply proxying to the system seemed reasonable and expedient.

When all was ready, I started the above listener, which automatically daemonizes itself. I then installed the post-update hook into my bare repository, and tested it out. And it runs! When I push to master, my site is automatically deployed, typically within 15-20 seconds from completion.

Caveats

This solution, of course, relies on a daemonized process. If that process were to terminate, I'd have no idea until I discovered my site didn't refresh after the most recent push. Clearly, some sort of monitor checking for the status of the daemon should be in place.

Also, note that I'm having this update on changes to the master branch; you may need to adapt it for your own needs, depending on your branching strategy.

Finally, this approach does not address issues that might require a roll-back. Ideally, the script should probably log what revision was current prior to the deployment, allowing roll-back to the previous state. Alternately, the deployment script should create a new clone of the site and swap symlinks to allow quick roll-back when required.

blog comments powered by Disqus