Using Ansible to deploy Drupal

Here's an example of an Ansible playbook which deploys Drupal. It's probably not perfect/maybe even 'wrong' in some way, I don't know. I'm teaching myself Ansible in the process :)

- hosts: all
  gather_facts: no
   - name: Backup database
     shell: "{{ item }}"
      - mkdir -p ~jenkins/dbbackups
      - drush @{{ shortname }}_{{ env }} sql-dump --skip-tables-key=common | gzip > ~jenkins/dbbackups/{{ shortname }}_{{ env }}_prior_to_{{ build }}.sql.gz

   - name: Clone repo as new build
      repo: "{{ repo }}"
      dest: /home/jenkins/{{ shortname}}_{{ env }}_{{ build }}
      version: "{{ branch }}"
      accept_hostkey: yes

   - name: Move build into position
     command: mv /home/jenkins/{{ shortname}}_{{ env }}_{{ build }} /var/www/{{ shortname}}_{{ env }}_{{ build }}
     sudo: true

   - name: Assign symlinks
     command: "{{ item }} chdir=/var/www/{{ shortname}}_{{ env }}_{{ build }}/www/sites/{{ url }}"
      - ln -s /var/www/shared/{{ shortname }}_{{ env}} settings.php
      - ln -s /var/www/shared/{{ shortname }}_{{ env}}_files files

   - name: Harden permissions
     shell: "{{ item }}"
      - find /var/www/{{ shortname}}_{{ env }}_{{ build }} -type d -print0 | xargs -0 -r chmod 555
      - find /var/www/{{ shortname}}_{{ env }}_{{ build }} -type f -print0 | xargs -0 -r chmod 444
     sudo: true

   - name: Test new build
     shell: chdir=/var/www/{{ shortname}}_{{ env }}_{{ build }}/www/sites/{{ url }} drush status

   - name: Maintenance mode on
     command: drush @{{ shortname }}_{{ env }} -y vset maintenance_mode 1

   - name: Apply updates
     shell: chdir=/var/www/{{ shortname}}_{{ env }}_{{ build }}/www/sites/{{ url }} drush -y updatedb
     register: result
     ignore_errors: true
   - debug: var=result.stderr.split('\n')

   - name: Revert database
     shell: if [ -f ~jenkins/dbbackups/{{ shortname}}_{{ env }}_prior_to_{{ build }}.sql.gz ]; then drush -y @{{ shortname }}_{{ env }} sql-drop; zcat ~jenkins/dbbackups/{{ shortname}}_{{ env }}_prior_to_{{ build }}.sql.gz | drush @{{ shortname }}_{{ env }} sql-cli; exit 1; fi
     when: result|failed

   - name: Maintenance mode off
     command: drush @{{ shortname }}_{{ env }} -y vset maintenance_mode 0

   - name: Clear cache
     command: drush @{{ shortname }}_{{ env }} -y cc all

   - name: Take new build live
     command: "{{ item }} chdir=/var/www"
      - unlink live.{{ shortname }}.{{ env }}
      - ln -s {{ shortname}}_{{ env }}_{{ build }} live.{{ shortname }}.{{ env }}
      - drush @{{ shortname }}_{{ env }} cc all
     sudo: true

   - name: Remove old builds to conserve disk space
     command: /usr/local/bin/ -d /var/www -r {{ shortname }} -b {{ env }} -k 5
     sudo: true

It is deceptively simple, but there are some sophisticated components of this approach to Drupal deployment which are characteristic perhaps of my philosophy on deployments more than anything else, and share a lot in common with traditional Capistrano-style deployments. Summed up, they are:

  • Abstract, abstract, abstract. This playbook can be driven this with Jenkins, and have it pass useful unique variables specific to the project in question (such as git repo etc, as well as the build ID). For example, a Jenkins 'Exec' command would be:
  • ansible-playbook --limit example /home/jenkins/ansible/playbooks/deploy.yml --extra-vars shortname=example env=master build=build_57 repo=<a href=""></a>:mig5/ branch=master
  • Always take a database backup first! We can use this later as a snapshot to revert back to if the 'drush updatedb' fails mid-schema change (e.g perhaps it successfully applies 4 hook_updates, but not the next one. Uh oh!)
  • Clone a totally separate new 'build' each time. No cd'ing into one monolithic checkout and running 'git pull'. Instead, perform some tests and then, once satisfied, switch a 'live' symlink to point to the new build. It is this 'live' symlink which is used as the DocumentRoot or 'root' path in your Apache or Nginx vhost configuration, respectively. Thus you don't need to change any config to serve the new build, and it is trivial to revert back to the previous build (just point the symlink back at it!)

    What about stuff that doesn't live in git? You will note that I symlink up a config which contains database credentials, as well as the 'files' directory, from somewhere outside the site root. This happens on every build.

  • Perform at least a basic 'drush status' against the new build. A simple PHP syntax error will cause the bootstrap to abort, and therefore so should your deployment, nice and early. Your site remains on the existing, previous build, unaffected. You could consider putting other tests at this phase of the deployment too. If it breaks the database, you could add another revert step here to restore the database snapshot.
  • Take the site offline before applying updates with drush updatedb. Take it out of maintenance mode again afterward, but note that visits in-between can actually cache the 'maintenance_mode' variable in Drupal's cache, resulting in the site appearing to stay down for longer than it really is. So, keep calm. Clear the cache too.
  • I have a script that removes all but the last X builds, to conserve disk space.

There are obviously some other 'rules' that I have standardised on, such as:

  • The repositories having the same directory structure (you could also use Drush Make to make this easier)
  • Drush aliases are defined by a shortname_environment syntax, as are certain things like the live symlink, the and the shared 'files' area, so that each site has its own files folder of course

In summary, Ansible is pretty neat for deployments. The above is based on a port from a Fabfile (Fabric) that I use. In many ways I still prefer Fabric, as it 'gets out of my way' when I want to do more complex conditionals, private functions etc.

For example, it feels easier to set up a new site from scratch in Fabric, because so much more work is required in terms of automation (generating random passwords etc).

I find Ansible a bit stiff in its 'task by task' approach, but for basic deployments this works well. Also (and maybe this is due to using a slightly older version - I am not a bleeding edge hipster), stderr output (which is where Drush sends some of its output) is seriously ugly, and Ansible doesn't seem to have a nice 'stdout_lines' equivalent for stderr_lines (am I wrong about this?). This might also be Drush's fault. It seems the \n line breaks are after every word - at least this is how it appears in the Jenkins console output:

TASK: [debug var=result.stdout_lines] *****************************************
ok: [] => {
    "result.stdout_lines": [
        "No           [success]",
        "Finished          [ok]",

Probably once I am more familiar with Ansible, I'll find ways of doing more complex Fabfile-ish stuff.

Meanwhile if you are interested in this approach to deployment, you should know that you can hire a Drupal-savvy sysadmin consultant for this sort of thing: i.e, me!