Alas, it took us some time to deliver the continuation on part 1 of our series. We were quite busy doing work for our customers. But here it is. So, thanks for all your patience!

While part 1 sketched the bigger picture of how to use Ansible in combination with Gitlab CI to automate delivery of your software to various environments, this article details how to deploy your Gitlab artifacts using Ansible. That is, we will take a look at a complete pipeline that - in the end - will deploy our Spring Boot application on a staging environment.

We will start with an overview of the IT landscape involved. This should bring us up to speed as well again:

Figure 1: Continuous Delivery
Figure 1: Continuous Delivery

Some parts of this sketch should be familiar. We have the developer that pushes code https://git-scm.com/) to a Gitlab server (step 0), which in turn triggers the pipeline (step 1). The pipeline sets all the rest in motion: its jobs are responsible for building and testing the application (step 2), publishing the built artifact to a Maven repository (step 3), triggering the Ansible powered deployment on the staging servers (step 4) which uses the repository of step 3 to download the artifact(s) (step 5) and, finally, (re)starts the updated application (step 6).

In what follows, we will take a closer look at the jobs within the pipeline. Remember, a pipeline is nothing but a series of jobs Gitlab will execute for us. The first job within the pipeline is to build and test the Spring boot application.

The pipeline for a given code repository is defined in a file called .gitlab-ci.yml. This is, what the definition of the steps 2 and 3 of our pipeline could look like:

variables:
  # This will supress any download for dependencies and plugins or upload messages which would clutter the console log.
  # `showDateTime` will show the passed time in milliseconds. You need to specify `--batch-mode` to make this work.
  MAVEN_OPTS: "-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
  # As of Maven 3.3.0 instead of this you may define these options in `.mvn/maven.config` so the same config is used
  # when running from the command line.
  # `installAtEnd` and `deployAtEnd`are only effective with recent version of the corresponding plugins.
  MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true"
  POSTGRES_DB: our_test_project_database
  POSTGRES_USER: our_test_project_user
  POSTGRES_PASSWORD: our_test_project_password

cache:
  paths:
    - /root/.m2/repository

stages:
  - build
  - test
  - release

validate:jdk8:
  stage: build
  script:
    - 'mvn $MAVEN_CLI_OPTS test-compile' 
  image: maven:3.5.0-jdk-8

deploy:jdk8:
  stage: test
  services:
    - postgres:9.6
  script:
    - 'mvn --settings settings.xml $MAVEN_CLI_OPTS -Dspring.profiles.active=gitlab deploy'
  image: maven:3.5.0-jdk-8

release_staging:
  stage: release
  image: williamyeh/ansible:centos7
  only: 
    - master
  tags:
    - ansible
  script:
    - 'ansible-playbook -i staging deploy.yml'

Let's walk through this step by step. At first, we declare some variables we use later on:

variables:
  # This will supress any download for dependencies and plugins or upload messages which would clutter the console log.
  # `showDateTime` will show the passed time in milliseconds. You need to specify `--batch-mode` to make this work.
  MAVEN_OPTS: "-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
  # As of Maven 3.3.0 instead of this you may define these options in `.mvn/maven.config` so the same config is used
  # when running from the command line.
  # `installAtEnd` and `deployAtEnd`are only effective with recent version of the corresponding plugins.
  MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true"
  POSTGRES_DB: our_test_project_database
  POSTGRES_USER: our_test_project_user
  POSTGRES_PASSWORD: our_test_project_password

Next, we declare a list of paths (actually, in this case it is just one) that should be cached between jobs. We do not want Maven to download all dependencies over and over again:

cache:
  paths:
    - /root/.m2/repository

After that is done, we declare three stages. A stage can be used to group different jobs. Jobs that belong to the same stage are executed in parallel. Our three stages are build, test, and release:

stages:
  - build
  - test
  - release

There is one other thing that stages do: Jobs belonging to a particular stage are only executed if the jobs of the preceeding stage have succeeded. In this case, jobs of the test stage would only be triggered if the jobs build stage have been executed successfully. That is quite handy.

Here is what the sequence of stages of the pipeline look like in Gitlab:

Figure 2: Overview of the pipeline.
Figure 2: Overview of the pipeline.

After all the preliminaries are dealt with, we can now go on defining the jobs that Gitlab should execute for us. The first is called validate:jdk8:

validate:jdk8:
  stage: build
  script:
    - 'mvn $MAVEN_CLI_OPTS test-compile' 
  image: maven:3.5.0-jdk-8

This job belongs to the build stage and has the following two properties: (1) It executes mvn test-compile with the $MVN_CLI_OPTS we have defined in the variables section. This will process the resources, compile the application and the tests. (2) It runs within a Docker image, in this case the maven:3.5.0-jdk-8 image found at Docker Hub.

The next job is the deploy:jdk8 job. It's definition looks like that:

deploy:jdk8:
  stage: test
  services:
    - postgres:9.6
  script:
    - 'mvn --settings settings.xml $MAVEN_CLI_OPTS -Dspring.profiles.active=gitlab deploy'
  image: maven:3.5.0-jdk-8

It is a job of the test stage. It runs Maven again and uses the same docker image as the validate:jdk8 job does. In addition to that, however, it needs a running PostgreSQL. It uses the variables set above in the variables section. Yes, it's that easy. The database Gitlab provides us with is available under the hostname postgres. We therefore change the JDBC connection string by selecting a Spring profile created for this purposes.

The last job is the release job we've all been waiting for. Let's take another look at it:

release_staging:
  stage: release
  image: williamyeh/ansible:centos7
  only: 
    - master
  tags:
    - ansible
  script:
    - 'ansible-playbook -i staging deploy.yml'

In this case, we release our Spring Boot application to a staging environment. This job is part of the release stage and is only triggered when the master branch changes. (Note: This is not true for the previously defined jobs. Those are triggered by every commited and pushed change to the repository.)

It is possible to not automatically (re)deploy the application for every change in the master branch. You can add when: manual to the release_staging entry in the .gitlab-ci.yml to let Gitlab know that deploying a change to the staging environment requires manual triggering by a developer. (This can be done by pressing the proper button within the pipeline section of that Gitlab project, i.e. the "play button" on the pipeline image below.) The whole entry would then look like this:

release_staging:
  stage: release
  image: williamyeh/ansible:centos7
  only: 
    - master
  when: manual
  tags:
    - ansible
  script:
    - 'ansible-playbook -i staging deploy.yml'

Figure 3: Pipeline with manual deployment.
Figure 3: Pipeline with manual deployment.

The actual work in this job is done by Ansible. The job itself does one thing only: Execute ansible-playbook.

Another thing to note is the tags element. A tag is a means to select a specific Gitlab runner for this job. A runner can have one or more tags. This job has the ansible tag. That means that the Gitlab runner for this tag is selected here.

The image we use here is a high-quality user-provided image of CentOS 7 providing Ansible for us. It executes the playbook we've talked about earlier.

The playbook is the next thing we should take a look at in detail. Here it is in its entirety:

---
- hosts: web
  become: true
  tasks:
    - name: Install python setuptools
      apt: name=python-setuptools state=present
    - name: Install pip
      easy_install: name=pip state=present
    - name: Install lxml
      pip: name=lxml
    - name: Download latest snapshot
      maven_artifact:
       group_id: our.springboot.application
       artifact_id: our_springboot_artifact_id
       repository_url: http://our_springboot_server.bevuta.com:8080/artifactory/our_springboot_application/
       username: our_springboot_user
       password: our_springboot_password
       dest: /home/our/springboot_application.jar
    - name: Copy systemd unit file
      copy: src=our_springboot_application.service dest=/etc/systemd/system/our_springboot_application.service
    - name: Start web interface
      systemd:
        state: restarted
        name: our_springboot_application
        enabled: yes
        daemon_reload: yes

The first part is rather simple: This playbook defines tasks for all hosts belonging to the group web and yes, Ansible should gain root rights on the target machine(s), that is why we set become: true.

The rest of it defines tasks Ansible should execute for us. At first, we install some Python tools Ansible needs to execute the maven_artifact task we ask it to do later on:

- name: Install python setuptools
  apt: name=python-setuptools state=present
- name: Install pip
  easy_install: name=pip state=present
- name: Install lxml
  pip: name=lxml

This is not very elegant, alas. Maybe you know of a better way to do this? Let us know!

After we've prepared the target machine(s), we download the artifact Gitlab has built for us (step 2) and uploaded to the Maven repository (step 3 of our pipeline, remember?):

- name: Download latest snapshot
  maven_artifact:
    group_id: our.springboot.application
    artifact_id: our_springboot_artifact_id
    repository_url: http://our_springboot_server.bevuta.com:8080/artifactory/our_springboot_application/
    username: our_springboot_user
    password: our_springboot_password
    dest: /home/our/springboot_application.jar

At last (and finally) we (automatically) start our Spring Boot application by enabling a systemd unit whose unit file we let Ansible create on the target machine(s):

- name: Copy systemd unit file
  copy: src=our_springboot_application.service dest=/etc/systemd/system/our_springboot_application.service
- name: Start web interface
  systemd:
    state: restarted
    name: our_springboot_application
    enabled: yes
    daemon_reload: yes

A unit file describes a so-called systemd unit, that is, "a service, a socket, a device, a mount point, an automount point, a swap file or partition, a start-up target, a watched file system path …" ( man/systemd.unit).

In our case, Ansible produces a unit file that describes to systemd how it should handle our_springboot_application, namely that it should be started automatically whenever the system boots.

And that's it. Our Spring Boot application has been build, tested, uploaded to a maven repository, downloaded, deployed and started. All that automatically.