Docker Swarm is arguably the simplest, yet still highly effective, orchestration solution for containerized applications.

The main issue with such a deployment is that the swarm stack is deployed all at once. In other words, services cannot be deployed into a stack individually. An individual service image can be updated, but not the service environment, volumes, etc.

Also, when dealing with multiple services in a stack, there is the challenge of knowing which version of the image to deploy for each service to a specific environment.

Azure DevOps allows one to select the exact versions of the artifacts produced by the CI pipeline. However, if a stack contains a large number of services, this task can become quite complicated.

In this blog, I will describe an approach that we implemented for one of our customers, providing them with fully automated CI/CD pipelines. This solution permitted the controlled deployment of a large number of services into Docker Swarm.

For the purpose of this demonstration, I will employ a three-service stack that I frequently use when educating individuals on Docker use. This stack contains two dummy REST services and a Single Page Application (SPA) frontend that utilizes these downstream services.

Our customer required a workflow similar to this, not exactly the same, but quite close:

Three Git repositories, ‘demo-policy’, ‘demo-customers’ and ‘demo-frontend’ contain the source code for the services. The ‘demo-integration-tests’ repository contains the integration test code, which should run every time any of the service’s CI pipelines succeed. The ‘demo-swarm-deployment’ repository contains the swarm deployment Compose files. Its connected pipeline should be triggered upon successful completion of the integration tests.

The idea behind the flow is that on any commit to any of the service repositories, the CI pipeline performs its tasks, eventually producing a Docker image. Upon successful completion of any of these pipelines, they trigger the integration tests. Once these tests successfully complete, it triggers the swarm deployment pipeline, which prepares current Compose files for the release pipeline.

The release pipeline uses the docker stack deploy command to deploy the stack to the desired swarm.

To track the versions of the images that need to be deployed to each environment, I utilize the artifactz.io service.

First, let’s configure the artifactz.io tenant. For the purpose of this demonstration, I will need four stages defined in artifactz.io:

I will also need to establish a flow that sequentially links these four stages:

Once a Docker Image has been produced, it is placed into the Development stage. It is then progressed through the flow, either automatically or via manual triggering.

Each stack in every environment will use image versions that are recorded against the corresponding stage at that specific moment in time.

One last requirement from artifactz.io is the API token, which should have both read and write access:

Make a note of the token once it’s created.

Next, set up your Azure DevOps organization and project. I installed the ‘Artifactz.io Azure DevOps Extension’ for convenience, which you can find here. We will also need a variable or variable group containing a secret variable with the API token we just created. I will refer to this as artifactz.apiToken in the future.

Everything is now in place. Let’s start configuring the CI pipelines for the demo services. I will use ‘demo-policy’ as an example; however, the setup for the other services will look similar. The pipeline YAML file, which is stored in the project, concludes with the following steps:

  - task: publish-artifact@1
    inputs:
      apiToken: $(artifactz.apiToken)
      stage: 'Development'
      flow: 'Standard'
      name: 'demo-policy'
      type: 'DockerImage'
      version: $(baseVersion).$(Build.BuildNumber)
  - script: |
      echo "##vso[build.updatebuildnumber]$(baseVersion).$(Build.BuildNumber)"
    displayName: 'Update build number'
  - task: push-artifact@1
    inputs:
      apiToken: $(artifactz.apiToken)
      stage: 'Development'
      name: 'demo-policy'

In the pipeline shown above, there are two steps that first publish the artifact to the flow and then push it from the Development to the Integration Test stage.

Despite this being a demo project with no additional tasks, real-world projects will quite likely involve at least one image scanning task.

The ‘demo-integration-tests’ pipeline is configured to be triggered if any of the service pipelines succeed. If the tests pass, it will promote all three images to the UAT stage at the end:

name: $(Rev:r)

trigger:
  branches:
    include:
      - master

resources:
  repositories:
  - repository: self
  pipelines:
  - pipeline: policy-ci
    source: ikolomiyets.demo-policy
    trigger:
      branches:
      - master
  - pipeline: customer-ci
    source: ikolomiyets.demo-customer
    trigger:
      branches:
      - master
  - pipeline: frontend-ci
    source: ikolomiyets.demo-frontend
    trigger:
      branches:
      - master


variables:
  - group: demo

stages:
- stage: Test
  displayName: Run integration tests
  jobs:
  - job: Test
    displayName: Test
    pool:
      vmImage: 'ubuntu-latest'
    steps:
    - task: GoTool@0
      inputs:
        version: '1.21.0'
    - script: |
        go version
        go get -v -t -d ./...
      displayName: 'Get dependencies and verify Go version'
    - script: |
        go run gotest.tools/gotestsum@latest --junitfile unit-tests.xml
      displayName: 'Test'
      env:
        ARTIFACTZ_TOKEN: $(artifactz-token)

    - task: PublishTestResults@2
      inputs:
        testResultsFormat: 'JUnit'
        testResultsFiles: '**/unit-tests.xml'
        failTaskOnFailedTests: true
      condition: succeededOrFailed()
    - task: push-artifact@1
      displayName: 'Push customers image version to the UAT stage'
      inputs:
        apiToken: $(artifactz.apiToken)
        stage: 'Integration Test'
        name: 'demo-customers'
    - task: push-artifact@1
      displayName: 'Push policy image version to the UAT stage'
      inputs:
        apiToken: $(artifactz.apiToken)
        stage: 'Integration Test'
        name: 'demo-policy'
    - task: push-artifact@1
      displayName: 'Push frontend image version to the UAT stage'
      inputs:
        apiToken: $(artifactz.apiToken)
        stage: 'Integration Test'
        name: 'demo-frontend'

In a real-world scenario, it might not necessarily be an automatic promotion, but for the purpose of this demonstration, it serves our purpose.

The culmination of the CI flow is the ‘demo-swarm-deployment’ pipeline, which simply packages the Compose files for release:

name: $(Rev:r)

trigger:
- master

resources:
  repositories:
    - repository: self
  pipelines:
    - pipeline: integration-tests-ci
      source: ikolomiyets.demo-integration-tests
      trigger:
        branches:
          - master

variables:
  - group: demo
  - name: vmImageName
    value: 'ubuntu-latest'

stages:
- stage: Build
  displayName: Prepare manifests
  jobs:
  - job: Prepare
    displayName: Prepare deployment manifests
    pool:
      vmImage: $(vmImageName)
    steps:
    - task: PublishBuildArtifacts@1
      inputs:
        PathtoPublish: '.'
        ArtifactName: 'swarm'
        publishLocation: 'Container'

The release pipeline will resemble the following:

There are three release stages for each environment, with a promotion stage in between that acts as a gate for the manual promotion of the code to the next environment. This is optional as one might prefer software to be automatically promoted to the next stage. In the current example, though, each stage can be triggered if approved by the user, thus preventing automatic promotion.

Let’s take a closer look at the UAT stage definition:

We’ll bypass the first four steps, which are self-explanatory. The three ‘Download…’ steps are only necessary if your agent is deploying a stack to a remote Docker instance.

The version retrieval steps are relatively straightforward:

Each of these steps retrieves a single artifact version at a specific stage and places it into a variable — policy.version, in this particular case.

The final step is a deployment script, which looks as follows:

echo Installing docker client certificate
mkdir -p ~/.docker
echo $(dockerCa.secureFilePath) 
echo $(dockerCert.secureFilePath) 
echo $(dockerKey.secureFilePath) 
cp $(dockerCa.secureFilePath) ~/.docker/ca.pem
cp $(dockerCert.secureFilePath) ~/.docker/cert.pem
cp $(dockerKey.secureFilePath) ~/.docker/key.pem
chmod 640 ~/.docker/key.pem

echo "Preparing versions variables"
[ $policy.version = "undefined" ] && policy.version=latest

[ $customers.version = "undefined" ] && customers.version=latest

[ $frontend.version = "undefined" ] && frontend.version=latest

cd $(Agent.ReleaseDirectory)/demo-swarm-deployment/swarm
echo "Deploying stack to a docker instance at $DOCKER_HOST"
echo "Deploying the following artifacts:"
echo "   policy     - $(policy.version)"
echo "   customer   - $(customers.version)"
echo "   frontend   - $(frontend.version)"

cat docker-compose.template| sed s/__DEMO_POLICY_TAG__/$(policy.version)/g|sed s/__DEMO_CUSTOMERS_TAG__/$(customers.version)/g|sed s/__DEMO_FRONTEND_TAG__/$(frontend.version)/g|docker stack deploy --compose-file docker-compose.uat.yaml --compose-file - demo

It comprises of three parts:

• set up of Docker client certificates (optional if script deploys stack to the local Docker Instance)
• ensure that image versions are not empty, if they are, use the latest image tag
• replace placeholders in the Docker Compose file with concrete image tags and deploy the stack to Docker Swarm

The Docker Compose template file in this case looks like this:

version: "3.7"
services:
  demo-policy:
    image: ikolomiyets/demo-policy:__DEMO_POLICY_TAG__
  demo-customer:
    image: ikolomiyets/demo-customers:__DEMO_CUSTOMERS_TAG__
  demo-frontend:
    image: ikolomiyets/demo-frontend:__DEMO_FRONTEND_TAG__
    configs:
    - source: nginx
      target: /etc/nginx/conf.d/default.conf
configs:
  nginx: 
    file: ./default.conf

The placeholders such as __DEMO_*_TAG__ will be replaced by the script, either with a concrete image tag or with latest, should the tag not be available for any reason.

Wrapping things up

The above demonstration presents a controlled manner to deploy multiple services into a single stack, ensuring the correct versions of the software are running in each environment.

You can find the sources for the pipeline here:

• demo-policy
• demo-customers
• demo-frontend
• demo-integration-tests
• demo-swarm-deployment