Speed Up Your CI With Docker (Github Actions)

Joël AZÉMAR
5 min readNov 21, 2021

How to share your Docker image across your jobs with GitHub Actions

This Article will use GitHub Actions as an example; if you are interested in doing the same with Travis-CI, please follow this Article.

The purpose is to create a Docker image from our source code and run our tests through that image. That way, we can speed up our test suite by running it in parallel. But how do we share the image created instead of creating it for each job? That is what we are looking at in this Article.

Github Actions is super popular at that time. If I like it a lot, I have to say it still has room for improvement; for instance, the lack of YAML functionality, like the fact we can’t anchor an alias to a node, makes the configuration lengthy and prone to errors. But as the project advances at an incredible pace, I have no doubt things will improve in no time.

UPDATE: Well, unfortunately, it seems that is not going to happen, check out this short article to see how to work around the lack of YAML Anchors.

We are going to explore two ways to achieve our goal here, Cache Registery and Artifact.

Using GitHub Actions Cache

The complete working file is here.

Github Actions give us access to a cache registry in this section. We are going to use it.

The configuration of GitHub Actions takes place in the file .github/workflows/main.yml. You can name the file whatever you want, and you can add as much as a workflow you need/want under the workflows directory. For the sake of simplicity, we will stick with the default name.

name: Docker Cached Imageon: [push,pull_request]env:
SERVICE_NAME: dummy_project

We declare our first section responsible for creating and storing the Docker Image we will use later. We are going step by step:

First of all, we need to check out the code.

jobs:  image-build-and-cache:  runs-on: ubuntu-latest  steps:
- name: Check Out Code
uses: actions/checkout@v2

To use the Cache, we need to declare it.

...env:
IMAGE_CACHE_DIR: /tmp/cache/docker-image
IMAGE_CACHE_KEY: cache-image
jobs:
...
steps:
...
- name: Create Cache Registry
id: cache-docker-images
uses: actions/cache@v2
with:
path: ${{ env.IMAGE_CACHE_DIR }}
key: ${{ runner.os }}-docker-images-${{ steps.branch-commits.outputs.last_sha }}

We define the key of our Cache depending on the last commit. That way, we ensure our Cache will expire each time we check out a new code. Indeed we want our Cache for the lifetime of the CI builds. To get that SHA, we have several options, Github provides useful environment variables like $GITHUB_SHA, but we will use one of the last features of the new workflows instead for the sake of learning.

name: Docker Cached Image...jobs:  image-build-and-cache:  ...  outputs:
branch: ${{ steps.branch-commits.outputs.branch }}
sha: ${{ steps.branch-commits.outputs.sha_short }}
steps:
...
- name: Get Current Branch and Commit
id: branch-commits
shell: bash
run: |
echo "##[set-output name=branch;]$(echo ${GITHUB_REF#refs/heads/})"
echo "::set-output name=sha_short::$(git rev-parse --short HEAD)"

We just declare our outputs for further use.

Now is the time to build and store our Docker Image.

...
env:
IMAGE_CACHE_DIR: /tmp/cache/docker-image
APP_IMAGE_TAG: dummy_project/app-ci:latest
RUBY_VERSION: 3.0.0
jobs:
...
steps:
...
- name: Build Docker Image from Source Code
if: steps.cache-docker-images.outputs.cache-hit != 'true'
uses: docker/build-push-action@v2
with:
build-args: |
ARG_RUBY_VERSION=${{ env.RUBY_VERSION }}
context: .
file: dockerfiles/Dockerfile-ci
tags: ${{ env.APP_IMAGE_TAG }}
outputs: type=docker,dest=${{ env.IMAGE_CACHE_DIR }}/image-app.tar

The build push action is going to create the Docker image and push the compressed image into the Cache; you can see the last command outputs: as an equivalent of:

docker save --output ${{ env.IMAGE_CACHE_DIR }}/image-app.tar ${{ env.SERVICE_NAME }}/app-ci:latest

We are now ready to run the other jobs using the just-created image.

name: Docker Cached Image...jobs:  image-build-and-cache:   ...    steps:      ...  test-app:    needs: image-build-and-cache    runs-on: ubuntu-latest    steps:      - name: Declare Cache Registry
id: cache-docker-images
uses: actions/cache@v2
with:
path: ${{ env.IMAGE_CACHE_DIR }}
key: ${{ runner.os }}-docker-images-${{ needs.image-build-and-cache.outputs.sha }}

The first thing to notice here is that the second job requires the first one, as you can see with the line:

needs: image-build-and-cache

That will ensure the first job is done before that job starts.

We need to declare our Cache before using it. Unfortunately, without the help of the definition of commons section, this will end up in an old fashion copy past of what we already defined in the first job.

Now we can reach the cache-store and load the previously created image.

name: Docker Cached Image...jobs:  image-build-and-cache:  ...  test-app:    needs: image-build-and-cache    ...    steps:      ...      - name: Docker load
run: |
docker load --input ${{ env.IMAGE_CACHE_DIR }}/image-app.tar

And that is it! We are now able to run whatever we need through our Docker container:

name: Docker Cached Image
...
jobs:
image-build-and-cache:
...
test-app:
needs: image-build-and-cache
...
steps:
...
- name: Run Test
run: |
docker run --rm \
--env DB_HOST=${{ env.SERVICE_NAME }}-db \
--env DB_PASSWORD=${{ env.DB_PASSWORD }} \
--env DB_USERNAME=root \
--network=${{ env.SERVICE_NAME }}-bridge-docker-network \
${{ env.SERVICE_NAME }}/app-ci:${{ env.VERSION }} /bin/bash -c "bin/rails test"

You might have noticed the use of the keyword :if when building the Docker Image. Why are we doing that? Well, it is not that useful if you have only one workflow. However, if you run multiple workflows, once one creates the image and caches it, the remaining workflows will use the Cache and not build the image repeatedly.

We can’t expire the Cache manually, but GitHub will do it for us, check out the politic of eviction.

Using GitHub Artifacts

The complete working file is here.

As an alternative to using the Cache, we can use Artifacts. For that, you need to respective change the two declarations of the Cache by those:

name: Docker Cached Image....jobs:  image-build-and-cache:  ...  steps:
...
- name: Upload docker image
uses: actions/upload-artifact@v2
with:
name: image-app.tar
path: /tmp/image-app.tar
retention-days: 1
... test-app: needs: image-build-and-cache ... steps: ...

- name: Download docker image
uses: actions/download-artifact@v2
with:
name: image-app.tar
path: /tmp
- name: Load docker image
...
- name: Run Test
...

That’s it, and we can now run countless jobs against our built image.

Complete example here.

--

--