Summary

When using Bitbucket Pipelines to build docker images, some use cases might require the image to be built to run in multiple CPU architectures, such as ARM64, ARMv7, x86_64, etc. With the introduction of docker buildx command, it's now possible to build images for different platforms using the QEMU processor emulator, which is essentially a software-based emulation tool that eliminates the requirement of having hosts that run on different CPU architectures. QEMU will translate the instructions of a given architecture to the instruction set of the host machine.

This article provides instructions on how to configure your bitbucket-pipelines.yml file using QEMU to build multi-architecture images in a Linux self-hosted runner.

Environment

Solution

Following are described the required steps to construct your bitbucket-pipelines.yml file to build multi-architecture images in a Linux docker self-hosted runners: 

1. Create and start a Linux docker self-hosted runner.
   If you haven't already, create a Linux docker self-hosted runner under your repository or workspace, and start the runner with the provided pre-configured command.
   
   

2. Define a custom docker daemon
   The default pipelines docker daemon does not allow the usage of --privileged flag. When using self-hosted runners, it's possible to define a custom docker daemon that does not have this restriction, such as the docker:dind :

   definitions:
     services:
       docker: # Define a custom docker daemon - can only be used with a self-hosted runner
         image: docker:dind

3. Enable docker CLI experimental Feature
   As buildx is considered an experimental feature in Docker v20(version used by Pipelines), you'll need to enable the experimental feature by exporting the environment variable DOCKER_CLI_EXPERIMENTAL :

   export DOCKER_CLI_EXPERIMENTAL=enabled

4. Download buildx plugin from the source and move it to the docker-cli plugins folder
   Buildx is a docker plugin that does not come by default in most of the images. If your docker images don't have buildx already installed, you can download it directly from the official releases page :

   https://github.com/docker/buildx/releases

   and move the file to the $HOME/.docker/cli-plugins/ folder as per the instruction of Buildx - Manual Download.

   mv docker-buildx $HOME/.docker/cli-plugins/ && chmod +x ~/.docker/cli-plugins/docker-buildx

5. Run the multiarch/qemu-user-static to setup the QEMU emulation environment
   The multiarch/qemu-user-static image already comes with the QEMU simulator for several architectures and with the scripts to setup those simulators into the host's kernel. This requires this image to be executed with the --privileged flag :

   docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

6. Create a new buildx builder :
   With the buildx plugin installed and QEMU emulation setup, it's now necessary to create a builder that buildx can use  : 

   docker buildx create --use

   The --use flag will make buildx already switch to the newly created builder. 
   
   

7. Build the multi-architecture image with buildx build
   Finally, we are ready to execute the buildx build command specifying the --platform flag with the desired target architectures to build against : 

   docker buildx build -t $DOCKER_HUB_USER/my_image_multi_arch:my_tag --push --platform linux/amd64,linux/arm64,linux/arm/v7 .

   By default, buildx will only save the resulting image on docker’s internal build cache, so it will not show up when executing the docker images command. To store the image, you will have to push it to the Docker registry (e.g DockerHub). The command above already includes the --push flag and assumes you have previously logged in to Dockerhub with the user $DOCKER_HUB_USER.
   
   

Below is an example of the resulting bitbucket-pipelines.yml file following the instructions above: 

pipelines:
  default:
    - step:
        name: My Runner
        image: atlassian/default-image:4
        runs-on:
          - linux
          - self.hosted
        services:
          - docker
        script:
          - echo "Executing on self-hosted runner" 
          - export DOCKER_CLI_EXPERIMENTAL=enabled # Enable usage of buildx in Docker version < 23
          - export BUILDX_VERSION=0.11.0  # define what BUILDX_VERSION to download and install
          - curl -fsSLO https://github.com/docker/buildx/releases/download/v${BUILDX_VERSION}/buildx-v${BUILDX_VERSION}.linux-amd64
          - mkdir -p $HOME/.docker/cli-plugins/ && mv buildx-v${BUILDX_VERSION}.linux-amd64 $HOME/.docker/cli-plugins/docker-buildx && chmod +x ~/.docker/cli-plugins/docker-buildx # download buildx and move it to the docker plugin folder
          - docker run --rm --privileged multiarch/qemu-user-static --reset -p yes; docker buildx create --use # setup the QEMU emulation environment
          - echo $DOCKER_HUB_PASS | docker login --username $DOCKER_HUB_USER --password-stdin # login to Dockerhub 
          - docker buildx build -t $DOCKER_HUB_USER/my_image_multi_arch:my_tag --push --platform linux/amd64,linux/arm64,linux/arm/v7 . # Build multi-architecture image for the platforms provided and push to Dockerhub
definitions:
  services:
    docker: # Define a custom docker daemon - can only be used with a self-hosted runner
      image: docker:dind