Purpose

The purpose of this KB article is to provide context as to why deployment to Google Cloud may fail from Pipeline and how to address this.

Diagnosis

You are running a build and one or more Google deployment steps are failing with the following error:

ERROR: Error response from daemon: authorization denied by plugin pipelines: 
--privileged=true is not allowed

Cause

• When a Docker service is defined in a Pipeline step, Pipeline will automatically mount the Docker client executable under the /usr/bin directory. This removes the requirement of the build image to have Docker client installed. At the time of writing this KB, the docker client version in Pipelines is v20.10.23
• Certain docker images that can be uses as build containers already have Docker client installed
• It has been noticed that some Docker images have Docker client v23 or v24 installed under the path /usr/local/bin
• If /usr/local/bin directory is configured before /usr/bin in the image's $PATH variable, the Pipeline will use the Docker client from /usr/local/bin directory

• Recently, the Google Cloud SDK image updated its Docker version from version 20.10.24 to version 24.0.2

• Since Docker Client v23 (see Docker v23.0 release notes), Docker has set Buildx and BuildKit as the default builder on the Linux environment. With this change when the docker build command executed, it's actually an alias to docker buildx build.
• The buildx command use a --privileged flag when building the image, and this flag is restricted in bitbucket pipelines for security reasons

Solution

To resolve this in Bitbucket Pipelines, you can perform one of the following workarounds:

1. Change the $PATH variable in the build step to configure /usr/bin directory before /usr/local/bin. This configuration will use the Pipeline provided Docker client.

   export PATH=/usr/bin:$PATH

   You can add this command at the beginning of your script in the Pipeline YML file.
   
   

2. Add the following to your Bitbucket YML configuration before the docker build command to force the build to use the legacy builder:

   export DOCKER_BUILDKIT=0

   
   NOTE: You also have the option to create it as a workspace variable (DOCKER_BUILDKIT) so that it applies to all of your repositories and you don't have to modify each YML file manually
   
   

To utilize the Buildkit feature without a workaround, you will need to configure your bitbucket-pipelines.yml to make use of self-hosted runners. Self hosted runner do not have the same privilege flag restrictions (as builds will be running on your own infrastructure):

• First enable the Docker daemon in the Pipeline step by adding it as a service on the build step or in the Global config:

  // Enable globally
  options:
  docker: true
  // Enable on build step
  pipelines:
    default:
      - step:
          script:
            - ...
          services:
            - docker

• Define a Docker service with the docker-in-docker image (dind). The current docker:dind image is already has Buildkit enabled by default. Configure it as shown below:
  

  definitions:
    services:
      docker:
        image: docker:dind

• Enable the Buildkit by setting the DOCKER_BUILDKIT=1 flag before your docker build commands. A complete example is shown below:

  image: atlassian/default-image:2
  definitions:
    services:
      docker:
       image: docker:dind
  options:
    docker: true
  pipelines:
    branches:
      master:
        - step:
            image: google/cloud-sdk:latest
  		  name: "Test build"
            services:
              - docker
            runs-on:
              - self.hosted
              - linux
            script:
              - export DOCKER_BUILDKIT=1
              - docker build .