Parallel steps

On this page

Still need help?

The Atlassian Community is here for you.

Ask the community

Introduction

Parallel steps enable you to build and test faster, by running a set of steps at the same time.

There is a limit of 10 for the total number of steps you can run in a pipeline, regardless of whether they are running in parallel or serial.

A way you might want to use this feature is:

  1. A single initial step to build the software
  2. A set of parallel steps to test your software:
    • unit tests
    • lint
    • integration test
    • browser test
  3. Final steps to deploy to a set of environments in order:
    1. test environment (automatically)
    2. staging (manual)
    3. production (manual)

Define your environment in Docker containers as much as possible and cache external build dependencies as much as possible in the first step, so you don't waste time doing it in each of your testing steps. Then, by running all the tests concurrently you'll get your results more quickly.

Configuration

Configuring steps to run in parallel is easy, just indent the parallel steps:

pipelines:
  default:
    - step:          # non-parallel step
        script:
          - ./build.sh
    - parallel:      # these 2 steps will run in parallel
        - step:
            script:
              - ./integration-tests.sh --batch 1
        - step:
            script:
              - ./integration-tests.sh --batch 2
    - step:          # non-parallel step
        script:
          - ./deploy.sh

Artifacts

Parallel steps can produce and consume artifacts. Things to be aware of:

  • Parallel steps can only use artifacts produced by previous steps, not by steps in the same parallel set.
  • Steps after the parallel set will get a combination of all files produced.
  • If parallel steps produce artifacts containing a file at the same location, conflicts are resolved on a per-file basis, the files generated by latest step in the YAML file will win.

The image below shows how artifacts can be combined.

Image showing that latest versions of files take precedence

Default variables

If you want to do some fancy logic in your scripts, the following variables are available during the parallel steps:

  • BITBUCKET_PARALLEL_STEP - zero-based index of the current step in the group, e.g. 0, 1, 2, …
  • BITBUCKET_PARALLEL_STEP_COUNT - total number of steps in the group, e.g. 5.

Build minutes

Be aware that the total number of build minutes used by a pipeline will not change if you make the steps parallel, but you'll be able to see the results sooner. The quicker you have the results, the quicker your iteration cycle can be.

Last modified on Nov 26, 2018

Was this helpful?

Yes
No
Provide feedback about this article
Powered by Confluence and Scroll Viewport.