2

We're running a self-hosted GitLab CE instance. I'm trying to construct a very particular ruleset that works in our project environment. I'm aware of the basic building blocks: branch pipelines and MR pipelines, workflow:rules for pipelines, rules for jobs, predefined $CI_* variables, etc. I'm having trouble bringing them all together in a way that would also avoid duplicate pipelines, a common problem that may require very specific rule definitions to overcome.

We have a main branch from which feature branches are created, and merged back via merge requests. Occasionally some trivial fixes are pushed directly to main.

We've defined a pipeline with three jobs, currently all set to when: manual until we figure out this scheme. The eventual goal is this:

  • pre-build tests (stage: .pre)
    • Always runs as the first job, when the pipeline runs.
    • Has allow_failure: false – if the tests don't pass, no further jobs should run.
  • build (stage: build)
    • Conditions detailed below.
  • deploy to production (stage: deploy)
    • Should only be run manually in all situations.
    • Has dependencies: [build]

The conditions we'd like to have for the creation of the entire pipeline:

  • A merge request for a feature branch is merged to main: run the pipeline for main.
  • Commits are pushed directly to main: run the pipeline for main.
  • (Side note: we want to prevent duplicate pipelines from the above two conditions.)
  • Commits are pushed to a new or existing feature branch: run the pipeline for the feature branch. Does not matter if it has an open MR or not.
  • All other cases (tags, new MR creation from/to any branch, etc.): DON'T run the pipeline.

If the pipeline runs, a couple of extra conditions for the build job specifically:

  • The pipeline is running for the main branch (via a push or an MR, doesn't matter): always build, assuming the tests passed.
  • The pipeline is running for a feature branch: must build manually.

Achieving this setup is probably not THAT many rules in total, but I'd like to not have to do hours of trial and error if this is something that an expert can readily help with.

Improve this question

1 Answer 1

Reset to default
2

I eventually figured it out. Here's how the above conditions are met, just in case they're of use to someone else down the line:

workflow:
  rules:
    # Explicitly enable merge request pipelines to be created (not done by default)
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    # Prevent duplicate pipelines from push events to branches that are the source
    # branch of one or more open merge requests. If $CI_OPEN_MERGE_REQUESTS is
    # non-empty on a branch pipeline, it means that the the above rule has already
    # caused (or will cause) a merge request pipeline to run. These rules are
    # separately evaluated for the merge request event resulting from the same push.
    - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
      when: never
    # Run a normal branch pipeline on a branch that isn't the source of an open MR
    - if: $CI_COMMIT_BRANCH


pre-build tests:
  stage: .pre
  when: always
  allow_failure: false
  # etc.

build:
  stage: build
  rules:
    # Automatically done assuming tests pass, but only on the default branch.
    # Also runs when a feature branch MR is merged to the default branch; the
    # pipeline source for that is again `push`, not `merge_request_event`.
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: on_success
    - when: manual
  # etc.

deploy to production:
  stage: deploy
  when: manual
  dependencies: [build]
  # etc.

The one part of the question that I couldn't do is this one:

new MR creation from/to any branch: DON'T run the pipeline

...while also satisfying this one (much more important):

Commits are pushed to a new or existing feature branch: run the pipeline for the feature branch.

A merge request pipeline is run when a merge request is created for an existing branch. When that branch was first pushed, a branch pipeline was run for a certain $CI_COMMIT_SHA. When the MR creation event pipeline runs, it builds for that same SHA. (However, this doesn't constitute a "duplicate pipeline" as described, because the pipelines are created by two different user actions—unlike a single push to an MR source branch without the $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS trick.)

There's no official way of preventing this; a related GitLab issue can be found here, open as of this writing. Thankfully, the practical impact of this is negligible in our case.

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.