18

I have a scheduled pipeline that copies some data from one server to another once a day.

The pipeline works fine on all branches but doesn't start if I select master branch (default branch) as the target branch.

I have tried on an exact copy of master and it worked fine.

I though it could be because master is protected but I tried on a protected copy of master and it worked.

I'm really not sure what's going on. When I click the "play" button next to the scheduled pipeline and it says that the job was successfully scheduled, but I cannot see any job in the job list.

Here some details on the .gitlab-ci.yml

stages:
  - copy_data
  - linting
  - test
  - deploy


lint:
  needs: []
  stage: linting
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH || $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == $CI_DEFAULT_BRANCH'
      changes:
        - "my_project/**/*.py"
  script:
    - ...

test:
  stage: test
  script:
    - ...
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH || $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == $CI_DEFAULT_BRANCH'

copy_database:on-schedule:
  stage: copy_data
  needs: []
  only:
    - schedules
    - $COPY_DB # this is set in the pipeline menu on gitlab
  script:
    - ...
  timeout: 3h
Improve this question

6 Answers 6

Reset to default
35

I finally figured out the problem! The issue is that gitlab was saying Successfully scheduled a pipeline to run. Go to the Pipelines page for details. while in reality there was an error.

To debug it I used the trick described here that is to run a manual pipeline and set CI_PIPELINE_SOURCE = "schedule". Running the pipeline in this way returned the error message and I was able to fix the issue.

If you are wondering what the error was, here is some more detail.

I had two pipelines optionally running on the master branch:

prepare_release:
  stage: prepare_release
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never
    - if: $CI_COMMIT_TAG
      when: never
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: manual
  script:
    - ...

create_release:
  image: registry.gitlab.com/gitlab-org/release-cli:latest
  stage: release
  needs:
    - job: prepare_release
      artifacts: true
  rules:
     // HERE THERE SHOULD BE A RULE TO PREVENT THIS FROM RUNNING ON SCHEDULE
       // - if: $CI_PIPELINE_SOURCE == "schedule"
    //  when: never
    - if: $CI_COMMIT_TAG
      when: never
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
  script:
    - ...

The second pipeline did not have a

    - if: $CI_PIPELINE_SOURCE == "schedule"
      when: never

And therefore tried running on the schedule. However since it needs the previous one and that wasn’t created, it errors. Mi mistake was to assume that the "needs" would take into account this rule from the parent job.

Improve this answer
Sign up to request clarification or add additional context in comments.

6 Comments

IMHO it should be a GitLab bug, when it says the pipeline run "successfully" while it errors under the hood. Is there a GitLab version when it is fixed?
To add on your answer, another option to skip a job on schedules is to add: ` except: - schedules `
Thanks for the tip on running a manual pipeline! Life saver
Good grief. Thank you so much for the manual pipeline tip. I was tearing my hair out.
@Orb Indeed, it would make sense for Gitlab to at least write a "Empty pipeline" entry in the pipeline execution log or something, so we at least know the scheduling happened.
|
4

It is because it is protected, but not just because it is protected. The ability to create pipelines on a protected branch is determined by the user's ability to merge or push to that branch. A scheduled pipeline is run with the permissions of the schedule owner (initially, the user who created it).

You either need to have your user given merge/push access on master, or have another user with that permission to take ownership of the schedule.

Improve this answer

1 Comment

Thanks for the answer. Unfortunately I don't think that's the issue. I'm the owner of the pipeline and have all permissions on that branch. I have tried unprotecting the master branch but it still does not work. I have a feeling it's due to other jobs having rules based on $CI_DEFAULT_BRANCH but I can't see what's wrong with it...
2 
copy_database:on-schedule:
  stage: copy_data
  needs: []
  rules:
    - if: $CI_PIPELINE_SOURCE == "schedule" || $COPY_DB
      when: always
  script:
    ...
Improve this answer

7 Comments

Hi, thanks for the suggestion. I've tried this and unfortunately it still doesn't run on master (it works on other branches though...)
Did you try this on the master branch? even running with COPY_BD defined?
Yeah, I did but no luck :/
Just created a project with the working feature: gitlab.com/MoLt1eS/cron-stackoverflow-answer Regular pipeline after comiting to master: gitlab.com/MoLt1eS/cron-stackoverflow-answer/-/pipelines/… Cron working as intended every 5 min, using master branch: gitlab.com/MoLt1eS/cron-stackoverflow-answer/-/pipelines/…
Cron config: gitlab.com/MoLt1eS/cron-stackoverflow-answer/-/… (I will disable after a few more runs)
|
0

Look in your gitlab-ci.yml for this entry:

  only:
  - master

And edit accordingly ; Worked for me.

Improve this answer

Comments

0

In our case, it was that while the schedule was on target branch master, the default branch of ther repo had recently been renamed to main.

Improve this answer

Comments

0

I had a similar issue with a scheduled pipeline not running.

This is the best way to debug (as of gitlab 15.8):

  • As an admin, impersonate the user that owns the schedule.
  • Leave Admin Mode (if you're still in admin mode after impersonating). This is important because projects required by the pipeline may not be accessible to the user without admin mode.
  • Go to CI/CD -> Pipelines -> Run pipeline.
  • Trigger pipeline with the same settings used in the schedule, but make sure to add variable CI_PIPELINE_SOURCE with value schedule.

An error message should indicate the underlying problem now.

PS: Running the pipeline from CI/CD -> Schedules may lie to you and say

Successfully scheduled a pipeline to run. …

… even though GitLab fails to schedule it.

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.