GitLab CI：根据文件更改有条件地运行作业步骤

Question

我正在使用 GitLab CI，我有一个想要修改的作业模板。目前，我的工作是这样的：

mwaa:dags:test:2.8.1:
  variables:
    AIRFLOW_VERSION: 2.8.1
    PYTHON_VERSION: 3.11
  extends:
    - .template_MWAA:DAGS:Test

此作业扩展了包含

before_script

、

script

和

after_script

部分的模板。我想修改工作以便：

1. 如果

   airflow-mwaa/requirements/**

   目录中有更改，则应运行所有部分（

   before_script

   、

   script

   和

   after_script

   ）。
2. 如果

   airflow-mwaa/requirements/**

   目录中没有任何更改，则应该只运行

   script

   部分。

以下是模板的相关部分：

before_script:
  - echo "Starting .install_dependencies template"
  - pip3 install -r "$CI_PROJECT_DIR/airflow-mwaa/requirements/$AIRFLOW_VERSION/requirements.txt"

script:
  - echo "Starting .template_MWAA:DAGS:Test template"
  - cd "$CI_PROJECT_DIR/airflow-mwaa"
  - airflow db init
  - ./tests-ci.sh

after_script:
  # remove older files from terraform providers cache
  - find "$PIP_CACHE_DIR" -mindepth 1 -mtime +14 -type f -delete
  - find "$PIP_CACHE_DIR" -mindepth 1 -mtime +14 -type d -empty -delete

  stage: Test
  extends:
    - .template_MWAA
  needs: []
  variables:
    PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
    PYTHONPATH: "$CI_PROJECT_DIR/airflow-mwaa/dags"
  cache:
    key: $CI_JOB_NAME_SLUG-python
    paths:
      - .cache/pip
  before_script:
    - echo "Starting .install_dependencies template"
    - pip3 install -r "$CI_PROJECT_DIR/airflow-mwaa/requirements/$AIRFLOW_VERSION/requirements.txt"
  script:
    - echo "Starting .template_MWAA:DAGS:Test template"
    - cd "$CI_PROJECT_DIR/airflow-mwaa"
    - airflow db init
    - ./tests-ci.sh
  after_script:
    # remove older files from terraform providers cache
    - find "$PIP_CACHE_DIR" -mindepth 1 -mtime +14 -type f -delete
    - find "$PIP_CACHE_DIR" -mindepth 1 -mtime +14 -type d -empty -delete
  coverage: '/^TOTAL\s+\d+\s+\d+\s+(\d+\%)$/'
  artifacts:
    reports:
      junit: $CI_PROJECT_DIR/airflow-mwaa/test_results/report.xml
  rules:
    - if: $CI_COMMIT_REF_PROTECTED == "true"
      when: on_success
      allow_failure: false
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      changes:
        - airflow-mwaa/dags/**/*
        - airflow-mwaa/tests/**/*
      allow_failure: false
      when: on_success
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      when: manual
      allow_failure: true
    - when: never

如何修改我的作业或模板以实现基于文件更改的条件执行？我不确定如何将

changes

条件与脚本部分的选择性执行结合起来。

任何帮助或指导将不胜感激！

Answer 1

我认为这样的事情可以工作：

# Single job just to generate a CHANGE_DETECTED variable
# See https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportsdotenv
change_detection:
  rules:
    - changes:
        airflow-mwaa/requirements/**
      variables:
        CHANGE_DETECTED: "true"
    - variables:
        CHANGE_DETECTED: "false"
  script:
    - echo "$CHANGE_DETECTED" > change_detection.env
  artifacts:
    report:
      dotenv: change_detection.env

.mwaa:dags:test:2.8.1:
  variables:
    AIRFLOW_VERSION: 2.8.1
    PYTHON_VERSION: 3.11
  needs:
    - change_detection # Here we get the CHANGE_DETECTED variable
  extends:
    - .template_MWAA:DAGS:Test

mwaa:dags:test:2.8.1_unchanged:
  extends:
    - .mwaa:dags:test:2.8.1
  # Override the before and after script by empty arrays 
  # to not run those for other jobs:
  before_script: []
  after_script: []
  rules:
    - if: $CHANGE_DETECTED == "true"
      when: never # Invert the rule to stop processing next rules
    - !reference [.template_MWAA:DAGS:Test, rules]

mwaa:dags:test:2.8.1_changed:
  extends:
    - .mwaa:dags:test:2.8.1
  rules:
    - if: $CHANGE_DETECTED == "false"
      when: never
    - !reference [.template_MWAA:DAGS:Test, rules]

但是，您几乎最终会得到更多的 YAML，而不仅仅是复制粘贴原始模板并在那里进行一些小的更改，这实际上可能会更具可读性（稍微有一点 DRY 成本）。

我假设对模板的更改受到限制（由其他人拥有，从某个地方导入等），但如果您可以更改模板，一切都会变得更容易，因为您可以只在 bash 脚本部分中使用变量并且做这样的事情：

  before_script:
    - |
      if [ "$CHANGE_DETECTED" == "true" ]; then
        echo "Starting .install_dependencies template"
        pip3 install -r "$CI_PROJECT_DIR/airflow-mwaa/requirements/$AIRFLOW_VERSION/requirements.txt"
      fi