使用ansible执行DAG管道

Question

我有 3 个 ansible 任务（A、B、C）。我希望任务 A 和 B 并发运行，并且 C 在 A 和 B 完成后立即执行（C 使用来自 A 和 B 的标准输出的数据）。因此，如果每个任务需要 30 秒完成，我预计整个管道将在 60 秒内执行（并发 A 和 B 为 30 秒，C 为 30 秒）。

tasks:
  - name: A
    shell:
      cmd: "sleep 30"
    async: 1000
    poll: 0
  - name: B
    shell:
      cmd: "sleep 30"
    async: 1000
    poll: 0
  - name: C
    shell:
      cmd: "sleep 30"
    # what to add here?

我尝试使用

async_status

+

until

（在 C 中），但我无法将其与实际的

shell

模块混合。

Answer 1

问：“我预计整个管道将在 60 秒内执行完毕（并发 A 和 B 为 30 秒，C 为 30 秒）”

由于开销（协议、通信、网络、处理等......），这种情况不会发生。

问：“我希望任务 A 和 B 并发运行，并且在 A 和 B 完成后立即执行 C（C 使用来自 A 和 B 的标准输出的数据）。”

一个最小的示例手册

---
- hosts: test
  become: false
  gather_facts: false

  tasks:

    - block:

      - name: A
        shell:
          cmd: "echo 'TASK A'; sleep 30"
        register: A
        async: 60
        poll: 0

      - name: B
        shell:
          cmd: "echo 'TASK B'; sleep 30"
        register: B
        async: 60
        poll: 0

    - name: Wait until Concurrent Remote Tasks are complete
      async_status:
        jid: '{{ item }}'
      # Inner loop
      loop:
        - '{{ A.ansible_job_id }}'
        - '{{ B.ansible_job_id }}'
      register: async_status
      # Outer loop
      until: async_status is finished
      delay: 30
      retries: 1

      # C uses data from A's and B's stdout
      # Therefore need to 'slurp' the Remote Results

    - name: Gather Remote Results
      slurp:
        src: "/home/ansible_user/.ansible_async/{{ item }}"
      register: async_tasks
      loop:
        - '{{ A.ansible_job_id }}'
        - '{{ B.ansible_job_id }}'

    - name: C
      shell:
        cmd: "echo 'Output from {{ _A }} and {{ _B }}'; sleep 30"
      register: C
      # Because the 'content' of Remote Results file is a Base64 encoded JSON string
      vars:
        _A: "{{ ((async_tasks.results[0].content | b64decode) | from_json).stdout }}"
        _B: "{{ ((async_tasks.results[1].content | b64decode) | from_json).stdout }}"

    - name: Final result
      debug:
        msg: "{{ C.stdout }}"

将导致

的预期行为和输出

PLAY [test] ********************************************************************************************
Wednesday 21 August 2024  09:19:24 +0200 (0:00:00.015)       0:00:00.015 ******

TASK [A] ***********************************************************************************************
changed: [test.example.com]
Wednesday 21 August 2024  09:19:26 +0200 (0:00:02.451)       0:00:02.467 ******

TASK [B] ***********************************************************************************************
changed: [test.example.com]
Wednesday 21 August 2024  09:19:27 +0200 (0:00:00.721)       0:00:03.188 ******
FAILED - RETRYING: [test.example.com]: Wait until Concurrent Remote Tasks are complete (1 retries left).

TASK [Wait until concurrent remote tasks are complete] *************************************************
changed: [test.example.com] => (item=j989601333376.1621265)
changed: [test.example.com] => (item=j524870789757.1621464)
Wednesday 21 August 2024  09:19:59 +0200 (0:00:32.200)       0:00:35.389 ******

TASK [Gather Remote Results] ***************************************************************************
ok: [test.example.com] => (item=j989601333376.1621265)
ok: [test.example.com] => (item=j524870789757.1621464)
Wednesday 21 August 2024  09:20:01 +0200 (0:00:01.423)       0:00:36.813 ******

TASK [C] ***********************************************************************************************
changed: [test.example.com]
Wednesday 21 August 2024  09:20:31 +0200 (0:00:30.666)       0:01:07.480 ******

TASK [Final result] ************************************************************************************
ok: [test.example.com] =>
  msg: Output from TASK A and TASK B

PLAY RECAP *********************************************************************************************
test.example.com : ok=6    changed=4    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Playbook run took 0 days, 0 hours, 1 minutes, 7 seconds
Wednesday 21 August 2024  09:20:31 +0200 (0:00:00.033)       0:01:07.513 ******
===============================================================================
Wait until Concurrent Remote Tasks are complete ------------------------------------------------- 32.20s
C ----------------------------------------------------------------------------------------------- 30.67s
A ------------------------------------------------------------------------------------------------ 2.45s
Gather Remote Results ---------------------------------------------------------------------------- 1.42s
B ------------------------------------------------------------------------------------------------ 0.72s
Final result ------------------------------------------------------------------------------------- 0.03s

real    1m14.259s
user    0m3.187s
sys     0m1.210s