Airflow worker进程未触发-调度程序抛出错误消息(Docker编写-Celery执行模式)

问题描述 投票:0回答:1

相同的设置在Mac上有效,并将图像和组合文件移动到CentOS服务器。最初,我在路由方面遇到问题,并通过使用firewalld更新规则解决了该问题。

scheduler_1  | Celery Task ID: ('parallel_dag', 'task_4', datetime.datetime(2020, 3, 17, 0, 0, tzinfo=<TimezoneInfo [UTC, GMT, +00:00:00, STD]>), 1)
scheduler_1  | Traceback (most recent call last):
scheduler_1  |   File "/usr/local/lib/python3.7/site-packages/kombu/utils/functional.py", line 42, in __call__
scheduler_1  |     return self.__value__
scheduler_1  | AttributeError: 'ChannelPromise' object has no attribute '__value__'
scheduler_1  |
scheduler_1  | During handling of the above exception, another exception occurred:
scheduler_1  |
scheduler_1  | Traceback (most recent call last):
scheduler_1  |   File "/usr/local/lib/python3.7/site-packages/kombu/transport/virtual/base.py", line 921, in create_channel
scheduler_1  |     return self._avail_channels.pop()
scheduler_1  | IndexError: pop from empty list
scheduler_1  |
scheduler_1  | During handling of the above exception, another exception occurred:
scheduler_1  |
scheduler_1  | Traceback (most recent call last):
scheduler_1  |   File "/usr/local/lib/python3.7/site-packages/airflow/executors/celery_executor.py", line 118, in send_task_to_executor
scheduler_1  |     result = task.apply_async(args=[command], queue=queue)
scheduler_1  |   File "/usr/local/lib/python3.7/site-packages/celery/app/task.py", line 570, in apply_async
scheduler_1  |     **options
scheduler_1  |   File "/usr/local/lib/python3.7/site-packages/celery/app/base.py", line 756, in send_task
scheduler_1  |     amqp.send_task_message(P, name, message, **options)
scheduler_1  |   File "/usr/local/lib/python3.7/site-packages/celery/app/amqp.py", line 552, in send_task_message
scheduler_1  |     **properties
scheduler_1  |   File "/usr/local/lib/python3.7/site-packages/kombu/messaging.py", line 181, in publish
scheduler_1  |     exchange_name, declare,
scheduler_1  |   File "/usr/local/lib/python3.7/site-packages/kombu/connection.py", line 510, in _ensured
scheduler_1  |     return fun(*args, **kwargs)
scheduler_1  |   File "/usr/local/lib/python3.7/site-packages/kombu/messaging.py", line 187, in _publish
scheduler_1  |     channel = self.channel
scheduler_1  |   File "/usr/local/lib/python3.7/site-packages/kombu/messaging.py", line 209, in _get_channel
scheduler_1  |     channel = self._channel = channel()
scheduler_1  |   File "/usr/local/lib/python3.7/site-packages/kombu/utils/functional.py", line 44, in __call__
scheduler_1  |     value = self.__value__ = self.__contract__()
scheduler_1  |   File "/usr/local/lib/python3.7/site-packages/kombu/messaging.py", line 224, in <lambda>
scheduler_1  |     channel = ChannelPromise(lambda: connection.default_channel)
scheduler_1  |   File "/usr/local/lib/python3.7/site-packages/kombu/connection.py", line 852, in default_channel
scheduler_1  |     self.ensure_connection(**conn_opts)
python docker-compose airflow airflow-scheduler
1个回答
0
投票

这类似于此错误https://issues.apache.org/jira/browse/AIRFLOW-6527。因此,

我必须将这些参数operation_timeoutsend_task_timeout属性值设置为更高的值。默认情况下,它设置为2。

就我而言,我正在使用docker启动该过程。因此,在entrypoint.sh中将您的值设置为大于2。

© www.soinside.com 2019 - 2024. All rights reserved.