启动后运行apache airflow worker

Question

我试图通过rc.local(centos 7)设置 "airflow worker "在系统启动后运行。

我已经以root身份安装了python和airflow。路径是 /root/airflow 和 /root/anaconda3.

我把这个添加到rc.local中。

#!/bin/bash
# THIS FILE IS ADDED FOR COMPATIBILITY PURPOSES
#
# It is highly advisable to create own systemd services or udev rules
# to run scripts during boot instead of using this file.
#
# In contrast to previous versions due to parallel execution during boot
# this script will NOT be run after all other services.
#
# Please note that you must run 'chmod +x /etc/rc.d/rc.local' to ensure
# that this script will be executed during boot.

touch /var/lock/subsys/local
exec 2> /home/centos/rc.local.log  # send stderr from rc.local to a log file
exec 1>&2                      # send stdout to the same log file
set -x                         # tell sh to display commands before execution
export C_FORCE_ROOT="true"
/root/anaconda3/bin/python /root/anaconda3/bin/airflow worker
exit 0

当我试着手动运行它时，它工作了(sh etcrc.local)

但是开机后运行时，却在日志文件中出现这个错误而崩溃。

它好像找不到airflow的路径，但我已经把它写完整了。

+ export C_FORCE_ROOT=true
+ C_FORCE_ROOT=true
+ /root/anaconda3/bin/python /root/anaconda3/bin/airflow worker
Traceback (most recent call last):
  File "/root/anaconda3/bin/airflow", line 37, in <module>
    args.func(args)
  File "/root/anaconda3/lib/python3.7/site-packages/airflow/utils/cli.py", line 75, in wrapper
    return f(*args, **kwargs)
  File "/root/anaconda3/lib/python3.7/site-packages/airflow/bin/cli.py", line 1129, in worker
    sp = _serve_logs(env, skip_serve_logs)
  File "/root/anaconda3/lib/python3.7/site-packages/airflow/bin/cli.py", line 1065, in _serve_logs
    sub_proc = subprocess.Popen(['airflow', 'serve_logs'], env=env, close_fds=True)
  File "/root/anaconda3/lib/python3.7/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/root/anaconda3/lib/python3.7/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'airflow': 'airflow'

Answer 1

可以先把这一行改成...

/root/anaconda3/bin/python /root/anaconda3/bin/airflow worker

改成

/root/anaconda3/bin/airflow worker

因为你只需要调用你需要的气流仓，并传递给它一个单一的服务。请记住，你可以传递更多的参数。但是调用Python的一个版本感觉没有必要。