执行远程 ssh 命令时使用 Fabric2 退出代码:127

问题描述 投票:0回答:2

我在 fabric2 命令模块中遇到了奇怪的行为。这些命令就像一个魅力:

connect = Connection(host=h, user=u ,connect_kwargs={"password":p})
connect.run('mkdir temp_streaming')
connect.put(m, "temp_streaming/mapper.py")
connect.put(r, "temp_streaming/reducer.py")

但是说到这个:

input_path = input("Please choose the input of your program (on your HDFS File System) ")
output_path = input("Please choose a name for your output folder ")

main_command = str('yarn jar '+ jar_path+' -files mapper.py,reducer.py -mapper temp_streaming/mapper.py -reducer temp_streaming/reducer.py -input '+ input_path + ' -output '+ output_path)

connect.run(main_command)

我收到这个错误,我无法弄清楚:

bash: yarn: command not found
Traceback (most recent call last):
  File "__main.py__", line 77, in <module>
    main()
  File "__main.py__", line 65, in main
   RunMapReduce(mapper, reducer, jar_path)
  File "__main.py__", line 46, in RunMapReduce
   connect.run(main_command)
 File "<decorator-gen-3>", line 2, in run
 File "/usr/local/lib/python3.5/dist-packages/fabric2/connection.py", line 30, in opens
return method(self, *args, **kwargs)
 File "/usr/local/lib/python3.5/dist-packages/fabric2/connection.py", line 586, in run
return self._run(runner, command, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/invoke/context.py", line 100, in _run
return runner.run(command, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/invoke/runners.py", line 268, in run
return self._run_body(command, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/invoke/runners.py", line 401, in _run_body
raise UnexpectedExit(result)
invoke.exceptions.UnexpectedExit: Encountered a bad command exit code!

Command: 'yarn jar /home/hadoop/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.8.4.jar -files mapper.py,reducer.py -mapper temp_streaming/mapper.py -reducer temp_streaming/reducer.py -input /books/kafka_metamorphosis.txt -output /test'

Exit code: 127

Stdout: already printed

Stderr: already printed

当我直接使用 ssh 登录服务器并运行它时,即使这个 bash 命令有效,它的行为就好像远程服务器不知道 yarn 命令一样。在我看来,它与这个库有关,但我无法通过文档解决我的问题。

python ssh fabric
2个回答
3
投票

我也遇到了这个问题:事实证明 Fabric 运行的 SSH 是一个非交互式 shell

包含

$PATH
bin 路径的
yarn
变量设置在
/etc/profile
~/.bash_profile
~/.bashrc
中,其中没有一个被获取。

这是链接,以及一些解决方案。

就我个人而言,我不喜欢玩这些文件。我刚刚所做的(在偶然发现此链接之前)是我使用了二进制文件的完整路径:

/usr/bin/yarn [the rest of your command]


1
投票

我设法让它与关键字参数一起工作

warn=True
,如下所示:

result = connection.run(command, warn=True)

这个选项在文档中并不明显,需要搜索升级指南才能找到它。尽管这似乎不是问题,但我强烈建议改进 Fabric2 文档,因为我认为这是一个极有可能出现的情况。

© www.soinside.com 2019 - 2024. All rights reserved.