如何使用python subprocess.run调用其他python文件和bash文件

Question

我正在使用 python subprocess.run 调用位于远离源的另一个目录中的 .sh 文件和 .py 文件。这是两个目录之间的唯一连接，使得导入不切实际。下面的代码已被简化到最低限度。

parent-dir/
├── dirA
│   ├── main.py
│   └── main_test.py
└── dirB
    ├── app.sh
    └── utility.py

主.py

from pathlib import Path
from typing import Tuple, Optional
import subprocess
import sys

dir_a_path = Path(__file__).resolve().parent
    
# get the parent folder
parent_dir = dir_a_path.parent
dir_b_path = str(parent_dir.joinpath("dirB"))


def execute_bash_command(command: str, cwd: Optional[str]=None) -> Tuple[int, str, str]:
    """Runs a bash command in a new shell
    exit_code, stdout, stderr = execute_bash_command(command)

    Args:
        command (str): the command to run
        cwd (Optional[str]): where to run the command line from. 
            Use this instead of 'cd some_dir/ && command'

    Raises:
        Exception: Exception

    Returns:
        Tuple[int, str, str]: [exit code, stdout, stderr]
    """
    try:
        print(f"Executing {command} from cwd: {cwd}")
        output = subprocess.run(command, cwd=cwd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, check=True, shell=True)
        stdout = output.stdout.decode("utf-8").strip()
        print(stdout)
        return (output.returncode, stdout, "")
    
    except subprocess.CalledProcessError as cpe:
        print(f"error: {cpe}")
        stdout = cpe.stdout.decode("utf-8").strip()
        stderr = cpe.stderr.decode("utf-8").strip()
        return (cpe.returncode, stdout, stderr)

def call_app() -> Tuple[int, str, str]:
    command = [
        "./app.sh",
        "install"
    ]
    command = " ".join(command)
    return execute_bash_command(command=command, cwd=dir_b_path)

def call_utility() -> Tuple[int, str, str]:
    command = [
        sys.executable,
        "greet"
    ]
    command = " ".join(command)
    return execute_bash_command(command=command, cwd=dir_b_path)

main_test.py

from main import call_app, call_utility

def test_call_app():
    code, stdout, stderr = call_app()
    assert code == 0
    assert stdout == "Installing application..."
    
def test_call_utility():
    code, stdout, stderr = call_utility()
    assert code == 0
    assert stdout == "running command greet"

def test_execute_bash_command():
    """ Creates dummy file """
    exit_code, stdout, _ = execute_bash_command("touch tmp/test.txt")
    assert exit_code == 0

app.sh

#!/bin/bash

if [ "$1" = "greet"]; then
    echo "Hello, world!"
elif [ "$1" = "install"]; then
    echo "Installing application..."
else
    echo "Usage: $0 {greet|install}"
    exit 1
fi

实用程序.py

import sys

if __name__ == "__main__":
    print(f"running command {sys.argv[1]}")

当我从终端

python utility.py greet

或

./app.sh install

运行时，它运行良好。问题是当我使用

main_test.py

执行

python -m pytest

时。我收到子进程以不同代码退出的错误。

错误：

Executing ./app.sh install from cwd: /mnt/c/Users/XXX/source/python-testing/parent-dir/dirB
error: Command './app.sh install' returned non-zero exit status 1.

或

Executing /usr/bin/python3 greet from cwd: /mnt/c/Users/XXX/source/python-testing/parent-dir/dirB
error: Command '/usr/bin/python3 greet' returned non-zero exit status 2.

从 powershell 和 WSL2 运行 pytest 时存在这些错误。

这些错误模仿了代码库上运行的实际错误。

为什么这没有按预期工作？在

cwd

中使用

subprocess.run

不是正确的方法吗？我陷入困境，因为我正在尝试重构一些令人讨厌的代码，这些代码神奇地工作，但不清楚，所以我希望得到一些指导。

Answer 1

我有几条意见：

• 关于

  subprocess.run

  • 该命令应该是字符串列表。我的经验使我相信字符串列表比单个字符串更麻烦。函数

    run

    将以任何方式分割字符串。
  • 不要使用

    shell=True

    ，因为它会导致意外行为和安全漏洞的来源
  • 我不放入

    check=True

    。这样，返回码被捕获，并且 stderr 被填充。我不必将调用放入 try/ except 块中
  • 我喜欢

    text=True

    选项，因为这样，我返回的 stdout 和 stderr 是文本，而不是字节
  • 我喜欢

    capture_output=True

    选项，而不是使用 stdout=PIPE、stderr=PIPE。更方便、更干净、更清晰
  • 我不返回返回代码、stdout、stdin，而是返回整个 CompletedProcess 对象并让调用者享受它的乐趣。

• 关于测试

  • 我在输出中添加了尾随换行符
  • 我稍微改变了

    test_execute_bash_command

    ，因为

    touch

    命令总是会失败
  • 我将

    main_test.py

    重命名为

    test_main.py

    ，因为

    pytest

    默认查找

    test_*.py

话虽如此，这里是 main.py：

import logging
import subprocess
import sys
from pathlib import Path
from typing import Optional

logging.basicConfig(level=logging.DEBUG)
DIR_B_PATH = Path(__file__).parent.parent / "dirB"


def execute_bash_command(
    command: list, cwd: Optional[Path] = None
) -> subprocess.CompletedProcess:
    logging.debug("Executing command %r from dir: %r", command, cwd)
    completed_process = subprocess.run(
        command,
        cwd=cwd,
        capture_output=True,
        text=True,
    )
    return completed_process


def call_app() -> subprocess.CompletedProcess:
    command = ["./app.sh", "install"]
    return execute_bash_command(command=command, cwd=DIR_B_PATH)


def call_utility() -> subprocess.CompletedProcess:
    command = [sys.executable, "utility.py", "greet"]
    return execute_bash_command(command, cwd=DIR_B_PATH)

这是 test_main.py：

from main import call_app, call_utility, execute_bash_command


def test_call_app():
    completed_process = call_app()
    assert completed_process.returncode == 0
    assert completed_process.stdout == "Installing application...\n"


def test_call_utility():
    completed_process = call_utility()
    assert completed_process.returncode == 0
    assert completed_process.stdout == "running command greet\n"


def test_execute_bash_command():
    """Creates dummy file"""
    completed_process = execute_bash_command(["ls"])
    assert completed_process.returncode == 0

测试结果：

dirA/test_main.py::test_call_app PASSED
dirA/test_main.py::test_call_utility PASSED
dirA/test_main.py::test_execute_bash_command PASSED