我有一个简单的脚本来模拟写入交错的 stdout 和 stderr 流的程序。
import sys
import time
for i in range(5):
print(int(time.time()), "This is Stdout")
print(int(time.time()), "Stderr", file=sys.stderr)
time.sleep(1)
当我使用 Python 运行这样的程序时
subprocess.Popen
我丢失了 stderr 和 stdout 的顺序。
import subprocess
file = open('./stdout1.log', 'w', encoding='utf-8')
subprocess.Popen('./stderrout.py', stdout=file, stderr=file)
输出
1720517080 Stderr
1720517081 Stderr
1720517082 Stderr
1720517083 Stderr
1720517084 Stderr
1720517080 This is Stdout
1720517081 This is Stdout
1720517082 This is Stdout
1720517083 This is Stdout
1720517084 This is Stdout
我知道 stderr 会更快地刷新或类似的东西,但是我如何保留顺序?
我还尝试使用
subprocess.STDOUT
进行 stderr 并使用 text
和 bufsize
参数,但没有运气。还尝试将 buffering
添加到文件句柄。 所有情况下的输出都是相同的。
这是完整的程序
import subprocess
file = open('./stdout1.log', 'w', encoding='utf-8') ; subprocess.Popen('./stderrout.py', stdout=file, stderr=file)
file = open('./stdout2.log', 'w', encoding='utf-8') ; subprocess.Popen('./stderrout.py', stdout=file, stderr=file, text=True, bufsize=0)
file = open('./stdout3.log', 'w', encoding='utf-8') ; subprocess.Popen('./stderrout.py', stdout=file, stderr=file, text=True, bufsize=1)
file = open('./stdout4.log', 'w', encoding='utf-8') ; subprocess.Popen('./stderrout.py', stdout=file, stderr=subprocess.STDOUT)
file = open('./stdout5.log', 'w', encoding='utf-8') ; subprocess.Popen('./stderrout.py', stdout=file, stderr=subprocess.STDOUT, text=True, bufsize=0)
file = open('./stdout6.log', 'w', encoding='utf-8') ; subprocess.Popen('./stderrout.py', stdout=file, stderr=subprocess.STDOUT, text=True, bufsize=1)
file = open('./stdout1b.log', 'w', encoding='utf-8', buffering=1) ; subprocess.Popen('./stderrout.py', stdout=file, stderr=file)
file = open('./stdout2b.log', 'w', encoding='utf-8', buffering=1) ; subprocess.Popen('./stderrout.py', stdout=file, stderr=file, text=True, bufsize=0)
file = open('./stdout3b.log', 'w', encoding='utf-8', buffering=1) ; subprocess.Popen('./stderrout.py', stdout=file, stderr=file, text=True, bufsize=1)
file = open('./stdout4b.log', 'w', encoding='utf-8', buffering=1) ; subprocess.Popen('./stderrout.py', stdout=file, stderr=subprocess.STDOUT)
file = open('./stdout5b.log', 'w', encoding='utf-8', buffering=1) ; subprocess.Popen('./stderrout.py', stdout=file, stderr=subprocess.STDOUT, text=True, bufsize=0)
file = open('./stdout6b.log', 'w', encoding='utf-8', buffering=1) ; subprocess.Popen('./stderrout.py', stdout=file, stderr=subprocess.STDOUT, text=True, bufsize=1)
有没有办法将两个流重定向到同一个文件而不丢失顺序?
如果不是,
bufsize
论证的意义何在?
发生这种情况是因为 Python 的 stdout 仅在连接到终端 (TTY) 时才具有行缓冲,这可能是性能/交互性的权衡。
$ python3 -c 'import sys; print(sys.stdout.line_buffering)'
True
$ python3 -c 'import sys; print(sys.stdout.line_buffering)' | cat
False
同时,stderr 始终是行缓冲的:
$ python3 -c 'import sys; print(sys.stderr.line_buffering)'
True
$ python3 -c 'import sys; print(sys.stderr.line_buffering)' | cat
True
因此,不需要的缓冲是作者而不是读者,这就是为什么
bufsize
没有帮助。
pty
将 Popen 的标准输出作为 TTY 文件描述符打开。大量借用另一个答案:
import errno
import os
import pty
import signal
import subprocess
# From https://stackoverflow.com/a/77387332/252218
def subprocess_tty(cmd, encoding="utf-8", timeout=10, **kwargs):
"""`subprocess.Popen` yielding stdout lines acting as a TTY"""
m, s = pty.openpty()
p = subprocess.Popen(cmd, stdout=s, stderr=s, **kwargs)
os.close(s)
try:
for line in open(m, encoding=encoding):
if not line: # EOF
break
yield line
except OSError as e:
if errno.EIO != e.errno: # EIO also means EOF
raise
finally:
if p.poll() is None:
p.send_signal(signal.SIGINT)
try:
p.wait(timeout)
except subprocess.TimeoutExpired:
p.terminate()
try:
p.wait(timeout)
except subprocess.TimeoutExpired:
p.kill()
p.wait()
# Run the writer command, and print all stdout and stderr lines interleaved.
for line in subprocess_tty(['python3', 'stderrout.py']):
print(line, end='')
输出:
1720520529 This is Stdout
1720520529 Stderr
1720520530 This is Stdout
1720520530 Stderr
1720520531 This is Stdout
1720520531 Stderr
1720520532 This is Stdout
1720520532 Stderr
1720520533 This is Stdout
1720520533 Stderr
或者,您可以使 stdout 和 stderr 都无缓冲。这会比较慢,尤其是在写入许多小字符串时。另一方面,这是一个更简单的解决方案,只需添加 -u
:
import subprocess
file = open('./stdout1.log', 'w', encoding='utf-8')
subprocess.Popen(['python3', '-u', './stderrout.py'], stdout=file, stderr=file)