subprocess.Popen() 不等待 s3 同步命令完成

Question

我正在使用

aws s3 sync

命令将文件从 s3 同步到本地，并使用 python 执行一些操作。由于没有

boto3

的

s3 sync

实现，我正在使用

subprocess.Popen()

运行cli命令。我也试过

.run

但它仍然不等待所有文件被下载并继续到下一行代码，只下载了几个文件。

我的同步功能是这样的：

    def sync_s3(self, aws_cli_path, s3_path, local_directory, aws_profile):
        print("Running sync command from s3")
        try:
            process = subprocess.Popen([aws_cli_path, "s3", "sync", s3_path, local_directory, "--profile", aws_profile], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
            while process.poll() is None:
                pass    
            stdout, stderr = process.communicate()

            if process.returncode != 0:
                raise Exception(stderr.decode())
            else:
                 print("S3 sync command completed successfully:", stdout.decode())
                 logging.info(f'S3 sync completed successfully: {stdout.decode()}')
        except Exception as e:
            print("Command failed with error:", e)
            logging.critical(f'S3 sync failed: {e}')
            sys.exit(701)

也尝试使用

subprocess.run()

但它仍然不等待文件下载。

Answer 1

根据文档，

subprocess.run()

应该等待命令完成。因此，你应该没问题：

process = subprocess.run([aws_cli_path, "s3", "sync", s3_path, local_directory, "--profile", aws_profile], capture_output=True)

if process.returncode != 0:
    raise Exception(process.stderr.decode())
else:
    print("S3 sync command completed successfully:", process.stdout.decode())
    logging.info(f'S3 sync completed successfully: {process.stdout.decode()}')

在我这边完美地工作。