Python S3 流式传输，从函数返回流

Question

是否可以从函数返回打开的写入流？ 我收到错误：

ValueError：对已关闭文件进行 I/O 操作。

import boto3
import io

s3 = boto3.client('s3')

# Download file from S3 and upload a copy

def download_s3():
    response = s3.get_object(
        Bucket="bedrock-data-files", 
        Key="test/filename.txt"
        )
    stream = io.BytesIO(response['Body'].read())
    return stream

def upload_s3():
    returnedStream = io.BytesIO()
    s3.upload_fileobj(
        returnedStream,
        "bedrock-data-files", 
        "test/copy.txt"
        )
    return returnedStream

downloadstream = download_s3()
uploadstream = upload_s3()
for chunk in downloadstream.read():
    uploadstream.write(chunk)

这个版本可以工作，但我需要在函数中创建写入流。

def download_s3():
    response = s3.get_object(
        Bucket="bedrock-data-files", 
        Key="test/filename.txt"
        )
    stream = io.BytesIO(response['Body'].read())
    return stream

downloadstream = download_s3()
print(downloadstream) # This line returns <_io.BytesIO object at 0x10679dbc0>
s3.upload_fileobj(
    downloadstream,
    "bedrock-data-files", 
    "test/copy.txt"
    )

Answer 1

我一直在进一步研究这门课，认为你应该能够做如下的事情：

def download_s3():
    response = s3.get_object(
        Bucket="bedrock-data-files", 
        Key="test/filename.txt"
        )
    data = io.BytesIO(response['Body'])
    return data # this is a bytes object containing all the data from response['Body']

downloaddata = download_s3()
uploadstream = upload_s3()
N = 1024 # set N to the number of bytes to read
         # if N is omitted then read will return all the data on the first call
while True:
    chunk = downloaddata.read(N): # read the next chunk of data
    if len(chunk) == 0: # if nothing is returned
        break;          # then end of data has been reached
    uploadstream.write(chunk)