ProcessPoolExecutor 失败

Question

我有一个简单的设置来测试并行执行，但无论我尝试什么，它都会失败。我正在 Jupyter Notebook 中工作。

这是一个模型示例：

from concurrent.futures import ProcessPoolExecutor

def worker(i):
    print(f"TASK: {i}")
    result = i * i
    print(f"RESULT: {result}")
    return result


futures = []

if __name__ == "__main__":
    with ProcessPoolExecutor(max_workers=4) as executor:
        for i in range(10):
            futures.append(executor.submit(worker, i))

    for i, future in enumerate(futures):
        try:
            result = future.result()
            print(result)
        except Exception as e:
            print(f"Error in future {i}: {e}")

我有超过 4 个可用核心，没有内存问题，但无论我尝试什么，它都会失败。

Error in future 0: A process in the process pool was terminated abruptly while the future was running or pending.
(and so on for the other futures)

解决方案：

将

worker

移动到python文件中并导入它，然后就可以正常工作了。

Answer 1

问题是它试图在 Jupyter Notebook 中运行。解决方案是将

worker

移动到python文件中并导入它，然后一切正常。