为什么 Windows 和 Mac 上的多重处理比单处理器花费的时间更长？

Question

据我所知，如果将大型任务划分为多个进程，完成任务所需的时间就会减少，但在我的代码中并没有发生这种情况。

from multiprocessing import Process
import time
import random

def calculate_squares(numbers):
    for num in numbers:
        square = num * num

if __name__ == "__main__":
    numbers = [random.randrange(1, 50, 1) for i in range(100000000)]
    quarter = len(numbers) // 4
    first_part = numbers[:quarter]
    second_part = numbers[quarter:2*quarter]
    third_part = numbers[2*quarter:3*quarter]
    fourth_part = numbers[3*quarter:]

    start_time = time.time()

    # Creating four processes
    p1 = Process(target=calculate_squares, args=(first_part,))
    p2 = Process(target=calculate_squares, args=(second_part,))
    p3 = Process(target=calculate_squares, args=(third_part,))
    p4 = Process(target=calculate_squares, args=(fourth_part,))

    # Start all processes
    p1.start()
    p2.start()
    p3.start()
    p4.start()

    # Wait for all processes to complete
    p1.join()
    p2.join()
    p3.join()
    p4.join()

    end_time = time.time()
    print(f"Execution time with four processes: {(end_time - start_time) * 10**3} ms")

    # Single process execution for comparison
    start_time = time.time()
    calculate_squares(numbers)
    end_time = time.time()
    print(f"Execution time for single process: {(end_time - start_time) * 10**3} ms")

这是代码并且

i5 13gen 的输出（Windows）：

Execution time with four processes: 4975.9681224823 ms
Execution time for single process: 3258.450984954834 ms

i3 8gen (linux) 的输出：

Execution time with four processes: 1378.4070014953613 ms
Execution time for single process: 2259.6046924591064 ms

Mac m1 的输出：

Execution time with four processes: 3175.5270957946777 ms
Execution time for single process: 2120.435953140259 ms

Answer 1

（我认为这应该在某个地方得到回答，但我找不到足够相似的问题来作为重复项结束，所以这里。）

@KlausD。说得对：在 Windows 上启动新进程的开销较高，默认情况下在 MacOS 上也是如此，因为整个解释器状态（包括所有变量）需要序列化并发送到其他进程。正如您所看到的，在 Linux 上，多处理方法更快。您可以在 Python 文档中阅读更多相关信息：上下文和启动方法。

我只使用 Linux，因此为了重现该问题，我添加了对

multiprocessing.set_start_method('spawn')

的调用。然后我们分别测量启动进程所需的时间：

from multiprocessing import Process, set_start_method
import time
import random

def calculate_squares(numbers):
    for num in numbers:
        square = num * num

if __name__ == "__main__":
    numbers = [random.randrange(1, 50, 1) for i in range(100000000)]
    quarter = len(numbers) // 4
    first_part = numbers[:quarter]
    second_part = numbers[quarter:2*quarter]
    third_part = numbers[2*quarter:3*quarter]
    fourth_part = numbers[3*quarter:]

    # Creating four processes
    set_start_method('spawn')
    start_time = time.time()
    p1 = Process(target=calculate_squares, args=(first_part,))
    p2 = Process(target=calculate_squares, args=(second_part,))
    p3 = Process(target=calculate_squares, args=(third_part,))
    p4 = Process(target=calculate_squares, args=(fourth_part,))
    end_time = time.time()
    print(f"Creation time for four processes: {(end_time - start_time) * 10**3} ms")

    # Start all processes
    start_time = time.time()
    p1.start()
    p2.start()
    p3.start()
    p4.start()
    end_time = time.time()
    print(f"Startup time for four processes: {(end_time - start_time) * 10**3} ms")

    # Wait for all processes to complete
    start_time = time.time()
    p1.join()
    p2.join()
    p3.join()
    p4.join()
    end_time = time.time()
    print(f"Execution time with four processes: {(end_time - start_time) * 10**3} ms")

    # Single process execution for comparison
    start_time = time.time()
    calculate_squares(numbers)
    end_time = time.time()
    print(f"Execution time for single process: {(end_time - start_time) * 10**3} ms")

我的 i7-13700K 上的输出：

Creation time for four processes: 0.040531158447265625 ms
Startup time for four processes: 1386.2104415893555 ms
Execution time with four processes: 424.9296188354492 ms
Execution time for single process: 1357.4330806732178 ms

所以很明显，大部分时间都花在将数据传输到新进程上，而不是实际处理上。

Linux 上默认使用更高效的

'fork'

方法：

Creation time for four processes: 0.0457763671875 ms
Startup time for four processes: 271.9295024871826 ms
Execution time with four processes: 363.21234703063965 ms
Execution time for single process: 1363.1033897399902 ms