托管字典在多处理中的行为不符合预期

问题描述 投票:0回答:1

据我目前所知,这可能只是 macOS 中的问题。

这是我的 MRE:

from multiprocessing import Pool, Manager
from functools import partial

def foo(d, n):
    d.setdefault("X", []).append(n)

def main():
    with Manager() as manager:
        d = manager.dict()
        with Pool() as pool:
            pool.map(partial(foo, d), range(5))
        print(d)

if __name__ == "__main__":
    main()

输出:

{'X': []}

预期输出:

{'X': [0, 1, 2, 3, 4]}

平台:

macOS 14.3.1
Python 3.12.2

也许我做的事情根本上是错误的,但我明白经理的全部目的就是准确处理这种情况。

编辑

还有另一个hack,恕我直言,这应该是不必要的,但即使这样也不起作用(产生相同的输出):

from multiprocessing import Pool, Manager

def ipp(d, lock):
    global gDict, gLock
    gDict = d
    gLock = lock

def foo(n):
    global gDict, gLock
    with gLock:
        gDict.setdefault("X", []).append(n)

def main():
    with Manager() as manager:
        d = manager.dict()
        lock = manager.Lock()
        with Pool(initializer=ipp, initargs=(d, lock)) as pool:
            pool.map(foo, range(5))
        print(d)

if __name__ == "__main__":
    main()
python multiprocessing
1个回答
0
投票

这是一个蹩脚的“答案”,我对此表示歉意,并在您发表评论后将其删除。我的明显印象是,答案与我提到的帖子未管理的“子列表”有关。

例如:

这似乎有效:

def foo(d, n):
    d.setdefault("X", []).append(n)

def main():
    with Manager() as manager:
        d = manager.dict()
        d["X"] = manager.list()
        with Pool() as pool:
            pool.map(partial(foo, d), range(5))
        print(list(d["X"]))

if __name__ == "__main__":
    main()

就像:

def foo(d, n):
    target = d.setdefault("X", [])
    target.append(n)
    d["X"] = target

def main():
    with Manager() as manager:
        d = manager.dict()
        with Pool() as pool:
            pool.map(partial(foo, d), range(5))
        print(d)

if __name__ == "__main__":
    main()

因此,更新“托管”对象似乎有效,而更新其中包含嵌套非托管列表的托管对象却不起作用。

© www.soinside.com 2019 - 2024. All rights reserved.