我最近开始使用 ZeroMQ,并观察到有时ZeroMQ 上下文的初始化可能需要长达 40 秒的时间。通常比这快得多。
例如:
import zmq
# the next line will sometimes take 40 seconds
# or so but is usually near instantaneous
with zmq.Context.instance() as context:
# connect / bind to socket, do stuff
pass
此行为似乎与我是否使用上下文管理器无关,例如以下行为与上面相同:
import zmq
# the next line will sometimes take 40 seconds
# or so but is usually near instantaneous
context = zmq.Context.instance()
# connect / bind to socket, do stuff
context.term()
它在 Ubuntu 18.04.2 上的 Python 3.6.8 中运行
我无法建立任何放缓的模式。
造成这种行为的可能原因是什么?
根据@user3666197的建议,我花了一些时间并设法在第二次运行时体现出速度减慢。此后一切恢复“正常”。
第二次运行测试脚本时速度开始变慢:
= zmq.Context()] took + 704 [us] to execute
1.~ 704 us
2.~ 131 766 816 us
3.~ 145 us
4.~ 137 us
当时在
htop
上查看内存根本没有显示内存已满,但交换已经处于活动状态。
使用@user3666197建议脚本的精简版本:
import sys
import zmq
clock = zmq.Stopwatch()
clock.start()
foo = zmq.Context()
bar = clock.stop()
print("{0:>2s}: {1:>9d}".format(sys.argv[1], bar))
foo.term()
并循环调用它,例如:
for x in $(seq 1 20);
do python3 test.zmq.py $x;
done
我可以在原始服务器上重复这个问题但尚未在另一台服务器上遇到它,这表明存在环境问题(或者与特定发行版版本和/或Python版本作为“好”服务器有关)年纪大了)。
目前这对我来说已经足够了,但我稍后会回来并收集足够的信息以向维护者提交一些内容。
导致这种行为(仅“有时”存在)的可能原因是什么? “
我无法建立任何放缓模式。
a)上面提到的
这 40 秒可能是有意义的,但需要 RAM 来交换 I/O 操作。我无法想象这种时间块的任何其他原因(在我使用 ZeroMQ 的最后 10 年里,我从未注意到 ZeroMQ 中有任何接近 3/4 分钟的延迟,从来没有)
b)现实检查
在提出任何索赔之前,始终测量各自的实际执行时间:
让我们开始对可重复和记录的测试用例进行系统和公平,好吗?
像“有时”这样的类别不值得,数据值得。首先,我们可以使用以下模板方法来区分任何实际延迟的根本原因:
M = "[{0:_>40s}] took +{1:_>9d} [us] to execute"
N = 0
from zmq import Stopwatch; aClk = Stopwatch()
#------------------------------
pass; aClk.start()
import zmq
pass; _ = aClk.stop(); print( M.format( "IMPORT", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context()
pass; _ = aClk.stop(); print( M.format( "= zmq.Context()", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
context = zmq.Context.instance()
pass; _ = aClk.stop(); print( M.format( "= zmq.Context.instance()", _ ) )
#------------------------------
pass; aClk.start()
context.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
with zmq.Context.instance() as aCtxInCtxMGR:
N +=1
pass; _ = aClk.stop(); print( M.format( "with … as aCtxInCtxMGR:", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 0 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 0 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 10 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 10 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 20 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 20 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 50 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 50 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
pass; aClk.start()
aCTX = zmq.Context( 99 )
pass; _ = aClk.stop(); print( M.format( "= zmq.Context( 99 )", _ ) )
#------------------------------
pass; aClk.start()
aCTX.term()
pass; _ = aClk.stop(); print( M.format( ".term()", _ ) )
#------------------------------
等等
广告貌似合理
[__________________________________IMPORT] took +_____1198 [us] to execute
[_________________________= zmq.Context()] took +_____1523 [us] to execute
[________________= zmq.Context.instance()] took +_____1348 [us] to execute
[_________________with … as aCtxInCtxMGR:] took +_____7909 [us] to execute
[_____________________= zmq.Context( 0 )] took +______966 [us] to execute
[_____________________= zmq.Context( 10 )] took +______944 [us] to execute
[_____________________= zmq.Context( 20 )] took +______936 [us] to execute
[_____________________= zmq.Context( 50 )] took +______962 [us] to execute
[_____________________= zmq.Context( 99 )] took +_____1015 [us] to execute
[__________________________________IMPORT] took +______879 [us] to execute
[_________________________= zmq.Context()] took +_____1076 [us] to execute
[_________________________________.term()] took +______881 [us] to execute
[________________= zmq.Context.instance()] took +_____1105 [us] to execute
[_________________________________.term()] took +______896 [us] to execute
[_________________with … as aCtxInCtxMGR:] took +_____1474 [us] to execute
[_____________________= zmq.Context( 0 )] took +_____1163 [us] to execute
[_________________________________.term()] took +______982 [us] to execute
[_____________________= zmq.Context( 10 )] took +_____2646 [us] to execute
[_________________________________.term()] took +______873 [us] to execute
[_____________________= zmq.Context( 20 )] took +_____2689 [us] to execute
[_________________________________.term()] took +______981 [us] to execute
[_____________________= zmq.Context( 50 )] took +_____2772 [us] to execute
[_________________________________.term()] took +______910 [us] to execute
[_____________________= zmq.Context( 99 )] took +_____1086 [us] to execute
[_________________________________.term()] took +_____1010 [us] to execute