shell：参数列表太长

Question

我想同时对具有多个请求的端点进行负载测试。我收到错误

Argument list too long

。我该如何解决这个问题？

import subprocess
import argparse
def main(**kwargs):
    my_parser = argparse.ArgumentParser()
    my_parser.add_argument('-s')
    my_parser.add_argument('-e')
    args = my_parser.parse_args()
    i = int(vars(args)['s'])
    cmd = ''
    f= open('test.txt')
    while i <= int(vars(args)['e']):
        for line in f:
            cmd += f'curl -X POST -H "Content-Type: application/json" https://some_api_ep -d {line} &'
        i +=1
    subprocess.run(cmd, shell=True)
if __name__ == "__main__":
    main()

$ python load_generator.py -s 1 -e 1
Traceback (most recent call last):
  File "load_generator.py", line 20, in <module>
    main()
  File "load_generator.py", line 17, in main
    subprocess.run(cmd, shell=True)
  File "/Users/someone/.pyenv/versions/3.6.7/lib/python3.6/subprocess.py", line 403, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/Users/someone/.pyenv/versions/3.6.7/lib/python3.6/subprocess.py", line 709, in __init__
    restore_signals, start_new_session)
  File "/Users/someone/.pyenv/versions/3.6.7/lib/python3.6/subprocess.py", line 1344, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
OSError: [Errno 7] Argument list too long: '/bin/sh'

总记录数

$ cat test.txt | wc -l
18732 

样本数据

$ cat test.txt
'{"ErrorCode": 55201, "ErrorClass": 8, "NodeID": 32, "Params0": 1, "Params1": 70451202, "Params2": 9530, "Params3": 7, "Mid": -1, "SystemName": None, "NodeName": "MSC", "Swrelease": "SWL [CRC:3816]", "Build": "AA"}' 
'{"ErrorCode": 55004, "ErrorClass": 8, "NodeID": 32, "Params0": 538990697, "Params1": 410814, "Params2": 410931, "Params3": 0, "Mid": -1, "SystemName": None, "NodeName": "MSC", "Swrelease": "SWL [CRC:3816]", "Build": "AA"}' 
'{"ErrorCode": 282, "ErrorClass": 0, "NodeID": 32, "Params0": 61, "Params1": 14, "Params2": 0, "Params3": 5, "Mid": -1, "SystemName": None, "NodeName": "MSC", "Swrelease": "SWL [CRC:3816]", "Build": "AA"}'

Answer 1

我最终使用了@barmar的建议，即创建一个

.sh

文件并通过

/bin/sh

运行它。

Answer 2

尝试改变这个：

    while i <= int(vars(args)['e']):
        for line in f:
            cmd += f'curl -X POST -H "Content-Type: application/json" https://some_api_ep -d {line} &'
        i +=1
    subprocess.run(cmd, shell=True)

至：

    while i <= int(vars(args)['e']):
        for line in f:
            cmd = f'curl -X POST -H "Content-Type: application/json" https://some_api_ep -d {line} &'
            subprocess.run(cmd, shell=True)
        i +=1

或者你可以从这个“如何在Python中启动后台进程？”

查看