如何在x分钟后重复停止程序并重新运行它在python中更改一个变量？

Question

我有一个用python编写的程序，如下所示：

for i in range(4954):
    ........
    save("/Downloads/pos_" + str(i) + ".h5")

事实是程序运行时间呈指数增长（我们使用time.time()观察它）

所以我需要的是运行程序10分钟，然后重新运行程序。但我只需要将i in for循环更改为停止的数字。我可以做一个cron工作，但我该怎么做才能改变i值？

Answer 1

您可以使用marshal.dump（https://docs.python.org/3/library/marshal.html）在停止程序时保存该值，然后在启动时通过marshal.load加载它。

老实说，我认为解决问题的根本原因，即解决指数运行时间是一种更好的方法。

Answer 2

如果循环中的每个迭代（每个i值）彼此独立，则可以将其与ProcessPoolExecutor并行化：

import concurrent.futures

def downloader(i):
    ...
    save("/Downloads/pos_" + str(i) + ".h5")

def main():
    i_range = list(range(4954))
    with concurrent.futures.ProcessPoolExecutor(10) as executor: # create 10 workers (subprocesses)
        for number, result in zip(i_range , executor.map(downloader, i_range)):
            print('File %d is scheduled to download!' % (number))

if __name__ == '__main__':
    main()

这需要Python3。

Answer 3

考虑使用发电机：

import time

def gen(n):
    for i in range(n):
        yield n

g = gen(4954)

time_limit = 600

while some_condition:
# whatever condition required for you to continually restart this section

    start = time.time()
    while time.time() - start < time_limit:
        ... some code...
        try: 
            save("/Downloads/pos_" + str(next(g)) + ".h5")
        except StopIteration:
            print('end of iterations')

g将记住它的迭代，并在下次调用next(g)时继续。

Answer 4

我的解决方案：

import os
import time
import threading

class test_class(object):
    def __init__(self):
        self.iter_number = 0
        self.new_iter_number = 0

        self.iter_file_dir = "/tmp_file.txt" # tmp file where sto the new iteration number remaining

        self.end_iteration = False # Flag to stop the iteration

        self.delay = 1 # Seconds that means 10 minutes

        with open(self.iter_file_dir, "r") as file:
            self.iter_number = int(file.readline())

        self.run()

    def for_function(self):
        for i in range(self.iter_number):
            save("/Downloads/pos_" + str(i) + ".h5")
            if(self.end_iteration):
                self.new_iter_number = self.iter_number - i
                break

    def run(self):
        threading.Thread(name="Iteration_Thread", target=self.for_function).start()

        time_remaining = self.delay-time.time()%self.delay
        time.sleep(time_remaining)

        self.end_iteration = True

        # Save the file with the new value
        with open(self.iter_file_dir, 'w') as f:
            f.write(str(self.new_iter_number))
test_class()

这段代码背后的想法很简单：

1. 在您想要的位置创建一个tmp_file.txt，并在其中写入for中需要的迭代次数。
2. 创建一个休眠X分钟的计时器，然后设置一个终止for循环的标志。
3. 之后在tmp_file.txt中保存新计算的迭代次数

每次cron运行程序时，它都会加载新的迭代值。显然，当在文件中完成迭代时，写入0，因此不再执行迭代。