重用生成器表达式

Question

生成器表达式是一个非常有用的工具，并且比列表推导式具有巨大的优势，即它不会为新数组分配内存。

我在生成器表达式方面面临的问题是，我只能使用这样的生成器一次，这最终让我不得不编写列表推导式：

>>> names = ['John', 'George', 'Paul', 'Ringo']
>>> has_o = (name for name in names if 'o' in name)
>>> for name in has_o:
...   print(name.upper())
...
JOHN
GEORGE
RINGO
>>> for name in has_o:
...   print(name.lower())
...
>>>

上面的代码说明了生成器表达式只能使用一次。这是当然的，因为生成器表达式返回生成器的一个实例，而不是定义一个可以一次又一次实例化的生成器函数。

有没有办法在每次使用生成器时克隆生成器，以便使其可重用，或者使生成器表达式语法返回生成器函数而不是单个实例？

Answer 1

使它成为

lambda

：

has_o = lambda names: (name for name in names if 'o' in name)
for name in has_o(["hello","rrrrr"]):
   print(name.upper())
for name in has_o(["hello","rrrrr"]):
   print(name.upper())

lambda

是一个单行代码，并且每次返回一个新的生成器。这里我选择能够传递输入列表，但是如果是固定的，你甚至不需要参数：

names = ["hello","rrrrr"]
has_o = lambda: (name for name in names if 'o' in name)
for name in has_o():
   print(name.upper())
for name in has_o():
   print(name.upper())

在最后一种情况下，请注意以下事实：如果

names

更改或重新分配，则

lambda

将使用新的

names

对象。您可以使用默认值技巧来修复名称重新分配：

has_o = lambda lst=names: (name for name in lst if 'o' in name)

并且您可以使用默认的值和复制技巧来修复

names

的后续修改（当您认为您的第一个目标是避免创建列表时，这不是超级有用:)）：

has_o = lambda lst=names[:]: (name for name in lst if 'o' in name)

（现在做出选择:)）

Answer 2

itertools.tee

允许您从一个可迭代对象中创建多个迭代器：

from itertools import tee

names = ['John', 'George', 'Paul', 'Ringo']
has_o_1, has_o_2 = tee((name for name in names if 'o' in name), 2)
print('iterable 1')
for name in has_o_1:
    print(name.upper())
print('iterable 2')
for name in has_o_2:
    print(name.upper())

输出：

iterable 1
JOHN
GEORGE
RINGO
iterable 2
JOHN
GEORGE
RINGO

Answer 3

这是对@Jean-FrançoisFabre 答案的增强。

从概念上讲，您要求的是一个强大的可迭代对象 （“稳健”是指可以对其进行多次

for...in

迭代，并且它们不会相互干扰）。 这是一个小助手类，可以干净地制作这样的东西：

class RobustIterableFromFunctionReturningIterator:
  def __init__(self, function_returning_iterator):
    self.function_returning_iterator = function_returning_iterator
  def __iter__(self):
    return self.function_returning_iterator()

然后你可以说：

names = ['John', 'George', 'Paul', 'Ringo']
has_o = RobustIterableFromFunctionReturningIterator(lambda: (name for name in names if 'o' in name))
for name in has_o:
   print(name.upper())
for name in has_o:
   print(name.upper())
for name0 in has_o:
  for name1 in has_o:
     print(name0.upper() + ' ' + name1.upper())

---

这个小助手类也更普遍有用；对于其他示例用途，请参阅我的回答：在可迭代对象上迭代多次

Answer 4

好的朋友，这里有一段代码可以让你的迭代器可重用。 每次迭代后它都会自动重置，因此您不必担心任何事情。 它的效率有多高，嗯，它是两个方法调用（tee() 的一个 next() 依次调用迭代器本身的 next()），以及在原始迭代器顶部额外的 try- except 块。 您必须决定微小的速度损失是否可以，或者使用 lambda 来重建迭代器，如其他答案所示。

from itertools import tee

class _ReusableIter:
    """
    This class creates a generator object that wraps another generator and makes it reusable
    again after each iteration is finished.
    It makes two "copies" (using tee()) of an original iterator and iterates over the first one.
    The second "copy" is saved for later use.
    After first iteration reaches its end, it makes two "copies" of the saved "copy", and
    the previous iterator is swapped with the new first "copy" which is iterated over while the second "copy" (a "copy" of the old "copy") waits for the
    end of a new iteration, and so on.
    After each iteration, the _ReusableIter() will be ready to be iterated over again.

    If you layer a _ReusableIter() over another _ReusableIter(), the result can lead you into an indefinite loop,
    or provoke some other unpredictable behaviours.
    This is caused by later explained problem with copying instances of _ReusableIter() with tee().
    Use ReusableIterator() factory function to initiate the object.
    It will prevent you from making a new layer over an already _ReusableIter()
    and return that object instead.

    If you use the _ReusableIter() inside nested loops the first loop
    will get the first element, the second the second, and the last nested loop will
    loop over the rest, then as the last loop is done, the iterator will be reset and
    you will enter the infinite loop. So avoid doing that if the mentioned behaviour is not desired.

    It makes no real sense to copy the _ReusableIter() using tee(), but if you think of doing it for some reason, don't.
    tee() will not do a good job and the original iterator will not really be copied.
    What you will get instead is an extra layer over THE SAME _ReusableIter() for every copy returned.

    TODO: A little speed improvement can be achieved here by implementing tee()'s algorithm directly into _ReusableIter()
    and dump the tee() completely.
    """
    def __init__ (self, iterator):
        self.iterator, self.copy = tee(iterator)
        self._next = self.iterator.next

    def reset (self):
        self.iterator, self.copy = tee(self.copy)
        self._next = self.iterator.next

    def next (self):
        try:
            return self._next()
        except StopIteration:
            self.reset()
            raise

    def __iter__ (self):
        return self

def ReusableIter (iterator):
    if isinstance(iterator, _ReusableIter):
        return iterator
    return _ReusableIter(iterator)

Usage:
>>> names = ['John', 'George', 'Paul', 'Ringo']
>>> has_o = ReusableIter(name for name in names if 'o' in name)
>>> for name in has_o:
>>>     print name
John
George
Ringo
>>> # And just use it again:
>>> for name in has_o:
>>>     print name
John
George
Ringo
>>>