如何计算其他代码使用的生成器中的项目

Question

我正在创建一个被另一个函数占用的生成器,但我仍然想知道生成了多少项:

lines = (line.rstrip('\n') for line in sys.stdin)
process(lines)
print("Processed {} lines.".format( ? ))

我能想到的最好的方法是用一个保持计数的类来包装发生器,或者可以将其内部转出并发送()内容.是否有一种优雅而有效的方法来查看当你生成多少项时发生器不是在Python 2中使用它的人吗？

编辑:这是我最终得到的:

class Count(Iterable):
    """Wrap an iterable (typically a generator) and provide a ``count``
    field counting the number of items.

    Accessing the ``count`` field before iteration is finished will
    invalidate the count.
    """
    def __init__(self, iterable):
        self._iterable = iterable
        self._counter = itertools.count()

    def __iter__(self):
        return itertools.imap(operator.itemgetter(0), itertools.izip(self._iterable, self._counter))

    @property
    def count(self):
        self._counter = itertools.repeat(self._counter.next())
        return self._counter.next()

Answer 1

如果您不在乎使用发电机,您可以这样做:

sum(1 for x in gen)

Answer 2

这是使用itertools.count()示例的另一种方式:

import itertools

def generator():
    for i in range(10):
       yield i

def process(l):
    for i in l:
        if i == 5:
            break

def counter_value(counter):
    import re
    return int(re.search('\d+', repr(counter)).group(0))

counter = itertools.count()
process(i for i, v in itertools.izip(generator(), counter))

print "Element consumed by process is : %d " % counter_value(counter)
# output: Element consumed by process is : 6

希望这有用.

Answer 3

通常,我只是将生成器变成一个列表并取其长度.如果你有理由认为这会消耗太多内存,你最好的选择似乎确实是你自己建议的包装类.但这并不算太糟糕:

class CountingIterator(object):
    def __init__(self, it):
        self.it = it
        self.count = 0
    def __iter__(self):
        return self
    def next(self):
        nxt = next(self.it)
        self.count += 1
        return nxt
    __next__ = next

(最后一行是为了向前兼容Python 3.x.)