生成器和迭代器 Python

生成器 generator

An object created by a generator function.
Each yield temporarily suspends processing, remembering the location execution state (including local variables and pending try-statements). When the generator iterator resumes, it picks up where it left off (in contrast to functions which start fresh on every invocation).

循环的过程中计算。每次循环都会产生返回值（通过yield可以抛出），极大地减少需要的内存。

生成器表达式

生成器表达式是迭代和列表解析的组合。
在生成一个列表[1,2,3,4,5,6,7,8,9,10]的时候：

# 列表推导生成式
liist1 = [i+1 for i in range(10)]

列表推导生成式可以直接生成列表并且储存所有数据 [1,2,3,4,5,6,7,8,9,10]

# 生成器
liist2 = (i+1 for i in range(10))

生成器则是生成了 <generator object <genexpr> at 0x00F02CA0>
逐个输出可以不断通过next(liist2)函数获得生成器的下一个返回值。在没有更多的元素可以生成时，会抛出StopIteration 错误。这种做法并不多见。
generator 也是一个可迭代对象，所以我们可以通过 for loop循环输出。

for i in liist2:
    print(i)

在这里 for loop 实际上就是通过不断调用next()函数实现的。等价于：

while True:
    try:
        x = next(liist2)
    # 遇到StopIteration就退出循环    
    except StopIteration:
        break

生成器函数

生成器函数会在每次执行完毕后利用 yield 自动挂起，并输出一个返回值，之后随时可以重新利用.__next__()继续执行，直至最后没有返回值引起StopIteration异常，结束迭代。

• 斐波那契数列
  在生成斐波那契数列时，如果想要输出每一个数, 每次计算后print()是一种方式：

#fibonacci数列1，1，2，3，5，8，12，21，34.
def fibo1(n):
    a,b = 0,1
    count = 0
    while count < n:
        a,b =b,a+b
        count += 1
        print(a)
    return 'Done!'

fibo1(10)

也可以调用 yield 完成：

#fibonacci数列1，1，2，3，5，8，12，21，34.
def fibo2(n):
    a,b = 0,1
    count = 0
    while count < n:
        yield b
        a,b =b,a+b
        count += 1
    return 'Done!'

res = fibo2(10)
for i in res:
    print(i)

使用生成器完成的好处是可以减少内存占用，每次运算时用多少占多少。

• 单线程并发
  yield 也可以单线程实现并发运算效果：

import time
def consumer(name):
    print(f"{name} 准备讲课啦!")
    while True:
       lesson = yield
 
       print("开始[%s]了,[%s]老师来讲课了!" %(lesson,name))
 
 
def producer():
    c = consumer('Tom')
    c2 = consumer('Jerry')
    c.__next__()
    c2.__next__()

    print("同学们开始上课 了!")
    tut = ['math','english','chinese','salksjmdlaksjd']
    for i in tut:
        time.sleep(1)
        print("到了两个同学!")
        c.send(i)
        c2.send(i)

producer()

输出：

Tom 准备讲课啦!
Jerry 准备讲课啦!
同学们开始上课 了!
到了两个同学!
开始[math]了,[Tom]老师来讲课了!
开始[math]了,[Jerry]老师来讲课了!
到了两个同学!
开始[english]了,[Tom]老师来讲课了!
开始[english]了,[Jerry]老师来讲课了!
到了两个同学!
开始[chinese]了,[Tom]老师来讲课了!
开始[chinese]了,[Jerry]老师来讲课了!
到了两个同学!
开始[salksjmdlaksjd]了,[Tom]老师来讲课了!
开始[salksjmdlaksjd]了,[Jerry]老师来讲课了!

---

可迭代的 Iterable 和 迭代器 Iterator

iterables

An object capable of returning its members one at a time. Examples of iterables include all sequence types (such as list, str, and tuple) and some non-sequence types like dict, file objects, and objects of any classes you define with an __iter__() method or with a __getitem__() method that implements Sequence semantics.
Iterables can be used in a for loop and in many other places where a sequence is needed (zip(), map(), …). When an iterable object is passed as an argument to the built-in function iter(), it returns an iterator for the object. This iterator is good for one pass over the set of values. When using iterables, it is usually not necessary to call iter() or deal with iterator objects yourself. The for statement does that automatically for you, creating a temporary unnamed variable to hold the iterator for the duration of the loop. See also iterator, sequence, and generator.

iterator

An object representing a stream of data. Repeated calls to the iterator’s __next__() method (or passing it to the built-in function next()) return successive items in the stream. When no more data are available a StopIteration exception is raised instead. At this point, the iterator object is exhausted and any further calls to its __next__() method just raise StopIteration again. Iterators are required to have an __iter__() method that returns the iterator object itself so every iterator is also iterable and may be used in most places where other iterables are accepted. One notable exception is code which attempts multiple iteration passes. A container object (such as a list) produces a fresh new iterator each time you pass it to the iter() function or use it in a for loop. Attempting this with an iterator will just return the same exhausted iterator object used in the previous iteration pass, making it appear like an empty container.

• 可迭代的 Iterable对象。 可以使用isinstance(x, Iterable) 判断。
  集合数据类型： list tuple dict set str
  生成器 generator：生成器 & 生成器函数yield

>>> from collections.abc import Iterable
>>> isinstance([], Iterable)
True
>>> isinstance({}, Iterable)
True
>>> isinstance('abc', Iterable)
True
>>> isinstance((x for x in range(10)), Iterable)
True
>>> isinstance(100, Iterable)
False

• 迭代器也可以通过isinstance(x, Iterator) 判断。

>>> from collections.abc import Iterator
>>> isinstance((x for x in range(10)), Iterator)
True
>>> isinstance([], Iterator)
False
>>> isinstance({}, Iterator)
False
>>> isinstance('abc', Iterator)
False

• list dict之类可以迭代但是不是迭代器
• 可作用于for循环的对象都是Iterable类型
• 可作用于next()函数的对象都是Iterator类型，它们表示一个惰性计算的序列
• 可以对list等Iterable但是不是Iterator的类型使用iter()函数，获得一个Iterator对象

>>> from collections.abc import Iterator
>>> isinstance(iter({}), Iterable)
True
>>> isinstance(iter({}), Iterator)
True

• 总结
  一个实现了iter方法的对象是可迭代的，一个实现next方法并且是可迭代的对象是迭代器。
  可以被next()函数调用并不断返回下一个值的对象称为迭代器：Iterator。

所以一个实现了iter方法和next方法的对象就是迭代器

---

Reference

[1] https://docs.python.org/3/glossary.html#term-generator

[2] python 生成器和迭代器有这篇就够了