Lazy generator pipelines

Now that you understand the generators individually, we'll arrange both of them into a lazy pipeline. We'll be using take() and distinct() together to fetch the first three unique items from a collection:

def run_pipeline():
items = [3, 6, 6, 2, 1, 1]
for item in take(3, distinct(items)):
print(item)

Notice that the distinct() generator only does just enough work to satisfy the demands of the take() generator which is iterating over it

  • It never gets as far as the last two items in the source list because they are not needed to produce the first three unique items. This lazy approach to computation is very powerful, but the complex control flows it produces can be difficult to debug. It's often useful during development to force evaluation of all of the generated values, and this is most easily achieved by inserting a call to the list() constructor:
take(3, list(distinct(items)))

This interspersed call to list() causes the distinct() generator to exhaustively process its source items before take() does its work. Sometimes when you're debugging lazily evaluated sequences, this can give you the insight you need to understand what's going on.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset