Generators and Iterators in Python

Understanding generators and iterators is essential for writing efficient and memory-friendly Python code.

What is an Iterator?

• An iterator is any object that implements the __iter__() and __next__() methods.
• It allows us to traverse through all the elements in a collection, one at a time.
• When there are no more elements, a StopIteration exception is raised.

nums = [1, 2, 3]
iterator = iter(nums)
print(next(iterator))  # Output: 1
print(next(iterator))  # Output: 2

• iter() returns the iterator object itself.
• next() fetches the next item.

What is a Generator?

• A generator is a simpler way to create iterators.
• It is written like a normal function but uses the yield statement to return data.
• Each yield temporarily suspends the function’s state, allowing it to resume from where it left off.

def count_up_to(max):
    count = 1
    while count <= max:
        yield count
        count += 1

counter = count_up_to(3)
print(next(counter))  # Output: 1
print(next(counter))  # Output: 2

• No need to manually implement __iter__() and __next__().

Benefits of Generators

• Memory Efficient: Generate items one at a time, not all at once.
• Faster Startup: You don’t have to wait for all data to be processed.
• Clean Syntax: Easier to write and maintain than traditional iterators.
• Infinite Sequences: Ideal for sequences that have no end.

Generator Expressions

Generator expressions are similar to list comprehensions but use parentheses () instead of brackets []. They generate items lazily.

gen = (x * x for x in range(5))
print(next(gen))  # Output: 0
print(next(gen))  # Output: 1

Summary Table

Generators are a powerful feature in Python to handle large data efficiently with minimal memory overhead.

Let me know if you’d like me to add a real-world use case next, like file reading or API data streaming!

Interview angle

• “What does yield actually do?” - suspends the function, returning a value and preserving local state, resuming on the next next(). That suspension is what makes lazy, constant-memory iteration possible.
• “What is yield from for?” - delegating to a sub-generator, forwarding values, exceptions and the return value. It replaces a manual loop and is the basis of generator-based coroutines.
• “How do you process a large file?” - iterate the file object directly, which yields lines lazily. .readlines() loads the whole file and is the memory bug this pattern exists to avoid.