Python Generators: Complete Guide with Examples
Master Python generators - powerful tools for creating memory-efficient iterators using the yield keyword. Learn generator functions, expressions, coroutines, and handle large datasets with lazy evaluation.
yield Keyword
Core generator feature
30+ Examples
Practical generator patterns
Memory Efficient
Lazy evaluation
Infinite Sequences
Stream processing
What is a Generator?
A generator is a special type of function that returns values one at a time instead of returning all values at once.
Generators use the keyword: yield
Why Use Generators?
Generators are useful because they:
- Save memory
- Work faster for large data
- Produce values lazily (on demand)
- Simplify iterator creation
Normal Function vs Generator
Normal Function
def numbers():
return [1, 2, 3]
print(numbers())Output: [1, 2, 3]
Generator Function
def numbers():
yield 1
yield 2
yield 3
print(numbers())Output: <generator object numbers at 0x...>
Accessing Generator Values
Use: next() and loops.
Using next()
def demo():
yield 10
yield 20
yield 30
g = demo()
print(next(g))
print(next(g))
print(next(g))Output:
10
20
30
What Happens Internally?
When yield executes:
- Function pauses
- State is saved
- Resumes from same point next time
Generator with Loop
def count():
for i in range(5):
yield i
g = count()
for value in g:
print(value)Output:
0
1
2
3
4
Difference Between return and yield
| return | yield |
|---|---|
| Ends function | Pauses function |
| Returns single value | Returns sequence one-by-one |
| Uses more memory | Memory efficient |
Generator Expression
Similar to list comprehension.
List Comprehension
nums = [x*x for x in range(5)]
print(nums)Output: [0, 1, 4, 9, 16]
Generator Expression
nums = (x*x for x in range(5))
print(nums)
for i in nums:
print(i)Output:
<generator object ...>
0
1
4
9
16
Memory Comparison
Generators are memory efficient because they do not store all values at once.
Infinite Generators
Generators can generate infinite values.
Example:
def infinite():
n = 1
while True:
yield n
n += 1
g = infinite()
print(next(g))
print(next(g))
print(next(g))Output:
1
2
3
Fibonacci Using Generator
def fibonacci(n):
a, b = 0, 1
for i in range(n):
yield a
a, b = b, a + b
for num in fibonacci(10):
print(num)Output:
0
1
1
2
3
5
8
13
21
34
Generator for Reading Large Files
def read_file(file_name):
with open(file_name) as file:
for line in file:
yield line
for line in read_file("data.txt"):
print(line)Advantages of Generators
- Faster execution
- Reduced memory usage
- Useful for large datasets
- Supports lazy evaluation
- Easy iterator implementation
Disadvantages of Generators
- Can iterate only once
- Slightly harder to debug
- Values are not stored permanently
Iterators vs Generators
| Iterator | Generator |
|---|---|
| Created using class | Created using function |
Uses __iter__() and __next__() | Uses yield |
| More complex | Simpler |
Comparison Table
| Feature | Iterator | Generator | Decorator |
|---|---|---|---|
| Purpose | Loop protocol | Easy iteration | Modify functions |
| Implementation | Class with __next__ |
Function with yield |
Function returning wrapper |
| Memory | Efficient | Very efficient | Minimal overhead |
| Use case | Custom iteration | Lazy evaluation | Cross-cutting concerns |
| State | Manual management | Automatic suspension | Wraps existing code |
Real-Life Example
Music Streaming App
Songs are loaded one-by-one instead of loading all songs into memory.
This is similar to generators.
zip() and enumerate() for cleaner loops and aligned iteration