A generator is a special type of iterator that allows Python to produce values one at a time instead of storing all the values in memory at once.
Generators are commonly created using a function with the yield keyword.
yield instead of return when it needs to produce a sequence of values one at a time.
A generator is an object that generates values one by one when they are requested.
Unlike a normal function, a generator function does not return all results at once.
It pauses its execution at each yield statement and continues from that point when the next value is requested.
def numbers():
yield 1
yield 2
yield 3
for number in numbers():
print(number)1 2 3
The yield keyword is used to produce a value from a generator.
When Python reaches yield, the generator pauses its execution and remembers its current state.
def demo():
yield 10
yield 20
yield 30Each time the generator requests another value, execution continues from where it previously paused.
def count():
yield 1
yield 2
yield 3
yield 4
yield 5
for value in count():
print(value)1 2 3 4 5
A generator is an iterator, so we can use the next() function with it.
def numbers():
yield 10
yield 20
yield 30
generator = numbers()
print(next(generator))
print(next(generator))
print(next(generator))10 20 30
A generator pauses when it reaches a yield statement.
def demo():
print("Start")
yield 10
print("Middle")
yield 20
print("End")
generator = demo()
print(next(generator))
print(next(generator))
Start 10 Middle 20
The generator continues from the point where it was previously paused.
| Normal Function | Generator Function |
|---|---|
Usually uses return. |
Uses yield. |
| Returns a result and ends. | Pauses and can continue later. |
| Can return a complete collection. | Produces values one at a time. |
| May require more memory for large collections. | Can be memory-efficient for large sequences. |
def even_numbers():
for number in range(2, 11, 2):
yield number
for number in even_numbers():
print(number)2 4 6 8 10
def squares(limit):
for number in range(1, limit + 1):
yield number * number
for value in squares(5):
print(value)1 4 9 16 25
A generator function can accept parameters just like a normal function.
def numbers(start, end):
for number in range(start, end + 1):
yield number
for value in numbers(5, 9):
print(value)5 6 7 8 9
def even_numbers(start, end):
for number in range(start, end + 1):
if number % 2 == 0:
yield number
for value in even_numbers(1, 10):
print(value)2 4 6 8 10
Python also provides generator expressions. Their syntax is similar to list comprehensions, but they use parentheses instead of square brackets.
numbers = (x * x for x in range(1, 6))
for number in numbers:
print(number)1 4 9 16 25
numbers_list = [x * x for x in range(1, 6)]
numbers_generator = (x * x for x in range(1, 6))| List Comprehension | Generator Expression |
|---|---|
Uses []. |
Uses (). |
| Creates the list immediately. | Produces values when requested. |
| Stores all generated values. | Can avoid storing all values at once. |
When a generator has no more values to produce, iteration ends.
Calling next() after the generator is exhausted raises StopIteration.
def numbers():
yield 1
yield 2
generator = numbers()
print(next(generator))
print(next(generator))
print(next(generator))1 2 StopIteration
Generators are especially useful when working with large amounts of data.
def numbers(limit):
for number in range(1, limit + 1):
yield number
for number in numbers(1000000):
if number > 5:
break
print(number)1 2 3 4 5
The generator produces values as needed instead of creating a million-value list first.
A generator can be memory-efficient because it produces one value at a time.
def numbers():
for number in range(1, 6):
yield number
generator = numbers()
for number in generator:
print(number)Only the value currently requested needs to be produced.
Generators can be useful for processing files line by line.
def read_lines(filename):
with open(filename, "r") as file:
for line in file:
yield line.strip()The function produces one line at a time rather than creating a separate list containing every line.
Generators can be combined so that the output of one stage becomes the input of another stage.
def numbers():
for number in range(1, 11):
yield number
def squares(values):
for value in values:
yield value * value
result = squares(numbers())
for value in result:
print(value)1 4 9 16 25 36 49 64 81 100
A generator function can contain multiple yield statements.
def colors():
yield "Red"
yield "Green"
yield "Blue"
for color in colors():
print(color)Red Green Blue
Generators also support the send() method, which can send a value into a paused generator.
def calculator():
total = 0
while True:
value = yield total
total += value
generator = calculator()
print(next(generator))
print(generator.send(10))
print(generator.send(20))0 10 30
The send() method resumes the generator and provides a value to the yield expression.
A generator can also use return to finish its execution.
def numbers():
yield 1
yield 2
return
for number in numbers():
print(number)1 2
The return statement ends the generator. A return value can also be carried by the resulting StopIteration exception.
| Generator | Custom Iterator |
|---|---|
Usually created using yield. |
Usually created by implementing __iter__() and __next__(). |
| Python manages much of the iterator state automatically. | The programmer manages the iterator state. |
| Usually requires less code. | Can require more code. |
| Is itself an iterator. | Implements the iterator protocol. |
def multiplication_table(number, limit):
for i in range(1, limit + 1):
yield number * i
table = multiplication_table(5, 10)
for value in table:
print(value)5 10 15 20 25 30 35 40 45 50
yield.yield keyword.yield pauses execution and preserves the generator's state.next() requests the next generated value.StopIteration indicates that iteration is complete.Question: Which keyword is used to create a generator function in Python?