SoFunction
Updated on 2025-04-12

Detailed explanation of the usage of yield function in Python

python official api address

1. Introduction to yield

In Python, the yield keyword is mainly used in generator functions. Its purpose is to enable functions to work like an iterator, that is, they can be traversed, but they do not load all the results into memory at once.

Basic usage

  • Define generator functions
def simple_generator():
    yield 1
    yield 2
    yield 3

gen = simple_generator()
print(next(gen))  # Output: 1print(next(gen))  # Output: 2print(next(gen))  # Output: 3
  • Use for loop to traverse the generatorThe generator object is iterable, so a for loop can be used to iterate over the value generated by the generator.
    In the following example, for loop through all the values ​​generated by the generator function and print them in turn.
def simple_generator():
    yield 1
    yield 2
    yield 3

for value in simple_generator():
    print(value)
  • Generator expressionsIn the following example, the generator expression generates a sequence of square numbers and prints all values ​​using a for loop.
gen_expr = (x * x for x in range(5))
for value in gen_expr:
    print(value)
  • Generator to list
def simple_generator():
    yield 1
    yield 2
    yield 3
gen = simple_generator()
print(list(gen))  # Output: [1, 2, 3]

Advanced Usage

3.1 yield send() method

The generator can not only pass yieldReturn value, can also be passedsend() Method receives external input.

send()The method allows sending a value to the generator, which will become the value of the previous yield expression.

This method can realize two-way communication between the generator and the outside.

def coroutine():
    while True:
        received = yield
        print(f"Received data: {received}")

co = coroutine()
next(co)  # Pre-excitation generator(10)  # Output: Received data: 10(20)  # Output: Received data: 20

In this example,coroutineThe generator function receives external data and prints it on each iteration.
next(co)Used to pre-excite the generator so that it is ready to receive data.

3.2 yield from method

Starting from Python 3.3, we introducedyield fromSyntax, which allows one generator to delegate another generator to generate values.

This delegate mechanism can simplify the use of nested generators and improve the readability and efficiency of code.

def base_code_pool():
    for i in range(3):
        yield f'BASE-{i + 1}'

def outsource_pool():
    for i in range(30):
        yield f'OUTS-{i + 1}'

def team_member_code():
    yield from base_code_pool()
    print('The internal resource number is used up, start using outsourcing')
    yield from outsource_pool()

team_member = team_member_code()
for i in range(5):
    print(next(team_member))

Running results:

BASE-1
BASE-2
BASE-3
Use out of internal resource numbers, start using outsourcing
OUTS-1
OUTS-2

In this example,team_member_codeGenerator function delegationbase_code_pool andoutsource_pool Generator to generate values.
whenbase_code_poolAfter the value of   is used up, the generator will continue tooutsource_poolGenerate values.

3.3 yield and yield from overlay

yieldandyield fromare two ways in Python to create generators, which allow functions to gradually return values ​​during iteration, rather than return all results at once.
This feature makes generators ideal for handling scenarios such as large datasets or infinite sequences, because they do not load all data into memory at once, saving memory resources.

Regarding whether it is possible to "overlay" the results of yield and yield from, we are actually talking about how to combine multiple generators or pass the results of one generator to another.

  • Use yield and yield from combination generatorWhen you want to combine two or more generators, you can useyieldTo produce elements in each generator one by one, or useyield fromcommand directly to the sub-generator, and let it be responsible for producing its own elements.
  • For example:
def generator_a():
    for i in range(3):
        yield i

def generator_b():
    for i in range(3, 6):
        yield i

def combined_generators():
    # Use yield to directly produce elements in each generator    for value in generator_a():
        yield value
    for value in generator_b():
        yield value
    
    # Or use yield from to delegate to the child generator    yield from generator_a()
    yield from generator_b()

In this example,combined_generatorsFunction pass yieldandyield fromThe results of the two generators are "overlaid".

Here, the meaning of "overlapping" refers to the output stream generated by the two generators sequentially connected, rather than the mathematical addition operation.

Canyieldandyield fromUse it in combination to implement combinations between generators, and even use these two statements within the same function at the same time.

This not only simplifies the code structure, but also provides more flexible control of the behavior of the generator.

Let's dive into how to use it effectively yieldandyield fromTo "overlay" the results of multiple generators.

  • The actual superposition of resultsIf we are talking about actual result overlays (such as numerical addition), then you need to explicitly write logic to achieve this.
    For example, if you want to add values ​​from different generators to get the sum, you can do this:
def sum_of_generators(gen1, gen2):
    return sum(gen1) + sum(gen2)

total = sum_of_generators(generator_a(), generator_b())
print(total)  # Output: 15 (0+1+2+3+4+5)# Output: 0 1 2 3 4 5for item in combined_generators():
    print(item)

Here,sum_of_generatorsThe function takes two generators as parameters, sums them separately and then adds them.
Note that this approach immediately consumes all elements of the two generators and calculates the sum, which is probably not the most efficient approach, especially for very large data sets.

  • Use yield and yield fromThe above example can be used toyieldandyield fromUse it in combination to implement combinations between generators, and even use these two statements within the same function at the same time.
    This not only simplifies the code structure, but also provides more flexible control of the behavior of the generator.
    Let's take a deeper look at how to use it effectivelyyieldandyield fromTo "overlay" the results of multiple generators.
    Consider a scenario where you have a main generator that needs to get values ​​from multiple sub-generators and may also produce some extra values ​​themselves.
    In this case, you can useyieldto directly produce these additional values, and useyield fromTo delegate to the sub-generator.
    For example:
def generator_a():
    for i in range(3):
        yield i

def generator_b():
    for i in range(3, 6):
        yield i

def combined_generators():
    # Use yield directly to produce additional values    yield "Start"
    
    # Delegate to the child generator using yield from    yield from generator_a()
    
    # Use yield directly to produce the intermediate value    yield "Middle"
    
    # Delegate to child generator using yield from b    yield from generator_b()
    
    # Finally, use yield directly to output the value    yield "End"

# Output: Start 0 1 2 Middle 3 4 5 Endfor item in combined_generators():
    print(item)

In this example,combined_generatorsFunctions show how to use it in the same generatoryield andyield from。

First, it passes yieldSent a string"Start"；

Then, useyield fromLeave control togenerator_a(), the latter is responsible for output0, 1, and 2；

Then, use it againyieldSend"Middle";After, it passes yield fromletgenerator_b()Output3, 4, and 5；

Finally, emit the string in the same way"End"。

This method allows you to easily add new generation logic without breaking the original logic, while also maintaining the clarity and readability of the code.

Handle overlays in complex situations

If you are facing more complex scenarios, such as if you need to process data from different generators in some form before outputting it, or if you need to decide whether to call a sub-generator based on certain conditions, then you can continue to expand this pattern.

For example, suppose you want to add the results of two generators as part of the final output:

def add_generators(gen1, gen2):
    iterator1 = iter(gen1)
    iterator2 = iter(gen2)
    try:
        while True:
            value1 = next(iterator1)
            value2 = next(iterator2)
            yield value1 + value2
    except StopIteration:
        pass

def enhanced_combined_generators():
    yield "Start"
    yield from add_generators(generator_a(), generator_b())
    yield "End"

# Output: Start 3 5 7 Endfor item in enhanced_combined_generators():
    print(item)

In summary,yieldandyield fromThe results can be "overlapping" through programming logic, but it depends on the specific behavior you want to achieve.

If it is a simple iterative output, you can directly use yield from to simplify the code;

And if it is a numerical or other type of accumulation, additional logic is required to complete the process.

Main application scenarios

  • Processing large data setsThe generator is great for handling large data sets because it can generate values ​​on demand when needed, rather than loading all values ​​into memory at once.
    This can significantly reduce memory usage and improve program performance.
def read_large_file(file_path):
    with open(file_path, 'r') as file:
        for line in file:
            yield line

for line in read_large_file('large_file.txt'):
    print(line, end='')

In this example, the generator functionread_large_fileRead large files line by line and print each line using for loop.
This method avoids loading the entire file into memory, thus saving memory.

  • Implement coroutines and concurrencyGenerators can be used to implement coroutine and concurrent programming modes.
    Coroutines are user-mode lightweight threads that can be used to simulate asynchronous operations and implement non-blocking I/O processing, etc.
def coroutine_example():
    while True:
        received = yield
        print(f"Processing data: {received}")

co = coroutine_example()
next(co)  # Pre-excitation generator('data1')  # Output: Processing data: data1('data2')  # Output: Processing data: data2

In this example,coroutine_example The generator function can receive external data and process it, implementing simple coroutine functions.

  • Data processing pipelineGenerators can be used to implement data processing pipelines, each generator function is responsible for a processing step.
    This pattern can break down complex tasks into simple steps, improving the readability and maintainability of the code.
def pipeline_stage1(data):
    for item in data:
        yield item * 2

def pipeline_stage2(data):
    for item in data:
        yield item + 1

data = range(5)
stage1 = pipeline_stage1(data)
stage2 = pipeline_stage2(stage1)

for value in stage2:
    print(value)

In this example, the data is processed by two generator functions.

First of all,pipeline_stage1Multiply by 2, then inpipeline_stage2Add 1.

def add_generators(gen1, gen2):
    iterator1 = iter(gen1)
    iterator2 = iter(gen2)
    try:
        while True:
            value1 = next(iterator1)
            value2 = next(iterator2)
            yield value1 + value2
    except StopIteration:
        pass

def enhanced_combined_generators():
    yield "Start"
    yield from add_generators(generator_a(), generator_b())
    yield "End"
# Output: Start 3 5 7 Endfor item in enhanced_combined_generators():
    print(item)
    Here is a helper function add_generators，It accepts two generators and takes out their elements one by one to generate new values。enhanced_combined_generators This new feature was added on the previous basis。In this way，You can build complex generator logic with great flexibility，Without writing a brand new generator for every possible change10。

To sum up, yield and yield from work well together to help developers build efficient and easy-to-maintain generator code. Whether it is simple connection or more complex operations, such as data transformation or conditional branching, the expected results can be achieved through reasonable design. It is important to understand the role of each statement and how they interact so that you can write Python code 2 that is both powerful and elegant. In addition, the introduction of yield from not only simplifies the code, but also enhances the ability to communicate between generators, especially when it comes to exception transfer14. Therefore, in practical programming practice, making full use of these two tools can greatly improve the quality and performance of the code.

5. Summary

  • yield function attributes
    • When a function contains the yield keyword, the function becomes a generator function.
    • Calling a generator function does not immediately execute the code inside the function body, but returns a generator object.
    • The generator object can be iterated. Each iteration, the generator function will continue to execute from where it last left until the next yield statement is encountered, then return a value and save the current state so that the execution will continue next time.
    • This behavior makes the generator very suitable for handling large amounts of data or streaming data, as it does not require loading all data into memory at once, saving memory resources.
  • yield vs return
    • return: When the function is executedreturnWhen a statement is made, it will immediately stop execution and return a value to the call.
      This means that the local variables of the function will be destroyed and the execution state of the function will not be saved.
      So if the same function is called again, it will be executed from scratch.
    • yield:andreturnDifferent, when executing to the yield statement, the function pauses execution, returns a value to the caller, and saves all the current local variable states.
      The next time the generator is called, the function will resume execution from the last paused place until the next yield statement or function ends.
  • Generator usage scenarios
    • Processing large data sets: Since the generator is lazy evaluation, it can generate data only when needed, so it is very suitable for handling large data sets, avoiding the memory shortage caused by loading all data at once.
    • Simplify the code structure: Using generators can simplify code structures, especially when complex state machines or recursive structures need to be handled.
    • Coroutines and concurrency: Although multiple concurrency models are already provided in Python's standard library, the generator can be implemented as a lightweight coroutine for implementing simple concurrency tasks.

This is the end of this article about the detailed explanation of the usage of yield function in Python. For more related Python yield function content, please search for my previous article or continue browsing the related articles below. I hope everyone will support me in the future!