SoFunction
Updated on 2025-03-06

Thoughts on the usage of yield return in C#

Preface

When we write C# code, we often need to deal with large amounts of data collections. In the traditional way, we often need to load the entire data collection into memory before doing it. However, if the data set is very large, this method will cause excessive memory usage and may even cause the program to crash.

In C#yield returnThe mechanism can help us solve this problem. By usingyield return, we can generate data sets on demand instead of generating the entire data set at once. This can greatly reduce memory usage and improve program performance.

In this article, we will discuss in depth in C#yield returnThe mechanism and usage of this powerful feature can help you better understand this powerful feature and use it flexibly in actual development.

How to use

We mentioned aboveyield returnGenerate the data set as needed instead of generating the entire data set at once. Next, with a simple example, we look at how it works so that we can deepen our understanding of it

foreach (var num in GetInts())
{
    ("External traversal:{0}", num);
}
IEnumerable<int> GetInts()
{
    for (int i = 0; i < 5; i++)
    {
        ("Internal traversal:{0}", i);
        yield return i;
    }
}

First, inGetIntsIn the method, we useyield returnKeywords to define an iterator. This iterator can generate sequences of integers on demand. On each cycle, useyield returnReturns the current integer. By 1foreachLoop to traverseGetIntsThe sequence of integers returned by the method. During iterationGetIntsThe method will be executed, but the entire sequence will not be loaded into memory. Instead, when needed, each element in the sequence is generated as needed. At each iteration, the information corresponding to the integer of the current iteration is output. So the output result is

Internal traversal: 0
External traversal: 0
Internal traversal: 1
External traversal: 1
Internal traversal: 2
External traversal: 2
Internal traversal: 3
External traversal: 3
Internal traversal: 4
External traversal: 4

As you can see, the integer sequence is generated on demand, and the corresponding information is output every time it is generated. This method can greatly reduce memory usage and improve program performance. Of course fromc# 8The way to start asynchronous iteration is also supported

await foreach (var num in GetIntsAsync())
{
    ("External traversal:{0}", num);
}
async IAsyncEnumerable<int> GetIntsAsync()
{
    for (int i = 0; i < 5; i++)
    {
        await ();
        ("Internal traversal:{0}", i);
        yield return i;
    }
}

Unlike the above, if we need to use an asynchronous method, we need to return itIAsyncEnumerableType, the execution results of this method are consistent with the execution results of the synchronous method above, so we will not show it. Our examples above are based on continuous iteration of loops, and in fact they useyield returnThe method can also be output as needed, which is suitable for flexible iteration. As shown in the following example

foreach (var num in GetInts())
{
    ("External traversal:{0}", num);
}
IEnumerable<int> GetInts()
{
    ("Internal traversal: 0");
    yield return 0;
    ("Internal traversal: 1");
    yield return 1;
    ("Internal traversal: 2");
    yield return 2;
}

foreachThe loop will be called every timeGetInts()method,GetInts()The method is used internallyyield returnThe keyword returns a result. Each time I go through the next oneyield return. So the result of the above code is

Internal traversal: 0
External traversal: 0
Internal traversal: 1
External traversal: 1
Internal traversal: 2
External traversal: 2

Explore the essence

We've shown aboveyield returnHow to use, it is a lazy loading mechanism that allows us to process data one by one, rather than reading all data into memory at once. Next, let’s explore how the magical operation is implemented so that everyone can understand the iterative system related to a clearer understanding.

foreach nature

First let's take a lookforeachWhy can it be traversed, that is, if it can beforeachWhat conditions do the traversed object need to meet? At this time, we can decompile the tool to see what the compiled code looks like. I believe everyone is most familiar with it.List<T>If we traverse the collection, we will useList<T>Examples to demonstrate

List<int> ints = new List<int>();
foreach(int item in ints)
{
    (item);
}

The above code is very simple, and we did not give it any initialization data, which can eliminate interference, allowing us to see the results of decompilation more clearly and eliminate other interferences. Its decompiled code is like this

List<int> list = new List<int>();
List<int>.Enumerator enumerator = ();
try
{
    while (())
    {
        int current = ;
        (current);
    }
}
finally
{
    ((IDisposable)enumerator).Dispose();
}

There are many tools that can decompile code, and I use more of them.ILSpy、dnSpy、dotPeekand onlinec#Decompile the website,indnSpyDecompiled code can also be debugged.

Through the above decompiled code we can seeforeachIt will be compiled into a fixed structure, which is the iterator pattern structure in the design pattern we often mention

Enumerator enumerator = ();
while (())
{
   var current = ;
}

Through this fixed structure, let's summarize itforeachHow it works

  • Can beforeachThe object needs to be includedGetEnumerator()method
  • Iterator object containsMoveNext()Methods andCurrentproperty
  • MoveNext()Method returnboolType, determine whether it can continue to iterate.CurrentThe property returns the current iteration result.

We can take a lookList<T>How is the iterable source code structure of class implemented

public class List<T> : IList<T>, IList, IReadOnlyList<T>
{
    public Enumerator GetEnumerator() => new Enumerator(this);
    IEnumerator<T> IEnumerable<T>.GetEnumerator() => Count == 0 ? SZGenericArrayEnumerator<T>.Empty : GetEnumerator();
    IEnumerator () => ((IEnumerable<T>)this).GetEnumerator();
    public struct Enumerator : IEnumerator<T>, IEnumerator
    {
        public T Current => _current!;
        public bool MoveNext()
        {
        }
    }
}

There are two core interfaces involvedIEnumerable<andIEnumerator,The two of them define the abstraction of the ability that can implement iteration, and the implementation method is as follows

public interface IEnumerable
{
    IEnumerator GetEnumerator();
}
public interface IEnumerator
{
    bool MoveNext();
    object Current{ get; }
    void Reset();
}

If the class is implementedIEnumerableThe interface and implementedGetEnumerator()The method can beforeach, the iterated object isIEnumeratorType, including oneMoveNext()Methods andCurrentproperty. The above interface is the original object method, and this operation is aimed atobjectType collection object. Most of our actual development process use generic collections, and of course there are corresponding implementation methods, as shown below

public interface IEnumerable<out T> : IEnumerable
{
    new IEnumerator<T> GetEnumerator();
}
public interface IEnumerator<out T> : IDisposable, IEnumerator
{
    new T Current{ get; }
}

Can beforeachIteration does not mean that it must be implementedIEnumerableInterface, this just provides us with an abstraction that can be iterated. As long as the class containsGetEnumerator()Method and return an iterator, which contains the returnboolType ofMoveNext()Methods and get the current iterative objectCurrentJust attributes.

yield return nature

We saw above that it can beforeachWhat is the essence of iteration, thenyield returnThe return value can beIEnumerable<T>There must be something strange about the reception instructions. Let's decompile our example above and look at the code after decompilation. In order to facilitate everyone to compare the decompilation results, I will paste the above example again here.

foreach (var num in GetInts())
{
    ("External traversal:{0}", num);
}
IEnumerable&lt;int&gt; GetInts()
{
    for (int i = 0; i &lt; 5; i++)
    {
        ("Internal traversal:{0}", i);
        yield return i;
    }
}

We will not show all its decompilation results here, we will only show the core logic

//Foreach compiled resultsIEnumerator&lt;int&gt; enumerator = GetInts().GetEnumerator();
try
{
    while (())
    {
        int current = ;
        ("External traversal:{0}", current);
    }
}
finally
{
    if (enumerator != null)
    {
        ();
    }
}
//The result after the GetInts method is compiledprivate IEnumerable&lt;int&gt; GetInts()
{
    &lt;GetInts&gt;d__1 &lt;GetInts&gt;d__ = new &lt;GetInts&gt;d__1(-2);
    &lt;GetInts&gt;d__.&lt;&gt;4__this = this;
    return &lt;GetInts&gt;d__;
}

Here we can seeGetInts()The original code in the method is gone, but one more<GetInts>d__1Type l, that is,yield returnThe essence isSyntactic sugar. Let's take a look<GetInts>d__1Class implementation

//The generated class implements the IEnumerable interface and also implements the IEnumerator interface//Indicate that it contains both the GetEnumerator() method, the MoveNext() method and the Current attributeprivate sealed class &lt;&gt;GetIntsd__1 : IEnumerable&lt;int&gt;, IEnumerable, IEnumerator&lt;int&gt;, IEnumerator, IDisposable
{
    private int &lt;&gt;1__state;
    //The current iteration result    private int &lt;&gt;2__current;
    private int &lt;&gt;l__initialThreadId;
    public C &lt;&gt;4__this;
    private int &lt;i&gt;5__1;
    //The results of the current iteration    int IEnumerator&lt;int&gt;.Current
    {
        get{ return &lt;&gt;2__current; }
    }
    //The results of the current iteration    object 
    {
        get{ return &lt;&gt;2__current; }
    }
    //The constructor contains state fields, and the direction change means that the state machine can realize core process flow through the state machine.    public &lt;GetInts&gt;d__1(int &lt;&gt;1__state)
    {
        this.&lt;&gt;1__state = &lt;&gt;1__state;
        &lt;&gt;l__initialThreadId = ;
    }
    //Core method MoveNext    private bool MoveNext()
    {
        int num = &lt;&gt;1__state;
        if (num != 0)
        {
            if (num != 1)
            {
                return false;
            }
            //Control status            &lt;&gt;1__state = -1;
            //Self-increase, i++ loop in the code            &lt;i&gt;5__1++;
        }
        else
        {
            &lt;&gt;1__state = -1;
            &lt;i&gt;5__1 = 0;
        }
        //Cycle termination condition i<5 in the above loop        if (&lt;i&gt;5__1 &lt; 5)
        {
            ("Internal traversal:{0}", &lt;i&gt;5__1);
            // Assign the current iteration result to the Current attribute            &lt;&gt;2__current = &lt;i&gt;5__1;
            &lt;&gt;1__state = 1;
            //It is stated that it can continue to iterate            return true;
        }
        //Iteration ends        return false;
    }
    //IEnumerator's MoveNext method    bool ()
    {
        return ();
    }
    //IEnumerable IEnumerable method    IEnumerator&lt;int&gt; IEnumerable&lt;int&gt;.IEnumerable()
    {
        //Instantiate <GetInts>d__1 instance        &lt;GetInts&gt;d__1 &lt;GetInts&gt;d__;
        if (&lt;&gt;1__state == -2 &amp;&amp; &lt;&gt;l__initialThreadId == )
        {
            &lt;&gt;1__state = 0;
            &lt;GetInts&gt;d__ = this;
        }
        else
        {
            //Initialize the state machine            &lt;GetInts&gt;d__ = new &lt;GetInts&gt;d__1(0);
            &lt;GetInts&gt;d__.&lt;&gt;4__this = &lt;&gt;4__this;
        }
        //Because <GetInts>d__1 implements the IEnumerator interface, you can return it directly        return &lt;GetInts&gt;d__;
    }
    IEnumerator ()
    {
        //Because <GetInts>d__1 implements the IEnumerator interface, it can be converted directly        return ((IEnumerable&lt;int&gt;)this).GetEnumerator();
    }
    void ()
    {
    }
    void ()
    {
    }
}

Through the class it generates, we can see that the class is implementedIEnumerableThe interface has also been implementedIEnumeratorThe interface indicates that it contains bothGetEnumerator()Methods, also includeMoveNext()Methods andCurrentproperty. This class can satisfy thefoeachThe core structure of iteration. We wrote it manuallyforThe code is includedMoveNext()In the method, it contains the defined state mechanism code and moves iteration to the next element according to the current state machine code. Let's briefly explain oursforThe code is translated toMoveNext()Execution process in the method

  • During the first iteration<>1__stateInitialized to 0, representing the first iterated element, at this timeCurrentThe initial value is 0, and the loop control variable is<i>5__1The initial value is also 0.
  • Determine whether the termination condition is met. If it is not met, the logic in the loop will be executed. And change the loader<>1__stateis 1, which means that the first iteration execution is completed.
  • Loop control variables<i>5__1Continue to increase and change the loader<>1__stateis -1, which represents sustainable iteration. And loop to execute the custom logic of the loop body.
  • If the iteration condition is not met, returnfalse, that is,MoveNext()Iterative conditions are not metwhile (())Logical termination.

We also show anotheryield returnThe way is to include multipleyield returnForm of

IEnumerable&lt;int&gt; GetInts()
{
    ("Internal traversal: 0");
    yield return 0;
    ("Internal traversal: 1");
    yield return 1;
    ("Internal traversal: 2");
    yield return 2;
}

The result of decompiling the above code is as follows. Here we only show the core methodMoveNext()Implementation

private bool MoveNext()
{
    switch (&lt;&gt;1__state)
    {
        default:
            return false;
        case 0:
            &lt;&gt;1__state = -1;
            ("Internal traversal: 0");
            &lt;&gt;2__current = 0;
            &lt;&gt;1__state = 1;
            return true;
        case 1:
            &lt;&gt;1__state = -1;
            ("Internal traversal: 1");
            &lt;&gt;2__current = 1;
            &lt;&gt;1__state = 2;
            return true;
        case 2:
            &lt;&gt;1__state = -1;
            ("Internal traversal: 2");
            &lt;&gt;2__current = 2;
            &lt;&gt;1__state = 3;
            return true;
        case 3:
            &lt;&gt;1__state = -1;
            return false;
    }
}

Through the compiled code, we can see that multipleyield returnThe form will be compiled intoswitch...caseThere are several forms ofyield returnIt will be compiled inton+1indivualcase, one extracaseIt representsMoveNext()Termination condition, that is, returnfalseconditions. OtherscaseThen returntrueIt means iterating can continue.

IAsyncEnumerable interface

We show the synchronization aboveyield returnWay,c# 8New additions startedIAsyncEnumerable<T>Interface is used to complete asynchronous iteration, that is, scenarios where the iterator logic contains asynchronous logic.IAsyncEnumerable<T>The implementation code of the interface is as follows

public interface IAsyncEnumerable<out T>
{
    IAsyncEnumerator<T> GetAsyncEnumerator(CancellationToken cancellationToken = default);
}
public interface IAsyncEnumerator<out T> : IAsyncDisposable
{
    ValueTask<bool> MoveNextAsync();
    T Current { get; }
}

Its biggest difference is synchronousIEnumeratorWhat is included isMoveNext()The method returnsbool，IAsyncEnumeratorThe interface containsMoveNextAsync()Asynchronous method, the return isValueTask<bool>type. So the above example code

await foreach (var num in GetIntsAsync())
{
    ("External traversal:{0}", num);
}

So hereawaitAlthough it is addedforeachThe above, but the actual function is to execute every iterationMoveNextAsync()method. It can be roughly understood as the following working method

IAsyncEnumerator<int> enumerator = ();
while (().GetAwaiter().GetResult())
{
   var current = ;
}

Of course, the actual compiled code does not look like this, we used in previous articles<Summary of research on async await state machine asynchronous operation>As explained in one articleasync awaitWill be compiled intoIAsyncStateMachineAsynchronous state machine, soIAsyncEnumerator<T>Combinedyield returnThe implementation is more complex and contains more code than the synchronization method. However, the implementation principle can be compared with the synchronization method. However, we will not show more about the implementation of the asynchronous state machine at the same time.Asynchronous yield returnAfter compilation, the students who are interested can learn about it.

foreach enhancement

c# 9Added the enhanced function of foreach, that is, through the form of an extension method, the original inclusionforeachIncreased Objects of CapabilitiesGetEnumerator()Methods so that ordinary classes do not haveforeachThe ability can also be used to iterate. It is used as follows

Foo foo = new Foo();
foreach (int item in foo)
{
    (item);
}
public class Foo
{
    public List&lt;int&gt; Ints { get; set; } = new List&lt;int&gt;();
}
public static class Bar
{
    //Define the extension method for Foo    public static IEnumerator&lt;int&gt; GetEnumerator(this Foo foo)
    {
        foreach (int item in )
        {
            yield return item;
        }
    }
}

This function is indeed relatively powerful and meets the principle of openness and closure. We can enhance the function of the code without modifying the original code. It can be said that it is very practical. Let's take a look at what the compiled result is

Foo foo = new Foo();
IEnumerator<int> enumerator = (foo);
try
{
    while (())
    {
        int current = ;
        (current);
    }
}
finally
{
    if (enumerator != null)
    {
        ();
    }
}

Here we see the extension methodGetEnumerator()It is essentially syntactic sugar, which will compile the extension ability intoExtended class.GetEnumerator (extended instance)way. That is the original way we write code, but the compiler helps us generate its calling method. Next, let's take a lookGetEnumerator()What does the extension method compile into

public static IEnumerator<int> GetEnumerator(Foo foo)
{
    <GetEnumerator>d__0 <GetEnumerator>d__ = new <GetEnumerator>d__0(0);
    <GetEnumerator>d__.foo = foo;
    return <GetEnumerator>d__;
}

Do you think this code looks familiar? It's good and aboveyield return natureThe syntax sugar generation method mentioned in this section is the same. When compiling, a corresponding class is generated. The class here is<GetEnumerator>d__0, Let's take a look at the structure of this class

private sealed class &lt;GetEnumerator&gt;d__0 : IEnumerator&lt;int&gt;, IEnumerator, IDisposable
{
    private int &lt;&gt;1__state;
    private int &lt;&gt;2__current;
    public Foo foo;
    private List&lt;int&gt;.Enumerator &lt;&gt;s__1;
    private int &lt;item&gt;5__2;
    int IEnumerator&lt;int&gt;.Current
    {
        get{ return &lt;&gt;2__current; }
    }
    object 
    {
        get{ return &lt;&gt;2__current; }
    }
    public &lt;GetEnumerator&gt;d__0(int &lt;&gt;1__state)
    {
        this.&lt;&gt;1__state = &lt;&gt;1__state;
    }
    private bool MoveNext()
    {
        try
        {
            int num = &lt;&gt;1__state;
            if (num != 0)
            {
                if (num != 1)
                {
                    return false;
                }
                &lt;&gt;1__state = -3;
            }
            else
            {
                &lt;&gt;1__state = -1;
                //Because of the Ints in the example we are using List<T>                &lt;&gt;s__1 = ();
                &lt;&gt;1__state = -3;
            }
            //Because the above extension method uses the foreach traversal method            //It has also been compiled into the actual production method            if (&lt;&gt;s__1.MoveNext())
            {
                &lt;item&gt;5__2 = &lt;&gt;s__1.Current;
                &lt;&gt;2__current = &lt;item&gt;5__2;
                &lt;&gt;1__state = 1;
                return true;
            }
            &lt;&gt;m__Finally1();
            &lt;&gt;s__1 = default(List&lt;int&gt;.Enumerator);
            return false;
        }
        catch
        {
            ((IDisposable)this).Dispose();
            throw;
        }
    }
    bool ()
    {
        return ();
    }
    void ()
    {
    }
    void ()
    {
    }
    private void &lt;&gt;m__Finally1()
    {
    }
}

Seeing the code generated by the compiler, we can seeyield returnThe generated code structure is the same, justMoveNext()The logic in it depends on the specific logic we write the code, and different logics generate different codes. Here we will not explain the code generated, because the logic is similar to the code we explained above.

Summarize

Through this article we have introducedc#In-houseyield returngrammar and explore some of the thoughts brought by it. We have shown through some simple examplesyield returnHow to use iterators to process large amounts of data on demand. At the same time, we analyzeforeachIterative andyield returnThe nature of grammar explains their implementation principles and underlying mechanisms. Fortunately, the knowledge involved is generally simple. If you read the relevant implementation code carefully, I believe you will understand the implementation principle behind it. I will not elaborate on it here.

This is the end of this article about thinking about the usage of yield return in C#. For more related C# yield return content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!