Go language Goroutines leak scenarios and prevention and control analysis

Scene

Go has many functions to automatically manage memory. for example:

• Whether variables are allocated to heap memory or stack memory, the compiler will judge through escape analysis;
• Automatic garbage collection of heap memory.

Even so, if we are not careful in encoding, we may still cause memory leaks. The most common one is goroutine leaks, such as the following function:

func goroutinueLeak() {
	ch := make(chan int)
	go func(ch chan int) {
    // Because ch has no data, this coroutine will block here.		val := <-ch
		(val)
	}(ch)
}

becausechThe data has not been sent, so the goroutine we enabled will keep blocking. Each callgoroutinueLeakA goroutine will be leaked. When you see it from the monitoring panel, the number of goroutines will gradually increase until the service of OOM.

Common causes of Goroutine leaks

The channel sending side causes blockage

Using context to set timeout is a common scenario. Just imagine, under what circumstances will the following function leak goroutine?

func contextLeak() error {
	ctx, cancel := ((), 1*)
	defer cancel()
	ch := make(chan int)
        //g1
	go func() {
		// Get data, such as network requests, may take a long time		val := RetriveData()
		ch <- val
	}()
	select {
	case <-():
		return ("timeout")
	case val := <-ch:
		(val)
	}
	return nil
}

RetriveData()If the timeout, thencontextLeak()An error will be returned and the execution of this function ends. And the coroutine we startedg1, since there is no receiver, it will block inch<-val。

The solution can be simple, for example,chAdd cache.

The channel receiver causes blockage

The function given at the beginninggoroutinueLeak, because the receiving end of the channel cannot receive data, resulting in blockage.

Here is another example. Is it possible for the function below to leak goroutinue?

func errorAssertionLeak() {
	ch := make(chan int)
        // g1
	go func() {
		val := <-ch
		(val)
	}()
        // RetriveSomeData means to obtain data, such as from the network	val, err := RetriveSomeData()
	if err != nil {
		return
	}
	ch <- val
	return nil
}

ifRetriveSomeData()ReturnederrNot fornil, then this function interrupts, and no data will be sent toch, which leads to coroutinesg1Will keep blocking.

How to prevent

A goroutine leak often takes a service to run for a period of time before it is discovered.

We can judge whether there is a goroutine leak by monitoring the number of goroutines; or use pprof (introduced in previous articles) to locate the leaked goroutine. But these are already repaired. The ideal situation is that we can discover it during the development process.

The recommended approach in this article is to use unit testing. Starting withgoroutinueLeakAs an example, let's write a single test:

func TestLeak(t *) {
	goroutinueLeak()
}

Execute go test and find that the test passed:

=== RUN   TestLeak
--- PASS: TestLeak (0.00s)
PASS
ok      example/leak    0.598s

This is because single test will not detect goroutine leakage by default.

We can join the Uber team in a single testuber-go/goleakBag:

import (
	"testing"
	"/goleak"
)
func TestLeak(t *) {
        // Add this line of code to automatically detect whether goroutine leaks	defer (t)
	goroutinueLeak()
}

At this time, execute go test and output:

=== RUN   TestLeak
    /xxx/leak_test.go:12: found unexpected goroutines:
        [Goroutine 21 in state chan receive, with example/.func1 on top of the stack:
        goroutine 21 [chan receive]:
        example/.func1(0x0)
            /xxx/:9 +0x27
        created by example/
            /xxx/:8 +0x7a
        ]
--- FAIL: TestLeak (0.46s)
FAIL
FAIL    example/leak    0.784s

At this time, the single test will not pass due to the goroutine leakage.

If you think every test case needs to be addeddefer (t)Too tedious (especially added to existing projects), goleak provides a method to use in TestMainVerifyTestMain, the above single test can be modified to:

func TestLeak(t *) {
	goroutinueLeak()
}
func TestMain(m *) {
	(m)
}

Summarize

Although my articles often mention single tests, I am not a big fan of unit tests. A solid foundation, sufficient test, and a responsible attitude are also very important.

Quote

• /blog/2018/1…
• /blog/2018/1…
• /uber-go/gol…

The above is the detailed content of the Go language Goroutines leak scenario and prevention and control analysis. For more information about Go Goroutines leak prevention and control, please follow my other related articles!