SoFunction
Updated on 2025-03-05

The use of Golang's fuzz testing tool

background

We often joke that programmers write bugs every day, which is indeed a fact. Not detecting a bug does not mean that there is no problem with the program. Traditional code reviews, static analysis, manual testing and automated unit tests cannot exhaust all input combinations, especially difficult to simulate some random, edge data.

In June last year, Go officially announced that the getip version has natively supported Fuzzing and started public beta, which will be released in mid-2022 with [Go version 1.18]. Go-fuzzing has discovered more than 200 bugs in the Go standard library so far (/dvyukov/go-fuzz#trophies). The upcoming [Go version 1.18] provides an excellent tool for self-testing of code.

Speaking of [Go version 1.18], the most concerned thing about generics should be generics, but I personally think go-fuzzing is also a highlight. Go 1.18 incorporates fuzz testing into the go test toolchain, and together with unit testing, performance benchmarks, etc., it has become an important member of the Go native testing toolchain.

This time, let’s talk about the go-fuzzing tool.

Development Environment

Upgrade to Go 1.18
Although Go 1.18 has not been officially released yet, you can download the RC version. And even if you use the old version of Go in the production environment, your personal local development environment can be upgraded to 1.18, and you can also use go-fuzzing to better self-test.

go-fuzzing

Official document: go fuzzing is to automatically test by continuously giving different inputs to a program, and to intelligently find failure examples by analyzing code coverage. This method can find some boundary problems as much as possible. Personal tests have indeed found some problems that are difficult to find in normal times.

fuzzing, also known as fuzz testing, is called fuzz testing or random testing in Chinese. It is essentially an automated testing technology. More specifically, it is an automated testing technology based on random inputs, which is often used to discover bugs and problems in the code that handles user inputs.

fuzz tests rules

func FuzzFoo(f *) {
    (5, "hello")

    (func(t *, i int, s string) {
        out, err := Foo(i, s)
        if err != nil && out != "" {
            ("%q, %v", out, err)
        }
    })
}
  • The function must start with Fuzz, the only parameter is *, and no return value
  • Fuzz tests must be executed under a file named *_test.go
  • fuzz target is a method that calls (*).Fuzz. The first parameter is *. The subsequent parameter is the parameter called fuzzing arguments. The method does not return a value
  • There can only be one fuzz target in each fuzz test
  • When calling(), the parameter type needs to be consistent with the order and type of fuzzing arguments.
  • fuzzing arguments only support the following types:
    • int, int8, int16, int32/rune, int64
    • uint, uint8/byte, uint16, uint32, uint64
    • string, []byte
    • float32, float64
    • bool

How to use go-fuzzing

1. First, you must define fuzzing arguments and write fuzzing targets through fuzzing arguments

2. Think about how to write a fuzzing target. The key is how to verify the correctness of the results. Because fuzzing arguments are given randomly, there must be a way to verify the results.

3. How to print out the error result when encountering a failed example

4. Generate a new test case based on the error result. This new test case will be used to debug the bugs found and can be left to CI for use.

Here is an example of summing numbers in a slice:

// slice_sum.go
func SliceSum(arr []int64) int64 {
  var sum int64

  for _, val := range arr {
    if val % 100000 != 0 {
      sum += val
    }
  }

  return sum
}

Step 1: Define fuzzing arguments

At least one fuzzing arguments needs to be given, otherwise go-fuzzing cannot generate test code.
This is the method of summing elements in a slice. Then we can use the number of elements n of the slice (simulate the number by yourself) as fuzzing arguments, and then go-fuzzing will automatically generate different parameters according to the running code coverage to simulate the test.

// slice_sum_test.go
func FuzzSliceSum(f *) {
  // 10. go-fuzzing is called corpus. The value of 10 is a value that makes go fuzzing cold-start. The specific amount does not matter  (10)

  (func(t *, n int) {
      // Limit 20 elements    n %= 20

    // Remaining treatment
  })
}

Step 2: Write a fuzzing target

The focus is to write verifiable fuzzing targets, not only to write test code based on given fuzzy parameters, but also to generate data that can verify the correctness of the results.
For the method of summing this slice element, it is to randomly generate slices of n elements, and then sum them to get the correct result.

package fuzz

import (
    "/stretchr/testify/assert"
    "math/rand"
    "testing"
    "time"
)

// slice_sum_test.go
func FuzzSliceSum(f *) {
  // Initialize random number seeds  (().UnixNano())
  // Material  (10)

  (func(t *, n int) {
    n %= 20

    var arr []int64
    var expect int64 // Expected value
    for i := 0; i < n; i++ {
      val := rand.Int63() % 1000000
      arr = append(arr, val)
      expect += val
    }

    // Comparison of the result of summing yourself with the result of summing the function    (t, expect, SliceSum(arr))
  })
}

Perform fuzz testing

➜  fuzz go test -fuzz=SliceSum
fuzz: elapsed: 0s, gathering baseline coverage: 0/52 completed
fuzz: elapsed: 0s, gathering baseline coverage: 52/52 completed, now fuzzing with 8 workers
fuzz: elapsed: 0s, execs: 9438 (34179/sec), new interesting: 2 (total: 54)
--- FAIL: FuzzSliceSum (0.28s)
    --- FAIL: FuzzSliceSum (0.00s)
        slice_sum_test.go:32: 
                Error Trace:    slice_sum_test.go:32
                                                        :556
                                                        :339
                                                        :337
                Error:          Not equal: 
                                expected: 5715923
                                actual  : 5315923
                Test:           FuzzSliceSum
    
    Failing input written to testdata/fuzz/FuzzSliceSum/8e8981ffa4ee4d93f475c807563f9d63854a6c913cdfb10a73191549318a2a51
    To re-run:
    go test -run=FuzzSliceSum/8e8981ffa4ee4d93f475c807563f9d63854a6c913cdfb10a73191549318a2a51
FAIL
exit status 1
FAIL    demo/fuzz       0.287s

In the above output, you can only see that the expected value is different from the actual value, but it is difficult to analyze errors.

Step 3: Print out an error example

If the above error output can be printed out an example of the error, it can be used as a test case for single test. We can't try it one by one, and there may not be only one wrong example.

Modify the fuzzy test code and add print:

package fuzz

import (
    "/stretchr/testify/assert"
    "math/rand"
    "testing"
    "time"
)

// slice_sum_test.go
func FuzzSliceSum(f *) {
    // Initialize random number seeds    (().UnixNano())
    // Material    (10)

    (func(t *, n int) {
        n %= 20

        var arr []int64
        var expect int64 // Expected value        var buf 
        ("\n")

        for i := 0; i < n; i++ {
            val := rand.Int63() % 1000000
            arr = append(arr, val)
            expect += val
            (("%d,\n", val))
        }

        // Comparison of the result of summing yourself with the result of summing the function        (t, expect, SliceSum(arr), ())
    })
}

Perform fuzzing test again

➜  fuzz go test -fuzz=SliceSum
fuzz: elapsed: 0s, gathering baseline coverage: 0/47 completed
fuzz: elapsed: 0s, gathering baseline coverage: 47/47 completed, now fuzzing with 8 workers
fuzz: elapsed: 0s, execs: 17109 (42507/sec), new interesting: 2 (total: 49)
--- FAIL: FuzzSliceSum (0.41s)
    --- FAIL: FuzzSliceSum (0.00s)
        slice_sum_test.go:34: 
                Error Trace:    slice_sum_test.go:34
                                                        :556
                                                        :339
                                                        :337
                Error:          Not equal: 
                                expected: 7575516
                                actual  : 7175516
                Test:           FuzzSliceSum
                Messages:       
                                92016,
                                642504,
                                400000,
                                489403,
                                472011,
                                811028,
                                315130,
                                298207,
                                57765,
                                542614,
                                136594,
                                351360,
                                867104,
                                918715,
                                515092,
                                665973,
    
    Failing input written to testdata/fuzz/FuzzSliceSum/9191ba4d7ea5420a9a76661d4e7d6a7a4e69ad4d5d8ef306ff78161a2acf1416
    To re-run:
    go test -run=FuzzSliceSum/9191ba4d7ea5420a9a76661d4e7d6a7a4e69ad4d5d8ef306ff78161a2acf1416
FAIL
exit status 1
FAIL    demo/fuzz       0.413s

Step 4: Write a new test case for single test based on the output error example

// After passing the single test, perform a fuzzy test to see if there are any other edge problemsfunc TestSliceSumFuzzCase1(t *) {
    arr := []int64{
        92016,
        642504,
        400000,
        489403,
        472011,
        811028,
        315130,
        298207,
        57765,
        542614,
        136594,
        351360,
        867104,
        918715,
        515092,
        665973,
    }
    // The expected value is obtained from the output of step 3    (t, int64(7575516), SliceSum(arr))
}

This allows easy debugging and can add effective test cases for single testing to ensure that this bug will not occur.

Production environment project Go version issue

What should I do if the Go version of the online project cannot be upgraded to 1.18?

The online version does not upgrade to 1.18, but we have no problem with local development and upgrading. You can add the following command comments to the head of the file:

slice_sum_test.go

//go:build go1.18
// +build go1.18

In this way, no matter which version we use online, we will not report any error, and we usually perform fuzzy testing locally

Note: The third line must be a blank line, otherwise it will become a package comment.

Some problems that cannot be reproduced, such as coroutine deadlock, the output is executed or stuck and then it will end after a while. I have not yet figured out this kind of long-term fuzzy test. If anyone knows, please tell me.

refer to
/dvyukov/go-fuzz#trophies
/blog/fuzz-beta

This is the end of this article about the use of Golang's fuzz testing tool. For more related Golang fuzz testing, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!