Go Map Concurrent Conflict Prevention and Resolution

background

Regarding Go Map, there are two features that need to be paid attention to:

• Map is unsafe for concurrent read and write, which is due to performance considerations;
• Errors caused by concurrent reading and writing of Map, cannot be usedrecovercapture.

The latter means that if there are concurrent reading and writing problems, the service will hang up.

Everyone may know these two characteristics, but even with this consensus, I have seen accidents caused by this problem.

The general situation of the accident is that a person encapsulates the reading and writing of the map and does not use the lock. Another person opens a coroutine to read and write maps. The small number of requests for the test environment may not necessarily lead to a crash, so this problem will only occur in the production environment.

In addition to relying on developers' consciousness and code review, how can we prevent this situation? I think it is also important to add parallel testing to unit tests.

Parallel unit testing

Unit tests are not concurrent by default. For example, the following single test can be passed:

func TestConcurrent(t *) {
    var m = map[string]int{}
    // Write a map    ("write", func(t *) {
      for i := 0; i < 10000; i++ {
        m["a"] = 1
      }
    })
    // Read map    ("read", func(t *) {
      for i := 0; i < 10000; i++ {
        _ = m["a"]
      }
    })
}

But our expectation is that if the above single test fails, how can we solve it?

There is oneParallelMethod, which means that the current test will run in parallel with other tests. If the parameters have-or-, a test may be run multiple times, and multiple running instances of the same test will not be run in parallel.

We will give the above single test,():

func TestConcurrent(t *) {
    var m = map[string]int{}
    ("write", func(t *) {
      // Add parallel      ()
      for i := 0; i < 10000; i++ {
        m["a"] = 1
      }
    })
    ("read", func(t *) {
      // Add parallel      ()
      for i := 0; i < 10000; i++ {
        _ = m["a"]
      }
    })
}

This execution will report an error:

fatal error: concurrent map read and map write

Supports concurrent maps

It is not troublesome to have Map support concurrent reading and writing. Common practices include:

• When operating map, add a read and write lock；
• use .

It may be used more by everyone. Here is a simple demo.

We add locks to the above single test and you will be able to pass this time.

func TestConcurrent(t *) {
	var m = map[string]int{}
	// Define the lock, and the zero value can be used	var mu 
	("write", func(t *) {
		()
		for i := 0; i < 10000; i++ {
			// Lock			()
			m["a"] = 1
			// Unlock			()
		}
	})
	("read", func(t *) {
		()
		for i := 0; i < 10000; i++ {
			// Lock			()
			_ = m["a"]
			// Unlock			()
		}
	})
}

This article focuses on the maps that are built into the Go standard library and support concurrent reading and writing:。

It's the thread-safe versionmap[interface{}]interface{}, the zero value can be used directly, the value cannot be copied. It is mainly used in the following scenarios:

• When the value of the same key is written less and read more;
• But multiple goroutines read, write or modify a series of different keys.

In the above two scenarios,Mutex(orRWMutex) map will greatly reduce the competition for locks.

There are not many methods provided, here are some. Note that any is an alias for interface{} in go 1.18.

Store, set key-value.

func (m *Map) Store(key, value any)

Load, read value according to key.

func (m *Map) Load(key any) (value any, ok bool)

Delete, delete a key.

func (m *Map) Delete(key any)

Range, iterate through all keys, iffReturning false will stop traversing.

func (m *Map) Range(f func(key, value any) bool)

There are also LoadAndDelete (read after reading), LoadOrStore (read key, set when it does not exist).

We give the above single test, use, the test can also be passed.

func TestConcurrent(t *) {
	// Can use zero value	var m 
	("write", func(t *) {
		()
		for i := 0; i < 10000; i++ {
			// Write			("a", 1)
		}
	})
	("read", func(t *) {
		()
		for i := 0; i < 10000; i++ {
			// read			v, ok := ("a")
			if ok {
				_ = v.(int)
			}
		}
	})
}

refer to

/sync#Map

The above is the detailed content of the prevention and resolution of concurrent conflicts on Go Map. For more information about concurrent conflicts on Go Map, please follow my other related articles!