background
It mainly records the practice of golang concurrent development in work and in various documents. Golang concurrency is mainly used
Channel: Use channel to control subcoroutines, WaitGroup: Use semaphore mechanism to control subcoroutines, Context: Use context to control subcoroutines. Using one or more of these three mechanisms can achieve good concurrency control. Regarding these three knowledge points,https:///jiaoben/The introduction is more detailed, here are only a few scenario usages.
practice
waitGroup performs different tasks concurrently
This writing method is suitable for concurrent processing of small cases, and funcA and funcB functions when different tasks are different.
wg := {}
var result1 interface{}
var result2 interface{}
(2)
go func() {
defer func() {
()
}()
result1 = funcA()
}()
go func() {
defer func() {
()
}()
result1 = funcB()
}()
() waitGroup performs the same task concurrently
Suppose there is a batch of urls that need to be crawled concurrently. At this time, it may be that the requested address is different and the function of the task is consistent. At this time, you can use the for loop to execute in batches. When using this method, you need to use a thread-safe structure to synchronize data. In addition, the biggest disadvantage of the following writing method is that it cannot be used for concurrency control. If there are too many requests, it is easy to fill the downstream, and too many coroutines are activated, which wastes resources. It is only suitable for small data sets.
import (
"fmt"
"io/ioutil"
"net/http"
"strings"
"sync"
)
func main() {
idList := []string{"x", "xx"}
wg := {}
dataMap := {} // It is thread-safe. If you use map to store the result, you will report an error. // Accept the return result can also be a channel, and the channel is also thread-safe for _, id := range idList {
(1)
go func(id string) {
defer func() {
()
}()
data := PostData(id)
(id, data)
}(id)
}
()
for _, id := range idList {
((id))
}
}
func PostData(id string) string {
url := "http:xx"
method := "POST"
payload := ((`{"id":"%s"}`, id))
client := &{}
req, err := (method, url, payload)
if err != nil {
(err)
return ""
}
("Content-Type", "application/json")
res, err := (req)
if err != nil {
(err)
return ""
}
defer ()
body, err := ()
if err != nil {
(err)
return ""
}
return string(body)
}Add channel to control concurrency
package main
import (
"fmt"
"net/http"
"sync"
)
func main() {
urls := []string{"", "", ""}
//Control concurrency is 2 concurrency := 2
sem := make(chan struct{}, concurrency)
var wg
for _, url := range urls {
(1)
go func(url string) {
sem <- struct{}{} // Get semaphore defer func() {
<-sem // Release the semaphore ()
}()
resp, err := (url)
if err != nil {
("Error fetching %s: %v\n", url, err)
return
}
defer ()
("Fetched %s with status code %d\n", url, )
}(url)
}
()
("All URLs fetched")
}Concurrent programming using channel
Next, we use a for loop to iterate through the URL slice and start a goroutine for each URL. In each goroutine, we first get a signal from the concurrency channel to indicate that the request can be started. We then send an http GET request and send the response result to the result channel. Finally, we release a signal that the request has been completed.
In the main function, we use another for loop to read all response results from the result channel and print them to the console.
It should be noted that we use an empty structure {} in the concurrency channel because we only need the channel to control the concurrency, and do not need to pass any data in the channel. In addition, we use the blocking characteristic of the channel to control the concurrency because when the channel is full, any operation that attempts to send data into the channel will be blocked until there is room for it.
package main
import (
"fmt"
"net/http"
)
func main() {
urls := []string{"", "", ""}
// Create a channel to control the concurrency to 2 concurrency := make(chan struct{}, 2)
// Create a channel to receive response results results := make(chan string, len(urls))
for _, url := range urls {
// Start a goroutine to request the url go func(url string) {
// Get a signal from the channel to indicate that the request can be started concurrency <- struct{}{}
// Send http GET request resp, err := (url)
if err != nil {
results <- ("%s -> error: %s", url, err)
} else {
results <- ("%s -> status: %s", url, )
()
}
// Release a signal indicating that the request has been completed <-concurrency
}(url)
}
// Read all response results from the result channel for i := 0; i < len(urls); i++ {
(<-results)
}
}Use context to perform concurrent control
The goruntine generated under this mechanism is tree-shaped and has dependencies.
func getData(ctx , result chan string, id string) {
for {
select {
case <-():
("running get Data")
return
default:
resultData := PostData(id)
result <- resultData
}
}
}
func main() {
idList := []string{"xx", "xxx"}
ctx := ()
var result = make(chan string, 2)
go getData(ctx, result, idList[0])
go getData(ctx, result, idList[1])
(<-result)
(<-result)
}
func PostData(id string) string {
url := "http://xxx"
method := "POST"
payload := ((`{"id":"%s"}`, id))
client := &{}
req, err := (method, url, payload)
if err != nil {
(err)
return ""
}
("Content-Type", "application/json")
res, err := (req)
if err != nil {
(err)
return ""
}
defer ()
body, err := ()
if err != nil {
(err)
return ""
}
return string(body)
}This is the end of this article about several methods of golang concurrent execution. For more related golang concurrent execution content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!