golang has well encapsulated http, making it very convenient when developing http-based services, but it is easy for us to ignore the underlying implementation details.
Here are some pitfalls I have stepped on and the corresponding solutions.
Call http service
The usual practice is as follows:
resp, err := ("/")
if err != nil {
// handle error
}
defer ()
body, err := ()
// ...
Trap 1: Response body is not closed in time
When the network program is running, after a while, the common problem is that an error occurs: "socket: too many open files", which is usually caused by the open file handle not being closed.
In the use of http, the easiest thing to ignore is that the body of the response returned by http must be closed, otherwise there will be memory leakage.
The problem that is even less easy to find is that if the content is not read out, the socket link will be leaked and subsequent services will not be used.
Here, is an interface of type, including read and close interfaces.
type Response struct {
// Body represents the response body.
//
// The response body is streamed on demand as the Body field
// is read. If the network connection fails or the server
// terminates the response, calls return an error.
//
// The http Client and Transport guarantee that Body is always
// non-nil, even on responses without a body or responses with
// a zero-length body. It is the caller's responsibility to
// close Body. The default HTTP client's Transport may not
// reuse HTTP/ "keep-alive" TCP connections if the Body is
// not read to completion and closed.
//
// The Body is automatically dechunked if the server replied
// with a "chunked" Transfer-Encoding.
Body
}
If there is no content read through or other interfaces, the socket link cannot be reused by subsequent connections, and the result is that the connection always exists.
Although the leak of the connection can be avoided after calling, we still recommend calling Close after obtaining the response, because between the place where the response is returned and ReadAll, if there is a conditional judgment, the interface returns in advance, it will still cause leakage.
defer ()
In addition, there is no need to shut down actively.
Trap 2: The default http transport setting is not suitable
In simple applications, using the default http client can meet the needs. In slightly more complex scenarios, you need to have a clearer understanding of the client.
type Client struct {
// Transport specifies the mechanism by which individual
// HTTP requests are made.
// If nil, DefaultTransport is used.
Transport RoundTripper
// Timeout specifies a time limit for requests made by this
// Client. The timeout includes connection time, any
// redirects, and reading the response body. The timer remains
// running after Get, Head, Post, or Do return and will
// interrupt reading of the .
//
// A Timeout of zero means no timeout.
//
// The Client cancels requests to the underlying Transport
// as if the Request's Context ended.
//
// For compatibility, the Client will also use the deprecated
// CancelRequest method on Transport if found. New
// RoundTripper implementations should use the Request's Context
// for cancelation instead of implementing CancelRequest.
Timeout
}
Here, we focus on the two fields Transport and Timeout. Transport records the transaction information of this request, as well as information related to connection multiplexing.
Timeout records the timeout of this call to avoid long waits when an exception occurs.
The default Transport definition we usually use is as follows:
var DefaultTransport RoundTripper = &Transport{
Proxy: ProxyFromEnvironment,
DialContext: (&{
Timeout: 30 * ,
KeepAlive: 30 * ,
DualStack: true,
}).DialContext,
MaxIdleConns: 100,
IdleConnTimeout: 90 * ,
TLSHandshakeTimeout: 10 * ,
ExpectContinueTimeout: 1 * ,
}
By default, it will retain open connections for future reuse. If the service wants to connect to many hosts, it will save many idle connections. IdleConnTimeout is used to recycle idle connections that have exceeded a certain period of time;
In fact, the MaxIdleConns of Defaulttransport is 100, which is still relatively small in many scenarios, especially when large systems need to be managed and modules are frequently interacted.
In addition, if the connection needs to access many resource nodes regularly, we know that the number of connections required on each resource node is greater than 2, then many short connections will appear, because for each resource machine, the default maximum number of connections of DefaultTransport is 2 and the maximum idle connection is 1.
type Transport struct {
// MaxIdleConnsPerHost, if non-zero, controls the maximum idle
// (keep-alive) connections to keep per-host. If zero,
// DefaultMaxIdleConnsPerHost is used.
MaxIdleConnsPerHost int
// MaxConnsPerHost optionally limits the total number of
// connections per host, including connections in the dialing,
// active, and idle states. On limit violation, dials will block.
//
// Zero means no limit.
//
// For HTTP/2, this currently only controls the number of new
// connections being created at a time, instead of the total
// number. In practice, hosts using HTTP/2 only have about one
// idle connection, though.
MaxConnsPerHost int
}
Long connections for HTTP and long connections for TCP
In http1.1, http maintains long connections by default for future reuse, but this long connection is usually time-limited, and to the settings in the Transport we opened above, the number of idle connections is the maximum limit. If this limit is exceeded, the remaining new connections become short connections.
The TCP protocol itself is a long connection. If there is no data transmission for a certain period of time, it will send a heartbeat to detect whether the connection is alive. If so, the connection will continue to be valid.
Summarize
The above is personal experience. I hope you can give you a reference and I hope you can support me more.