Current limiters are very important components to improve service stability and can be used to limit request rates and protect services to avoid service overload. There are many ways to implement the current limiter, and the common current limiting algorithms areFixed window, sliding window, leaky bucket, token bucket
Simply put, a token bucket is to imagine a fixed-sized bucket. The system will place a token into the bucket at a constant rate, and if the bucket is full, it will not be placed for the time being. When there are fewer requests, you can "storage" some tokens first to deal with sudden traffic. If there are remaining tokens in the bucket, you can keep getting it. If there are no tokens left, you need to wait until the token is placed in the bucket.
After understanding the principle of the token bucket, some students really want to implement a current limiter to their own project. em... How to say, building a wheel is indeed conducive to improving their own level, but if it is applied to commercial projects, there is actually no need to build the wheel by themselves. Golang has already made the wheel for us...~!
The official extension library provided by Golang has its own implementation of the current limiting algorithm, that is,/x/time/rate. This current limiter is also implemented based on Token Bucket.
The internal structure of the current limiter
time/ratePackedLimiterThe type defines the current limiter, and all current limiting functions are based onLimiterThe internal structure of type implementation is as follows:
type Limiter struct {
mu
limit Limit
burst int // The size of the token bucket tokens float64
last // The last time to update tokens lastEvent // The time of the last speed limiter event (pass or limit are speed limiter events)}The functions of its main fields are:
limit：
limitThe field indicates the rate of putting a token into the bucket. Its type is Limit, which is the type alias of int64.set uplimitYou can use numbers to specify how many tokens to put into the bucket per second, or you can specify the time interval for putting tokens into the bucket., in fact, after specifying the number of tokens per second, the time interval for each token can be calculated.burst: The size of the token bucket.
tokens: Token in the bucket.
last: The last time I put the token in the bucket.
lastEvent: The time of the last speed limiter event (pass or limit are speed limiter events)
You can see it intimer/rateIn the current limiter implementation, there is no timer and queue to really put tokens into the bucket every once in a while, but only the remaining tokens in the bucket are represented by counting. Each time you spend tokens, the number of tokens in the bucket will be updated based on the time difference of the last update token number.。
Have a rough understandingtime/rateAfter the internal implementation of the current limiter, we will focus on the specific usage of this component in the following content:
Construct the current limiter
We can construct a current limiter object using the following method:
limiter := (10, 100);
Here are two parameters:
The first parameter is
r Limit, the current limiter is set to thelimitField, representing how many tokens can be generated into the Token bucket per second. Limit is actually an alias for float64.The second parameter is
b int, b represents the capacity of the Token bucket, which is the set current limiter LimiterburstField.
Then, for the above example, the token bucket size of the current limiter it constructs is 100, and the token is placed into the bucket at a rate of 10 tokens per second.
Except forr LimitIn addition to directly specifying the number of tokens generated per second, you can also use the Every method to specify the interval for placing tokens into the bucket, for example:
limit := (100 * ); limiter := (limit, 100);
The above means that a token is placed into the bucket every 100ms. Essentially, it is also to put 10 into the bucket in one second.
Use current limiter
Limited provides three types of methods for program consumption tokens, one token can be consumed at a time, or multiple tokens can be consumed at one time. Each method represents the different corresponding means when the token is insufficient. It can block and wait for the token in the bucket to replenish, or it can directly return to the token failed.
Wait/WaitN
func (lim *Limiter) Wait(ctx ) (err error) func (lim *Limiter) WaitN(ctx , n int) (err error)
Wait is actuallyWaitN(ctx,1)。
When using the Wait method to consume a token, if the token array in the bucket is insufficient (less than N), the Wait method will block for a period of time until the token meets the condition. If sufficient, return directly.
As you can see here, the Wait method has a context parameter. We can set the context's Deadline or Timeout to determine the maximum time for this Wait.
// Wait until you get the token in the bucketerr := (())
if err != nil {
("Error: ", err)
}
// Set a one-second waiting timeoutctx, _ := ((), * 1)
err := (ctx)
if err != nil {
("Error: ", err)
}Allow/AllowN
func (lim *Limiter) Allow() bool func (lim *Limiter) AllowN(now , n int) bool
Allow is actually rightAllowN((),1)A simplified function.
The AllowN method indicates that as of a certain moment, whether the number in the current bucket is at least n, and if it is satisfied, it returns true, and n tokens are consumed from the bucket at the same time. On the contrary, if the token in the bucket is not consumed, it returns false.
The corresponding online usage scenario is that if the request rate exceeds the limit, the overclocked request will be directly discarded.
if ((), 2) {
("event allowed")
} else {
("event not allowed")
}Reserve/ReserveN
func (lim *Limiter) Reserve() *Reservation func (lim *Limiter) ReserveN(now , n int) *Reservation
Reserve is equivalent toReserveN((), 1)。
The usage of ReserveN is relatively complicated. When the call is completed, no matter whether the token is sufficient, a return will be returned.*ReservationObject. You can call the object'sDelay()Method, the parameter type returned by this method is, reflects the time you need to wait, and you must wait until the waiting time before the next work can be carried out. If you don't want to wait, you can callCancel()method, this method will return the token.
To give a simple example, we can use the Reserve method like this.
r := ()
f !() {
// Not allowed to act! Did you remember to set to be > 0 ?
return
}
(())
Act() // Execute related logicDynamic adjustment of rate and bucket size
Limiter supports dynamic adjustment of the rate and bucket size after creation:
SetLimit(Limit) Changes the rate of putting into the token
SetBurst(int) Change the size of the Token bucket
With these two methods, the Token bucket size and rate can be dynamically changed according to existing environments and conditions and our needs.
Summarize
Today we have summarized the use of Golang's official current limiter, which is a current limiter implemented by token bucket calculation. inWait/WaitN，Allow/AllowNThese two sets of methods are often used. The former is to use the token when consuming the token. If the token in the bucket is insufficient, the program can wait for the new token in the bucket to be placed (it is best to set the waiting time). The latter is to choose to directly discard the request when the token in the bucket is insufficient.
In addition to the current limiter implementation provided by Golang, Uber's open source current limiteruber-go/ratelimitIt is also a good choice. Unlike Golang's official current limiter, Uber's current limiter is implemented through a leak bucket algorithm, but the traditional leak bucket algorithm has been improved. Interested students can experience it on their own.
This is the end of this article about the detailed explanation of the usage of Go official current limiter. For more related contents of Go current limiter, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!