SoFunction
Updated on 2025-04-06

Some common ways to implement current limiting in Gateway

How to implement current limiting in Gateway

Current limiting in API gateways (such as Spring Cloud Gateway, Kong, Nginx, etc.) is to control the frequency of service requests, thereby avoiding system overload and ensuring stability and availability. Current limiting can be achieved through a variety of strategies, common methods include limiting based on the number of requests, time windows, IP addresses, etc. Below isSpring Cloud GatewaySome common ways to implement current limiting:

1. Current limit based on the number of requests

This method prevents excessive requests from accessing the service by limiting the number of requests per unit time. For example, limiting can only process up to 100 requests per second.

  • Spring Cloud Gateway implementation:Spring Cloud Gateway provides built-in current limiting function that can be used throughRequestRateLimiterFilters to implement current limiting based on the number of requests. Sample configuration:
spring:
  cloud:
    gateway:
      routes:
        - id: rate_limiter_route
          uri: 
          predicates:
            - Path=/api/**
          filters:
            - name: RequestRateLimiter
              args:
                : 10  # Number of requests per second                : 20  # Burst capacity，That is, the maximum number of requests allowed in a short period of time
  • replenishRate: The number of requests allowed per second.
  • burstCapacity: Burst capacity, indicating the maximum number of requests that can be processed in a short period of time. Requests exceeding this number will be discarded or rejected.

2. Current limit based on token bucket algorithm (Token Bucket)

The token bucket algorithm is an algorithm that smooths request traffic. It generates tokens at a fixed rate and stores them in a bucket. When the request arrives, it needs to obtain the token to be processed. If the token bucket is empty, the request is rejected.

Spring Cloud Gateway implementation:useRequestRateLimiterImplement current limiting based on token buckets.

Sample configuration:

spring:
  cloud:
    gateway:
      routes:
        - id: token_bucket_route
          uri: 
          predicates:
            - Path=/api/**
          filters:
            - name: RequestRateLimiter
              args:
                : 10  # Number of tokens generated per second                : 20  # Capacity of token bucket

The configuration here is similar to the request count limit above, but the token bucket algorithm is implemented behind it.

3. Current limit based on IP

The request frequency of each IP can be restricted according to the client IP address to avoid excessive access to the service by a certain client. Redis is usually used to implement IP-based current limiting.

Sample configuration (using Redis):

spring:
  cloud:
    gateway:
      routes:
        - id: ip_rate_limiter_route
          uri: 
          predicates:
            - Path=/api/**
          filters:
            - name: RequestRateLimiter
              args:
                : 10
                : 20
                : 1
                : "#{@ipKeyResolver}"

In this example,keyResolverSpecifies how to limit the current based on the client IP address, and you can customize an IP resolveripKeyResolver, so that the number of requests per IP will be limited.

4. Current limit based on time window

Time window current limiting is to limit the number of requests according to a fixed time window (such as per minute, hour, etc.). The maximum number of requests per unit time is controlled through the time window.

Sample configuration:

spring:
  cloud:
    gateway:
      routes:
        - id: time_window_route
          uri: 
          predicates:
            - Path=/api/**
          filters:
            - name: RequestRateLimiter
              args:
                : 5  # Number of requests per second                : 10  # Burst capacity                : 1  # Number of tokens consumed per request

This way limits the request to a certain time window. If the number of requests in the window exceeds the limit, additional requests will be rejected.

5. Customize the current limiting strategy

Spring Cloud Gateway also allows customizationKeyResolver、RateLimiterSo for achieving more flexible current limiting strategies. For example, different current limiting strategies are made based on user ID, API path, etc.

CustomizeKeyResolverExample:

@Bean
public KeyResolver ipKeyResolver() {
    return exchange -> (().getRemoteAddress().getHostString());
}

This example uses the client IP address to limit the current.

6. Other gateway current limiting tools

In addition to Spring Cloud Gateway, other gateways such asKong、NginxAlso provides rich current limiting functions:

  • KongUse the plug-in mechanism to achieve current limiting, and support current limiting based on the number of requests, IP, API Key, etc.
  • NginxCan be passedlimit_reqandlimit_connmodule to perform current limit control.

Summarize

existSpring Cloud GatewayAmong them, common current limiting methods include:

  • Rate Limiting based on the number of requests
  • Token Bucket based on token bucket algorithm
  • IP-based current limit
  • Current limit based on time window

Can be passedRequestRateLimiterFilters implement these strategies, combined withRedisetc. to share and persist current limiting state. When choosing a current limiting strategy, it needs to be determined based on business needs, traffic mode, and system load capacity.

Analyze Gateway current limiting code

 
@Component
public class RateLimitConfig {
    @Bean
    KeyResolver userKeyResolver() {
	return exchange -> (().getQueryParams().getFirst("token"));
    }
    @Bean
    KeyResolver ipKeyResolver() {
	return exchange -> (().getRemoteAddress().getHostName());
    }
    @Bean
    KeyResolver apiKeyResolver() {
	return exchange -> (().getPath().value());
    }
    @Primary
    @Bean
    KeyResolver appIpKeyResolver() {
	return exchange -> {
	    Route route = (Route) ().get(ServerWebExchangeUtils.GATEWAY_ROUTE_ATTR);
	    return (().getRemoteAddress().getHostName().concat(().getHost()));
	};
    }
}

This code implements the current limit configuration of Spring Cloud Gateway, and the specific implementation is differentKeyResolverto provide a unique identifier for the request, which is used to distinguish different request sources or policies.KeyResolveris the core component of current limiting, which determines the "key" of current limiting (i.e., the unique identification of each request), thus ensuring that different requests are current limiting according to different rules.

The following is a detailed analysis of the code:

1.KeyResolverThe role of

KeyResolveris part of the current limiting mechanism in Spring Cloud Gateway, used to generate the only "key" related to the request. When limiting the current, the number of requests for each limiting unit (such as per user, per IP, per API, etc.) is calculated based on these "keys". The current limiting rules will control the flow based on these keys.

2.userKeyResolver

@Bean KeyResolver userKeyResolver() { return exchange -> (().getQueryParams().getFirst("token")); }
  • Function:ShouldKeyResolverRequest-based query parameterstokenGenerates a unique current limiting key.
  • Application scenarios: When the client usestokenWhen the parameter (probably an API Key or user identity identifier), it can be based on thistokenLimit traffic to each user. Each user (or eachtoken) requests will be counted separately.
  • Specific implementation: Get from the requested query parameterstokenThe first value of the parameter. If this parameter is not available, returnnullOr a null value will cause the current limiting function to not take effect.

3.ipKeyResolver

@Bean KeyResolver ipKeyResolver() { return exchange -> (().getRemoteAddress().getHostName()); }
  • Function:ShouldKeyResolverGenerate current limit keys based on the client's IP address.
  • Application scenarios: Use this when you need to limit the request frequency of each IP addressKeyResolver. For example, the number of requests per IP can be limited within a certain time window, thereby avoiding excessive requests from a single IP to put pressure on the system.
  • Specific implementation:pass().getRemoteAddress()Get the remote IP address of the client and usegetHostName()Get the hostname of the IP. In theory,getHostName()An attempt will be made to resolve the IP address to the hostname, but in some cases it may directly return the IP address.

4.apiKeyResolver

@Bean KeyResolver apiKeyResolver() { return exchange -> (().getPath().value()); }
  • Function:ShouldKeyResolverGenerates unique current limiting keys based on the requested API path.
  • Application scenarios: Use this when you need to limit the request frequency of a certain API pathKeyResolver. For example, limit/api/v1/loginor/api/v1/registerThe frequency of access to the path can prevent a specific API from being over-accessed.
  • Specific implementation:pass().getPath().value()Get the path to the request. This path is usually the path part of the URI, such as/api/v1/resource。

5.appIpKeyResolver

@Primary
@Bean KeyResolver appIpKeyResolver()
{ 
    return exchange -> 
    { 
        Route route = (Route) ().get(ServerWebExchangeUtils.GATEWAY_ROUTE_ATTR); return (().getRemoteAddress().getHostName().concat(().getHost())); 
    };
    }
  • Function:ShouldKeyResolverCombines the client's IP address and routed URI to generate the current limit key.
  • Application scenarios: Use this when you want to limit the flow based on the client IP and the specific services you accessKeyResolver. For example, the frequency of access to a particular API can be restricted for each IP.
  • Specific implementation：
    • pass().getRemoteAddress().getHostName()Get the client's IP address.
    • pass().get(ServerWebExchangeUtils.GATEWAY_ROUTE_ATTR)Get the routing information for the current request.
    • ().getHost()Get the host name (or domain name) of the requested service and splice it with the client IP as the only key to limit the current.
    • useconcat()Method combines the client IP and the target service host name into a string to ensure the independence of current limiting.

6.@Primaryannotation

@Primary @Bean KeyResolver appIpKeyResolver()
{ 
    ... 
}
  • effect: There are multiple same type@Beanhour,@PrimaryThe annotation indicates that the use of this is preferredBean. If there is any other placeKeyResolverSpring will use the injection requirements by defaultappIpKeyResolver()The instance returned by the method.
  • Scene: When there are multipleKeyResolverWhen, if no explicitly specified, which one should be usedKeyResolver, Spring will automatically select the mark as@PrimaryThe one. It's marked hereappIpKeyResolverFor the defaultKeyResolver。

Summarize

This code defines multipleKeyResolver, and provides optional implementations for different current limiting strategies:

  • userKeyResolver: Request-basedtokenCurrent limit, suitable for current limit based on user or identity.
  • ipKeyResolver: Current limit based on client IP address, suitable for limiting the number of requests per IP.
  • apiKeyResolver: Current limit based on request path, suitable for restricting access to a specific API.
  • appIpKeyResolver: The URI combination of URI based on client IP address and route is suitable for finer granular current limiting requirements, ensuring that requests from the same IP to different APIs can be counted independently.

@PrimaryAnnotation ensuresappIpKeyResolveris the default current limit key generator, and it will be used first unless otherwise specified.

TheseKeyResolverWill be withRequestRateLimiterUse equal current limiting functions in conjunction with each other, thereby achieving refined current limiting control at the API gateway layer.

Why is there no specified current limit amount, the amount of access allowed per second

In this code, although different definitions are definedKeyResolverto generate a current limit identifier (e.g.token、IPorAPI path), but there is really no actual current limit rules (such as the number of visits allowed per second). The specific control of current limit is through Spring Cloud GatewayRequestRateLimiterImplemented by a filter, which is used to limit the requests according to specified rules.

To achieve the allowed access per second, you need to configureRequestRateLimiterFilter and specify current limit rules for it (such as maximum visits per second and maximum burst traffic).

Complete current limit configuration example

First, you need to create aRequestRateLimiterFilter and configure the current limiting policy in the filter. Usually, you can configure it by:

import ;
import ;
import ;
import ;
import ;
@Configuration
public class GatewayConfig {
    @Bean
    public RequestRateLimiter filter(KeyResolver keyResolver) {
        // Configure current limit: up to 10 requests per second, and up to 20 burst traffic        return new RequestRateLimiter()
                .setRateLimiter(new MyRateLimiter(10, 20))  // 10 requests per second, up to 20 burst traffic                .setKeyResolver(keyResolver); // Set the KeyResolver for current limit    }
    // Customize a simple RateLimiter implementation class    public static class MyRateLimiter implements RateLimiter {
        private final int replenishRate;
        private final int burstCapacity;
        public MyRateLimiter(int replenishRate, int burstCapacity) {
             = replenishRate;
             = burstCapacity;
        }
        @Override
        public boolean isAllowed(String key) {
            // Implement your own current limit logic here, you can use the token bucket algorithm, leaky bucket algorithm, etc.            return true;  // Just an example, it actually needs to implement current limiting logic        }
        // Configure the maximum number of requests per second (10 times per second)        public int getReplenishRate() {
            return replenishRate;
        }
        // Configure burst capacity        public int getBurstCapacity() {
            return burstCapacity;
        }
    }
}

How to convert current limit rules withKeyResolverCooperate?

In this code,RequestRateLimiterThe current limiting rules (such as 10 requests per second and burst capacity is 20) will be compared with the previously definedKeyResolverCooperate, limit specifictoken, IP or API path access frequency.

  • KeyResolverSelect: You can use the one you defined earlieruserKeyResolver、ipKeyResolverorapiKeyResolveras the basis for current limit.
  • Current limit logic:passMyRateLimiterClass implements current limiting logic (such as token buckets, leaky buckets and other algorithms). Each current-limited "key" (e.g.tokenorIP) will limit the number of visits according to configured rules (such as 10 times per second, 20 burst traffic).

Sample configuration: PresstokenCurrent limit

Suppose you want to be based ontokenTo perform current limiting, you can configure it like this:

@Bean
public RequestRateLimiter filter(KeyResolver userKeyResolver) {
    return new RequestRateLimiter()
            .setRateLimiter(new MyRateLimiter(10, 20))  // 10 requests per second, up to 20 burst traffic            .setKeyResolver(userKeyResolver);  // Use the token-based KeyResolver defined previously}

Current limit configuration items

  • replenishRate: The number of requests allowed per second. For example,10Indicates that up to 10 requests can be processed per second.
  • burstCapacity: Burst capacity, indicating the maximum number of requests that can be allowed in a short period of time. For example,20Indicates that an instant access is allowed up to 20 times, but it usually returns smoothly to the normal request rate.
  • KeyResolver: You have defined multipleKeyResolver, they determine the granularity of current limiting. You can use user(token), IP address, path, etc. to limit the current.

Summarize

This code itself only defines the current limitIdentifier(passKeyResolver). True current limit control needs to be passedRequestRateLimiterFilters are implemented and specify the specific current limitingLimit amount(such as how many requests are allowed per second). You can usereplenishRateandburstCapacityConfigure the allowed visits and burst traffic per second.

This is the end of this article about some common ways to implement Gateway's current limit. For more related Gateway's current limit content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!