SoFunction
Updated on 2025-04-13

Interpretation of SpringCloud Load Balancing spring-cloud-starter-loadbalancer

Brief description

spring-cloud-starter-loadbalancer is a component in Spring Cloud that provides client load balancing capabilities. In early Spring Cloud, Netflix Ribbon was widely used as a client load balancer, but over time and Netflix Ribbon entered maintenance mode, the Spring Cloud community began to move towards a more flexible and easier to maintain alternative.

spring-cloud-starter-loadbalancer is built on Spring 5's WebClient and uses Reactor (the core library of Spring 5's reactive programming model) to implement asynchronous non-blocking load balancing requests. It combines very well with Spring Cloud's service discovery and configuration and can be easily used with service discovery components such as Eureka, Consul, Nacos, etc.

When adding spring-cloud-starter-loadbalancer to a Spring Boot application, you can use the loadBalancer method to create a WebClient instance with load balancing. This WebClient instance will automatically obtain a list of service instances from service discovery and use built-in load balancing algorithms (such as polling, randomization, etc.) to select a service instance to send the request.

For example, if you are using Eureka as a service discovery and want to send a GET request to a service named "my-service", you can do this:

import ;  
import ;  
import ;  
import ;  
import ;  
  
@Service  
public class MyServiceClient {  
  
    @Autowired  
    private  webClientBuilder;  
  
    @Bean  
    @LoadBalanced  
    public  loadBalancedWebClientBuilder() {  
        return ();  
    }  
  
    public String getSomethingFromMyService() {  
        // Note that we directly use "my-service" as the URI, rather than the specific service instance address        return ()  
                .get()  
                .uri("http://my-service/some-endpoint")  
                .retrieve()  
                .bodyToMono()  
                .block(); // Note: The block() method blocks the current thread and is usually only used in non-reactive contexts    }  
}

Main features

  • Based on WebClient: Tightly integrated with Spring 5's WebClient, providing reactive HTTP client functionality.
  • Service Discovery Integration: Integrated with Spring Cloud's service discovery components (such as Eureka, Consul, Nacos, etc.) to automatically obtain a list of service instances.
  • Built-in load balancing algorithm: Provides built-in load balancing algorithms, such as polling (Round Robin), Random, etc.
  • Reactive programming: Supports reactive programming models, allowing non-blocking I/O operations and asynchronous processing.
  • flexibility: Provides more flexibility and scalability compared to Ribbon, making it easier to customize load balancing behavior.
  • Integrate with Spring Cloud Gateway: Tightly integrated with Spring Cloud Gateway, providing it with load balancing.

use

  1. Add dependencies: Add spring-cloud-starter-loadbalancer dependency in Maven or Gradle projects.
		<!-- SpringCloud Loadbalancer -->
        <dependency>
            <groupId></groupId>
            <artifactId>spring-cloud-starter-loadbalancer</artifactId>
        </dependency>
  1. Configure WebClient: Use the @LoadBalanced annotation to mark a bean to configure it to support load balancing.
  2. Send a request: When sending a request through WebClient, use the service name (rather than the specific service instance address) as the host part of the URI.
  3. Custom load balancing algorithm: If needed, you can customize the load balancing algorithm and apply it to the WebClient by configuring or programmatically.

Notes:

  • Blocking call: Although WebClient is reactive, in some cases (such as when interacting with synchronous code), the block() method may be required to block the current thread and wait for a response. However, block() should be avoided in the reactive context.
  • Configuration: The behavior of the load balancer can be customized through configuration, including selecting the load balancing algorithm, setting the timeout time, etc.
  • Service Discovery: Make sure that the application has correctly configured the service discovery components (such as Eureka, Consul, etc.) so that spring-cloud-starter-loadbalancer can get a list of service instances.
  • Version compatibility: Note the version compatibility of spring-cloud-starter-loadbalancer with other Spring Cloud components to ensure that they work together.

Load balancing algorithm

1. Polling Load Balancing Policy (Round Robin)

  • describe: This is the default load balancing policy, which sends requests to each service instance in the service instance list in sequence.
  • Features：
    • Simple and easy to implement.
    • Each service instance receives approximately the same number of requests (in ideal cases).
    • The current load status or performance of the service instance is not considered.

2. Random load balancing strategy (Random)

  • describe: This policy will randomly select a service instance to send the request.
  • Features：
    • In multiple requests, each service instance may be selected.
    • It increases randomness compared to the polling policy, but the number of requests received by each service instance may be unequal.
    • Also, the current load status or performance of the service instance is not considered.

3. Customize load balancing strategies

  • describe: In addition to the built-in load balancing policy, spring-cloud-starter-loadbalancer also supports custom load balancing policies.
  • Features：
    • Developers can implement their own load balancing algorithms according to actual needs.
    • Decisions can be made by considering the current load status, performance, and geographical location of the service instance.
    • Provides greater flexibility and customization.

4. Nacos Weight Load Balancer

  • describe: When used with the Nacos service discovery component, you can use the weight load balancer provided by Nacos.
  • Features：
    • A service instance can configure a weight value, and the higher the weight value, the more requests it receives.
    • The weight value can be dynamically adjusted based on factors such as the performance of the service instance, resource usage, etc.
    • It provides finer granular control and can be flexibly configured according to actual needs.

5. Custom algorithm

Define your own load balancing algorithm by implementing a custom ReactorLoadBalancer.

  1. Define a custom load balancer: Need to implement the ReactorLoadBalancer interface or extend existing implementations (such as RoundRobinLoadBalancer).
  2. Implement the choice method: This is the core of the load balancing algorithm, which receives a request (usually a Request object) and a list of service instances (ServiceInstanceListSupplier), and returns a Mono, representing the selected service instance.
  3. Configure a custom load balancer: The custom load balancer needs to be configured as the default load balancer for Spring Cloud. This is usually done by registering a ReactorLoadBalancer bean.

Example:

import ;  
import ;  
import ;  
import ;  
import ;  
import ;  
import ;  
import ;  
  
import ;  
  
@Configuration  
public class CustomLoadBalancerConfig {  
  
    @Bean  
    public ReactiveLoadBalancer&lt;ServiceInstance&gt; customLoadBalancer(  
            ReactiveLoadBalancerFactory&lt;ServiceInstance&gt; factory,  
            ObjectProvider&lt;List&lt;ServiceInstance&gt;&gt; serviceInstances) {  
  
        return new ReactiveLoadBalancer&lt;ServiceInstance&gt;() {  
  
            @Override  
            public Mono&lt;Response&lt;ServiceInstance&gt;&gt; choose(Request request) {  
                // Here is a custom load balancing algorithm implementation                // For example, we can simply return the first instance in the service instance list                return (())  
                        .flatMapMany(List::stream)  
                        .firstElement() // Or you can implement your own choice logic                        .map(Response::just);  
            }  
  
            // Other necessary methods (such as recordStats, filter, etc.) can be implemented as needed        };  
    }  
  
    // If you want to configure a custom load balancer for a specific service,    // You can distinguish and return different ReactiveLoadBalancer instances through ServiceId    // For example, public ReactiveLoadBalancer<ServiceInstance> customLoadBalancerForServiceX(...) {...}}

spring-cloud-starter-loadbalancer provides a variety of load balancing algorithms, including polling, randomization, and custom policies.

These algorithms can be selected and configured according to actual needs to meet different load balancing needs. At the same time, integration with the Nacos service discovery component also provides the functionality of a weight load balancer, further increasing the flexibility and customizability of load balancing.

Developers can choose suitable load balancing algorithms based on their business scenarios and needs, and configure and optimize them appropriately to achieve more efficient and reliable microservice calls.

Reactive programming

Starting with the Spring Cloud Greenwich version, Spring Cloud introduced support for Project Reactor and transformed the load balancer from a traditional blocking (based on Ribbon) to a reactive one (based on spring-cloud-starter-loadbalancer).

Reactive programming is an asynchronous, non-blocking programming paradigm that uses streams and propagation of change to process data. In reactive programming, data is not passed through traditional calling and return mechanisms, but is passed between components through an asynchronous data stream.

In spring-cloud-starter-loadbalancer, reactive programming is mainly reflected in the following aspects:

  1. Non-blocking calls: Unlike traditional Ribbon-based blocking load balancers, spring-cloud-starter-loadbalancer uses a reactive programming model to perform non-blocking load balancing requests. This means that it does not block the thread waiting for the response, but processes the request and response asynchronously.
  2. Responsive Type: The load balancer API uses reactive types, such as Mono and Flux, which are reactive types provided by Project Reactor. Mono is used to represent an asynchronous sequence of 0 or 1 element, while Flux is used to represent an asynchronous sequence of 0 to N elements.
  3. Backpressure: Reactive programming supports backpressure mechanism, i.e. consumers can control the speed at which producers generate data. This is very useful when handling large numbers of concurrent requests, and can avoid consumers being unable to process due to producers generating data too quickly.
  4. Error handling: Reactive programming provides rich error handling mechanisms, such as onErrorResume, retry and other operators, which can handle exceptions gracefully when errors occur.
  5. Combination and conversion:Mono and Flux provide rich operators for combining and converting asynchronous data streams. This allows for flexible processing of load balancing requests and responses to meet a variety of complex business needs.

Simple example:

import ;  
import ;  
import ;  
import ;  
import ;  
  
import ;  
  
@RestController  
public class MyController {  
  
    private final LoadBalancerClient loadBalancerClient;  
  
    public MyController(LoadBalancerClient loadBalancerClient) {  
         = loadBalancerClient;  
    }  
  
    @GetMapping("/call-service")  
    public Mono&lt;String&gt; callService() {  
        // Get a service instance        ServiceInstance serviceInstance = ("my-service").block();  
  
        // Use WebClient to initiate a reactive request        WebClient webClient = ()  
                .baseUrl(().toString())  
                .build();  
  
        return ()  
                .uri("/some-endpoint")  
                .retrieve()  
                .bodyToMono();  
    }  
}

Note: The block() method is used in the example above to synchronize the fetching of service instances, which may not be best practice in practical applications. In general, the asynchronous nature of reactive programming should be maintained throughout the call chain. However, to simplify the example, the block() method is used here. In practical applications, both the acquisition of the service instance and the initiation of the request should be converted into reactive operations.

Integrate with OpenFeign

In Spring Cloud applications, spring-cloud-starter-loadbalancer is usually used with spring-cloud-starter-openfeign or other HTTP clients such as WebClient to support load balancing calls to service-discovered clients.

When using OpenFeign declarative HTTP client, spring-cloud-starter-loadbalancer is automatically integrated to provide load balancing. Simply include the corresponding dependencies in the or file and specify the service name using the @FeignClient annotation on the Feign client interface.

  • Maven dependencies
&lt;dependencies&gt;  
   &lt;!-- ... Other dependencies ... --&gt;  
   &lt;dependency&gt;  
       &lt;groupId&gt;&lt;/groupId&gt;  
       &lt;artifactId&gt;spring-cloud-starter-openfeign&lt;/artifactId&gt;  
   &lt;/dependency&gt;  
   &lt;dependency&gt;  
       &lt;groupId&gt;&lt;/groupId&gt;  
       &lt;artifactId&gt;spring-cloud-starter-loadbalancer&lt;/artifactId&gt;  
   &lt;/dependency&gt;  
   &lt;!-- ... Other dependencies ... --&gt;  
&lt;/dependencies&gt;
  • Feign Client
@FeignClient(name = "my-service")  
public interface MyServiceClient {  
   // Define HTTP methods   @GetMapping("/some-endpoint")  
   Mono&lt;String&gt; getSomething();  
}

WebClient Integration

Using WebClient as an HTTP client, you can use spring-cloud-starter-loadbalancer to implement load balancing calls between services. A bean needs to be created and use LoadBalancerExchangeFilterFunction to automatically handle service discovery and load balancing.

  • Configure WebClient Bean
@Bean  
public  webClientBuilder(LoadBalancerClient loadBalancerClient) {  
    return ()  
            .baseUrl("lb://my-service") // Enable load balancing using the 'lb://' prefix            .filter(new LoadBalancerExchangeFilterFunction(loadBalancerClient));  
}
  • Use WebClient to initiate a request
@Autowired  
private  webClientBuilder;  
  
public Mono<String> callService() {  
    WebClient webClient = ();  
    return ()  
            .uri("/some-endpoint")  
            .retrieve()  
            .bodyToMono();  
}

Notes:

  • Make sure Spring Cloud version supports spring-cloud-starter-loadbalancer.
  • From using Ribbon to spring-cloud-starter-loadbalancer, be aware of configuration differences and API changes.
  • When using WebClient, make sure that the correct URL prefix (lb://) is used to enable load balancing.
  • When customizing the load balancer, ensure that the implementation is thread-safe and capable of handling concurrent requests.

Summarize

The above is personal experience. I hope you can give you a reference and I hope you can support me more.