Source code

If you would like to try it by yourself, you may always take a look at my source code example. In order to do that you need to clone my repository sample-spring-microservices-kubernetes. Then just follow my instructions The good idea is to read the article Microservices with Spring Cloud Kubernetes before you move on.

Step 1. Enable Spring Cloud Kubernetes Load Balancer

You need to include a single Spring Boot Starter to enable Spring Cloud Kubernetes Load Balancer. It is spring-cloud-starter-kubernetes-loadbalancer. Consequently, you also need to have a REST client on the classpath. Spring RestTemplate is automatically included with the Spring Web module. We will also use the OpenFeign client, so you should include the right starter.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-kubernetes-loadbalancer</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>

Step 2. Implement and use the REST client

Spring Cloud OpenFeign is a declarative REST client. Therefore, you need to create an interface with methods and Spring MVC annotations. It is important to set the right name inside @FeignClient annotation. This name needs to be the same as the name of the target Kubernetes Service. In the following code example, you see an implementation of the employee service client inside department-service.

@FeignClient(name = "employee")
public interface EmployeeClient {

   @GetMapping("/department/{departmentId}")
   List<Employee> findByDepartment(@PathVariable("departmentId") String departmentId);
	
}

OpenFeign client is enabled after annotating the main class with @EnableFeignClients. After that you can inject it to the @RestController class. Finally, you may use its method findByDepartment to find employees assigned to the particular department.

@RestController
public class DepartmentController {

    private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);

    DepartmentRepository repository;
    EmployeeClient employeeClient;

    public DepartmentController(DepartmentRepository repository, EmployeeClient employeeClient) {
        this.repository = repository;
        this.employeeClient = employeeClient;
    }

    @GetMapping("/{id}/with-employees")
    public Department findByIdWithEmployees(@PathVariable("id") String id) {
        LOGGER.info("Department findByIdWithEmployees: id={}", id);
        Optional<Department> optDepartment = repository.findById(id);
        if (optDepartment.isPresent()) {
            Department department = optDepartment.get();
            department.setEmployees(employeeClient.findByDepartment(department.getId()));
            return department;
        }
        return null;
    }

    @GetMapping("/organization/{organizationId}/with-employees")
    public List<Department> findByOrganizationWithEmployees(@PathVariable("organizationId") Long organizationId) {
        LOGGER.info("Department find: organizationId={}", organizationId);
        List<Department> departments = repository.findByOrganizationId(organizationId);
        departments.forEach(d -> d.setEmployees(employeeClient.findByDepartment(d.getId())));
        return departments;
    }

}

Opposite to the OpenFeign, the RestTemplate is a low-level HTTP client. We need to enable Spring Cloud load balancing for it. To do that just annotate RestTemplate bean with @LoadBalanced.

@Bean
@LoadBalanced
RestTemplate restTemplate() {
    return new RestTemplate();
}

Here’s the similar implementation of @RestController class, but this time with the RestTemplate.

public class DepartmentWithRestTemplateController {


    private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);

    DepartmentRepository repository;
    RestTemplate restTemplate;

    public DepartmentWithRestTemplateController(DepartmentRepository repository, RestTemplate restTemplate) {
        this.repository = repository;
        this.restTemplate = restTemplate;
    }

    @GetMapping("/{id}/with-employees")
    public Department findByIdWithEmployees(@PathVariable("id") String id) {
        LOGGER.info("Department findByIdWithEmployees: id={}", id);
        Optional<Department> optDepartment = repository.findById(id);
        if (optDepartment.isPresent()) {
            Department department = optDepartment.get();
            department.setEmployees(findEmployeesByDepartment(department.getId()));
            return department;
        }
        return null;
    }


    @GetMapping("/organization/{organizationId}/with-employees")
    public List<Department> findByOrganizationWithEmployees(@PathVariable("organizationId") Long organizationId) {
        LOGGER.info("Department find: organizationId={}", organizationId);
        List<Department> departments = repository.findByOrganizationId(organizationId);
        departments.forEach(d -> d.setEmployees(findEmployeesByDepartment(d.getId())));
        return departments;
    }

    private List<Employee> findEmployeesByDepartment(String departmentId) {
        Employee[] employees = restTemplate
                .getForObject("http://employee//department/{departmentId}", Employee[].class, departmentId);
        return Arrays.asList(employees);
    }

}

Step 3. Deploy Spring Cloud applications on Kubernetes

The project is ready to be used with Skaffold. Therefore, you don’t have to worry about the deployment process. You just need to run a single command skaffold dev --port-forward to deploy our applications on Kubernetes. But before deploying them on the cluster, we perform a short overview of the communication process. We have three microservices. Each of them is running in two instances. They are using Mongo as a backend store. There is also the gateway application. It is built on top of Spring Cloud Gateway. It provides a single API endpoint to all downstream services. Of course, all our applications are using Spring Cloud Load Balancer for traffic management. To clarify, you can take a look at the picture below.

Here’s the Kubernetes deployment manifest for the employee-service. It sets two running pods in replicas parameter, and references ConfigMap and Secret to inject Mongo credentials.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: employee
  labels:
    app: employee
spec:
  replicas: 2
  selector:
    matchLabels:
      app: employee
  template:
    metadata:
      labels:
        app: employee
    spec:
      containers:
      - name: employee
        image: piomin/employee
        ports:
        - containerPort: 8080
        env:
        - name: MONGO_DATABASE
          valueFrom:
            configMapKeyRef:
              name: mongodb
              key: database-name
        - name: MONGO_USERNAME
          valueFrom:
            secretKeyRef:
              name: mongodb
              key: database-user
        - name: MONGO_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mongodb
              key: database-password

The most important element in the communication between applications is Kubernetes Service object. The name of service must be the same as the hostname used by the RestTemplate or OpenFeign client. In our case that name is employee.

apiVersion: v1
kind: Service
metadata:
  name: employee
  labels:
    app: employee
    spring-boot: "true"
spec:
  ports:
  - port: 8080
    protocol: TCP
  selector:
    app: employee
  type: ClusterIP

Let’s verify the status after deploying all the applications. Here’s the list of running pods.

Here’s the list of Kubernetes Endpoints.

Step 4. Communication in the POD mode

By default, Spring Cloud Kubernetes Load Balancer uses the POD mode. In this mode, it gets the list of Kubernetes endpoints to detect the IP address of all the application pods. In that case, the only thing you need to do is to disable the Ribbon load balancer. Spring Cloud Kubernetes is still using it as a default client-side load balancer. To disable it you need to set the following property.

spring:
  cloud:
    loadbalancer:
      ribbon:
        enabled: false

After adding a test data we may send some requests to the endpoint GET /{id}/with-employees. It finds a department by the id. Then it communicates with the employee-service endpoint GET /department/{departmentId} to search all the employees assigned to the current department. The department-service is exposed on port 8081, since I enabled option port-forward on Skaffold.

$ curl http://localhost:8081/5f5896b3cb8caf7e6f6b9e1c/with-employees
{"id":"5f5896b3cb8caf7e6f6b9e1c","organizationId":"1","name":"test1","employees":[{"id":"5f5896e26092716e54f60a9a","name":"test1","age":30,"position":"d
eveloper"},{"id":"5f5896f29625e62c7d373906","name":"test2","age":40,"position":"tester"},{"id":"5f5897266092716e54f60a9b","name":"test3","age":45,"posit
ion":"developer"}]}

Let’s take a look on the logs from employee-service. I repeated the request visible above several times. The requests are load balanced between two available instances of employee-service. Skaffold prints the id of every pod in the logs, so you can verify that everything works fine.

Step 5. Communication across multiple namespaces

By default, Spring Cloud Kubernetes allows load balancing within the same namespace. You may enable discovery across multiple namespaces. To do that you need to use the property following property.

spring:
  cloud:
    kubernetes:
      discovery:
        all-namespaces: true

After setting that, we will deploy the employee-service application in the different namespace than department-service. To do that you need to set parameter -n on the skaffold command.

$ skaffold dev -n test-a

Here’s the current list of running pods.

You just need to send the same request as before curl http://localhost:8081/5f5896b3cb8caf7e6f6b9e1c/with-employees. Of course, the Kubernetes service name must be unique across all the namespaces.

Step 6. Load balancer on Spring Cloud Gateway

Spring Cloud Gateway uses the same load balancing mechanism as other Spring Cloud applications. To enable it on Kubernetes we need to include dependency spring-cloud-starter-kubernetes-loadbalancer. We should also enable Spring Cloud DiscoveryClient and disable Ribbon. Here’s the configuration of the gateway-service.

spring:
  application:
    name: gateway
  cloud:
    loadbalancer:
      ribbon:
        enabled: false
    gateway:
      discovery:
        locator:
          enabled: true
      routes:
        - id: employee
          uri: lb://employee
          predicates:
            - Path=/employee/**
          filters:
            - RewritePath=/employee/(?.*), /$\{path}
        - id: department
          uri: lb://department
          predicates:
            - Path=/department/**
          filters:
            - RewritePath=/department/(?.*), /$\{path}
        - id: organization
          uri: lb://organization
          predicates:
            - Path=/organization/**
          filters:
            - RewritePath=/organization/(?.*), /$\{path}

A gateway acts as an entry point to our system. It performs routing and load balancing to all the downstream services. Therefore, we can call any of our applications using already defined routes. Since gateway-service is available on port 8080, I can call any of the endpoints using the following requests.

We can expose API documentation of all the microservices on the gateway. To do that we may use the SpringFox project. First, we need to include SpringFox Starter to Maven dependencies.

<dependency>
   <groupId>io.springfox</groupId>
   <artifactId>springfox-boot-starter</artifactId>
   <version>3.0.0</version>
</dependency>

Swagger documentation is usually generated for a single application. Because we want to expose documentation of all our microservices we need to override the Swagger resource provider. The custom implementation is visible below. It uses a discovery mechanism to the names of running services.

@Configuration
public class GatewayApi {

   @Autowired
   RouteDefinitionLocator locator;

   @Primary
   @Bean
   public SwaggerResourcesProvider swaggerResourcesProvider() {
      return () -> {
         List<SwaggerResource> resources = new ArrayList<>();
         Flux<RouteDefinition> definitions = locator.getRouteDefinitions();
         definitions
               .filter(routeDefinition -> !routeDefinition.getId().startsWith("ReactiveCompositeDiscoveryClient_"))
               .subscribe(routeDefinition -> resources.add(createResource(routeDefinition.getId(), "2.0")));
         try {
            Thread.sleep(1000);
         } catch (InterruptedException e) {
            e.printStackTrace();
         }
         return resources;
      };
   }

   private SwaggerResource createResource(String location, String version) {
      SwaggerResource swaggerResource = new SwaggerResource();
      swaggerResource.setName(location);
      swaggerResource.setLocation("/" + location + "/v2/api-docs");
      swaggerResource.setSwaggerVersion(version);
      return swaggerResource;
   }

}

Here’s the Swagger UI available at http://localhost:8080/swagger-ui/index.html on my local machine.

Step 7. Enabling circuit breaker

We can use a circuit breaker component with Spring Cloud Load Balancer. The default Spring Cloud implementation is based on Resilience4j. In order to enable it for your application, you need to include the following dependency.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-circuitbreaker-resilience4j</artifactId>
</dependency>

The next step is to provide a configuration of a circuit breaker.

@Bean
Customizer<Resilience4JCircuitBreakerFactory> defaultCustomizer() {
   return factory -> factory.configureDefault(id ->
         new Resilience4JConfigBuilder(id)
               .timeLimiterConfig(TimeLimiterConfig.custom()
                     .timeoutDuration(Duration.ofMillis(1000))
                     .build())
               .circuitBreakerConfig(CircuitBreakerConfig.custom()
                     .slidingWindowSize(10)
                     .failureRateThreshold(66.6F)
                     .slowCallRateThreshold(66.6F)
                     .build())
               .build()
   );
}

We need to inject Resilience4JCircuitBreakerFactory to the Spring controller class. Then we are creating the circuit breaker instance using an injected factory. Finally, the client calling method is running inside the circuit breaker run method.

@RestController
public class DepartmentController {

    private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);

    DepartmentRepository repository;
    EmployeeClient employeeClient;
    Resilience4JCircuitBreakerFactory circuitBreakerFactory;

    public DepartmentController(DepartmentRepository repository, EmployeeClient employeeClient,
                                Resilience4JCircuitBreakerFactory circuitBreakerFactory) {
        this.repository = repository;
        this.employeeClient = employeeClient;
        this.circuitBreakerFactory = circuitBreakerFactory;
    }

    @GetMapping("/{id}/with-employees-and-delay")
    public Department findByIdWithEmployeesAndDelay(@PathVariable("id") String id) {
        LOGGER.info("Department findByIdWithEmployees: id={}", id);
        Optional<Department> optDepartment = repository.findById(id);
        if (optDepartment.isPresent()) {
            Department department = optDepartment.get();
            Resilience4JCircuitBreaker circuitBreaker = circuitBreakerFactory.create("delayed-circuit");
            List<Employee> employees = circuitBreaker.run(() ->
                    employeeClient.findByDepartmentWithDelay(department.getId()));
            department.setEmployees(employees);
            return department;
        }
        return null;
    }
}

Conclusion

Load balancing is one of the key patterns in a microservices architecture. Spring Cloud Load Balancer is replacing the Ribbon client. By default, load balancing in Kubernetes is based on Services. Therefore, you need to use additional tools for more advanced routing mechanisms. Spring Cloud Kubernetes comes with some interesting features. One of them is the ability to load balance across multiple namespaces. You can also use the additional Spring Cloud components like a circuit breaker. In comparison with tools like Istio, it is still not much. Is it a chance for improvement? We will see. Nevertheless, Spring Cloud Kubernetes is currently one of the most popular Spring Cloud projects. It may be a good choice if you are migrating your Spring Cloud microservices architecture to Kubernetes. Support for the load balancer may be an important step during such a migration.

The post Spring Cloud Kubernetes Load Balancer Guide appeared first on Piotr's TechBlog.

If you are interested in more detailed explanation of Spring Cloud components used for inter-service communication you should refer to the third part of my online course Microservices With Spring Boot And Spring Cloud: Part 3 – Inter-service communication.

Example

You can find a source code snippets related to this article in my GitHub repository https://github.com/piomin/course-spring-microservices.git. That repository is also used for my online course, so I decided to extend it with the new examples. All the required changes were performed in directory inter-communication/inter-caller-service inside that repository. The code is written in Kotlin.
There are three applications, which are a part of our sample system: discovery-server (Spring Cloud Netflix Eureka), inter-callme-service (Spring Boot application that expose REST API), and finally inter-caller-service (Spring Boot application that calls endpoints exposed by inter-callme-service).

How to start with Spring Cloud Load Balancer

To enable Spring Cloud Load Balancer for our application we first need to include the following starter to Maven dependencies. That module may be also included together with some other Spring Cloud starters.

<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-loadbalancer</artifactId>
</dependency>

Because Ribbon is still used as a default client-side load balancer for REST-based communication between applications we need to disable it in application properties. Here’s a fragment of application.yml file.

spring:
  application:
    name: inter-caller-service
  cloud:
    loadbalancer:
      ribbon:
        enabled: false

For discovery integration we also need to include spring-cloud-starter-netflix-eureka-client. To use RestTemplate with a client-side load balancer we should define the bean visible below and annotate it with @LoadBalanced. As you on the code below I’m also setting interceptor on RestTemplate, but more about it in the next section.

@Bean
@LoadBalanced
fun template(): RestTemplate = RestTemplateBuilder()
		.interceptors(responseTimeInterceptor())
		.build()

Adapt traffic to average response time

Spring Cloud Load Balancer provides a simple round robin rule for load balancing between multiple instances of a single service. Our goal here is to implement a rule, which measures each application response time and gives a weight according to that time. The longer the response time, the less weight it will get. The rule should randomly pick an instance where the possibility is determined by its weight. To record response time of each call we need to set an already mentioned interceptor that implements ClientHttpRequestInterceptor. Interceptor is executed on every request (1). Since the implementation is very typical, one line requires explanation (2). I’m getting the address of the target application from a thread scoped variable existing in Slf4J MDC. Of course I could also implement a simple thread scoped context based on ThreadLocal, but MDC is used here just for simplification.

class ResponseTimeInterceptor(private val responseTimeHistory: ResponseTimeHistory) : ClientHttpRequestInterceptor {

    private val logger: Logger = LoggerFactory.getLogger(ResponseTimeInterceptor::class.java)

    override fun intercept(request: HttpRequest, array: ByteArray,
                           execution: ClientHttpRequestExecution): ClientHttpResponse {
        val startTime: Long = System.currentTimeMillis()
        val response: ClientHttpResponse = execution.execute(request, array) // 1
        val endTime: Long = System.currentTimeMillis()
        val responseTime: Long = endTime - startTime
        logger.info("Response time: instance->{}, time->{}", MDC.get("address"), responseTime)
        responseTimeHistory.addNewMeasure(MDC.get("address"), responseTime) // 2
        return response
    }
}

Of course, counting an average response time is just a part of our job. The most important is the implementation of our custom load balancer, which is visible below. It should implement interface ReactorServiceInstanceLoadBalancer. It need to inject ServiceInstanceListSupplier bean to fetch a list of available instances of a given service in overridden method choose. While choosing the right instance we are analyzing the average response time for each instance saved in ResponseTimeHistory by ResponseTimeInterceptor. In the beginning our load balancer acts like a simple round robin.

class WeightedTimeResponseLoadBalancer(
        private val serviceInstanceListSupplierProvider: ObjectProvider<ServiceInstanceListSupplier>,
        private val serviceId: String,
        private val responseTimeHistory: ResponseTimeHistory) : ReactorServiceInstanceLoadBalancer {

    private val logger: Logger = LoggerFactory.getLogger(WeightedTimeResponseLoadBalancer::class.java)
    private val position: AtomicInteger = AtomicInteger()

    override fun choose(request: Request<*>?): Mono<Response<ServiceInstance>> {
        val supplier: ServiceInstanceListSupplier = serviceInstanceListSupplierProvider
                .getIfAvailable { NoopServiceInstanceListSupplier() }
        return supplier.get().next()
                .map { serviceInstances: List<ServiceInstance> -> getInstanceResponse(serviceInstances) }
    }

    private fun getInstanceResponse(instances: List<ServiceInstance>): Response<ServiceInstance> {
        return if (instances.isEmpty()) {
            EmptyResponse()
        } else {
            val address: String? = responseTimeHistory.getAddress(instances.size)
            val pos: Int = position.incrementAndGet()
            var instance: ServiceInstance = instances[pos % instances.size]
            if (address != null) {
                val found: ServiceInstance? = instances.find { "${it.host}:${it.port}" == address }
                if (found != null)
                    instance = found
            }
            logger.info("Current instance: [address->{}:{}, stats->{}ms]", instance.host, instance.port,
                    responseTimeHistory.stats["${instance.host}:${instance.port}"])
            MDC.put("address", "${instance.host}:${instance.port}")
            DefaultResponse(instance)
        }
    }
}

Here’s the implementation of ResponseTimeHistory bean, which is responsible for storing measures and selecting the instance of service based on computed weight.

class ResponseTimeHistory(private val history: MutableMap<String, Queue<Long>> = mutableMapOf(),
                          val stats: MutableMap<String, Long> = mutableMapOf()) {

    private val logger: Logger = LoggerFactory.getLogger(ResponseTimeHistory::class.java)

    fun addNewMeasure(address: String, measure: Long) {
        var list: Queue<Long>? = history[address]
        if (list == null) {
            history[address] = LinkedList<Long>()
            list = history[address]
        }
        logger.info("Adding new measure for->{}, measure->{}", address, measure)
        if (measure == 0L)
            list!!.add(1L)
        else list!!.add(measure)
        if (list.size > 9)
            list.remove()
        stats[address] = countAvg(address)
        logger.info("Counting avg for->{}, stat->{}", address, stats[address])
    }

    private fun countAvg(address: String): Long {
        val list: Queue<Long>? = history[address]
        return list?.sum()?.div(list.size) ?: 0
    }

    fun getAddress(numberOfInstances: Int): String? {
        if (stats.size < numberOfInstances)
            return null
        var sum: Long = 0
        stats.forEach { sum += it.value }
        var r: Long = Random.nextLong(100)
        var current: Long = 0
        stats.forEach {
            val weight: Long = (sum - it.value)*100 / sum
            logger.info("Weight for->{}, value->{}, random->{}", it.key, weight, r)
            current += weight
            if (r <= current)
                return it.key
        }
        return null
    }

}

Customizing Spring Cloud Load Balancer

The implementation of our mechanism for weighted response time rule is ready, so the last step is to apply it to Spring Cloud Load Balancer. To do that we need to create a dedicated configuration class with ReactorLoadBalancer bean declaration as shown below.

class CustomCallmeClientLoadBalancerConfiguration(private val responseTimeHistory: ResponseTimeHistory) {

    @Bean
    fun loadBalancer(environment: Environment, loadBalancerClientFactory: LoadBalancerClientFactory):
            ReactorLoadBalancer<ServiceInstance> {
        val name: String? = environment.getProperty("loadbalancer.client.name")
        return WeightedTimeResponseLoadBalancer(
                loadBalancerClientFactory.getLazyProvider(name, ServiceInstanceListSupplier::class.java),
                name!!, responseTimeHistory)
    }
}

The custom configuration may be passed to a load balancer using annotation @LoadBalancerClient. The name of client should be the same as registered in discovery. This part of code is currently commented out in the GitHub repository, so if you would like to enable it for testing just uncomment it.

@SpringBootApplication
@LoadBalancerClient(value = "inter-callme-service", configuration = [CustomCallmeClientLoadBalancerConfiguration::class])
class InterCallerServiceApplication {

    @Bean
    fun responseTimeHistory(): ResponseTimeHistory = ResponseTimeHistory()

    @Bean
    fun responseTimeInterceptor(): ResponseTimeInterceptor = ResponseTimeInterceptor(responseTimeHistory())

    // THE REST OF IMPLEMENTATION...
}

Customizing instance list supplier

Currently Spring Cloud Load Balancer does not support a static list of instances set in configuration properties (unlike Netflix Ribbon). We can easily add such a mechanism. The static list of instances for every service will be defined as shown below.

spring:
  application:
    name: inter-caller-service
  cloud:
    loadbalancer:
      ribbon:
        enabled: false
      instances:
        - name: inter-callme-service
          servers: localhost:59600, localhost:59800

As the first step, we should define a class that implements interface ServiceInstanceListSupplier and overrides two methods: getServiceId() and get(). The following implementation of ServiceInstanceListSupplier takes the list of service addresses from application properties through @ConfigurationProperties.

class StaticServiceInstanceListSupplier(private val properties: LoadBalancerConfigurationProperties,
                                        private val environment: Environment) : ServiceInstanceListSupplier {

    override fun getServiceId(): String = environment.getProperty("loadbalancer.client.name")!!

    override fun get(): Flux<MutableList<ServiceInstance>> {
        val serviceConfig: LoadBalancerConfigurationProperties.ServiceConfig? =
                properties.instances.find { it.name == serviceId }
        val list: MutableList<ServiceInstance> =
                serviceConfig!!.servers.split(",", ignoreCase = false, limit = 0)
                        .map { StaticServiceInstance(serviceId, it) }.toMutableList()
        return Flux.just(list)
    }

}

Here’s the implementation of configuration class with properties.

@Configuration
@ConfigurationProperties("spring.cloud.loadbalancer")
class LoadBalancerConfigurationProperties {

    val instances: MutableList<ServiceConfig> = mutableListOf()

    class ServiceConfig {
        var name: String = ""
        var servers: String = ""
    }

}

The same as for the previous sample we should also register our implementation of ServiceInstanceListSupplier as a bean inside custom configuration class.

class CustomCallmeClientLoadBalancerConfiguration) {

    @Bean
    fun discoveryClientServiceInstanceListSupplier(discoveryClient: ReactiveDiscoveryClient, environment: Environment,
        zoneConfig: LoadBalancerZoneConfig, context: ApplicationContext,
        properties: LoadBalancerConfigurationProperties): ServiceInstanceListSupplier {
        val delegate = StaticServiceInstanceListSupplier(properties, environment)
        val cacheManagerProvider = context.getBeanProvider(LoadBalancerCacheManager::class.java)
        return if (cacheManagerProvider.ifAvailable != null) {
            CachingServiceInstanceListSupplier(delegate, cacheManagerProvider.ifAvailable)
        } else delegate
    }
}

Testing Spring Cloud Load Balancer

To test the solution implemented for the purpose of this article you should:

1. Run the instance of discovery server (only if StaticServiceInstanceListSupplier is disabled)
2. Run two instances of inter-callme-service (for one selected instance activate random delay using VM parameter -Dspring.profiles.active=delay)
3. Run instance of inter-caller-service, which is available on port 8080
4. Send some test requests to inter-caller-service using command, for example curl -X POST http://localhost:8080/caller/random-send/12345

Our test scenario is visualized in the following picture.

Conclusion

Currently, Spring Cloud Load Balancer does not offer such many interesting features for inter-service communication as the Netflix Ribbon client. Of course, it is still being actively developed by the Spring Team. The good news is that we can easily customize Spring Cloud Load Balancer to add some custom features. In this article I demonstrated how to provide more advanced load balancing algorithms or create custom instances of list suppliers.

The post A Deep Dive Into Spring Cloud Load Balancer appeared first on Piotr's TechBlog.