Source code

If you would like to try it by yourself, you may always take a look at my source code example. In order to do that you need to clone my repository sample-spring-microservices-kubernetes. Then just follow my instructions The good idea is to read the article Microservices with Spring Cloud Kubernetes before you move on.

Step 1. Enable Spring Cloud Kubernetes Load Balancer

You need to include a single Spring Boot Starter to enable Spring Cloud Kubernetes Load Balancer. It is spring-cloud-starter-kubernetes-loadbalancer. Consequently, you also need to have a REST client on the classpath. Spring RestTemplate is automatically included with the Spring Web module. We will also use the OpenFeign client, so you should include the right starter.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-kubernetes-loadbalancer</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>

Step 2. Implement and use the REST client

Spring Cloud OpenFeign is a declarative REST client. Therefore, you need to create an interface with methods and Spring MVC annotations. It is important to set the right name inside @FeignClient annotation. This name needs to be the same as the name of the target Kubernetes Service. In the following code example, you see an implementation of the employee service client inside department-service.

@FeignClient(name = "employee")
public interface EmployeeClient {

   @GetMapping("/department/{departmentId}")
   List<Employee> findByDepartment(@PathVariable("departmentId") String departmentId);
	
}

OpenFeign client is enabled after annotating the main class with @EnableFeignClients. After that you can inject it to the @RestController class. Finally, you may use its method findByDepartment to find employees assigned to the particular department.

@RestController
public class DepartmentController {

    private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);

    DepartmentRepository repository;
    EmployeeClient employeeClient;

    public DepartmentController(DepartmentRepository repository, EmployeeClient employeeClient) {
        this.repository = repository;
        this.employeeClient = employeeClient;
    }

    @GetMapping("/{id}/with-employees")
    public Department findByIdWithEmployees(@PathVariable("id") String id) {
        LOGGER.info("Department findByIdWithEmployees: id={}", id);
        Optional<Department> optDepartment = repository.findById(id);
        if (optDepartment.isPresent()) {
            Department department = optDepartment.get();
            department.setEmployees(employeeClient.findByDepartment(department.getId()));
            return department;
        }
        return null;
    }

    @GetMapping("/organization/{organizationId}/with-employees")
    public List<Department> findByOrganizationWithEmployees(@PathVariable("organizationId") Long organizationId) {
        LOGGER.info("Department find: organizationId={}", organizationId);
        List<Department> departments = repository.findByOrganizationId(organizationId);
        departments.forEach(d -> d.setEmployees(employeeClient.findByDepartment(d.getId())));
        return departments;
    }

}

Opposite to the OpenFeign, the RestTemplate is a low-level HTTP client. We need to enable Spring Cloud load balancing for it. To do that just annotate RestTemplate bean with @LoadBalanced.

@Bean
@LoadBalanced
RestTemplate restTemplate() {
    return new RestTemplate();
}

Here’s the similar implementation of @RestController class, but this time with the RestTemplate.

public class DepartmentWithRestTemplateController {


    private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);

    DepartmentRepository repository;
    RestTemplate restTemplate;

    public DepartmentWithRestTemplateController(DepartmentRepository repository, RestTemplate restTemplate) {
        this.repository = repository;
        this.restTemplate = restTemplate;
    }

    @GetMapping("/{id}/with-employees")
    public Department findByIdWithEmployees(@PathVariable("id") String id) {
        LOGGER.info("Department findByIdWithEmployees: id={}", id);
        Optional<Department> optDepartment = repository.findById(id);
        if (optDepartment.isPresent()) {
            Department department = optDepartment.get();
            department.setEmployees(findEmployeesByDepartment(department.getId()));
            return department;
        }
        return null;
    }


    @GetMapping("/organization/{organizationId}/with-employees")
    public List<Department> findByOrganizationWithEmployees(@PathVariable("organizationId") Long organizationId) {
        LOGGER.info("Department find: organizationId={}", organizationId);
        List<Department> departments = repository.findByOrganizationId(organizationId);
        departments.forEach(d -> d.setEmployees(findEmployeesByDepartment(d.getId())));
        return departments;
    }

    private List<Employee> findEmployeesByDepartment(String departmentId) {
        Employee[] employees = restTemplate
                .getForObject("http://employee//department/{departmentId}", Employee[].class, departmentId);
        return Arrays.asList(employees);
    }

}

Step 3. Deploy Spring Cloud applications on Kubernetes

The project is ready to be used with Skaffold. Therefore, you don’t have to worry about the deployment process. You just need to run a single command skaffold dev --port-forward to deploy our applications on Kubernetes. But before deploying them on the cluster, we perform a short overview of the communication process. We have three microservices. Each of them is running in two instances. They are using Mongo as a backend store. There is also the gateway application. It is built on top of Spring Cloud Gateway. It provides a single API endpoint to all downstream services. Of course, all our applications are using Spring Cloud Load Balancer for traffic management. To clarify, you can take a look at the picture below.

Here’s the Kubernetes deployment manifest for the employee-service. It sets two running pods in replicas parameter, and references ConfigMap and Secret to inject Mongo credentials.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: employee
  labels:
    app: employee
spec:
  replicas: 2
  selector:
    matchLabels:
      app: employee
  template:
    metadata:
      labels:
        app: employee
    spec:
      containers:
      - name: employee
        image: piomin/employee
        ports:
        - containerPort: 8080
        env:
        - name: MONGO_DATABASE
          valueFrom:
            configMapKeyRef:
              name: mongodb
              key: database-name
        - name: MONGO_USERNAME
          valueFrom:
            secretKeyRef:
              name: mongodb
              key: database-user
        - name: MONGO_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mongodb
              key: database-password

The most important element in the communication between applications is Kubernetes Service object. The name of service must be the same as the hostname used by the RestTemplate or OpenFeign client. In our case that name is employee.

apiVersion: v1
kind: Service
metadata:
  name: employee
  labels:
    app: employee
    spring-boot: "true"
spec:
  ports:
  - port: 8080
    protocol: TCP
  selector:
    app: employee
  type: ClusterIP

Let’s verify the status after deploying all the applications. Here’s the list of running pods.

Here’s the list of Kubernetes Endpoints.

Step 4. Communication in the POD mode

By default, Spring Cloud Kubernetes Load Balancer uses the POD mode. In this mode, it gets the list of Kubernetes endpoints to detect the IP address of all the application pods. In that case, the only thing you need to do is to disable the Ribbon load balancer. Spring Cloud Kubernetes is still using it as a default client-side load balancer. To disable it you need to set the following property.

spring:
  cloud:
    loadbalancer:
      ribbon:
        enabled: false

After adding a test data we may send some requests to the endpoint GET /{id}/with-employees. It finds a department by the id. Then it communicates with the employee-service endpoint GET /department/{departmentId} to search all the employees assigned to the current department. The department-service is exposed on port 8081, since I enabled option port-forward on Skaffold.

$ curl http://localhost:8081/5f5896b3cb8caf7e6f6b9e1c/with-employees
{"id":"5f5896b3cb8caf7e6f6b9e1c","organizationId":"1","name":"test1","employees":[{"id":"5f5896e26092716e54f60a9a","name":"test1","age":30,"position":"d
eveloper"},{"id":"5f5896f29625e62c7d373906","name":"test2","age":40,"position":"tester"},{"id":"5f5897266092716e54f60a9b","name":"test3","age":45,"posit
ion":"developer"}]}

Let’s take a look on the logs from employee-service. I repeated the request visible above several times. The requests are load balanced between two available instances of employee-service. Skaffold prints the id of every pod in the logs, so you can verify that everything works fine.

Step 5. Communication across multiple namespaces

By default, Spring Cloud Kubernetes allows load balancing within the same namespace. You may enable discovery across multiple namespaces. To do that you need to use the property following property.

spring:
  cloud:
    kubernetes:
      discovery:
        all-namespaces: true

After setting that, we will deploy the employee-service application in the different namespace than department-service. To do that you need to set parameter -n on the skaffold command.

$ skaffold dev -n test-a

Here’s the current list of running pods.

You just need to send the same request as before curl http://localhost:8081/5f5896b3cb8caf7e6f6b9e1c/with-employees. Of course, the Kubernetes service name must be unique across all the namespaces.

Step 6. Load balancer on Spring Cloud Gateway

Spring Cloud Gateway uses the same load balancing mechanism as other Spring Cloud applications. To enable it on Kubernetes we need to include dependency spring-cloud-starter-kubernetes-loadbalancer. We should also enable Spring Cloud DiscoveryClient and disable Ribbon. Here’s the configuration of the gateway-service.

spring:
  application:
    name: gateway
  cloud:
    loadbalancer:
      ribbon:
        enabled: false
    gateway:
      discovery:
        locator:
          enabled: true
      routes:
        - id: employee
          uri: lb://employee
          predicates:
            - Path=/employee/**
          filters:
            - RewritePath=/employee/(?.*), /$\{path}
        - id: department
          uri: lb://department
          predicates:
            - Path=/department/**
          filters:
            - RewritePath=/department/(?.*), /$\{path}
        - id: organization
          uri: lb://organization
          predicates:
            - Path=/organization/**
          filters:
            - RewritePath=/organization/(?.*), /$\{path}

A gateway acts as an entry point to our system. It performs routing and load balancing to all the downstream services. Therefore, we can call any of our applications using already defined routes. Since gateway-service is available on port 8080, I can call any of the endpoints using the following requests.

We can expose API documentation of all the microservices on the gateway. To do that we may use the SpringFox project. First, we need to include SpringFox Starter to Maven dependencies.

<dependency>
   <groupId>io.springfox</groupId>
   <artifactId>springfox-boot-starter</artifactId>
   <version>3.0.0</version>
</dependency>

Swagger documentation is usually generated for a single application. Because we want to expose documentation of all our microservices we need to override the Swagger resource provider. The custom implementation is visible below. It uses a discovery mechanism to the names of running services.

@Configuration
public class GatewayApi {

   @Autowired
   RouteDefinitionLocator locator;

   @Primary
   @Bean
   public SwaggerResourcesProvider swaggerResourcesProvider() {
      return () -> {
         List<SwaggerResource> resources = new ArrayList<>();
         Flux<RouteDefinition> definitions = locator.getRouteDefinitions();
         definitions
               .filter(routeDefinition -> !routeDefinition.getId().startsWith("ReactiveCompositeDiscoveryClient_"))
               .subscribe(routeDefinition -> resources.add(createResource(routeDefinition.getId(), "2.0")));
         try {
            Thread.sleep(1000);
         } catch (InterruptedException e) {
            e.printStackTrace();
         }
         return resources;
      };
   }

   private SwaggerResource createResource(String location, String version) {
      SwaggerResource swaggerResource = new SwaggerResource();
      swaggerResource.setName(location);
      swaggerResource.setLocation("/" + location + "/v2/api-docs");
      swaggerResource.setSwaggerVersion(version);
      return swaggerResource;
   }

}

Here’s the Swagger UI available at http://localhost:8080/swagger-ui/index.html on my local machine.

Step 7. Enabling circuit breaker

We can use a circuit breaker component with Spring Cloud Load Balancer. The default Spring Cloud implementation is based on Resilience4j. In order to enable it for your application, you need to include the following dependency.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-circuitbreaker-resilience4j</artifactId>
</dependency>

The next step is to provide a configuration of a circuit breaker.

@Bean
Customizer<Resilience4JCircuitBreakerFactory> defaultCustomizer() {
   return factory -> factory.configureDefault(id ->
         new Resilience4JConfigBuilder(id)
               .timeLimiterConfig(TimeLimiterConfig.custom()
                     .timeoutDuration(Duration.ofMillis(1000))
                     .build())
               .circuitBreakerConfig(CircuitBreakerConfig.custom()
                     .slidingWindowSize(10)
                     .failureRateThreshold(66.6F)
                     .slowCallRateThreshold(66.6F)
                     .build())
               .build()
   );
}

We need to inject Resilience4JCircuitBreakerFactory to the Spring controller class. Then we are creating the circuit breaker instance using an injected factory. Finally, the client calling method is running inside the circuit breaker run method.

@RestController
public class DepartmentController {

    private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);

    DepartmentRepository repository;
    EmployeeClient employeeClient;
    Resilience4JCircuitBreakerFactory circuitBreakerFactory;

    public DepartmentController(DepartmentRepository repository, EmployeeClient employeeClient,
                                Resilience4JCircuitBreakerFactory circuitBreakerFactory) {
        this.repository = repository;
        this.employeeClient = employeeClient;
        this.circuitBreakerFactory = circuitBreakerFactory;
    }

    @GetMapping("/{id}/with-employees-and-delay")
    public Department findByIdWithEmployeesAndDelay(@PathVariable("id") String id) {
        LOGGER.info("Department findByIdWithEmployees: id={}", id);
        Optional<Department> optDepartment = repository.findById(id);
        if (optDepartment.isPresent()) {
            Department department = optDepartment.get();
            Resilience4JCircuitBreaker circuitBreaker = circuitBreakerFactory.create("delayed-circuit");
            List<Employee> employees = circuitBreaker.run(() ->
                    employeeClient.findByDepartmentWithDelay(department.getId()));
            department.setEmployees(employees);
            return department;
        }
        return null;
    }
}

Conclusion

Load balancing is one of the key patterns in a microservices architecture. Spring Cloud Load Balancer is replacing the Ribbon client. By default, load balancing in Kubernetes is based on Services. Therefore, you need to use additional tools for more advanced routing mechanisms. Spring Cloud Kubernetes comes with some interesting features. One of them is the ability to load balance across multiple namespaces. You can also use the additional Spring Cloud components like a circuit breaker. In comparison with tools like Istio, it is still not much. Is it a chance for improvement? We will see. Nevertheless, Spring Cloud Kubernetes is currently one of the most popular Spring Cloud projects. It may be a good choice if you are migrating your Spring Cloud microservices architecture to Kubernetes. Support for the load balancer may be an important step during such a migration.

The post Spring Cloud Kubernetes Load Balancer Guide appeared first on Piotr's TechBlog.

If you are interested in more detailed explanation of Spring Cloud components used for inter-service communication you should refer to the third part of my online course Microservices With Spring Boot And Spring Cloud: Part 3 – Inter-service communication.

Example

You can find a source code snippets related to this article in my GitHub repository https://github.com/piomin/course-spring-microservices.git. That repository is also used for my online course, so I decided to extend it with the new examples. All the required changes were performed in directory inter-communication/inter-caller-service inside that repository. The code is written in Kotlin.
There are three applications, which are a part of our sample system: discovery-server (Spring Cloud Netflix Eureka), inter-callme-service (Spring Boot application that expose REST API), and finally inter-caller-service (Spring Boot application that calls endpoints exposed by inter-callme-service).

How to start with Spring Cloud Load Balancer

To enable Spring Cloud Load Balancer for our application we first need to include the following starter to Maven dependencies. That module may be also included together with some other Spring Cloud starters.

<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-loadbalancer</artifactId>
</dependency>

Because Ribbon is still used as a default client-side load balancer for REST-based communication between applications we need to disable it in application properties. Here’s a fragment of application.yml file.

spring:
  application:
    name: inter-caller-service
  cloud:
    loadbalancer:
      ribbon:
        enabled: false

For discovery integration we also need to include spring-cloud-starter-netflix-eureka-client. To use RestTemplate with a client-side load balancer we should define the bean visible below and annotate it with @LoadBalanced. As you on the code below I’m also setting interceptor on RestTemplate, but more about it in the next section.

@Bean
@LoadBalanced
fun template(): RestTemplate = RestTemplateBuilder()
		.interceptors(responseTimeInterceptor())
		.build()

Adapt traffic to average response time

Spring Cloud Load Balancer provides a simple round robin rule for load balancing between multiple instances of a single service. Our goal here is to implement a rule, which measures each application response time and gives a weight according to that time. The longer the response time, the less weight it will get. The rule should randomly pick an instance where the possibility is determined by its weight. To record response time of each call we need to set an already mentioned interceptor that implements ClientHttpRequestInterceptor. Interceptor is executed on every request (1). Since the implementation is very typical, one line requires explanation (2). I’m getting the address of the target application from a thread scoped variable existing in Slf4J MDC. Of course I could also implement a simple thread scoped context based on ThreadLocal, but MDC is used here just for simplification.

class ResponseTimeInterceptor(private val responseTimeHistory: ResponseTimeHistory) : ClientHttpRequestInterceptor {

    private val logger: Logger = LoggerFactory.getLogger(ResponseTimeInterceptor::class.java)

    override fun intercept(request: HttpRequest, array: ByteArray,
                           execution: ClientHttpRequestExecution): ClientHttpResponse {
        val startTime: Long = System.currentTimeMillis()
        val response: ClientHttpResponse = execution.execute(request, array) // 1
        val endTime: Long = System.currentTimeMillis()
        val responseTime: Long = endTime - startTime
        logger.info("Response time: instance->{}, time->{}", MDC.get("address"), responseTime)
        responseTimeHistory.addNewMeasure(MDC.get("address"), responseTime) // 2
        return response
    }
}

Of course, counting an average response time is just a part of our job. The most important is the implementation of our custom load balancer, which is visible below. It should implement interface ReactorServiceInstanceLoadBalancer. It need to inject ServiceInstanceListSupplier bean to fetch a list of available instances of a given service in overridden method choose. While choosing the right instance we are analyzing the average response time for each instance saved in ResponseTimeHistory by ResponseTimeInterceptor. In the beginning our load balancer acts like a simple round robin.

class WeightedTimeResponseLoadBalancer(
        private val serviceInstanceListSupplierProvider: ObjectProvider<ServiceInstanceListSupplier>,
        private val serviceId: String,
        private val responseTimeHistory: ResponseTimeHistory) : ReactorServiceInstanceLoadBalancer {

    private val logger: Logger = LoggerFactory.getLogger(WeightedTimeResponseLoadBalancer::class.java)
    private val position: AtomicInteger = AtomicInteger()

    override fun choose(request: Request<*>?): Mono<Response<ServiceInstance>> {
        val supplier: ServiceInstanceListSupplier = serviceInstanceListSupplierProvider
                .getIfAvailable { NoopServiceInstanceListSupplier() }
        return supplier.get().next()
                .map { serviceInstances: List<ServiceInstance> -> getInstanceResponse(serviceInstances) }
    }

    private fun getInstanceResponse(instances: List<ServiceInstance>): Response<ServiceInstance> {
        return if (instances.isEmpty()) {
            EmptyResponse()
        } else {
            val address: String? = responseTimeHistory.getAddress(instances.size)
            val pos: Int = position.incrementAndGet()
            var instance: ServiceInstance = instances[pos % instances.size]
            if (address != null) {
                val found: ServiceInstance? = instances.find { "${it.host}:${it.port}" == address }
                if (found != null)
                    instance = found
            }
            logger.info("Current instance: [address->{}:{}, stats->{}ms]", instance.host, instance.port,
                    responseTimeHistory.stats["${instance.host}:${instance.port}"])
            MDC.put("address", "${instance.host}:${instance.port}")
            DefaultResponse(instance)
        }
    }
}

Here’s the implementation of ResponseTimeHistory bean, which is responsible for storing measures and selecting the instance of service based on computed weight.

class ResponseTimeHistory(private val history: MutableMap<String, Queue<Long>> = mutableMapOf(),
                          val stats: MutableMap<String, Long> = mutableMapOf()) {

    private val logger: Logger = LoggerFactory.getLogger(ResponseTimeHistory::class.java)

    fun addNewMeasure(address: String, measure: Long) {
        var list: Queue<Long>? = history[address]
        if (list == null) {
            history[address] = LinkedList<Long>()
            list = history[address]
        }
        logger.info("Adding new measure for->{}, measure->{}", address, measure)
        if (measure == 0L)
            list!!.add(1L)
        else list!!.add(measure)
        if (list.size > 9)
            list.remove()
        stats[address] = countAvg(address)
        logger.info("Counting avg for->{}, stat->{}", address, stats[address])
    }

    private fun countAvg(address: String): Long {
        val list: Queue<Long>? = history[address]
        return list?.sum()?.div(list.size) ?: 0
    }

    fun getAddress(numberOfInstances: Int): String? {
        if (stats.size < numberOfInstances)
            return null
        var sum: Long = 0
        stats.forEach { sum += it.value }
        var r: Long = Random.nextLong(100)
        var current: Long = 0
        stats.forEach {
            val weight: Long = (sum - it.value)*100 / sum
            logger.info("Weight for->{}, value->{}, random->{}", it.key, weight, r)
            current += weight
            if (r <= current)
                return it.key
        }
        return null
    }

}

Customizing Spring Cloud Load Balancer

The implementation of our mechanism for weighted response time rule is ready, so the last step is to apply it to Spring Cloud Load Balancer. To do that we need to create a dedicated configuration class with ReactorLoadBalancer bean declaration as shown below.

class CustomCallmeClientLoadBalancerConfiguration(private val responseTimeHistory: ResponseTimeHistory) {

    @Bean
    fun loadBalancer(environment: Environment, loadBalancerClientFactory: LoadBalancerClientFactory):
            ReactorLoadBalancer<ServiceInstance> {
        val name: String? = environment.getProperty("loadbalancer.client.name")
        return WeightedTimeResponseLoadBalancer(
                loadBalancerClientFactory.getLazyProvider(name, ServiceInstanceListSupplier::class.java),
                name!!, responseTimeHistory)
    }
}

The custom configuration may be passed to a load balancer using annotation @LoadBalancerClient. The name of client should be the same as registered in discovery. This part of code is currently commented out in the GitHub repository, so if you would like to enable it for testing just uncomment it.

@SpringBootApplication
@LoadBalancerClient(value = "inter-callme-service", configuration = [CustomCallmeClientLoadBalancerConfiguration::class])
class InterCallerServiceApplication {

    @Bean
    fun responseTimeHistory(): ResponseTimeHistory = ResponseTimeHistory()

    @Bean
    fun responseTimeInterceptor(): ResponseTimeInterceptor = ResponseTimeInterceptor(responseTimeHistory())

    // THE REST OF IMPLEMENTATION...
}

Customizing instance list supplier

Currently Spring Cloud Load Balancer does not support a static list of instances set in configuration properties (unlike Netflix Ribbon). We can easily add such a mechanism. The static list of instances for every service will be defined as shown below.

spring:
  application:
    name: inter-caller-service
  cloud:
    loadbalancer:
      ribbon:
        enabled: false
      instances:
        - name: inter-callme-service
          servers: localhost:59600, localhost:59800

As the first step, we should define a class that implements interface ServiceInstanceListSupplier and overrides two methods: getServiceId() and get(). The following implementation of ServiceInstanceListSupplier takes the list of service addresses from application properties through @ConfigurationProperties.

class StaticServiceInstanceListSupplier(private val properties: LoadBalancerConfigurationProperties,
                                        private val environment: Environment) : ServiceInstanceListSupplier {

    override fun getServiceId(): String = environment.getProperty("loadbalancer.client.name")!!

    override fun get(): Flux<MutableList<ServiceInstance>> {
        val serviceConfig: LoadBalancerConfigurationProperties.ServiceConfig? =
                properties.instances.find { it.name == serviceId }
        val list: MutableList<ServiceInstance> =
                serviceConfig!!.servers.split(",", ignoreCase = false, limit = 0)
                        .map { StaticServiceInstance(serviceId, it) }.toMutableList()
        return Flux.just(list)
    }

}

Here’s the implementation of configuration class with properties.

@Configuration
@ConfigurationProperties("spring.cloud.loadbalancer")
class LoadBalancerConfigurationProperties {

    val instances: MutableList<ServiceConfig> = mutableListOf()

    class ServiceConfig {
        var name: String = ""
        var servers: String = ""
    }

}

The same as for the previous sample we should also register our implementation of ServiceInstanceListSupplier as a bean inside custom configuration class.

class CustomCallmeClientLoadBalancerConfiguration) {

    @Bean
    fun discoveryClientServiceInstanceListSupplier(discoveryClient: ReactiveDiscoveryClient, environment: Environment,
        zoneConfig: LoadBalancerZoneConfig, context: ApplicationContext,
        properties: LoadBalancerConfigurationProperties): ServiceInstanceListSupplier {
        val delegate = StaticServiceInstanceListSupplier(properties, environment)
        val cacheManagerProvider = context.getBeanProvider(LoadBalancerCacheManager::class.java)
        return if (cacheManagerProvider.ifAvailable != null) {
            CachingServiceInstanceListSupplier(delegate, cacheManagerProvider.ifAvailable)
        } else delegate
    }
}

Testing Spring Cloud Load Balancer

To test the solution implemented for the purpose of this article you should:

1. Run the instance of discovery server (only if StaticServiceInstanceListSupplier is disabled)
2. Run two instances of inter-callme-service (for one selected instance activate random delay using VM parameter -Dspring.profiles.active=delay)
3. Run instance of inter-caller-service, which is available on port 8080
4. Send some test requests to inter-caller-service using command, for example curl -X POST http://localhost:8080/caller/random-send/12345

Our test scenario is visualized in the following picture.

Conclusion

Currently, Spring Cloud Load Balancer does not offer such many interesting features for inter-service communication as the Netflix Ribbon client. Of course, it is still being actively developed by the Spring Team. The good news is that we can easily customize Spring Cloud Load Balancer to add some custom features. In this article I demonstrated how to provide more advanced load balancing algorithms or create custom instances of list suppliers.

The post A Deep Dive Into Spring Cloud Load Balancer appeared first on Piotr's TechBlog.

Spring Cloud video course

You may also find a video guide, where I’m showing all the features described here with more details. It is available on my YouTube channel and consists of four parts:

Part 1 – Introduction to Spring Boot (~35 minutes)
Part 2 – Distributed Configuration and Service Discovery (~36 minutes)
Part 3 – Inter-service communication (~34 minutes)
Part 4 – API Gateway (~22 minutes)

Architecture

The diagram visible below illustrates the architecture of our sample system. Here we have the characteristic elements for microservices like API gateway, discovery server, and configuration server. In the next sections of this article, I’ll show how to use Spring Cloud components that provide an implementation of those patterns. Currently, the main component for adding the API gateway to your system is Spring Cloud Gateway.
Spring Cloud provides integrations to several solutions that may be used as a discovery server: Netflix Eureka, HashiCorp Consul, Alibaba Nacos, or Apache ZooKeeper. The most common choice is between the first two of them. While Spring Cloud Netflix Eureka is dedicated just for discovery, Spring Cloud Consul may realize both discovery feature basing on Consul Services, and distributed configuration feature basing on Consul Key/Value engine.

In turn, Spring Cloud Config is responsible just for providing a mechanism for configuration management. However, it may also be integrated with third-party tools like Vault from HashiCorp.
We will figure out how to integrate our applications with discovery and configuration servers on the example of two simple Spring Boot applications callme-service and caller-service. The application caller-service is also calling endpoints exposed by the callme-service. We will enable such mechanisms on the caller-service like client-side load balancer with new Spring Cloud Load Balancer, and circuit breaker with new Spring Cloud Circuit Breaker built on top of Resilience4J.

Service discovery

Switching between discovery servers on the client-side is very easy with Spring Cloud due to the DiscoveryClient abstraction. This switch comes down to the replacement of a single dependency in Maven pom.xml. So if you are using Eureka you should add the following starter in your microservice.

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>

On the other hand, if you are using Consul you should add the following starter in your microservice.

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-consul-discovery</artifactId>
</dependency>

The situation is a little bit more complicated if you are defining some non-default configuration settings for a discovery client. In that case, you need to use properties specific just for Eureka, or just for Consul. For example, if you are running more than one instance of a single application on the same host with dynamic HTTP server port feature enabled (option server.port=0), you have to set a unique id of every instance. Here’s the property used for Eureka’s client.

eureka:
  instance:
    instanceId: ${spring.cloud.client.hostname}:${spring.application.name}:${random.value}

For the Consul client the same configuration looks as shown below.

spring:
  cloud:
    consul:
      discovery:
        instanceId: ${spring.cloud.client.hostname}:${spring.application.name}:${random.value}

You can easily configure and run Eureka discovery in your microservices architecture using Spring Cloud Netflix Eureka Server module. You just need to create the Spring Boot application that includes that module.

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>

We also need to enable Eureka for the application by annotating the main class with @EnableEurekaServer.

@SpringBootApplication
@EnableEurekaServer
class DiscoveryServerApplication

fun main(args: Array<String>) {
    runApplication<DiscoveryServerApplication>(*args)
}

The most convenient way to run Consul on the local machine is by using its Docker image. We can Consul on Docker container in development mode by executing the following command.

$ docker run -d --name=consul -e CONSUL_BIND_INTERFACE=eth0 -p 8500:8500 consul:1.7.2

Distributed configuration with Spring cloud

The next important element in our architecture is a configuration server. The most popular solution in Spring Cloud that provides mechanisms for distributed configuration is Spring Cloud Config. Spring Cloud Config provides server-side and client-side support for externalized configuration in a distributed system.
With the config server, you have a central place to manage external properties for applications across all environments. Some other solutions may be used in microservices-based architecture as a configuration server: Consul, ZooKeeper, or Alibaba Nacos. However, all these solutions are not strictly dedicated to distributed configuration, they can act as a discovery server as well.
Spring Cloud Config may integrate with different tools for storing data. The default implementation of the server storage backend uses git, but we can use some other tools like HashiCorp Vault for managing secrets and protecting sensitive data or just a simple file system. Such different backends may be together in the single config server application. We just need to activate the appropriate profiles in application properties with spring.profiles.active. We may override some default, for example, change the address of a Vault server or set an authentication token.

spring:
  application:
    name: config-server
  profiles:
    active: native,vault
  cloud:
    config:
      server:
        native:
          searchLocations: classpath:/config-repo
        vault:
          host: 192.168.99.100
          authentication: TOKEN
          token: spring-microservices-course

The same as for Consul we should use Docker to run an instance of Vault in development mode. We can set a static root token for authentication using the environment variable VAULT_DEV_ROOT_TOKEN_ID.

$ docker run -d --name vault --cap-add=IPC_LOCK -e 'VAULT_DEV_ROOT_TOKEN_ID=spring-microservices-course' -p 8200:8200 vault:1.4.0

When using Spring Cloud Config together with the discovery we may choose between two available approaches called Config First Bootstrap and Discovery First Bootstrap. In Discovery First Bootstrap a config server is registering itself in discovery service. Thanks to that each microservice can localize a config server basing on its registration id.
Since a configuration is injected in the bootstrap phase we need to use bootstrap.yml for setting properties on the client-side. To enable “discovering” for the config server on the client side we should set the property spring.cloud.config.discovery.enabled to true. We should also override registered service id of config server if it is different from auto-configured configserver (in our case it is config-server). Of course, we can also use Consul as a configuration properties source.

spring:
  application:
    name: callme-service
  cloud:
    config:
      discovery:
        enabled: true
        serviceId: config-server
    consul:
      host: 192.168.99.100
      config:
        format: YAML

Inter-service communication

Currently, there are three Spring components for inter-service communication over HTTP that have integration with service discovery: synchronous RestTemplate, reactive WebClient and declarative REST client OpenFeign. The RestTemplate component is available within the Spring Web module, WebClient within Spring WebFlux. To include Spring Cloud OpenFeign we need to include a dedicated starter.

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>

To use RestTemplate or WebClient for communication with discovery support, we need to register the beans and annotate them with @LoadBalanced. It is also worth setting the proper timeouts for such communication, especially if you are not using a circuit breaker.

@SpringBootApplication
@EnableFeignClients
class InterCallerServiceApplication {

    @Bean
    @LoadBalanced
    fun template(): RestTemplate = RestTemplateBuilder()
        .setReadTimeout(Duration.ofMillis(100))
        .setConnectTimeout(Duration.ofMillis(100))
        .build()

    @Bean
    @LoadBalanced
    fun clientBuilder(): WebClient.Builder {
        val tcpClient: TcpClient = TcpClient.create()
            .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 100)
            .doOnConnected { conn ->
                conn.addHandlerLast(ReadTimeoutHandler(100, TimeUnit.MILLISECONDS))
            }
        val connector = ReactorClientHttpConnector(HttpClient.from(tcpClient))
        return WebClient.builder().clientConnector(connector)
    }
    
}

Although Ribbon has been moved to maintenance mode almost 1.5 years ago it is still used as a default client-side load balancer in the newest stable version of Spring Cloud. Since Spring Cloud Load Balancer is included in common dependencies it is available in your application. Therefore, the only thing you need to do is to disable Ribbon in the configuration using spring.cloud.loadbalancer.ribbon.enabled property.
Currently, we don’t have many options for load balancer customization. One of them is the ability to configure client cache settings. By default, each client is caching the list of target services and refreshing them every 30 seconds. Such an interval may be too long in your situation.
We can easily change it in the configuration as shown below and set it to for example 1 second. If your load balancer is integrated with Eureka discovery you also need to decrease an interval of the fetching registry, which is by default 30 seconds. After those, both changes your client can refresh the list of currently running services almost immediately.

spring:
  cloud:
    loadbalancer:
      cache:
        ttl: 1s
      ribbon:
        enabled: false
eureka:
  client:
    registryFetchIntervalSeconds: 1

Circuit breaker

The circuit breaker is a popular design pattern used in a microservices architecture. It is designed to detect failures and encapsulates the logic of preventing a failure from constantly recurring. Spring Cloud provides an abstraction for using different circuit breaker implementations. Currently, we may use Netflix Hystrix, Sentinel, Spring Retry, and Resilience4J. To enable Spring Cloud Circuit Breaker based on Resilience4J we need to include the following dependency.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-circuitbreaker-resilience4j</artifactId>
</dependency>

Here’s the code responsible for registering Customizer bean, that configures a circuit breaker behavior.

@Bean
fun defaultCustomizer(): Customizer<Resilience4JCircuitBreakerFactory> {
  return Customizer { factory: Resilience4JCircuitBreakerFactory ->
    factory.configureDefault { id: String? ->
      Resilience4JConfigBuilder(id)
        .timeLimiterConfig(TimeLimiterConfig.custom()
          .timeoutDuration(Duration.ofMillis(500))
          .build())
        .circuitBreakerConfig(CircuitBreakerConfig.custom()
          .slidingWindowSize(10)
          .failureRateThreshold(33.3F)
          .slowCallRateThreshold(33.3F)
        .build())
      .build()
    }
  }
}

The settings of the circuit breaker have been visualized in the picture below. The sliding window size sets the number of requests which are used for calculating the error rate. If we have more than 3 errors in the window of size 10 a circuit is open.

In the last step, we need to create a circuit breaker instance using Resilience4JCircuitBreakerFactory bean and enable it for the HTTP client as shown below.

@RestController
@RequestMapping("/caller")
class CallerController(private val template: RestTemplate, private val factory: Resilience4JCircuitBreakerFactory) {

  private var id: Int = 0

  @PostMapping("/random-send/{message}")
  fun randomSend(@PathVariable message: String): CallmeResponse? {
    val request = CallmeRequest(++id, message)
    val circuit = factory.create("random-circuit")
    return circuit.run { template.postForObject("http://inter-callme-service/callme/random-call",
      request, CallmeResponse::class.java) }
  }
    
}

Spring Cloud API gateway

The last missing element in our microservices architecture is an API Gateway. Spring Cloud Gateway is the project that helps us in implementing such a component. Currently, it is the second most popular Spring Cloud project just after Spring Cloud Netflix. It has around 2k stars on GitHub. It is built on top of the Spring WebFlux and Reactor project. It works reactively and requires Netty as a runtime framework.
The main goal of API gateway is to hide the complexity of the microservices system from an external client by providing an effective way of routing to APIs, but it can also solve some problems around security or resiliency. The main component used for configuring Spring Cloud Gateway is a route.
It is defined by an ID, a destination URI, a collection of predicates, and a collection of filters. A route is matched if the aggregate predicate is true. With filters, you can modify requests and responses before or after sending the downstream request.
With a predefined set of gateway filters, we may enable such mechanisms like path rewriting, rate limiting, discovery client, circuit breaking, fallback, or routing metrics. To enable all these features on the gateway we first need to include the following dependencies.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
   <groupId>org.jetbrains.kotlin</groupId>
   <artifactId>kotlin-reflect</artifactId>
</dependency>
<dependency>
   <groupId>org.jetbrains.kotlin</groupId>
   <artifactId>kotlin-stdlib</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-data-redis-reactive</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-circuitbreaker-reactor-resilience4j</artifactId>
</dependency>

To enable all the previously listed features we don’t have to implement much code. Almost everything is configurable in application properties.

spring:
  application:
    name: api-gateway
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true 
          lowerCaseServiceId: true
      routes:
        - id: inter-callme-service
          uri: lb://inter-callme-service
          predicates:
            - Path=/api/callme/**
          filters:
            - RewritePath=/api(?/?.*), $\{path}
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 20
                redis-rate-limiter.burstCapacity: 40
            - name: CircuitBreaker
              args:
                name: sampleSlowCircuitBreaker
                fallbackUri: forward:/fallback/test
        - id: inter-caller-service
          uri: lb://inter-caller-service
          predicates:
            - Path=/api/caller/**
          filters:
            - StripPrefix=1
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 20
                redis-rate-limiter.burstCapacity: 40
    loadbalancer:
      ribbon:
        enabled: false
  redis:
    host: 192.168.99.100

management:
  endpoints.web.exposure.include: '*'
  endpoint:
    health:
      show-details: always

Some settings still need to be configured in the code. It is a configuration of the circuit breaker, that is based on Resilience4J project, where we need to register the bean Customizer. We also have to define a key for rate-limiting responsible setting a strategy of choosing requests for counting the limits.

@SpringBootApplication
class ApiGatewayApplication {

  @Bean
  fun keyResolver(): KeyResolver = KeyResolver { _ -> Mono.just("1") }

  @Bean
  fun defaultCustomizer(): Customizer<ReactiveResilience4JCircuitBreakerFactory> {
    return Customizer { factory: ReactiveResilience4JCircuitBreakerFactory ->
      factory.configureDefault { id: String? ->
        Resilience4JConfigBuilder(id)
          .timeLimiterConfig(TimeLimiterConfig.custom()
            .timeoutDuration(Duration.ofMillis(500))
            .build())
          .circuitBreakerConfig(CircuitBreakerConfig.custom()
            .slidingWindowSize(10)
            .failureRateThreshold(33.3F)
            .slowCallRateThreshold(33.3F)
            .build())
          .build()
      }
    }
  }

}

fun main(args: Array<String>) {
   runApplication<ApiGatewayApplication>(*args)
}

Conclusion

In this article, you may take a quick introduction to using the latest Spring Cloud components for building microservices architecture. For more details, you may refer to my video course published on YouTube.

The post A New Era Of Spring Cloud appeared first on Piotr's TechBlog.

Release Train

The Release Train version used in this article is Hoxton.M2. To access it we need to use the Spring Milestone repository.

<repositories>
   <repository>
      <id>spring-milestones</id>
      <name>Spring Milestones</name>
      <url>https://repo.spring.io/milestone</url>
   </repository>
</repositories>
<pluginRepositories>
   <pluginRepository>
      <id>spring-milestones</id>
      <name>Spring Milestones</name>
      <url>https://repo.spring.io/milestone</url>
   </pluginRepository>
</pluginRepositories>

Here’s declaration of release train inside dependency management section:

<parent>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-parent</artifactId>
   <version>2.2.0.RC1</version>
   <relativePath/>
</parent>
<properties>
   <java.version>1.8</java.version>
   <spring-cloud.version>Hoxton.M2</spring-cloud.version>
</properties>
<dependencyManagement>
   <dependencies>
      <dependency>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-dependencies</artifactId>
         <version>${spring-cloud.version}</version>
         <type>pom</type>
         <scope>import</scope>
      </dependency>
   </dependencies>
</dependencyManagement>

Dependencies

Besides standard Spring Boot Web Starter we need to include starters for Consul discovery and config client, and of course spring-cloud-loadbalancer library. We should remember about excluding Ribbon and Hystrix artifacts from spring-cloud-starter-consul-discovery starter.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-consul-discovery</artifactId>
   <exclusions>
      <exclusion>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-netflix-hystrix</artifactId>
      </exclusion>
      <exclusion>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
      </exclusion>
   </exclusions>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-consul-config</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-loadbalancer</artifactId>
</dependency>

Sample Applications

We have three sample applications. First callme-service exposes some REST endpoints, second caller-service communicates with the first using Consul discovery, RestTemplate and Spring Cloud Loadbalancer, and third gateway-service acts as API gateway in our architecture.

The repository with source code snippets is available on GitHub: https://github.com/piomin/sample-spring-cloud-microservices-future.git. For some configuration details related to Consul discovery please refer to my previous article. In fact we don’t have to provide any additional code to make it working. We just have to declare RestTemplate bean annotated with @LoadBalanced:

@SpringBootApplication
public class CallerApplication {

   public static void main(String[] args) {
      SpringApplication.run(CallerApplication.class, args);
   }

   @LoadBalanced
   @Bean
   RestTemplate template() {
      return new RestTemplate();
   }
}

Then we may call the target service using its name taken from the discovery server:

@RestController
@RequestMapping("/caller")
public class CallerController {

   private static final Logger LOGGER = LoggerFactory.getLogger(CallerController.class);

   @Autowired
   Environment environment;
   @Autowired
   RestTemplate template;

   @GetMapping
   public String call() {
      String url = "http://callme-service/callme";
      String callmeResponse = template.getForObject(url, String.class);
      LOGGER.info("Response: {}", callmeResponse);
      return "I'm Caller running on port " + environment.getProperty("local.server.port")
            + " calling-> " + callmeResponse;
   }
   
}

Summary

In this article, I show you a working example of communication between microservices using the new Spring Cloud Load Balancer. I also tried to use that load balancer instead of Spring Cloud Netflix Ribbon inside my application with Spring Cloud Gateway – with no success. According to the newest documentation of Spring Cloud Circuit Breaker built on top of Resilience4J I tried to include circuit breaker in the communication between caller-service and callme-service – also with no success. These new Spring Cloud features are still under active development. I hope to use them successfully in my applications soon. For more advanced Spring Cloud load balancing usage please refer to the article A Deep Dive Into Spring Cloud Load Balancer.

The post Using Spring Cloud Load Balancer In Microservices Communication appeared first on Piotr's TechBlog.

Service Discovery

Eureka is the only one important Spring Cloud Netflix module that has not been moved to maintenance mode. However, I would not say that it is actively developed. The last commit in the repository maintained by Netflix is from 11th January. Some time ago they have started working on Eureka 2, but it seems these works have been abandoned or they just have postponed open sourcing the newest version code to the future. Here https://github.com/Netflix/eureka/tree/2.x you can find an interesting comment about it: “The 2.x branch is currently frozen as we have had some internal changes w.r.t. to eureka2, and do not have any time lines for open sourcing of the new changes.”. So, we have two possibilities. Maybe, Netflix will decide to open source those internal changes as a version 2 of Eureka server. It is worth remembering that Eureka is a battle proven solution used at Scale by Netflix directly, and probably by many other organizations through Spring Cloud.
The second option is to choose another discovery server. Currently, Spring Cloud supports discovery based on various tools: ZooKeeper, Consul, Alibaba Nacos, Kubernetes. In fact, Kubernetes is based on etcd. Support for etcd is also being developed by Spring Cloud, but it is still in the incubation stage, and it is not known if it will ever be promoted to the official release train. In my opinion, there is one leader amongst these solutions – HashiCorp’s Consul.
Consul is now described as a service mesh solution providing a full featured control plane with service discovery, configuration, and segmentation functionality. It can be used as a discovery server or a key/value store in your microservices-based architecture. The integration with Consul is implemented by the Spring Cloud Consul project. To enable Consul client for your application you just need to include the following dependency to your Maven pom.xml:

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-consul-discovery</artifactId>
</dependency>

By default, Spring tries to connect with Consul on the address localhost:8500. If you need to override this address you should set the appropriate properties inside application.yml:

spring:  
  cloud:
    consul:
      host: 192.168.99.100
      port: 8500

You can easily test this solution with local instance of Consul started as the Docker container:

$ docker run -d --name consul -p 8500:8500 consul

As you see Consul discovery implementation with Spring Cloud is very easy – the same as for Eureka. Consul has one undoubted advantage over Eureka – it is continuously maintained and developed by HashiCorp. Its popularity is growing fast. It is a part of the biggest HashiCorp ecosystem, which includes Vault, Nomad and Terraform. In contrast to Eureka, Consul can be used not only for service discovery, but also as a configuration server in your microservices-based architecture.

Distributed Configuration

Netflix Archaius is an interesting solution for managing externalized configuration in microservices architecture. Although it offers some interesting features like dynamic and typed properties or support for dynamic data sources such as URLs, JDBC or AWS DynamoDB, Spring Cloud has also decided to move it to the maintenance mode. However, the popularity of Spring Cloud Archaius was limited, due to the existence of a similar project fully created by the Pivotal team and community – Spring Cloud Config. Spring Cloud Config supports multiple source repositories including Git, JDBC, Vault or simple files. You can find many examples of using this project for providing distributed configuration for your microservices in my previous posts. Today, I’m not going to talk about it. We will discuss an alternative solution – also supported by Spring Cloud.
As I have mentioned in the end of the previous section Consul can also be used as a configuration server. If you use Eureka as a discovery server, using Spring Cloud Config as a configuration server is a natural choice, because Eureka simply does not provide such features. This is not the case if you decide to use Consul. Now it makes sense to choose between two solutions: Spring Cloud Consul Config and Spring Cloud Config. Of course, both of them have their advantages and disadvantages. For example, you can easily build a cluster with Consul nodes, while with Spring Cloud Config you must rely on external discovery.
Now, let’s see how to use Spring Cloud Consul for managing external configuration in your application. To enable it on the application side you just need to include the following dependency to your Maven pom.xml:

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-consul-config</artifactId>
</dependency>

The same as for service discovery, If you would like to override some default client settings you need to set properties spring.cloud.consul.*. However, such a configuration must be provided inside bootstrap.yml.

spring:  
  application:
    name: callme-service
  cloud:
    consul:
      host: 192.168.99.100
      port: 8500

The name of the property source created on Consul should be the same as the application name provided in bootstrap.yml inside the config folder. You should create key server.port with value 0, to force Spring Boot to generate listening port number randomly. Supposing you need to set the application default listening port you should the following configuration.

When enabling dynamic port number generation you also need to override application instance id to be unique across a single machine. This feature is required if you are running multiple instances of a single service in the same machine. We will do it for callme-service, so we need to override the property spring.cloud.consul.discovery.instance-id with our value as shown below.

Then, you should see the following log on your application startup.

API Gateway

The successor of Spring Cloud Netflix Zuul is Spring Cloud Gateway. This project started around two years ago, and now is the second most popular Spring Cloud project with 1.4k stars on GitHub. It provides an API Gateway built on top of the Spring Ecosystem, including: Spring 5, Spring Boot 2 and Project Reactor. It is running on Netty, and does not work with traditional servlet containers like Tomcat or Jetty. It allows us to define routes, predicates and filters.
API gateway, the same as every Spring Cloud microservice may be easily integrated with service discovery based on Consul. We just need to include the appropriate dependencies inside pom.xml. We will use the latest development version of Spring Cloud libraries – 2.2.0.BUILD-SNAPSHOT. Here’s the list of required dependencies:

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-consul-discovery</artifactId>
   <version>2.2.0.BUILD-SNAPSHOT</version>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-consul-config</artifactId>
   <version>2.2.0.BUILD-SNAPSHOT</version>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
   <version>2.2.0.BUILD-SNAPSHOT</version>
</dependency>

The gateway configuration will also be served by Consul. Because we have pretty more configuration settings than for sample microservices, we will store it as YAML file. To achieve that we should create a YAML file available under path /config/gateway-service/data on Consul Key/Value. The configuration visible below enables service discovery integration and defines routes to the downstream services. Each route contains the name of the target service under which it is registered in service discovery, matching path and rewrite path used for call endpoint exposed by the downstream service. The following configuration is load on startup by our API gateway:

spring:
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true
      routes:
        - id: caller-service
          uri: lb://caller-service
          predicates:
            - Path=/caller/**
          filters:
            - RewritePath=/caller/(?<path>.*), /$\{path}
        - id: callme-service
          uri: lb://callme-service
          predicates:
            - Path=/callme/**
          filters:
            - RewritePath=/callme/(?<path>.*), /$\{path}

Here’s the same configuration visible on Consul.

The last step is to force gateway-service to read configuration stored as YAML. To do that we need to set property spring.cloud.consul.config.format to YAML. Here’s the full configuration provided inside bootstrap.yml.

spring:
  application:
    name: gateway-service
  cloud:
    consul:
      host: 192.168.99.100
      config:
        format: YAML

Client-side Load Balancer

In version 2.2.0.BUILD-SNAPSHOT of Spring Cloud Commons Ribbon is still the main auto-configured load balancer for HTTP clients. Although the Spring Cloud team has announced that Spring Cloud Load Balancer will be the successor of Ribbon, we currently won’t find much information about that project in documentation and on the web. We may expect that the same as for Netflix Ribbon, any configuration will be transparent for us, especially if we use a discovery client. Currently, spring-cloud-loadbalancer module is a part of Spring Cloud Commons project. You may include it directly to your application by declaring the following dependency in pom.xml:

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-loadbalancer</artifactId>
   <version>2.2.0.BUILD-SNAPSHOT</version>
</dependency>

For the test purposes it is worth to exclude some Netflix modules included together with <code>spring-cloud-starter-consul-discovery</code> starter. Now, we are sure that Ribbon is not used in background as load balancer. Here’s the list of exclusions I set for my sample application:

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-consul-discovery</artifactId>
   <version>2.2.0.BUILD-SNAPSHOT</version>
   <exclusions>
      <exclusion>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-netflix-core</artifactId>
      </exclusion>
      <exclusion>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-starter-netflix-archaius</artifactId>
      </exclusion>
      <exclusion>
         <groupId>com.netflix.ribbon</groupId>
         <artifactId>ribbon</artifactId>
      </exclusion>
      <exclusion>
         <groupId>com.netflix.ribbon</groupId>
         <artifactId>ribbon-core</artifactId>
      </exclusion>
      <exclusion>
         <groupId>com.netflix.ribbon</groupId>
         <artifactId>ribbon-httpclient</artifactId>
      </exclusion>
      <exclusion>
         <groupId>com.netflix.ribbon</groupId>
         <artifactId>ribbon-loadbalancer</artifactId>
      </exclusion>
   </exclusions>
</dependency>

Treat my example just as a playground. Certainly the targeted approach is going to be much easier. First, we should annotate our main or configuration class with @LoadBalancerClient. As always, the name of the client should be the same as the name of the target service registered in the registry. The annotation should also contain the class with client configuration.

@SpringBootApplication
@LoadBalancerClients({
   @LoadBalancerClient(name = "callme-service", configuration = ClientConfiguration.class)
})
public class CallerApplication {

   public static void main(String[] args) {
      SpringApplication.run(CallerApplication.class, args);
   }

   @Bean
   RestTemplate template() {
      return new RestTemplate();
   }

}

Here’s our load balancer configuration class. It contains the declaration of a single @Bean. I have chosen RoundRobinLoadBalancer type.

public class ClientConfiguration {

   @Bean
   public RoundRobinLoadBalancer roundRobinContextLoadBalancer(LoadBalancerClientFactory clientFactory, Environment env) {
      String serviceId = clientFactory.getName(env);
      return new RoundRobinLoadBalancer(serviceId, clientFactory
            .getLazyProvider(serviceId, ServiceInstanceSupplier.class), -1);
   }

}

Finally, here’s the implementation of caller-service controller. It uses LoadBalancerClientFactory directly to find a list of available instances of callme-service. Then it selects a single instance, get its host and port, and sets in as a target URL.

@RestController
@RequestMapping("/caller")
public class CallerController {

   @Autowired
   Environment environment;
   @Autowired
   RestTemplate template;
   @Autowired
   LoadBalancerClientFactory clientFactory;

   @GetMapping
   public String call() {
      RoundRobinLoadBalancer lb = clientFactory.getInstance("callme-service", RoundRobinLoadBalancer.class);
      ServiceInstance instance = lb.choose().block().getServer();
      String url = "http://" + instance.getHost() + ":" + instance.getPort() + "/callme";
      String callmeResponse = template.getForObject(url, String.class);
      return "I'm Caller running on port " + environment.getProperty("local.server.port")
            + " calling-> " + callmeResponse;
   }

}

Summary

The following picture illustrates the architecture of our sample system. We have two instances of callme-service, a single instance of caller-service, which uses Spring Cloud Balancer to find the list of available instances of callme-service. The ports are generated dynamically. The API gateway is hiding the complexity of our system from an external client. It is available on port 8080, and is forwarding requests to the downstream basing on request context path.

After starting, all the microservices you should be registered on your Consul node.

Now, you can try to endpoint exposed by caller-service through gateway: http://localhost:8080/caller. You should something like that:

The sample application source code is available on GitHub in repository https://github.com/piomin/sample-spring-cloud-microservices-future.git.

The post The Future of Spring Cloud Microservices After Netflix Era appeared first on Piotr's TechBlog.