Source code

If you would like to try it by yourself, you may always take a look at my source code example. In order to do that you need to clone my repository sample-spring-microservices-kubernetes. Then just follow my instructions The good idea is to read the article Microservices with Spring Cloud Kubernetes before you move on.

Step 1. Enable Spring Cloud Kubernetes Load Balancer

You need to include a single Spring Boot Starter to enable Spring Cloud Kubernetes Load Balancer. It is spring-cloud-starter-kubernetes-loadbalancer. Consequently, you also need to have a REST client on the classpath. Spring RestTemplate is automatically included with the Spring Web module. We will also use the OpenFeign client, so you should include the right starter.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-kubernetes-loadbalancer</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>

Step 2. Implement and use the REST client

Spring Cloud OpenFeign is a declarative REST client. Therefore, you need to create an interface with methods and Spring MVC annotations. It is important to set the right name inside @FeignClient annotation. This name needs to be the same as the name of the target Kubernetes Service. In the following code example, you see an implementation of the employee service client inside department-service.

@FeignClient(name = "employee")
public interface EmployeeClient {

   @GetMapping("/department/{departmentId}")
   List<Employee> findByDepartment(@PathVariable("departmentId") String departmentId);
	
}

OpenFeign client is enabled after annotating the main class with @EnableFeignClients. After that you can inject it to the @RestController class. Finally, you may use its method findByDepartment to find employees assigned to the particular department.

@RestController
public class DepartmentController {

    private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);

    DepartmentRepository repository;
    EmployeeClient employeeClient;

    public DepartmentController(DepartmentRepository repository, EmployeeClient employeeClient) {
        this.repository = repository;
        this.employeeClient = employeeClient;
    }

    @GetMapping("/{id}/with-employees")
    public Department findByIdWithEmployees(@PathVariable("id") String id) {
        LOGGER.info("Department findByIdWithEmployees: id={}", id);
        Optional<Department> optDepartment = repository.findById(id);
        if (optDepartment.isPresent()) {
            Department department = optDepartment.get();
            department.setEmployees(employeeClient.findByDepartment(department.getId()));
            return department;
        }
        return null;
    }

    @GetMapping("/organization/{organizationId}/with-employees")
    public List<Department> findByOrganizationWithEmployees(@PathVariable("organizationId") Long organizationId) {
        LOGGER.info("Department find: organizationId={}", organizationId);
        List<Department> departments = repository.findByOrganizationId(organizationId);
        departments.forEach(d -> d.setEmployees(employeeClient.findByDepartment(d.getId())));
        return departments;
    }

}

Opposite to the OpenFeign, the RestTemplate is a low-level HTTP client. We need to enable Spring Cloud load balancing for it. To do that just annotate RestTemplate bean with @LoadBalanced.

@Bean
@LoadBalanced
RestTemplate restTemplate() {
    return new RestTemplate();
}

Here’s the similar implementation of @RestController class, but this time with the RestTemplate.

public class DepartmentWithRestTemplateController {


    private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);

    DepartmentRepository repository;
    RestTemplate restTemplate;

    public DepartmentWithRestTemplateController(DepartmentRepository repository, RestTemplate restTemplate) {
        this.repository = repository;
        this.restTemplate = restTemplate;
    }

    @GetMapping("/{id}/with-employees")
    public Department findByIdWithEmployees(@PathVariable("id") String id) {
        LOGGER.info("Department findByIdWithEmployees: id={}", id);
        Optional<Department> optDepartment = repository.findById(id);
        if (optDepartment.isPresent()) {
            Department department = optDepartment.get();
            department.setEmployees(findEmployeesByDepartment(department.getId()));
            return department;
        }
        return null;
    }


    @GetMapping("/organization/{organizationId}/with-employees")
    public List<Department> findByOrganizationWithEmployees(@PathVariable("organizationId") Long organizationId) {
        LOGGER.info("Department find: organizationId={}", organizationId);
        List<Department> departments = repository.findByOrganizationId(organizationId);
        departments.forEach(d -> d.setEmployees(findEmployeesByDepartment(d.getId())));
        return departments;
    }

    private List<Employee> findEmployeesByDepartment(String departmentId) {
        Employee[] employees = restTemplate
                .getForObject("http://employee//department/{departmentId}", Employee[].class, departmentId);
        return Arrays.asList(employees);
    }

}

Step 3. Deploy Spring Cloud applications on Kubernetes

The project is ready to be used with Skaffold. Therefore, you don’t have to worry about the deployment process. You just need to run a single command skaffold dev --port-forward to deploy our applications on Kubernetes. But before deploying them on the cluster, we perform a short overview of the communication process. We have three microservices. Each of them is running in two instances. They are using Mongo as a backend store. There is also the gateway application. It is built on top of Spring Cloud Gateway. It provides a single API endpoint to all downstream services. Of course, all our applications are using Spring Cloud Load Balancer for traffic management. To clarify, you can take a look at the picture below.

Here’s the Kubernetes deployment manifest for the employee-service. It sets two running pods in replicas parameter, and references ConfigMap and Secret to inject Mongo credentials.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: employee
  labels:
    app: employee
spec:
  replicas: 2
  selector:
    matchLabels:
      app: employee
  template:
    metadata:
      labels:
        app: employee
    spec:
      containers:
      - name: employee
        image: piomin/employee
        ports:
        - containerPort: 8080
        env:
        - name: MONGO_DATABASE
          valueFrom:
            configMapKeyRef:
              name: mongodb
              key: database-name
        - name: MONGO_USERNAME
          valueFrom:
            secretKeyRef:
              name: mongodb
              key: database-user
        - name: MONGO_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mongodb
              key: database-password

The most important element in the communication between applications is Kubernetes Service object. The name of service must be the same as the hostname used by the RestTemplate or OpenFeign client. In our case that name is employee.

apiVersion: v1
kind: Service
metadata:
  name: employee
  labels:
    app: employee
    spring-boot: "true"
spec:
  ports:
  - port: 8080
    protocol: TCP
  selector:
    app: employee
  type: ClusterIP

Let’s verify the status after deploying all the applications. Here’s the list of running pods.

Here’s the list of Kubernetes Endpoints.

Step 4. Communication in the POD mode

By default, Spring Cloud Kubernetes Load Balancer uses the POD mode. In this mode, it gets the list of Kubernetes endpoints to detect the IP address of all the application pods. In that case, the only thing you need to do is to disable the Ribbon load balancer. Spring Cloud Kubernetes is still using it as a default client-side load balancer. To disable it you need to set the following property.

spring:
  cloud:
    loadbalancer:
      ribbon:
        enabled: false

After adding a test data we may send some requests to the endpoint GET /{id}/with-employees. It finds a department by the id. Then it communicates with the employee-service endpoint GET /department/{departmentId} to search all the employees assigned to the current department. The department-service is exposed on port 8081, since I enabled option port-forward on Skaffold.

$ curl http://localhost:8081/5f5896b3cb8caf7e6f6b9e1c/with-employees
{"id":"5f5896b3cb8caf7e6f6b9e1c","organizationId":"1","name":"test1","employees":[{"id":"5f5896e26092716e54f60a9a","name":"test1","age":30,"position":"d
eveloper"},{"id":"5f5896f29625e62c7d373906","name":"test2","age":40,"position":"tester"},{"id":"5f5897266092716e54f60a9b","name":"test3","age":45,"posit
ion":"developer"}]}

Let’s take a look on the logs from employee-service. I repeated the request visible above several times. The requests are load balanced between two available instances of employee-service. Skaffold prints the id of every pod in the logs, so you can verify that everything works fine.

Step 5. Communication across multiple namespaces

By default, Spring Cloud Kubernetes allows load balancing within the same namespace. You may enable discovery across multiple namespaces. To do that you need to use the property following property.

spring:
  cloud:
    kubernetes:
      discovery:
        all-namespaces: true

After setting that, we will deploy the employee-service application in the different namespace than department-service. To do that you need to set parameter -n on the skaffold command.

$ skaffold dev -n test-a

Here’s the current list of running pods.

You just need to send the same request as before curl http://localhost:8081/5f5896b3cb8caf7e6f6b9e1c/with-employees. Of course, the Kubernetes service name must be unique across all the namespaces.

Step 6. Load balancer on Spring Cloud Gateway

Spring Cloud Gateway uses the same load balancing mechanism as other Spring Cloud applications. To enable it on Kubernetes we need to include dependency spring-cloud-starter-kubernetes-loadbalancer. We should also enable Spring Cloud DiscoveryClient and disable Ribbon. Here’s the configuration of the gateway-service.

spring:
  application:
    name: gateway
  cloud:
    loadbalancer:
      ribbon:
        enabled: false
    gateway:
      discovery:
        locator:
          enabled: true
      routes:
        - id: employee
          uri: lb://employee
          predicates:
            - Path=/employee/**
          filters:
            - RewritePath=/employee/(?.*), /$\{path}
        - id: department
          uri: lb://department
          predicates:
            - Path=/department/**
          filters:
            - RewritePath=/department/(?.*), /$\{path}
        - id: organization
          uri: lb://organization
          predicates:
            - Path=/organization/**
          filters:
            - RewritePath=/organization/(?.*), /$\{path}

A gateway acts as an entry point to our system. It performs routing and load balancing to all the downstream services. Therefore, we can call any of our applications using already defined routes. Since gateway-service is available on port 8080, I can call any of the endpoints using the following requests.

We can expose API documentation of all the microservices on the gateway. To do that we may use the SpringFox project. First, we need to include SpringFox Starter to Maven dependencies.

<dependency>
   <groupId>io.springfox</groupId>
   <artifactId>springfox-boot-starter</artifactId>
   <version>3.0.0</version>
</dependency>

Swagger documentation is usually generated for a single application. Because we want to expose documentation of all our microservices we need to override the Swagger resource provider. The custom implementation is visible below. It uses a discovery mechanism to the names of running services.

@Configuration
public class GatewayApi {

   @Autowired
   RouteDefinitionLocator locator;

   @Primary
   @Bean
   public SwaggerResourcesProvider swaggerResourcesProvider() {
      return () -> {
         List<SwaggerResource> resources = new ArrayList<>();
         Flux<RouteDefinition> definitions = locator.getRouteDefinitions();
         definitions
               .filter(routeDefinition -> !routeDefinition.getId().startsWith("ReactiveCompositeDiscoveryClient_"))
               .subscribe(routeDefinition -> resources.add(createResource(routeDefinition.getId(), "2.0")));
         try {
            Thread.sleep(1000);
         } catch (InterruptedException e) {
            e.printStackTrace();
         }
         return resources;
      };
   }

   private SwaggerResource createResource(String location, String version) {
      SwaggerResource swaggerResource = new SwaggerResource();
      swaggerResource.setName(location);
      swaggerResource.setLocation("/" + location + "/v2/api-docs");
      swaggerResource.setSwaggerVersion(version);
      return swaggerResource;
   }

}

Here’s the Swagger UI available at http://localhost:8080/swagger-ui/index.html on my local machine.

Step 7. Enabling circuit breaker

We can use a circuit breaker component with Spring Cloud Load Balancer. The default Spring Cloud implementation is based on Resilience4j. In order to enable it for your application, you need to include the following dependency.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-circuitbreaker-resilience4j</artifactId>
</dependency>

The next step is to provide a configuration of a circuit breaker.

@Bean
Customizer<Resilience4JCircuitBreakerFactory> defaultCustomizer() {
   return factory -> factory.configureDefault(id ->
         new Resilience4JConfigBuilder(id)
               .timeLimiterConfig(TimeLimiterConfig.custom()
                     .timeoutDuration(Duration.ofMillis(1000))
                     .build())
               .circuitBreakerConfig(CircuitBreakerConfig.custom()
                     .slidingWindowSize(10)
                     .failureRateThreshold(66.6F)
                     .slowCallRateThreshold(66.6F)
                     .build())
               .build()
   );
}

We need to inject Resilience4JCircuitBreakerFactory to the Spring controller class. Then we are creating the circuit breaker instance using an injected factory. Finally, the client calling method is running inside the circuit breaker run method.

@RestController
public class DepartmentController {

    private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);

    DepartmentRepository repository;
    EmployeeClient employeeClient;
    Resilience4JCircuitBreakerFactory circuitBreakerFactory;

    public DepartmentController(DepartmentRepository repository, EmployeeClient employeeClient,
                                Resilience4JCircuitBreakerFactory circuitBreakerFactory) {
        this.repository = repository;
        this.employeeClient = employeeClient;
        this.circuitBreakerFactory = circuitBreakerFactory;
    }

    @GetMapping("/{id}/with-employees-and-delay")
    public Department findByIdWithEmployeesAndDelay(@PathVariable("id") String id) {
        LOGGER.info("Department findByIdWithEmployees: id={}", id);
        Optional<Department> optDepartment = repository.findById(id);
        if (optDepartment.isPresent()) {
            Department department = optDepartment.get();
            Resilience4JCircuitBreaker circuitBreaker = circuitBreakerFactory.create("delayed-circuit");
            List<Employee> employees = circuitBreaker.run(() ->
                    employeeClient.findByDepartmentWithDelay(department.getId()));
            department.setEmployees(employees);
            return department;
        }
        return null;
    }
}

Conclusion

Load balancing is one of the key patterns in a microservices architecture. Spring Cloud Load Balancer is replacing the Ribbon client. By default, load balancing in Kubernetes is based on Services. Therefore, you need to use additional tools for more advanced routing mechanisms. Spring Cloud Kubernetes comes with some interesting features. One of them is the ability to load balance across multiple namespaces. You can also use the additional Spring Cloud components like a circuit breaker. In comparison with tools like Istio, it is still not much. Is it a chance for improvement? We will see. Nevertheless, Spring Cloud Kubernetes is currently one of the most popular Spring Cloud projects. It may be a good choice if you are migrating your Spring Cloud microservices architecture to Kubernetes. Support for the load balancer may be an important step during such a migration.

The post Spring Cloud Kubernetes Load Balancer Guide appeared first on Piotr's TechBlog.

Spring Cloud video course

You may also find a video guide, where I’m showing all the features described here with more details. It is available on my YouTube channel and consists of four parts:

Part 1 – Introduction to Spring Boot (~35 minutes)
Part 2 – Distributed Configuration and Service Discovery (~36 minutes)
Part 3 – Inter-service communication (~34 minutes)
Part 4 – API Gateway (~22 minutes)

Architecture

The diagram visible below illustrates the architecture of our sample system. Here we have the characteristic elements for microservices like API gateway, discovery server, and configuration server. In the next sections of this article, I’ll show how to use Spring Cloud components that provide an implementation of those patterns. Currently, the main component for adding the API gateway to your system is Spring Cloud Gateway.
Spring Cloud provides integrations to several solutions that may be used as a discovery server: Netflix Eureka, HashiCorp Consul, Alibaba Nacos, or Apache ZooKeeper. The most common choice is between the first two of them. While Spring Cloud Netflix Eureka is dedicated just for discovery, Spring Cloud Consul may realize both discovery feature basing on Consul Services, and distributed configuration feature basing on Consul Key/Value engine.

In turn, Spring Cloud Config is responsible just for providing a mechanism for configuration management. However, it may also be integrated with third-party tools like Vault from HashiCorp.
We will figure out how to integrate our applications with discovery and configuration servers on the example of two simple Spring Boot applications callme-service and caller-service. The application caller-service is also calling endpoints exposed by the callme-service. We will enable such mechanisms on the caller-service like client-side load balancer with new Spring Cloud Load Balancer, and circuit breaker with new Spring Cloud Circuit Breaker built on top of Resilience4J.

Service discovery

Switching between discovery servers on the client-side is very easy with Spring Cloud due to the DiscoveryClient abstraction. This switch comes down to the replacement of a single dependency in Maven pom.xml. So if you are using Eureka you should add the following starter in your microservice.

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>

On the other hand, if you are using Consul you should add the following starter in your microservice.

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-consul-discovery</artifactId>
</dependency>

The situation is a little bit more complicated if you are defining some non-default configuration settings for a discovery client. In that case, you need to use properties specific just for Eureka, or just for Consul. For example, if you are running more than one instance of a single application on the same host with dynamic HTTP server port feature enabled (option server.port=0), you have to set a unique id of every instance. Here’s the property used for Eureka’s client.

eureka:
  instance:
    instanceId: ${spring.cloud.client.hostname}:${spring.application.name}:${random.value}

For the Consul client the same configuration looks as shown below.

spring:
  cloud:
    consul:
      discovery:
        instanceId: ${spring.cloud.client.hostname}:${spring.application.name}:${random.value}

You can easily configure and run Eureka discovery in your microservices architecture using Spring Cloud Netflix Eureka Server module. You just need to create the Spring Boot application that includes that module.

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>

We also need to enable Eureka for the application by annotating the main class with @EnableEurekaServer.

@SpringBootApplication
@EnableEurekaServer
class DiscoveryServerApplication

fun main(args: Array<String>) {
    runApplication<DiscoveryServerApplication>(*args)
}

The most convenient way to run Consul on the local machine is by using its Docker image. We can Consul on Docker container in development mode by executing the following command.

$ docker run -d --name=consul -e CONSUL_BIND_INTERFACE=eth0 -p 8500:8500 consul:1.7.2

Distributed configuration with Spring cloud

The next important element in our architecture is a configuration server. The most popular solution in Spring Cloud that provides mechanisms for distributed configuration is Spring Cloud Config. Spring Cloud Config provides server-side and client-side support for externalized configuration in a distributed system.
With the config server, you have a central place to manage external properties for applications across all environments. Some other solutions may be used in microservices-based architecture as a configuration server: Consul, ZooKeeper, or Alibaba Nacos. However, all these solutions are not strictly dedicated to distributed configuration, they can act as a discovery server as well.
Spring Cloud Config may integrate with different tools for storing data. The default implementation of the server storage backend uses git, but we can use some other tools like HashiCorp Vault for managing secrets and protecting sensitive data or just a simple file system. Such different backends may be together in the single config server application. We just need to activate the appropriate profiles in application properties with spring.profiles.active. We may override some default, for example, change the address of a Vault server or set an authentication token.

spring:
  application:
    name: config-server
  profiles:
    active: native,vault
  cloud:
    config:
      server:
        native:
          searchLocations: classpath:/config-repo
        vault:
          host: 192.168.99.100
          authentication: TOKEN
          token: spring-microservices-course

The same as for Consul we should use Docker to run an instance of Vault in development mode. We can set a static root token for authentication using the environment variable VAULT_DEV_ROOT_TOKEN_ID.

$ docker run -d --name vault --cap-add=IPC_LOCK -e 'VAULT_DEV_ROOT_TOKEN_ID=spring-microservices-course' -p 8200:8200 vault:1.4.0

When using Spring Cloud Config together with the discovery we may choose between two available approaches called Config First Bootstrap and Discovery First Bootstrap. In Discovery First Bootstrap a config server is registering itself in discovery service. Thanks to that each microservice can localize a config server basing on its registration id.
Since a configuration is injected in the bootstrap phase we need to use bootstrap.yml for setting properties on the client-side. To enable “discovering” for the config server on the client side we should set the property spring.cloud.config.discovery.enabled to true. We should also override registered service id of config server if it is different from auto-configured configserver (in our case it is config-server). Of course, we can also use Consul as a configuration properties source.

spring:
  application:
    name: callme-service
  cloud:
    config:
      discovery:
        enabled: true
        serviceId: config-server
    consul:
      host: 192.168.99.100
      config:
        format: YAML

Inter-service communication

Currently, there are three Spring components for inter-service communication over HTTP that have integration with service discovery: synchronous RestTemplate, reactive WebClient and declarative REST client OpenFeign. The RestTemplate component is available within the Spring Web module, WebClient within Spring WebFlux. To include Spring Cloud OpenFeign we need to include a dedicated starter.

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>

To use RestTemplate or WebClient for communication with discovery support, we need to register the beans and annotate them with @LoadBalanced. It is also worth setting the proper timeouts for such communication, especially if you are not using a circuit breaker.

@SpringBootApplication
@EnableFeignClients
class InterCallerServiceApplication {

    @Bean
    @LoadBalanced
    fun template(): RestTemplate = RestTemplateBuilder()
        .setReadTimeout(Duration.ofMillis(100))
        .setConnectTimeout(Duration.ofMillis(100))
        .build()

    @Bean
    @LoadBalanced
    fun clientBuilder(): WebClient.Builder {
        val tcpClient: TcpClient = TcpClient.create()
            .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 100)
            .doOnConnected { conn ->
                conn.addHandlerLast(ReadTimeoutHandler(100, TimeUnit.MILLISECONDS))
            }
        val connector = ReactorClientHttpConnector(HttpClient.from(tcpClient))
        return WebClient.builder().clientConnector(connector)
    }
    
}

Although Ribbon has been moved to maintenance mode almost 1.5 years ago it is still used as a default client-side load balancer in the newest stable version of Spring Cloud. Since Spring Cloud Load Balancer is included in common dependencies it is available in your application. Therefore, the only thing you need to do is to disable Ribbon in the configuration using spring.cloud.loadbalancer.ribbon.enabled property.
Currently, we don’t have many options for load balancer customization. One of them is the ability to configure client cache settings. By default, each client is caching the list of target services and refreshing them every 30 seconds. Such an interval may be too long in your situation.
We can easily change it in the configuration as shown below and set it to for example 1 second. If your load balancer is integrated with Eureka discovery you also need to decrease an interval of the fetching registry, which is by default 30 seconds. After those, both changes your client can refresh the list of currently running services almost immediately.

spring:
  cloud:
    loadbalancer:
      cache:
        ttl: 1s
      ribbon:
        enabled: false
eureka:
  client:
    registryFetchIntervalSeconds: 1

Circuit breaker

The circuit breaker is a popular design pattern used in a microservices architecture. It is designed to detect failures and encapsulates the logic of preventing a failure from constantly recurring. Spring Cloud provides an abstraction for using different circuit breaker implementations. Currently, we may use Netflix Hystrix, Sentinel, Spring Retry, and Resilience4J. To enable Spring Cloud Circuit Breaker based on Resilience4J we need to include the following dependency.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-circuitbreaker-resilience4j</artifactId>
</dependency>

Here’s the code responsible for registering Customizer bean, that configures a circuit breaker behavior.

@Bean
fun defaultCustomizer(): Customizer<Resilience4JCircuitBreakerFactory> {
  return Customizer { factory: Resilience4JCircuitBreakerFactory ->
    factory.configureDefault { id: String? ->
      Resilience4JConfigBuilder(id)
        .timeLimiterConfig(TimeLimiterConfig.custom()
          .timeoutDuration(Duration.ofMillis(500))
          .build())
        .circuitBreakerConfig(CircuitBreakerConfig.custom()
          .slidingWindowSize(10)
          .failureRateThreshold(33.3F)
          .slowCallRateThreshold(33.3F)
        .build())
      .build()
    }
  }
}

The settings of the circuit breaker have been visualized in the picture below. The sliding window size sets the number of requests which are used for calculating the error rate. If we have more than 3 errors in the window of size 10 a circuit is open.

In the last step, we need to create a circuit breaker instance using Resilience4JCircuitBreakerFactory bean and enable it for the HTTP client as shown below.

@RestController
@RequestMapping("/caller")
class CallerController(private val template: RestTemplate, private val factory: Resilience4JCircuitBreakerFactory) {

  private var id: Int = 0

  @PostMapping("/random-send/{message}")
  fun randomSend(@PathVariable message: String): CallmeResponse? {
    val request = CallmeRequest(++id, message)
    val circuit = factory.create("random-circuit")
    return circuit.run { template.postForObject("http://inter-callme-service/callme/random-call",
      request, CallmeResponse::class.java) }
  }
    
}

Spring Cloud API gateway

The last missing element in our microservices architecture is an API Gateway. Spring Cloud Gateway is the project that helps us in implementing such a component. Currently, it is the second most popular Spring Cloud project just after Spring Cloud Netflix. It has around 2k stars on GitHub. It is built on top of the Spring WebFlux and Reactor project. It works reactively and requires Netty as a runtime framework.
The main goal of API gateway is to hide the complexity of the microservices system from an external client by providing an effective way of routing to APIs, but it can also solve some problems around security or resiliency. The main component used for configuring Spring Cloud Gateway is a route.
It is defined by an ID, a destination URI, a collection of predicates, and a collection of filters. A route is matched if the aggregate predicate is true. With filters, you can modify requests and responses before or after sending the downstream request.
With a predefined set of gateway filters, we may enable such mechanisms like path rewriting, rate limiting, discovery client, circuit breaking, fallback, or routing metrics. To enable all these features on the gateway we first need to include the following dependencies.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
   <groupId>org.jetbrains.kotlin</groupId>
   <artifactId>kotlin-reflect</artifactId>
</dependency>
<dependency>
   <groupId>org.jetbrains.kotlin</groupId>
   <artifactId>kotlin-stdlib</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-data-redis-reactive</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-circuitbreaker-reactor-resilience4j</artifactId>
</dependency>

To enable all the previously listed features we don’t have to implement much code. Almost everything is configurable in application properties.

spring:
  application:
    name: api-gateway
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true 
          lowerCaseServiceId: true
      routes:
        - id: inter-callme-service
          uri: lb://inter-callme-service
          predicates:
            - Path=/api/callme/**
          filters:
            - RewritePath=/api(?/?.*), $\{path}
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 20
                redis-rate-limiter.burstCapacity: 40
            - name: CircuitBreaker
              args:
                name: sampleSlowCircuitBreaker
                fallbackUri: forward:/fallback/test
        - id: inter-caller-service
          uri: lb://inter-caller-service
          predicates:
            - Path=/api/caller/**
          filters:
            - StripPrefix=1
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 20
                redis-rate-limiter.burstCapacity: 40
    loadbalancer:
      ribbon:
        enabled: false
  redis:
    host: 192.168.99.100

management:
  endpoints.web.exposure.include: '*'
  endpoint:
    health:
      show-details: always

Some settings still need to be configured in the code. It is a configuration of the circuit breaker, that is based on Resilience4J project, where we need to register the bean Customizer. We also have to define a key for rate-limiting responsible setting a strategy of choosing requests for counting the limits.

@SpringBootApplication
class ApiGatewayApplication {

  @Bean
  fun keyResolver(): KeyResolver = KeyResolver { _ -> Mono.just("1") }

  @Bean
  fun defaultCustomizer(): Customizer<ReactiveResilience4JCircuitBreakerFactory> {
    return Customizer { factory: ReactiveResilience4JCircuitBreakerFactory ->
      factory.configureDefault { id: String? ->
        Resilience4JConfigBuilder(id)
          .timeLimiterConfig(TimeLimiterConfig.custom()
            .timeoutDuration(Duration.ofMillis(500))
            .build())
          .circuitBreakerConfig(CircuitBreakerConfig.custom()
            .slidingWindowSize(10)
            .failureRateThreshold(33.3F)
            .slowCallRateThreshold(33.3F)
            .build())
          .build()
      }
    }
  }

}

fun main(args: Array<String>) {
   runApplication<ApiGatewayApplication>(*args)
}

Conclusion

In this article, you may take a quick introduction to using the latest Spring Cloud components for building microservices architecture. For more details, you may refer to my video course published on YouTube.

The post A New Era Of Spring Cloud appeared first on Piotr's TechBlog.