In general, we will cover the following topics in this article:

• Using Spring Boot 3 in cloud-native development
• Provide service discovery for all microservices with Spring Cloud Netflix Eureka. Anticipating your questions – yes, Eureka is still there. It’s the last of Netflix microservices components still available in Spring Cloud
• Spring Cloud OpenFeign in inter-service communication
• Distributed configuration with Spring Cloud Config
• API Gateway pattern with Spring Cloud Gateway including a global OpenAPI documentation with the Springdoc project
• Collecting traces with Micrometer OpenTelemetry and Zipkin

Fortunately, the migration from Spring Boot 2 to 3 is not a painful process. You can even check it out in my example repository, which was originally written in Spring Boot 2. The list of changes is not large. However, times have changed during the last five years… And we will begin our considerations from that point.

Running Environment

Here are the results of my quick 1-day voting poll run on Twitter. I assume that those results are meaningful since around 900 people voted. As you probably expect, currently, the first-choice platform for running your Spring Boot microservices is Kubernetes. I don’t have a survey conducted five years ago, but the results would probably be significantly different. Even if you had Kubernetes in your organization 5 years ago, you were probably starting a migration of your apps or at least it was in progress. Of course, there might be some exceptions, but I’m thinking about the vast majority.

You could migrate to Kubernetes during that time, but also Kubernetes ecosystem has changed a lot. There are many useful tools and platform services you may easily integrate with your apps. We can at least mention Kubernetes native solutions like service mesh (e.g. Istio) or serverless (e.g. Knative). The main question here is: if I’m running microservices on Kubernetes are Spring Cloud components still relevant? The answer is: in most cases no. Of course, you can still use Eureka for service discovery, Spring Cloud Config for a distributed configuration, or Spring Cloud Gateway for the API gateway pattern. However, you can easily replace them with Kubernetes built-in mechanisms and additional platform services.

To conclude, this article is not aimed at Kubernetes users. It shows how to easily run microservices architecture anywhere. If you are looking for staff mainly related to Kubernetes you can read my articles about the best practices for Java apps and microservices there.

Source Code

If you would like to try it by yourself, you may always take a look at my source code. In order to do that you need to clone my GitHub repository. Then you should just follow my instructions.

Before we proceed to the source code, let’s take a look at the following diagram. It illustrates the architecture of our sample system. We have three independent Spring Boot 3 microservices, which register themself in service discovery, fetch properties from the configuration service, and communicate with each other. The whole system is hidden behind the API gateway. Our Spring Boot 3 microservices send traces to the Zipkin instance using the Micrometer OTEL project.

Currently, the newest version of Spring Cloud is 2022.0.1. This version of spring-cloud-dependencies should be declared as a BOM for dependency management.

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-dependencies</artifactId>
      <version>2022.0.1</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

Step 1: Configuration Server with Spring Cloud Config

To enable Spring Cloud Config feature for an application, we should first include spring-cloud-config-server to your project dependencies.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-config-server</artifactId>
</dependency>

Then enable running the embedded configuration server during application boot use @EnableConfigServer annotation.

@SpringBootApplication
@EnableConfigServer
public class ConfigApplication {

   public static void main(String[] args) {
      new SpringApplicationBuilder(ConfigApplication.class).run(args);
   }

}

By default Spring Cloud Config Server stores the configuration data inside the Git repository. We will change that behavior by activating the native mode. In this mode, Spring Cloud Config Server reads property sources from the classpath. We place all the YAML property files inside src/main/resources/config. Here’s the config server application.yml file. It activates the native mode and overrides a default port to 8088.

server:
  port: 8088
spring:
  profiles:
    active: native

The YAML filename will be the same as the name of the service. For example, the YAML file of discovery-service is located here: src/main/resources/config/discovery-service.yml. Besides a default profile, we will also define the custom docker profile. Therefore the name of the config file will contain the docker suffix. On the default profile, we are connecting services through localhost with dynamically assigned ports. So, the typical configuration file for the default profile will look like that:

server:
  port: 0

eureka:
  client:
    serviceUrl:
      defaultZone: http://localhost:8061/eureka/

Here’s the typical configuration file for the default profile:

server:
  port: 8080

eureka:
  client:
    serviceUrl:
      defaultZone: http://discovery-service:8061/eureka/

In order to connect the config server on the client side we need to include the following module in Maven dependencies:

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-config</artifactId>
</dependency>

Depending on the running environment (localhost or docker) we need to provide different addresses for the config server:

spring:
  config:
    import: "optional:configserver:http://config-service:8088"
    activate:
      on-profile: docker
---
spring:
  application:
    name: discovery-service
  config:
    import: "optional:configserver:http://localhost:8088"

Step 2: Discovery Server with Spring Cloud Netflix Eureka

Of course, you can replace Eureka with any other discovery server supported by Spring Cloud. It can be Consul, Alibaba Nacos, or Zookeeper. The best way to run the Eureka server is just to embed it into the Spring Boot app. In order to do that, we first need to include the following Maven dependency:

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>

Then we need to set the @EnableEurekaServer annotation on the main class.

@SpringBootApplication
@EnableEurekaServer
public class DiscoveryApplication {

   public static void main(String[] args) {
      new SpringApplicationBuilder(DiscoveryApplication.class).run(args);
   }

}

There is nothing new with that. As I already mentioned, the configuration files, discovery-service.yml or discovery-service-docker.yml, should be placed inside config-service module. We have changed Eureka’s running port from the default value (8761) to 8061. For the standalone Eureka instance, we have to disable registration and omit to fetch the registry. We just want to activate a single-node, demo discovery server.

server:
  port: 8061

eureka:
  instance:
    hostname: localhost
  client:
    registerWithEureka: false
    fetchRegistry: false
    serviceUrl:
      defaultZone: http://${eureka.instance.hostname}:${server.port}/eureka/

Once you have successfully started the application you may visit Eureka Dashboard available under the address http://localhost:8061/.

Step 3: Build Apps with Spring Boot 3 and Spring Cloud

Let’s take a look at a list of required Maven modules for our microservices. Each app has to get a configuration from the config-service and needs to register itself in the discovery-service. It also exposes REST API, automatically generates API documentation, and export tracing info to the Zipkin instance. We use the springdoc-openapi v2 library dedicated to Spring Boot 3. It generates documentation in both JSON and YAML formats available under the v3/api-docs path (or /v3/api-docs.yaml for the YAML format). In order to export traces to the Zipkin server, we will include the opentelemetry-exporter-zipkin module.

<dependencies>
  <dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-config</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
  </dependency>
  <dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-tracing-bridge-otel</artifactId>
  </dependency>
  <dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-exporter-zipkin</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springdoc</groupId>
    <artifactId>springdoc-openapi-starter-webmvc-api</artifactId>
    <version>2.0.2</version>
  </dependency>
</dependencies>

For the apps that call other services, we also need to include a declarative REST client. We will use Spring Cloud OpenFeign.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>

OpenFeign client automatically integrates with the service discovery. We just to set the name under which it is registered in Eureka inside the @FeingClient annotation. In order to create a client, we need to define an interface containing all the endpoints it has to call.

@FeignClient(name = "employee-service")
public interface EmployeeClient {

   @GetMapping("/organization/{organizationId}")
   List<Employee> findByOrganization(@PathVariable("organizationId") Long organizationId);
	
}

During the demo, we will send all the traces to Zipkin. It requires setting the value of the probability parameter to 1.0. In order to override the default URL of Zipkin we need to use the management.zipkin.tracing.endpoint property.

management:
  tracing:
    sampling:
      probability: 1.0
  zipkin:
    tracing:
      endpoint: http://zipkin:9411/api/v2/spans

Here’s the implementation of the @RestController in department-service. It injects the repository bean to interact with the database, and the Feign client bean to communicate with employee-service. The rest of the code is pretty simple.

@RestController
public class DepartmentController {

  private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);

  DepartmentRepository repository;
  EmployeeClient employeeClient;

  public DepartmentController(DepartmentRepository repository, EmployeeClient employeeClient) {
    this.repository = repository;
    this.employeeClient = employeeClient;
  }

  @PostMapping("/")
  public Department add(@RequestBody Department department) {
    LOGGER.info("Department add: {}", department);
    return repository.add(department);
  }
	
  @GetMapping("/{id}")
  public Department findById(@PathVariable("id") Long id) {
    LOGGER.info("Department find: id={}", id);
    return repository.findById(id);
  }
	
  @GetMapping("/")
  public List<Department> findAll() {
    LOGGER.info("Department find");
    return repository.findAll();
  }
	
  @GetMapping("/organization/{organizationId}")
  public List<Department> findByOrganization(@PathVariable("organizationId") Long organizationId) {
    LOGGER.info("Department find: organizationId={}", organizationId);
    return repository.findByOrganization(organizationId);
  }
	
  @GetMapping("/organization/{organizationId}/with-employees")
  public List<Department> findByOrganizationWithEmployees(@PathVariable("organizationId") Long organizationId) {
    LOGGER.info("Department find: organizationId={}", organizationId);
    List<Department> departments = repository.findByOrganization(organizationId);
    departments.forEach(d -> d.setEmployees(employeeClient.findByDepartment(d.getId())));
    return departments;
  }
	
}

As you see there are almost no differences in the app implementation between Spring Boot 2 and 3. The only thing you would have to do is to change all the javax.persistence to the jakarta.persistance.

Step 4: API Gateway with Spring Cloud Gateway

A gateway-service is the last app in our microservices architecture with Spring Boot 3. Beginning from Spring Boot 2 Spring Cloud Gateway replaced Netflix Zuul. We can also install it on Kubernetes using, for example, the Helm chart provided by VMWare Tanzu.

We will create a separate application with the embedded gateway. In order to do that we need to include Spring Cloud Gateway Starter in the Maven dependencies. Since our gateway has to interact with discovery and config services, it also includes Eureka Client Starter and Spring Cloud Config Starter. We don’t want to use it just as a proxy to the downstream services, but also we expose there OpenAPI documentation generated by all the apps. Since Spring Cloud Gateway is built on top of Spring WebFlux, we need to include Springdoc starters dedicated to that project.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-config</artifactId>
</dependency>
<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-tracing-bridge-otel</artifactId>
</dependency>
<dependency>
  <groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-exporter-zipkin</artifactId>
</dependency>
<dependency>
  <groupId>org.springdoc</groupId>
  <artifactId>springdoc-openapi-starter-webflux-api</artifactId>
  <version>2.0.2</version>
</dependency>
<dependency>
  <groupId>org.springdoc</groupId>
  <artifactId>springdoc-openapi-starter-webflux-ui</artifactId>
  <version>2.0.2</version>
</dependency>

In order to expose OpenAPI documentation from multiple v3/api-docs endpoints we need to use the GroupedOpenApi object. It should provide a way to switch between documentation generated by employee-service, department-service and organization-service. Those services run on dynamic addresses (or at least random ports). In that case, we will use the RouteDefinitionLocator bean to grab the current URL of each service. Then we just need to filter a list of routes to find only those related to our three microservices. Finally, we create the GroupedOpenApi containing a service name and path.

@SpringBootApplication
public class GatewayApplication {

   private static final Logger LOGGER = LoggerFactory
      .getLogger(GatewayApplication.class);

   public static void main(String[] args) {
      SpringApplication.run(GatewayApplication.class, args);
   }

   @Autowired
   RouteDefinitionLocator locator;

   @Bean
   public List<GroupedOpenApi> apis() {
      List<GroupedOpenApi> groups = new ArrayList<>();
      List<RouteDefinition> definitions = locator
         .getRouteDefinitions().collectList().block();
      assert definitions != null;
      definitions.stream().filter(routeDefinition -> routeDefinition
         .getId()
         .matches(".*-service"))
         .forEach(routeDefinition -> {
            String name = routeDefinition.getId()
               .replaceAll("-service", "");
            groups.add(GroupedOpenApi.builder()
               .pathsToMatch("/" + name + "/**").group(name).build());
         });
      return groups;
   }

}

Here’s the configuration of gateway-service. We should enable integration with the discovery server by setting the property spring.cloud.gateway.discovery.locator.enabled to true. Then we may proceed to define the route rules. We use the Path Route Predicate Factory for matching the incoming requests, and the RewritePath GatewayFilter Factory for modifying the requested path to adapt it to the format exposed by downstream services. The uri parameter specifies the name of the target service registered in the discovery server. For example, organization-service is available on the gateway under the /organization/** path thanks to the predicate Path=/organization/**, and the rewrite path from /organization/** to the /**.

spring:
  output:
    ansi:
      enabled: always
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true
      routes:
      - id: employee-service
        uri: lb://employee-service
        predicates:
        - Path=/employee/**
        filters:
        - RewritePath=/employee/(?<path>.*), /$\{path}
      - id: department-service
        uri: lb://department-service
        predicates:
        - Path=/department/**
        filters:
        - RewritePath=/department/(?<path>.*), /$\{path}
      - id: organization-service
        uri: lb://organization-service
        predicates:
        - Path=/organization/**
        filters:
        - RewritePath=/organization/(?<path>.*), /$\{path}
      - id: openapi
        uri: http://localhost:${server.port}
        predicates:
        - Path=/v3/api-docs/**
        filters:
        - RewritePath=/v3/api-docs/(?<path>.*), /$\{path}/v3/api-docs

springdoc:
  swagger-ui:
    urls:
      - name: employee
        url: /v3/api-docs/employee
      - name: department
        url: /v3/api-docs/department
      - name: organization
        url: /v3/api-docs/organization

As you see above, we are also creating a dedicated route for Springdoc OpenAPI. It rewrites the path for the /v3/api-docs context to serve it properly in the Swagger UI.

Step 5: Running Spring Boot 3 Microservices

Finally, we can run all our microservices. With the current configuration in the repository, you can start them directly on your laptop or with Docker containers.

Option 1: Starting directly on the laptop

In total, we have 6 apps to run: 3 microservices, a discovery server, a config server, and a gateway. We also need to run Zipkin to collect and store traces from communication between the services. In the first step, we should start the config-service. We can use Spring Boot Maven plugin for that. Just go to the config-service directory and the following command. It is exposed on the 8088 port.

$ mvn spring-boot:run

We should repeat the same step for all the other apps. The discovery-service is listening on the 8061 port, while the gateway-service on the 8060 port. Microservices will start on the dynamically generated port number thanks to the server.port=0 property in config. In the final step, we can run Zipkin using its Docker container with the following command:

$ docker run -d --name zipkin -p 9411:9411 openzipkin/zipkin

Option 2: Build images and run them with Docker Compose

In the first step, we will build the whole Maven project and Docker images for all the apps. I created a profile build-image that needs to be activated to build images. It mostly uses the build-image step provided by the Spring Boot Maven Plugin. However, for config-service and discovery-service I’m using Jib because it is built on top of the base image with curl installed. For both these services Docker compose needs to verify health checks before starting other containers.

$ mvn clean package -Pbuild-image

The docker-compose.yml is available in the repository root directory. The whole file is visible below. We need to run config-service before all other apps since it provides property sources. Secondly, we should start discovery-service. In both these cases, we are defining a health check that tests the HTTP endpoint using curl inside the container. Once we start and verify config-service and discovery-service we may run gateway-service and all the microservices. All the apps are running with the docker Spring profile activated thanks to the SPRING_PROFILES_ACTIVE environment variable. It corresponds to the spring.profiles.active param that may be defined in configuration properties.

version: "3.7"
services:
  zipkin:
    container_name: zipkin
    image: openzipkin/zipkin
    ports:
      - "9411:9411"
  config-service:
    image: piomin/config-service:1.1-SNAPSHOT
    ports:
      - "8088:8088"
    healthcheck:
      test: curl --fail http://localhost:8088/employee/docker || exit 1
      interval: 5s
      timeout: 2s
      retries: 3
  discovery-service:
    image: piomin/discovery-service:1.1-SNAPSHOT
    ports:
      - "8061:8061"
    depends_on:
      config-service:
        condition: service_healthy
    links:
      - config-service
    healthcheck:
      test: curl --fail http://localhost:8061/eureka/v2/apps || exit 1
      interval: 4s
      timeout: 2s
      retries: 3
    environment:
      SPRING_PROFILES_ACTIVE: docker
  employee-service:
    image: piomin/employee-service:1.2-SNAPSHOT
    ports:
      - "8080"
    depends_on:
      discovery-service:
        condition: service_healthy
    links:
      - config-service
      - discovery-service
      - zipkin
    environment:
      SPRING_PROFILES_ACTIVE: docker
  department-service:
    image: piomin/department-service:1.2-SNAPSHOT
    ports:
      - "8080"
    depends_on:
      discovery-service:
        condition: service_healthy
    links:
      - config-service
      - discovery-service
      - employee-service
      - zipkin
    environment:
      SPRING_PROFILES_ACTIVE: docker
  organization-service:
    image: piomin/organization-service:1.2-SNAPSHOT
    ports:
      - "8080"
    depends_on:
      discovery-service:
        condition: service_healthy
    links:
      - config-service
      - discovery-service
      - employee-service
      - department-service
      - zipkin
    environment:
      SPRING_PROFILES_ACTIVE: docker
  gateway-service:
    image: piomin/gateway-service:1.1-SNAPSHOT
    ports:
      - "8060:8060"
    depends_on:
      discovery-service:
        condition: service_healthy
    environment:
      SPRING_PROFILES_ACTIVE: docker
    links:
      - config-service
      - discovery-service
      - employee-service
      - department-service
      - organization-service
      - zipkin

Finally, let’s run all the apps using Docker Compose:

$ docker-compose up

Try it out

Once you start all the apps you can perform some test calls to the services through the gateway-service. It listening on the 8060 port. There is some test data automatically generated during startup. You can call the following endpoint to test all the services and communication between them:

$ curl http://localhost:8060/employee/
$ curl http://localhost:8060/department/organization/1
$ curl http://localhost:8060/department/organization/1/with-employees
$ curl http://localhost:8060/organization/
$ curl http://localhost:8060/organization/1/with-departments

Here are the logs generated by the apps during the calls visible above:

Let’s display Swagger UI exposed on the gateway. You can easily switch between contexts for all three microservices as you see below:

We can go to the Zipkin dashboard to verify the collected traces:

Final Thoughts

Treat this article as a quick guide to the most common components related to microservices with Spring Boot 3. I focused on showing you some new features since my last article on this topic. You could read how to implement tracing with Micrometer OpenTelemetry, generate API docs with Springdoc, or build Docker images with Spring Boot Maven Plugin.

The post Microservices with Spring Boot 3 and Spring Cloud appeared first on Piotr's TechBlog.

Before we start with a list of security “golden rules”, let’s analyze a typical microservices architecture. We will focus on components important for building a secure solution.

The picture visible below illustrates a typical microservices architecture built with Spring Cloud. There is an API gateway built on top of Spring Cloud Gateway. Since it is an entry point to our system, we will enable some important security mechanisms on it. There are several microservices hidden behind the gateway. There is also a discovery server, which allows localizing IP addresses using the name of services. And finally, there are some components that do not take part in communication directly. It is just a proposition of a few selected tools. You may choose other solutions providing the same features. Vault is a tool for securely storing and accessing secrets. Keycloak is an open-source identity and access management solution. Spring Cloud Config Server provides an HTTP API for external configuration. It may integrate with several third-party tools including Vault.

Let’s begin. Here’s our list of Spring Security best practices.

1. Enable rate limiting on the API gateway

An API gateway is an important pattern in microservice-based architecture. It acts as a single entry point into the whole system. It is responsible not only for request routing but also for several other things including security. Consequently, one of the most essential components of security we should enable on the gateway is rate limiting. It protects your API against DoS attacks, which can tank a server with unlimited HTTP requests.

Spring provides its own implementation of the API gateway pattern called Spring Cloud Gateway. On the other hand, Spring Cloud Gateway comes with a built-in implementation of a rate-limiting component. To sum up, you just need to include a single dependency to build a gateway application. Then you have to provide some configuration settings to enable rate limiting for a single route.

In order to enable a rate limiter on a gateway, we need to use a component called RequestRateLimiter GatewayFilter factory. It uses a RateLimiter implementation to determine if the current request is allowed to proceed. Otherwise, the gateway returns a status of HTTP 429 - Too Many Requests. The RequestRateLimiter implementation uses Redis as a backend.

Let’s take a look at a typical configuration of routes handled by Spring Cloud Gateway. There are three parameters that can be used to configure the rate limiter: replenishRate, burstCapacity, and requestedTokens. The replenishRate property is how many requests per second a single user may send. The burstCapacity property is the maximum number of requests a user can send in a single second. With the requestedTokens property, we may set the cost of a single token.

spring:
  cloud:
    gateway:
      routes:
        - id: account-service
          uri: http://localhost:8090
          predicates:
            - Path=/account/**
          filters:
            - RewritePath=/account/(?<path>.*), /$\{path}
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 1
                redis-rate-limiter.burstCapacity: 60
                redis-rate-limiter.requestedTokens: 15

Then we should define a KeyResolver bean. A rate limiter defines all parameters per a single key returned by the resolver.

@Bean
KeyResolver authUserKeyResolver() {
   return exchange -> ReactiveSecurityContextHolder.getContext()
            .map(ctx -> ctx.getAuthentication().getPrincipal().toString());
}

For more details, you may refer to the article Secure Rate Limiting with Spring Cloud Gateway.

2. Generate and propagate certificates dynamically

Should we use SSL in microservice-to-microservice communication? Of course yes. But the question is how will you handle certificates used by your microservices. There are several best practices related to SSL certificate management. For example, you should not issue certificates for long time periods. You should also automatically renew or refresh them.

There are some tools that can help in following best practices. One of the most popular of them is Vault from Hashicorp. It provides the PKI secrets engine, which is responsible for generating dynamic X.509 certificates. The simplest way to try Vault is to run it locally on a Docker container.

$ docker run --cap-add=IPC_LOCK -d --name vault -p 8200:8200 vault

Then, we may enable and configure the PKI engine in some simple steps. You can do it using Vault CLI or UI. For some more detailed information about it read my article SSL with Spring WebFlux and Vault PKI. For now, let’s enable PKI with TTL and then configure CA using CLI as shown below.

$ vault secrets enable pki
$ vault secrets tune -max-lease-ttl=8760h pki
$ vault write pki/root/generate/internal \
    common_name=my-website.com \
    ttl=8760h

In the next step, we will use Spring VaultTemplate to issue a certificate dynamically. The fragment of code visible below shows how to create a certificate request with 12h TTL and localhost as a Common Name. Firstly, let’s build such a request using the VaultCertificateRequest object. Then we will invoke the issueCertificate method on the VaultPkiOperations object. The generated CertificateBundle contains both a certificate and a private key.

private CertificateBundle issueCertificate() throws Exception {
   VaultPkiOperations pkiOperations = vaultTemplate.opsForPki("pki");
   VaultCertificateRequest request = VaultCertificateRequest.builder()
        .ttl(Duration.ofHours(12))
        .commonName("localhost")
        .build();
   VaultCertificateResponse response = pkiOperations
      .issueCertificate("default", request);
   CertificateBundle certificateBundle = response.getRequiredData();
   return certificateBundle;
}

Finally, we just need to use the method for generating a certificate in Vault on runtime. The default behavior of our web server needs to be overridden. To do that, we need to create a Spring @Component that implements WebServerFactoryCustomizer. Depending on the web server we need to customize a different WebServerFactory. Typically, for Spring MVC it is Tomcat and Netty for Spring WebFlux. Inside the customize method, we are generating a certificate and store it inside the keystore (cert + private key) and truststore (CA).

@Component
public class GatewayServerCustomizer implements 
         WebServerFactoryCustomizer<NettyReactiveWebServerFactory> {
   @SneakyThrows
   @Override
   public void customize(NettyReactiveWebServerFactory factory) {
      String keyAlias = "vault";
      CertificateBundle bundle = issueCertificate();
      KeyStore keyStore = bundle.createKeyStore(keyAlias);
      String keyStorePath = saveKeyStoreToFile("server-key.pkcs12", keyStore);
      Ssl ssl = new Ssl();
      ssl.setEnabled(true);
      ssl.setClientAuth(Ssl.ClientAuth.NEED);
      ssl.setKeyStore(keyStorePath);
      ssl.setKeyAlias(keyAlias);
      ssl.setKeyStoreType(keyStore.getType());
      ssl.setKeyPassword("");
      ssl.setKeyStorePassword("123456");
      X509Certificate caCert = bundle.getX509IssuerCertificate();
      log.info("CA-SerialNumber: {}", caCert.getSerialNumber());
      KeyStore trustStore = KeyStore.getInstance("pkcs12");
      trustStore.load(null, null);
      trustStore.setCertificateEntry("ca", caCert);
      String trustStorePath = saveKeyStoreToFile("server-trust.pkcs12", trustStore);
      ssl.setTrustStore(trustStorePath);
      ssl.setTrustStorePassword("123456");
      ssl.setTrustStoreType(trustStore.getType());
      factory.setSsl(ssl);
      factory.setPort(8443);
   }
}

3. Use SSL in microservices communication

Since using SSL on the edge of a microservices-based system is obvious, inter-service communication is sometimes considered to be non-secure. My recommendation is always the same. Better use SSL or not But we should think about securing at least components that store sensitive data. One of them will probably be a config server. That’s obviously one of the Spring Security best practices. For Spring Boot microservices we can use a component called Spring Cloud Config Server. Since it is built on top of Spring MVC we may easily enable a secure connection on the server-side.

server:
  port: ${PORT:8888}
  ssl:
    enabled: true
    client-auth: need
    key-store: classpath: server-key.jks
    key-store-password: 123456
    key-alias: configserver
    trust-store: classpath: server-trust.jks
    trust-store-password: 123456

On the client side, we use a component called Spring Cloud Config Client. Since it is responsible for connecting with the server, we also need to handle SSL there. To do that we need to override the RestTemplate SSL configuration on the ConfigServicePropertySourceLocator bean. The fragment of code visible below uses a self-signed certificate, but we can easily implement here a strategy described in the previous section.

@Configuration
public class SSLConfigServiceBootstrapConfiguration {
   @Autowired
   ConfigClientProperties properties;
   @Bean
   public ConfigServicePropertySourceLocator configServicePropertySourceLocator() throws Exception {
      final char[] password = "123456".toCharArray();
      final ClassPathResource resource = new ClassPathResource("account.jks");
      SSLContext sslContext = SSLContexts.custom()
         .loadKeyMaterial(resource.getFile(), password, password)
         .loadTrustMaterial(resource.getFile(), password, new TrustSelfSignedStrategy()).build();
      CloseableHttpClient httpClient = HttpClients.custom()
         .setSSLContext(sslContext)
         .setSSLHostnameVerifier((s, sslSession) -> true)
         .build();
      HttpComponentsClientHttpRequestFactory requestFactory = new HttpComponentsClientHttpRequestFactory(httpClient);
      ConfigServicePropertySourceLocator configServicePropertySourceLocator = new ConfigServicePropertySourceLocator(properties);
      configServicePropertySourceLocator.setRestTemplate(new RestTemplate(requestFactory));
      return configServicePropertySourceLocator;
   }
}

In the latest version of Spring Cloud Config, we can enable TLS traffic encryption in configuration. We just need to define the right settings using properties with a prefix spring.cloud.config.tls.*.

What about encrypting communication between applications and a discovery server? You can choose between several available discovery servers supported in Spring Cloud. But let’s assume we use Eureka. Similarly to Spring Cloud Config, we use a high-level client to communicate with a server. So, in that case, we need to define a bean DiscoveryClientOptionalArgs, and also override SSL settings on the HTTP client there. The Eureka client uses the Jersey HTTP client, so we need to create an instance of EurekaJerseyClientBuilder to override the SSL configuration.

@Bean
public DiscoveryClient.DiscoveryClientOptionalArgs discoveryClientOptionalArgs() throws Exception {
   DiscoveryClient.DiscoveryClientOptionalArgs args = new DiscoveryClient.DiscoveryClientOptionalArgs();
   final char[] password = "123456".toCharArray();
   final ClassPathResource resource = new ClassPathResource("account.jks");
   SSLContext sslContext = SSLContexts.custom()
	.loadKeyMaterial(resource.getFile(), password, password)
	.loadTrustMaterial(resource.getFile(), password, new TrustSelfSignedStrategy()).build();
   EurekaJerseyClientBuilder builder = new EurekaJerseyClientBuilder();	 
   builder.withClientName("account-client");
   builder.withMaxTotalConnections(10);
   builder.withMaxConnectionsPerHost(10);
   builder.withCustomSSL(sslContext);
   args.setEurekaJerseyClient(builder.build());
   return args;
}

Finally, we may configure HTTPS in communication between microservices. Since we use RestTemplate or WebClient instances directly on the client side it is relatively easy to implement secure communication in that case.

4. Keep configuration data encrypted

The current one of the best practices for Spring microservices security is related to a configuration server. We should encrypt at least sensitive data like passwords or secrets stored there. Spring Cloud Config Server provides a built-in mechanism for that. But we can also use Vault as a backend store for Spring Cloud Config Server, where all data is encrypted by default.

We will start with a default encrypt mechanism provided by Spring Cloud Config Server. Firstly, we need to enable it in the configuration properties.

spring:
  cloud:
    config:
      server:
        encrypt:
          enabled: true

Then, we have to configure a key store responsible for encrypting our sensitive data.

encrypt:
  keyStore:
    location: classpath:/config.jks
    password: 123456
    alias: config
    secret: 123456

Finally, we can set encrypted data instead of plain string with {cipher} prefix.

spring:  
  application:
    name: account-service
  security:
    user:
      password: '{cipher}AQBhpDVYHANrg59OGY7ioSbMdOrH7ZA0vfa2VqIvfxJK5vQp...'

Alternatively, you can use it as a configuration data backend. To do that you should enable a Spring profile called vault.

spring.profiles.active=vault

Then, we may add an example secret.

$ vault write secret/hello value=world
$ vault read secret/hello

5. Restrict access to the API resources

In the previous sections, we discussed such topics as authentication, traffic, and data encryption. But another important aspect of securing your applications is authorization and access to the API resources. If you think about web app authorization, the first approach that probably comes to your mind is OAuth 2.0 or OpenID Connect. OAuth 2.0 is the industry-standard protocol for authorization. Of course, it is supported by Spring Security. There are also multiple OAuth2 providers you can integrate your application with. One of them is Keycloak. I will use it in the example in this article. Firstly, let’s run Keycloak on a Docker container. By default, it exposes API and a web console on the port 8080.

$ docker run -d --name keycloak -p 8888:8080 \
   -e KEYCLOAK_USER=spring \
   -e KEYCLOAK_PASSWORD=spring123 \
   jboss/keycloak

We are going to enable and configure OAuth 2.0 support on the API gateway. Besides spring-cloud-starter-gateway dependency, we need to include spring-boot-starter-oauth2-client and spring-cloud-starter-security to activate the TokenRelay filter. Then we have to provide the Spring Security configuration settings for the OAuth2 client.

spring:
  security:
    oauth2:
      client:
        provider:
          keycloak:
            token-uri: http://localhost:8080/auth/realms/master/protocol/openid-connect/token
            authorization-uri: http://localhost:8080/auth/realms/master/protocol/openid-connect/auth
            userinfo-uri: http://localhost:8080/auth/realms/master/protocol/openid-connect/userinfo
            user-name-attribute: preferred_username
        registration:
          keycloak-with-test-scope:
            provider: keycloak
            client-id: spring-with-test-scope
            client-secret: c6480137-1526-4c3e-aed3-295aabcb7609
            authorization-grant-type: authorization_code
            redirect-uri: "{baseUrl}/login/oauth2/code/keycloak"
          keycloak-without-test-scope:
            provider: keycloak
            client-id: spring-without-test-scope
            client-secret: f6fc369d-49ce-4132-8282-5b5d413eba23
            authorization-grant-type: authorization_code
            redirect-uri: "{baseUrl}/login/oauth2/code/keycloak"

In the last step, we need to configure the Spring Security filter. Since Spring Cloud Gateway is built on top of Spring WebFlux, we need to annotate the configuration bean with @EnableWebFluxSecurity. Inside the filterChain method we are going to enable authorization for all the exchanges. We will also set OAuth2 as a default login method and finally disable CSRF.

@Configuration
@EnableWebFluxSecurity
public class SecurityConfig {
   @Bean
   public SecurityWebFilterChain filterChain(ServerHttpSecurity http) {
      http.authorizeExchange(exchanges -> exchanges.anyExchange().authenticated())
         .oauth2Login(withDefaults());
      http.csrf().disable();
      return http.build();
   }
}

As I mentioned before, we will have a token relay pattern between the gateway and microservices. A Token Relay is where an OAuth2 consumer acts as a Client and forwards the incoming token to outgoing resource requests. So, now let’s enable global method security and OAuth2 resources server for the downstream services.

@Configuration
@EnableWebSecurity
@EnableGlobalMethodSecurity(prePostEnabled = true)
public class SecurityConfig extends WebSecurityConfigurerAdapter {
   protected void configure(HttpSecurity http) throws Exception {
      http.authorizeRequests(authorize -> authorize.anyRequest().authenticated())
         .oauth2ResourceServer(OAuth2ResourceServerConfigurer::jwt);
   }
}

After that, it is possible to configure role-based access using @PreAuthorize and @PostAuthorize. Let’s take a look at the implementation of the REST controller class. It is a single ping method. That method may be accessed only by the client with the TEST scope. For more implementation details you may refer to the article Spring Cloud Gateway OAuth2 with Keycloak.

@RestController
@RequestMapping("/callme")
public class CallmeController {
   @PreAuthorize("hasAuthority('SCOPE_TEST')")
   @GetMapping("/ping")
   public String ping() {
      SecurityContext context = SecurityContextHolder.getContext();
      Authentication authentication = context.getAuthentication();
      return "Scopes: " + authentication.getAuthorities();
   }
}

6. Dynamically generate credentials to the external systems

Does your application connect to external systems like databases or message brokers? How do you store the credentials used by your application? Of course, we can always encrypt sensitive data, but if we work with many microservices having separate databases it may not be a very comfortable solution. Here comes Vault with another handy mechanism. Its database secrets engine generates database credentials dynamically based on configured roles. We may also take advantage of dynamically generated credentials for RabbitMQ, Nomad, and Consul.

Firstly, let’s enable the Vault database engine, which is disabled by default.

$ vault secrets enable database

Let’s assume our application connects to the Postgres database. Therefore, we need to configure a Vault plugin for PostgreSQL database and then provide connection settings and credentials.

$ vault write database/config/postgres \
  plugin_name=postgresql-database-plugin \
  allowed_roles="default" \
  connection_url="postgresql://{{username}}:{{password}}@localhost:5432?sslmode=disable" \
  username="postgres" \
  password="postgres123456"

Then we need to create a database role. The name of the role should be the same as the name passed in field allowed_roles in the previous step. We also have to set a target database name and SQL statement that creates users with privileges.

$ vault write database/roles/default db_name=postgres \
  creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';GRANT SELECT, UPDATE, INSERT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";GRANT USAGE,  SELECT ON ALL SEQUENCES IN SCHEMA public TO \"{{name}}\";" \
  default_ttl="1h" max_ttl="24h"

Thanks to Spring Cloud Vault project we can easily integrate any Spring Boot application with the Vault databases engine. Two dependencies need to be included in Maven pom.xml to enable that support. Of course, we also need dependencies for the JPA and Postgres driver.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-vault-config</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-vault-config-databases</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
   <groupId>org.postgresql</groupId>
   <artifactId>postgresql</artifactId>
</dependency>

The only thing we have to do is to configure integration with Vault via Spring Cloud Vault. The following configuration settings should be placed in bootstrap.yml (no application.yml). You may consider running the application on Nomad.

spring:
  cloud:
    vault:
      uri: http://localhost:8200
      token: ${VAULT_TOKEN}
      postgresql:
        enabled: true
        role: default
        backend: database
  datasource:
    url: jdbc:postgresql://localhost:5432/posgtres

The important part of the configuration visible above is under the property spring.cloud.vault.postgresql. Following Spring Cloud documentation “Username and password are stored in spring.datasource.username and spring.datasource.password so using Spring Boot will pick up the generated credentials for your DataSource without further configuration”. For more details about integration between Spring Cloud and Vault database engine, you may refer to my article Secure Spring Cloud Microservices with Vault and Nomad.

7. Always be up to date

This one of the best practices may be applied anywhere not only as a rule to Spring microservices security. We usually use open-source libraries in our applications, so it is important to include the latest versions of them. They may contain critical updates for publicly disclosed vulnerabilities contained within a project’s dependencies. There are also several dependency scanners like Snyk or OWASP.

Final thoughts

That is my private list of best practices for Spring Boot microservices security. Of course, most of them are not related just to a single framework, and we apply them for any other framework or toolkit. Do you have your own list of Spring Security best practices? Don’t afraid to share it in the comments.

The post Spring Microservices Security Best Practices appeared first on Piotr's TechBlog.

You can easily configure rate limiting with Spring Cloud Gateway. For a basic introduction to this feature, you may refer to my article Rate Limiting in Spring Cloud Gateway with Redis. Similarly, today we will also use Redis as a backend for a rate limiter. Moreover, we will configure an HTTP basic authentication. Of course, you can provide some more advanced authentication mechanisms like an X509 certificate or OAuth2 login. If you think about it, read my article Spring Cloud Gateway OAuth2 with Keycloak.

Source Code

If you would like to try it by yourself, you may always take a look at my source code. In order to do that you need to clone my repository sample-spring-cloud-gateway. Then you should go to the src/test/java directory, and just follow my instructions in the next sections.

1. Dependencies

Let’s start with dependencies. Since we will create an integration test, we need some additional libraries. Firstly, we will use the Testcontainers library. It allows us to run Docker containers during the JUnit test. We will use it for running Redis and a mock server, which is responsible for mocking a downstream service. Of course, we need to include a starter with Spring Cloud Gateway and Spring Data Redis. To implement an HTTP basic authentication we also need to include Spring Security. Here’s a full list of required dependencies in Maven pom.xml.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-data-redis-reactive</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-security</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-test</artifactId>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.testcontainers</groupId>
   <artifactId>mockserver</artifactId>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.mock-server</groupId>
   <artifactId>mockserver-client-java</artifactId>
   <scope>test</scope>
</dependency>

2. Configure an HTTP Basic Authentication

In order to configure an HTTP basic authentication, we need to create the @Configuration bean annotated with @EnableWebFluxSecurity. That’s because Spring Cloud Gateway is built on top of Spring WebFlux and Netty. Also, we will create a set of test users with MapReactiveUserDetailsService.

@Configuration
@EnableWebFluxSecurity
public class SecurityConfig {

   @Bean
   public SecurityWebFilterChain filterChain(ServerHttpSecurity http) {
      http.authorizeExchange(exchanges -> 
         exchanges.anyExchange().authenticated())
            .httpBasic();
      http.csrf().disable();
      return http.build();
   }

   @Bean
   public MapReactiveUserDetailsService users() {
      UserDetails user1 = User.builder()
            .username("user1")
            .password("{noop}1234")
            .roles("USER")
            .build();
      UserDetails user2 = User.builder()
            .username("user2")
            .password("{noop}1234")
            .roles("USER")
            .build();
      UserDetails user3 = User.builder()
            .username("user3")
            .password("{noop}1234")
            .roles("USER")
            .build();
      return new MapReactiveUserDetailsService(user1, user2, user3);
   }
}

3. Configure Spring Cloud Gateway Rate Limiter key

A request rate limiter feature needs to be enabled using the component called GatewayFilter. This filter takes an optional keyResolver parameter. The KeyResolver interface allows you to create pluggable strategies derive the key for limiting requests. In our case, it will be a user login. Once a user has been successfully authenticated, its login is stored in the Spring SecurityContext. In order to retrieve the context for a reactive application, we should use ReactiveSecurityContextHolder.

@Bean
KeyResolver authUserKeyResolver() {
   return exchange -> ReactiveSecurityContextHolder.getContext()
           .map(ctx -> ctx.getAuthentication()
              .getPrincipal().toString());
}

4. Test Scenario

In the test scenario, we are going to simulate incoming traffic. Every single request needs to have a Authorization header with the user credentials. A single user may send 4 requests per minute. After exceeding that limit Spring Cloud Gateway will return the HTTP code HTTP 429 - Too Many Requests. The traffic is addressed to the downstream service. Therefore, we are running a mock server using Testcontainers.

5. Testing Spring Cloud Gateway secure rate limiter

Finally, we may proceed to the test implementation. I will use JUnit4 since I used it before for the other examples in the sample repository. We have three parameters used for rate limiter configuration: replenishRate, burstCapacity and requestedTokens. Since we also allow less than 1 request per second we need to set the right values for burstCapacity and requestedTokens. In short, the requestedTokens property sets how many tokens a request costs. On the other hand, the burstCapacity property is the maximum number of requests (or cost) that is allowed for a user.

During the test we randomly set the username between user1, user2 and user3. The test is repeated 20 times.

@SpringBootTest(webEnvironment = 
   SpringBootTest.WebEnvironment.DEFINED_PORT,
                properties = {"rateLimiter.secure=true"})
@RunWith(SpringRunner.class)
public class GatewaySecureRateLimiterTest {

   private static final Logger LOGGER = 
      LoggerFactory.getLogger(GatewaySecureRateLimiterTest.class);
   private Random random = new Random();

   @Rule
   public TestRule benchmarkRun = new BenchmarkRule();

   @ClassRule
   public static MockServerContainer mockServer = 
      new MockServerContainer();
   @ClassRule
   public static GenericContainer redis = 
      new GenericContainer("redis:5.0.6").withExposedPorts(6379);

   @Autowired
   TestRestTemplate template;

   @BeforeClass
   public static void init() {
      System.setProperty("spring.cloud.gateway.routes[0].id", "account-service");
      System.setProperty("spring.cloud.gateway.routes[0].uri", "http://" + mockServer.getHost() + ":" + mockServer.getServerPort());
      System.setProperty("spring.cloud.gateway.routes[0].predicates[0]", "Path=/account/**");
      System.setProperty("spring.cloud.gateway.routes[0].filters[0]", "RewritePath=/account/(?<path>.*), /$\\{path}");
      System.setProperty("spring.cloud.gateway.routes[0].filters[1].name", "RequestRateLimiter");
      System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.redis-rate-limiter.replenishRate", "1");
      System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.redis-rate-limiter.burstCapacity", "60");
      System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.redis-rate-limiter.requestedTokens", "15");
      System.setProperty("spring.redis.host", redis.getHost());
      System.setProperty("spring.redis.port", "" + redis.getMappedPort(6379));
      new MockServerClient(mockServer.getContainerIpAddress(), mockServer.getServerPort())
            .when(HttpRequest.request()
                    .withPath("/1"))
            .respond(response()
                    .withBody("{\"id\":1,\"number\":\"1234567890\"}")
                    .withHeader("Content-Type", "application/json"));
   }

   @Test
   @BenchmarkOptions(warmupRounds = 0, concurrency = 1, benchmarkRounds = 20)
   public void testAccountService() {
      String username = "user" + (random.nextInt(3) + 1);
      HttpHeaders headers = createHttpHeaders(username,"1234");
      HttpEntity<String> entity = new HttpEntity<String>(headers);
      ResponseEntity<Account> r = template
         .exchange("/account/{id}", HttpMethod.GET, entity, Account.class, 1);
      LOGGER.info("Received({}): status->{}, payload->{}, remaining->{}",
            username, r.getStatusCodeValue(), r.getBody(), r.getHeaders().get("X-RateLimit-Remaining"));
    }

   private HttpHeaders createHttpHeaders(String user, String password) {
      String notEncoded = user + ":" + password;
      String encodedAuth = Base64.getEncoder().encodeToString(notEncoded.getBytes());
      HttpHeaders headers = new HttpHeaders();
      headers.setContentType(MediaType.APPLICATION_JSON);
      headers.add("Authorization", "Basic " + encodedAuth);
      return headers;
   }

}

Let’s run the test. Thanks to the junit-benchmarks library we may configure the number of rounds for the test. Each time I’m logging the response from the gateway that includes username, HTTP status, payload, and a header X-RateLimit-Remaining that shows a number of remaining tokens. Here’s the result.

The post Secure Rate Limiting with Spring Cloud Gateway appeared first on Piotr's TechBlog.

Spring Cloud Gateway is a very useful product. You may take an advantage of the many interesting features it provides. One of them is rate-limiting. You may read more about that in the article Rate Limiting in Spring Cloud Gateway with Redis. It is also worth learning about a circuit breaking and fault tolerance. You may find interesting pieces of information about it in the articles Circuit Breaking in Spring Cloud Gateway with Resilience4j and Timeouts and Retries in Spring Cloud Gateway.

Source code

If you would like to try it by yourself, you may always take a look at my source code. In order to do that you need to clone my repository sample-spring-security-microservices. Then you should go to the gateway directory, and just follow my instructions If you are interested in more details about Spring Security you should read its documentation.

Enable OAuth2 in Spring Cloud Gateway

To enable OAuth2 support for the Spring Cloud Gateway application we need to add some dependencies. Of course, the spring-cloud-starter-gateway dependency is required to enable the gateway feature. We also need to include spring-boot-starter-oauth2-client enabling Spring Security’s client support for OAuth 2.0 Authorization Framework and OpenID Connect Core 1.0. Finally, we have to add spring-cloud-starter-security to activate the TokenRelay filter.

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-oauth2-client</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-security</artifactId>
</dependency>

In the next step, we need to provide the configuration settings for the OAuth2 client. Because we are integrating with Keycloak we should set the name of registrationId (spring.security.oauth2.client.provider.[registrationId]) to keycloak. Then we need to set the uris of token, authorization and userinfo endpoints. On the other hand, we can set a value for a single issuer endpoint. The last important property in that section is user-name-attribute. Keycloak is returning user login inside the preferred_username attribute.

We will define two different clients for authorization. The first of them spring-cloud-gateway contains the scope allowed by our test method, while the second spring-cloud-gateway-2 does not.

spring:
  security:
    oauth2:
      client:
        provider:
          keycloak:
            token-uri: http://localhost:8080/auth/realms/master/protocol/openid-connect/token
            authorization-uri: http://localhost:8080/auth/realms/master/protocol/openid-connect/auth
            userinfo-uri: http://localhost:8080/auth/realms/master/protocol/openid-connect/userinfo
            user-name-attribute: preferred_username
        registration:
          keycloak-with-test-scope:
            provider: keycloak
            client-id: spring-with-test-scope
            client-secret: c6480137-1526-4c3e-aed3-295aabcb7609
            authorization-grant-type: authorization_code
            redirect-uri: "{baseUrl}/login/oauth2/code/keycloak"
          keycloak-without-test-scope:
            provider: keycloak
            client-id: spring-without-test-scope
            client-secret: f6fc369d-49ce-4132-8282-5b5d413eba23
            authorization-grant-type: authorization_code
            redirect-uri: "{baseUrl}/login/oauth2/code/keycloak"

In the last step, we need to configure Spring Security. Since Spring Cloud Gateway is built on top of Spring WebFlux, we need to annotate the configuration bean with @EnableWebFluxSecurity. Inside the springSecurityFilterChain method we are going to enable authorization for all the exchanges. We will also set OAuth2 as a default login method and finally disable CSRF.

@Configuration
@EnableWebFluxSecurity
public class SecurityConfig {

   @Bean
   public SecurityWebFilterChain springSecurityFilterChain(ServerHttpSecurity http) {
      http.authorizeExchange(exchanges -> exchanges.anyExchange().authenticated())
         .oauth2Login(withDefaults());
      http.csrf().disable();
      return http.build();
   }

}

Run and configure Keycloak

We are running Keycloak on a Docker container. By default, Keycloak exposes API and a web console on port 8080. However, that port number must be different than the Spring Cloud Gateway application port, so we are overriding it with 8888. We also need to set a username and password to the admin console.

$ docker run -d --name keycloak -p 8888:8080 \
   -e KEYCLOAK_USER=spring \
   -e KEYCLOAK_PASSWORD=spring123 \
   jboss/keycloak

Then we need to create two clients with the same names as defined inside the gateway configuration. Both of them need to have confidential in the “Access Type” section, a valid redirection URI set. We may use a simple wildcard while setting the redirection address as shown below.

The client spring-with-test-scope will have the scope TEST assigned. In contrast, the second client spring-without-test-scope will not have the scope TEST assigned.

Enable OAuth2 Resourse in Spring Cloud Gateway

Now, we may proceed to the implementation of the downstream application. In order to run it, you need to switch to the callme directory in the source code. First, we need to include some Maven dependencies. The spring-boot-starter-web starter provides web support for Spring Boot application. With spring-boot-starter-security we enable Spring Security for our microservice. The spring-security-oauth2-resource-server contains Spring Security’s support for OAuth 2.0 Resource Servers. It is also used to protect APIs via OAuth 2.0 Bearer Tokens. Finally, the spring-security-oauth2-jose module contains Spring Security’s support for the JOSE (Javascript Object Signing and Encryption) framework. The JOSE framework provides a method to securely transfer claims between parties. It supports JWT and JWS (JSON Web Signature).

<dependencies>
   <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-web</artifactId>
   </dependency>
   <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-security</artifactId>
   </dependency>
   <dependency>
      <groupId>org.springframework.security</groupId>
      <artifactId>spring-security-oauth2-resource-server</artifactId>
   </dependency>
   <dependency>
      <groupId>org.springframework.security</groupId>
      <artifactId>spring-security-oauth2-jose</artifactId>
   </dependency>
</dependencies>

In the next step, we need to configure a connection to the authorization server. A resource server will use the property spring.security.oauth2.resourceserver.jwt.issuer-uri to discover the authorization server public keys and then validate incoming JWT tokens.

spring:
  application:
    name: callme
  security:
    oauth2:
      resourceserver:
        jwt:
          issuer-uri: http://localhost:8080/auth/realms/master

We should also provide a Spring Security configuration. First, we need to annotate the Configuration bean with @EnableWebSecurity. Then, we should enable annotation-based security for the controller methods. It allows simple role-based access with @PreAuthorize and @PostAuthorize. In order to enable a method security feature we need to use annotation @EnableGlobalMethodSecurity. Finally, we just need to configure Spring Security to authorize all the incoming requests and validate JWT tokens.

@Configuration
@EnableWebSecurity
@EnableGlobalMethodSecurity(prePostEnabled = true)
public class SecurityConfig extends WebSecurityConfigurerAdapter {

   protected void configure(HttpSecurity http) throws Exception {
      http.authorizeRequests(authorize -> authorize.anyRequest().authenticated())
            .oauth2ResourceServer(OAuth2ResourceServerConfigurer::jwt);
   }
}

Finally, let’s take a look at the implementation of the REST controller class. It a single ping method. That method may be accessed only by the client with the TEST scope. It returns a list of assigned scopes from the Authentication bean.

@RestController
@RequestMapping("/callme")
public class CallmeController {

   @PreAuthorize("hasAuthority('SCOPE_TEST')")
   @GetMapping("/ping")
   public String ping() {
      SecurityContext context = SecurityContextHolder.getContext();
      Authentication authentication = context.getAuthentication();
      return "Scopes: " + authentication.getAuthorities();
   }
}

Configure routing on Spring Cloud Gateway

The last step before proceeding to the tests is to configure routing on the Spring Cloud Gateway application. Since the downstream service (callme) is running on port 8040 we need to set the uri to http://127.0.0.1:8040. In order to forward the access token to the callme-service we have to enable a global filter TokenRelay. Just to be sure that everything works as expected, we will remove the Cookie with the session id. The session id is generated on the gateway after performing OAuth2Login.

spring:
  application:
    name: gateway
  cloud:
    gateway:
      default-filters:
        - TokenRelay
      routes:
        - id: callme-service
          uri: http://127.0.0.1:8040
          predicates:
            - Path=/callme/**
          filters:
            - RemoveRequestHeader=Cookie

Finally, let’s take a look at the gateway main class. I added there two useful endpoints. First of them GET / is returning the HTTP session id. The second of them GET /token will return the current JWT access token. After the successful Spring Cloud Gateway OAuth2 login, you will see the result from the index method.

@SpringBootApplication
@RestController
public class GatewayApplication {

   private static final Logger LOGGER = LoggerFactory.getLogger(GatewayApplication.class);

   public static void main(String[] args) {
      SpringApplication.run(GatewayApplication.class, args);
   }

   @GetMapping(value = "/token")
   public Mono<String> getHome(@RegisteredOAuth2AuthorizedClient OAuth2AuthorizedClient authorizedClient) {
      return Mono.just(authorizedClient.getAccessToken().getTokenValue());
   }

   @GetMapping("/")
   public Mono<String> index(WebSession session) {
      return Mono.just(session.getId());
   }

}

Spring Cloud Gateway OAuth2 testing scenario

First, let’s take a look at the picture that illustrates our use case. We are calling POST /login endpoint on the gateway (1). After receiving the login request Spring Cloud Gateway try to obtain the access token from the authorization server (2). Then Keycloak is returning the JWT access token. As a result, Spring Cloud Gateway is calling the userinfo endpoint (3). After receiving the response it is creating a web session and Authentication bean. Finally, the gateway application is returning a session id to the external client (4). The external client is using a cookie with session-id to authorize requests. It calls GET ping from the callme application (5). The gateway application is forwarding the request to the downstream service (6). However, it removes the cookie and replaces it with a JWT access token. The callme application verifies an incoming token (7). Finally, it returns 200 OK response if the client is allowed to call endpoint (8). Otherwise, it returns 403 Forbidded.

We may start testing in the web browser. First, let’s call the login endpoint. We have to available clients keycloak-with-test-scope and keycloak-without-test-scope. We will use the client keycloak-with-test-scope.

Then, the gateway redirects us to the Keycloak login page. We can use the credentials provided during the creation of the Keycloak container.

After a successful login, the gateway will perform the OAuth2 authorization procedure. Finally, it redirects us to the main page. The main page is just a method index inside the controller. It is returning the current session id.

We can also use another endpoint implemented on the gateway – GET /token. It is returning the current JWT access token.

$ curl http://localhost:8080/token -H "Cookie: SESSION=9bf852f1-6e00-42f8-a9a2-3cbdced33993"
eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICI0RWpwdkVtQ1ZDZ1VDUm41Y2NJeXRiank0RnR0RXpBRXVrMURoZDRTT0RFIn0.eyJleHAiOjE2MDIyMzM5MTksImlhdCI6MTYwMjIz
MzAxOSwiYXV0aF90aW1lIjoxNjAyMjMzMDE5LCJqdGkiOiIyYWQzYjczNy1mZTdhLTQ3NGUtODhhYy01MGZjYzEzOTlhYTQiLCJpc3MiOiJodHRwOi8vbG9jYWxob3N0OjgwODAvYXV0aC9yZWFsbXMv
bWFzdGVyIiwiYXVkIjpbIm1hc3Rlci1yZWFsbSIsImFjY291bnQiXSwic3ViIjoiOWVhMDAyYmQtOTQ4Ni00Njk0LWFhYzUtN2IyY2QwNzc2MTZiIiwidHlwIjoiQmVhcmVyIiwiYXpwIjoic3ByaW5n
LWNsb3VkLWdhdGV3YXkiLCJzZXNzaW9uX3N0YXRlIjoiMDRhNzQ4YzUtOTA1My00ZmZmLWJjYzctNWY5MThjMzYwZGE4IiwiYWNyIjoiMSIsInJlYWxtX2FjY2VzcyI6eyJyb2xlcyI6WyJjcmVhdGUt
cmVhbG0iLCJST0xFX1RFTExFUiIsIm9mZmxpbmVfYWNjZXNzIiwiYWRtaW4iLCJURUxMRVIiLCJ1bWFfYXV0aG9yaXphdGlvbiJdfSwicmVzb3VyY2VfYWNjZXNzIjp7Im1hc3Rlci1yZWFsbSI6eyJy
b2xlcyI6WyJ2aWV3LWlkZW50aXR5LXByb3ZpZGVycyIsInZpZXctcmVhbG0iLCJtYW5hZ2UtaWRlbnRpdHktcHJvdmlkZXJzIiwiaW1wZXJzb25hdGlvbiIsImNyZWF0ZS1jbGllbnQiLCJtYW5hZ2Ut
dXNlcnMiLCJxdWVyeS1yZWFsbXMiLCJ2aWV3LWF1dGhvcml6YXRpb24iLCJxdWVyeS1jbGllbnRzIiwicXVlcnktdXNlcnMiLCJtYW5hZ2UtZXZlbnRzIiwibWFuYWdlLXJlYWxtIiwidmlldy1ldmVu
dHMiLCJ2aWV3LXVzZXJzIiwidmlldy1jbGllbnRzIiwibWFuYWdlLWF1dGhvcml6YXRpb24iLCJtYW5hZ2UtY2xpZW50cyIsInF1ZXJ5LWdyb3VwcyJdfSwiYWNjb3VudCI6eyJyb2xlcyI6WyJtYW5h
Z2UtYWNjb3VudCIsIm1hbmFnZS1hY2NvdW50LWxpbmtzIiwidmlldy1wcm9maWxlIl19fSwic2NvcGUiOiJwcm9maWxlIGVtYWlsIFRFU1QiLCJlbWFpbF92ZXJpZmllZCI6ZmFsc2UsInByZWZlcnJl
ZF91c2VybmFtZSI6InBpb21pbiJ9.X8XfIHiAiR1YMjiJza75aC294qLwi83RrUU2phorM7FP4phq3h-lx80Zu5xqTrMqMC1-RbHBnX-oUTbs4ViS3DziZlDvoRajdkrh6UTiK5oWgoRW-4qsH5L4X1W
bRfoBZgyHFRSnhaCO4CLgjCyEgeLUR5A-JWY-OMYQIOAxxHB2GwE3MNFfLWeqpmS1AWU8fL0giFFXFDfa1_XZEKgnqe1S75Ps_z8B1sfNfvNpz8daJ8omzXrt6I6TSa0FE3iiZ7Qx18mtkbx-iPuFqDD
RT6DGU-Hing9LnGuOt3Yas-WYdN7PKBigvIZv0LyvRFcilRJQBjOdVfEddL3OQ0rmEg

Just to check, you can decode a JWT token on the https://jwt.io site.

Finally, let’s call the endpoint exposed by the callme application. We are setting session Cookie in the request header. The endpoint is returning a list of scopes assigned to the current user. Only user with scope TEST is allowed to call the method.

$ curl http://localhost:8080/callme/ping -H "Cookie: SESSION=9bf852f1-6e00-42f8-a9a2-3cbdced33993"
Scopes: [SCOPE_profile, SCOPE_email, SCOPE_TEST]

Conclusion

In this article we were discussing important aspects related to microservices security. I showed you how to enable Spring Cloud Gateway OAuth2 support and integrate it with Keycloak. We were implementing such mechanisms like OAuth2 login, token relay, and OAuth2 resource server. The token relay mechanisms will be completely migrated from Spring Cloud Security to Spring Cloud Gateway. Enjoy

The post Spring Cloud Gateway OAuth2 with Keycloak appeared first on Piotr's TechBlog.

Source code

If you would like to try it by yourself, you may always take a look at my source code example. In order to do that you need to clone my repository sample-spring-microservices-kubernetes. Then just follow my instructions The good idea is to read the article Microservices with Spring Cloud Kubernetes before you move on.

Step 1. Enable Spring Cloud Kubernetes Load Balancer

You need to include a single Spring Boot Starter to enable Spring Cloud Kubernetes Load Balancer. It is spring-cloud-starter-kubernetes-loadbalancer. Consequently, you also need to have a REST client on the classpath. Spring RestTemplate is automatically included with the Spring Web module. We will also use the OpenFeign client, so you should include the right starter.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-kubernetes-loadbalancer</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>

Step 2. Implement and use the REST client

Spring Cloud OpenFeign is a declarative REST client. Therefore, you need to create an interface with methods and Spring MVC annotations. It is important to set the right name inside @FeignClient annotation. This name needs to be the same as the name of the target Kubernetes Service. In the following code example, you see an implementation of the employee service client inside department-service.

@FeignClient(name = "employee")
public interface EmployeeClient {

   @GetMapping("/department/{departmentId}")
   List<Employee> findByDepartment(@PathVariable("departmentId") String departmentId);
	
}

OpenFeign client is enabled after annotating the main class with @EnableFeignClients. After that you can inject it to the @RestController class. Finally, you may use its method findByDepartment to find employees assigned to the particular department.

@RestController
public class DepartmentController {

    private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);

    DepartmentRepository repository;
    EmployeeClient employeeClient;

    public DepartmentController(DepartmentRepository repository, EmployeeClient employeeClient) {
        this.repository = repository;
        this.employeeClient = employeeClient;
    }

    @GetMapping("/{id}/with-employees")
    public Department findByIdWithEmployees(@PathVariable("id") String id) {
        LOGGER.info("Department findByIdWithEmployees: id={}", id);
        Optional<Department> optDepartment = repository.findById(id);
        if (optDepartment.isPresent()) {
            Department department = optDepartment.get();
            department.setEmployees(employeeClient.findByDepartment(department.getId()));
            return department;
        }
        return null;
    }

    @GetMapping("/organization/{organizationId}/with-employees")
    public List<Department> findByOrganizationWithEmployees(@PathVariable("organizationId") Long organizationId) {
        LOGGER.info("Department find: organizationId={}", organizationId);
        List<Department> departments = repository.findByOrganizationId(organizationId);
        departments.forEach(d -> d.setEmployees(employeeClient.findByDepartment(d.getId())));
        return departments;
    }

}

Opposite to the OpenFeign, the RestTemplate is a low-level HTTP client. We need to enable Spring Cloud load balancing for it. To do that just annotate RestTemplate bean with @LoadBalanced.

@Bean
@LoadBalanced
RestTemplate restTemplate() {
    return new RestTemplate();
}

Here’s the similar implementation of @RestController class, but this time with the RestTemplate.

public class DepartmentWithRestTemplateController {


    private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);

    DepartmentRepository repository;
    RestTemplate restTemplate;

    public DepartmentWithRestTemplateController(DepartmentRepository repository, RestTemplate restTemplate) {
        this.repository = repository;
        this.restTemplate = restTemplate;
    }

    @GetMapping("/{id}/with-employees")
    public Department findByIdWithEmployees(@PathVariable("id") String id) {
        LOGGER.info("Department findByIdWithEmployees: id={}", id);
        Optional<Department> optDepartment = repository.findById(id);
        if (optDepartment.isPresent()) {
            Department department = optDepartment.get();
            department.setEmployees(findEmployeesByDepartment(department.getId()));
            return department;
        }
        return null;
    }


    @GetMapping("/organization/{organizationId}/with-employees")
    public List<Department> findByOrganizationWithEmployees(@PathVariable("organizationId") Long organizationId) {
        LOGGER.info("Department find: organizationId={}", organizationId);
        List<Department> departments = repository.findByOrganizationId(organizationId);
        departments.forEach(d -> d.setEmployees(findEmployeesByDepartment(d.getId())));
        return departments;
    }

    private List<Employee> findEmployeesByDepartment(String departmentId) {
        Employee[] employees = restTemplate
                .getForObject("http://employee//department/{departmentId}", Employee[].class, departmentId);
        return Arrays.asList(employees);
    }

}

Step 3. Deploy Spring Cloud applications on Kubernetes

The project is ready to be used with Skaffold. Therefore, you don’t have to worry about the deployment process. You just need to run a single command skaffold dev --port-forward to deploy our applications on Kubernetes. But before deploying them on the cluster, we perform a short overview of the communication process. We have three microservices. Each of them is running in two instances. They are using Mongo as a backend store. There is also the gateway application. It is built on top of Spring Cloud Gateway. It provides a single API endpoint to all downstream services. Of course, all our applications are using Spring Cloud Load Balancer for traffic management. To clarify, you can take a look at the picture below.

Here’s the Kubernetes deployment manifest for the employee-service. It sets two running pods in replicas parameter, and references ConfigMap and Secret to inject Mongo credentials.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: employee
  labels:
    app: employee
spec:
  replicas: 2
  selector:
    matchLabels:
      app: employee
  template:
    metadata:
      labels:
        app: employee
    spec:
      containers:
      - name: employee
        image: piomin/employee
        ports:
        - containerPort: 8080
        env:
        - name: MONGO_DATABASE
          valueFrom:
            configMapKeyRef:
              name: mongodb
              key: database-name
        - name: MONGO_USERNAME
          valueFrom:
            secretKeyRef:
              name: mongodb
              key: database-user
        - name: MONGO_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mongodb
              key: database-password

The most important element in the communication between applications is Kubernetes Service object. The name of service must be the same as the hostname used by the RestTemplate or OpenFeign client. In our case that name is employee.

apiVersion: v1
kind: Service
metadata:
  name: employee
  labels:
    app: employee
    spring-boot: "true"
spec:
  ports:
  - port: 8080
    protocol: TCP
  selector:
    app: employee
  type: ClusterIP

Let’s verify the status after deploying all the applications. Here’s the list of running pods.

Here’s the list of Kubernetes Endpoints.

Step 4. Communication in the POD mode

By default, Spring Cloud Kubernetes Load Balancer uses the POD mode. In this mode, it gets the list of Kubernetes endpoints to detect the IP address of all the application pods. In that case, the only thing you need to do is to disable the Ribbon load balancer. Spring Cloud Kubernetes is still using it as a default client-side load balancer. To disable it you need to set the following property.

spring:
  cloud:
    loadbalancer:
      ribbon:
        enabled: false

After adding a test data we may send some requests to the endpoint GET /{id}/with-employees. It finds a department by the id. Then it communicates with the employee-service endpoint GET /department/{departmentId} to search all the employees assigned to the current department. The department-service is exposed on port 8081, since I enabled option port-forward on Skaffold.

$ curl http://localhost:8081/5f5896b3cb8caf7e6f6b9e1c/with-employees
{"id":"5f5896b3cb8caf7e6f6b9e1c","organizationId":"1","name":"test1","employees":[{"id":"5f5896e26092716e54f60a9a","name":"test1","age":30,"position":"d
eveloper"},{"id":"5f5896f29625e62c7d373906","name":"test2","age":40,"position":"tester"},{"id":"5f5897266092716e54f60a9b","name":"test3","age":45,"posit
ion":"developer"}]}

Let’s take a look on the logs from employee-service. I repeated the request visible above several times. The requests are load balanced between two available instances of employee-service. Skaffold prints the id of every pod in the logs, so you can verify that everything works fine.

Step 5. Communication across multiple namespaces

By default, Spring Cloud Kubernetes allows load balancing within the same namespace. You may enable discovery across multiple namespaces. To do that you need to use the property following property.

spring:
  cloud:
    kubernetes:
      discovery:
        all-namespaces: true

After setting that, we will deploy the employee-service application in the different namespace than department-service. To do that you need to set parameter -n on the skaffold command.

$ skaffold dev -n test-a

Here’s the current list of running pods.

You just need to send the same request as before curl http://localhost:8081/5f5896b3cb8caf7e6f6b9e1c/with-employees. Of course, the Kubernetes service name must be unique across all the namespaces.

Step 6. Load balancer on Spring Cloud Gateway

Spring Cloud Gateway uses the same load balancing mechanism as other Spring Cloud applications. To enable it on Kubernetes we need to include dependency spring-cloud-starter-kubernetes-loadbalancer. We should also enable Spring Cloud DiscoveryClient and disable Ribbon. Here’s the configuration of the gateway-service.

spring:
  application:
    name: gateway
  cloud:
    loadbalancer:
      ribbon:
        enabled: false
    gateway:
      discovery:
        locator:
          enabled: true
      routes:
        - id: employee
          uri: lb://employee
          predicates:
            - Path=/employee/**
          filters:
            - RewritePath=/employee/(?.*), /$\{path}
        - id: department
          uri: lb://department
          predicates:
            - Path=/department/**
          filters:
            - RewritePath=/department/(?.*), /$\{path}
        - id: organization
          uri: lb://organization
          predicates:
            - Path=/organization/**
          filters:
            - RewritePath=/organization/(?.*), /$\{path}

A gateway acts as an entry point to our system. It performs routing and load balancing to all the downstream services. Therefore, we can call any of our applications using already defined routes. Since gateway-service is available on port 8080, I can call any of the endpoints using the following requests.

We can expose API documentation of all the microservices on the gateway. To do that we may use the SpringFox project. First, we need to include SpringFox Starter to Maven dependencies.

<dependency>
   <groupId>io.springfox</groupId>
   <artifactId>springfox-boot-starter</artifactId>
   <version>3.0.0</version>
</dependency>

Swagger documentation is usually generated for a single application. Because we want to expose documentation of all our microservices we need to override the Swagger resource provider. The custom implementation is visible below. It uses a discovery mechanism to the names of running services.

@Configuration
public class GatewayApi {

   @Autowired
   RouteDefinitionLocator locator;

   @Primary
   @Bean
   public SwaggerResourcesProvider swaggerResourcesProvider() {
      return () -> {
         List<SwaggerResource> resources = new ArrayList<>();
         Flux<RouteDefinition> definitions = locator.getRouteDefinitions();
         definitions
               .filter(routeDefinition -> !routeDefinition.getId().startsWith("ReactiveCompositeDiscoveryClient_"))
               .subscribe(routeDefinition -> resources.add(createResource(routeDefinition.getId(), "2.0")));
         try {
            Thread.sleep(1000);
         } catch (InterruptedException e) {
            e.printStackTrace();
         }
         return resources;
      };
   }

   private SwaggerResource createResource(String location, String version) {
      SwaggerResource swaggerResource = new SwaggerResource();
      swaggerResource.setName(location);
      swaggerResource.setLocation("/" + location + "/v2/api-docs");
      swaggerResource.setSwaggerVersion(version);
      return swaggerResource;
   }

}

Here’s the Swagger UI available at http://localhost:8080/swagger-ui/index.html on my local machine.

Step 7. Enabling circuit breaker

We can use a circuit breaker component with Spring Cloud Load Balancer. The default Spring Cloud implementation is based on Resilience4j. In order to enable it for your application, you need to include the following dependency.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-circuitbreaker-resilience4j</artifactId>
</dependency>

The next step is to provide a configuration of a circuit breaker.

@Bean
Customizer<Resilience4JCircuitBreakerFactory> defaultCustomizer() {
   return factory -> factory.configureDefault(id ->
         new Resilience4JConfigBuilder(id)
               .timeLimiterConfig(TimeLimiterConfig.custom()
                     .timeoutDuration(Duration.ofMillis(1000))
                     .build())
               .circuitBreakerConfig(CircuitBreakerConfig.custom()
                     .slidingWindowSize(10)
                     .failureRateThreshold(66.6F)
                     .slowCallRateThreshold(66.6F)
                     .build())
               .build()
   );
}

We need to inject Resilience4JCircuitBreakerFactory to the Spring controller class. Then we are creating the circuit breaker instance using an injected factory. Finally, the client calling method is running inside the circuit breaker run method.

@RestController
public class DepartmentController {

    private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);

    DepartmentRepository repository;
    EmployeeClient employeeClient;
    Resilience4JCircuitBreakerFactory circuitBreakerFactory;

    public DepartmentController(DepartmentRepository repository, EmployeeClient employeeClient,
                                Resilience4JCircuitBreakerFactory circuitBreakerFactory) {
        this.repository = repository;
        this.employeeClient = employeeClient;
        this.circuitBreakerFactory = circuitBreakerFactory;
    }

    @GetMapping("/{id}/with-employees-and-delay")
    public Department findByIdWithEmployeesAndDelay(@PathVariable("id") String id) {
        LOGGER.info("Department findByIdWithEmployees: id={}", id);
        Optional<Department> optDepartment = repository.findById(id);
        if (optDepartment.isPresent()) {
            Department department = optDepartment.get();
            Resilience4JCircuitBreaker circuitBreaker = circuitBreakerFactory.create("delayed-circuit");
            List<Employee> employees = circuitBreaker.run(() ->
                    employeeClient.findByDepartmentWithDelay(department.getId()));
            department.setEmployees(employees);
            return department;
        }
        return null;
    }
}

Conclusion

Load balancing is one of the key patterns in a microservices architecture. Spring Cloud Load Balancer is replacing the Ribbon client. By default, load balancing in Kubernetes is based on Services. Therefore, you need to use additional tools for more advanced routing mechanisms. Spring Cloud Kubernetes comes with some interesting features. One of them is the ability to load balance across multiple namespaces. You can also use the additional Spring Cloud components like a circuit breaker. In comparison with tools like Istio, it is still not much. Is it a chance for improvement? We will see. Nevertheless, Spring Cloud Kubernetes is currently one of the most popular Spring Cloud projects. It may be a good choice if you are migrating your Spring Cloud microservices architecture to Kubernetes. Support for the load balancer may be an important step during such a migration.

The post Spring Cloud Kubernetes Load Balancer Guide appeared first on Piotr's TechBlog.

Spring Cloud video course

You may also find a video guide, where I’m showing all the features described here with more details. It is available on my YouTube channel and consists of four parts:

Part 1 – Introduction to Spring Boot (~35 minutes)
Part 2 – Distributed Configuration and Service Discovery (~36 minutes)
Part 3 – Inter-service communication (~34 minutes)
Part 4 – API Gateway (~22 minutes)

Architecture

The diagram visible below illustrates the architecture of our sample system. Here we have the characteristic elements for microservices like API gateway, discovery server, and configuration server. In the next sections of this article, I’ll show how to use Spring Cloud components that provide an implementation of those patterns. Currently, the main component for adding the API gateway to your system is Spring Cloud Gateway.
Spring Cloud provides integrations to several solutions that may be used as a discovery server: Netflix Eureka, HashiCorp Consul, Alibaba Nacos, or Apache ZooKeeper. The most common choice is between the first two of them. While Spring Cloud Netflix Eureka is dedicated just for discovery, Spring Cloud Consul may realize both discovery feature basing on Consul Services, and distributed configuration feature basing on Consul Key/Value engine.

In turn, Spring Cloud Config is responsible just for providing a mechanism for configuration management. However, it may also be integrated with third-party tools like Vault from HashiCorp.
We will figure out how to integrate our applications with discovery and configuration servers on the example of two simple Spring Boot applications callme-service and caller-service. The application caller-service is also calling endpoints exposed by the callme-service. We will enable such mechanisms on the caller-service like client-side load balancer with new Spring Cloud Load Balancer, and circuit breaker with new Spring Cloud Circuit Breaker built on top of Resilience4J.

Service discovery

Switching between discovery servers on the client-side is very easy with Spring Cloud due to the DiscoveryClient abstraction. This switch comes down to the replacement of a single dependency in Maven pom.xml. So if you are using Eureka you should add the following starter in your microservice.

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>

On the other hand, if you are using Consul you should add the following starter in your microservice.

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-consul-discovery</artifactId>
</dependency>

The situation is a little bit more complicated if you are defining some non-default configuration settings for a discovery client. In that case, you need to use properties specific just for Eureka, or just for Consul. For example, if you are running more than one instance of a single application on the same host with dynamic HTTP server port feature enabled (option server.port=0), you have to set a unique id of every instance. Here’s the property used for Eureka’s client.

eureka:
  instance:
    instanceId: ${spring.cloud.client.hostname}:${spring.application.name}:${random.value}

For the Consul client the same configuration looks as shown below.

spring:
  cloud:
    consul:
      discovery:
        instanceId: ${spring.cloud.client.hostname}:${spring.application.name}:${random.value}

You can easily configure and run Eureka discovery in your microservices architecture using Spring Cloud Netflix Eureka Server module. You just need to create the Spring Boot application that includes that module.

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>

We also need to enable Eureka for the application by annotating the main class with @EnableEurekaServer.

@SpringBootApplication
@EnableEurekaServer
class DiscoveryServerApplication

fun main(args: Array<String>) {
    runApplication<DiscoveryServerApplication>(*args)
}

The most convenient way to run Consul on the local machine is by using its Docker image. We can Consul on Docker container in development mode by executing the following command.

$ docker run -d --name=consul -e CONSUL_BIND_INTERFACE=eth0 -p 8500:8500 consul:1.7.2

Distributed configuration with Spring cloud

The next important element in our architecture is a configuration server. The most popular solution in Spring Cloud that provides mechanisms for distributed configuration is Spring Cloud Config. Spring Cloud Config provides server-side and client-side support for externalized configuration in a distributed system.
With the config server, you have a central place to manage external properties for applications across all environments. Some other solutions may be used in microservices-based architecture as a configuration server: Consul, ZooKeeper, or Alibaba Nacos. However, all these solutions are not strictly dedicated to distributed configuration, they can act as a discovery server as well.
Spring Cloud Config may integrate with different tools for storing data. The default implementation of the server storage backend uses git, but we can use some other tools like HashiCorp Vault for managing secrets and protecting sensitive data or just a simple file system. Such different backends may be together in the single config server application. We just need to activate the appropriate profiles in application properties with spring.profiles.active. We may override some default, for example, change the address of a Vault server or set an authentication token.

spring:
  application:
    name: config-server
  profiles:
    active: native,vault
  cloud:
    config:
      server:
        native:
          searchLocations: classpath:/config-repo
        vault:
          host: 192.168.99.100
          authentication: TOKEN
          token: spring-microservices-course

The same as for Consul we should use Docker to run an instance of Vault in development mode. We can set a static root token for authentication using the environment variable VAULT_DEV_ROOT_TOKEN_ID.

$ docker run -d --name vault --cap-add=IPC_LOCK -e 'VAULT_DEV_ROOT_TOKEN_ID=spring-microservices-course' -p 8200:8200 vault:1.4.0

When using Spring Cloud Config together with the discovery we may choose between two available approaches called Config First Bootstrap and Discovery First Bootstrap. In Discovery First Bootstrap a config server is registering itself in discovery service. Thanks to that each microservice can localize a config server basing on its registration id.
Since a configuration is injected in the bootstrap phase we need to use bootstrap.yml for setting properties on the client-side. To enable “discovering” for the config server on the client side we should set the property spring.cloud.config.discovery.enabled to true. We should also override registered service id of config server if it is different from auto-configured configserver (in our case it is config-server). Of course, we can also use Consul as a configuration properties source.

spring:
  application:
    name: callme-service
  cloud:
    config:
      discovery:
        enabled: true
        serviceId: config-server
    consul:
      host: 192.168.99.100
      config:
        format: YAML

Inter-service communication

Currently, there are three Spring components for inter-service communication over HTTP that have integration with service discovery: synchronous RestTemplate, reactive WebClient and declarative REST client OpenFeign. The RestTemplate component is available within the Spring Web module, WebClient within Spring WebFlux. To include Spring Cloud OpenFeign we need to include a dedicated starter.

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>

To use RestTemplate or WebClient for communication with discovery support, we need to register the beans and annotate them with @LoadBalanced. It is also worth setting the proper timeouts for such communication, especially if you are not using a circuit breaker.

@SpringBootApplication
@EnableFeignClients
class InterCallerServiceApplication {

    @Bean
    @LoadBalanced
    fun template(): RestTemplate = RestTemplateBuilder()
        .setReadTimeout(Duration.ofMillis(100))
        .setConnectTimeout(Duration.ofMillis(100))
        .build()

    @Bean
    @LoadBalanced
    fun clientBuilder(): WebClient.Builder {
        val tcpClient: TcpClient = TcpClient.create()
            .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 100)
            .doOnConnected { conn ->
                conn.addHandlerLast(ReadTimeoutHandler(100, TimeUnit.MILLISECONDS))
            }
        val connector = ReactorClientHttpConnector(HttpClient.from(tcpClient))
        return WebClient.builder().clientConnector(connector)
    }
    
}

Although Ribbon has been moved to maintenance mode almost 1.5 years ago it is still used as a default client-side load balancer in the newest stable version of Spring Cloud. Since Spring Cloud Load Balancer is included in common dependencies it is available in your application. Therefore, the only thing you need to do is to disable Ribbon in the configuration using spring.cloud.loadbalancer.ribbon.enabled property.
Currently, we don’t have many options for load balancer customization. One of them is the ability to configure client cache settings. By default, each client is caching the list of target services and refreshing them every 30 seconds. Such an interval may be too long in your situation.
We can easily change it in the configuration as shown below and set it to for example 1 second. If your load balancer is integrated with Eureka discovery you also need to decrease an interval of the fetching registry, which is by default 30 seconds. After those, both changes your client can refresh the list of currently running services almost immediately.

spring:
  cloud:
    loadbalancer:
      cache:
        ttl: 1s
      ribbon:
        enabled: false
eureka:
  client:
    registryFetchIntervalSeconds: 1

Circuit breaker

The circuit breaker is a popular design pattern used in a microservices architecture. It is designed to detect failures and encapsulates the logic of preventing a failure from constantly recurring. Spring Cloud provides an abstraction for using different circuit breaker implementations. Currently, we may use Netflix Hystrix, Sentinel, Spring Retry, and Resilience4J. To enable Spring Cloud Circuit Breaker based on Resilience4J we need to include the following dependency.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-circuitbreaker-resilience4j</artifactId>
</dependency>

Here’s the code responsible for registering Customizer bean, that configures a circuit breaker behavior.

@Bean
fun defaultCustomizer(): Customizer<Resilience4JCircuitBreakerFactory> {
  return Customizer { factory: Resilience4JCircuitBreakerFactory ->
    factory.configureDefault { id: String? ->
      Resilience4JConfigBuilder(id)
        .timeLimiterConfig(TimeLimiterConfig.custom()
          .timeoutDuration(Duration.ofMillis(500))
          .build())
        .circuitBreakerConfig(CircuitBreakerConfig.custom()
          .slidingWindowSize(10)
          .failureRateThreshold(33.3F)
          .slowCallRateThreshold(33.3F)
        .build())
      .build()
    }
  }
}

The settings of the circuit breaker have been visualized in the picture below. The sliding window size sets the number of requests which are used for calculating the error rate. If we have more than 3 errors in the window of size 10 a circuit is open.

In the last step, we need to create a circuit breaker instance using Resilience4JCircuitBreakerFactory bean and enable it for the HTTP client as shown below.

@RestController
@RequestMapping("/caller")
class CallerController(private val template: RestTemplate, private val factory: Resilience4JCircuitBreakerFactory) {

  private var id: Int = 0

  @PostMapping("/random-send/{message}")
  fun randomSend(@PathVariable message: String): CallmeResponse? {
    val request = CallmeRequest(++id, message)
    val circuit = factory.create("random-circuit")
    return circuit.run { template.postForObject("http://inter-callme-service/callme/random-call",
      request, CallmeResponse::class.java) }
  }
    
}

Spring Cloud API gateway

The last missing element in our microservices architecture is an API Gateway. Spring Cloud Gateway is the project that helps us in implementing such a component. Currently, it is the second most popular Spring Cloud project just after Spring Cloud Netflix. It has around 2k stars on GitHub. It is built on top of the Spring WebFlux and Reactor project. It works reactively and requires Netty as a runtime framework.
The main goal of API gateway is to hide the complexity of the microservices system from an external client by providing an effective way of routing to APIs, but it can also solve some problems around security or resiliency. The main component used for configuring Spring Cloud Gateway is a route.
It is defined by an ID, a destination URI, a collection of predicates, and a collection of filters. A route is matched if the aggregate predicate is true. With filters, you can modify requests and responses before or after sending the downstream request.
With a predefined set of gateway filters, we may enable such mechanisms like path rewriting, rate limiting, discovery client, circuit breaking, fallback, or routing metrics. To enable all these features on the gateway we first need to include the following dependencies.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
   <groupId>org.jetbrains.kotlin</groupId>
   <artifactId>kotlin-reflect</artifactId>
</dependency>
<dependency>
   <groupId>org.jetbrains.kotlin</groupId>
   <artifactId>kotlin-stdlib</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-data-redis-reactive</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-circuitbreaker-reactor-resilience4j</artifactId>
</dependency>

To enable all the previously listed features we don’t have to implement much code. Almost everything is configurable in application properties.

spring:
  application:
    name: api-gateway
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true 
          lowerCaseServiceId: true
      routes:
        - id: inter-callme-service
          uri: lb://inter-callme-service
          predicates:
            - Path=/api/callme/**
          filters:
            - RewritePath=/api(?/?.*), $\{path}
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 20
                redis-rate-limiter.burstCapacity: 40
            - name: CircuitBreaker
              args:
                name: sampleSlowCircuitBreaker
                fallbackUri: forward:/fallback/test
        - id: inter-caller-service
          uri: lb://inter-caller-service
          predicates:
            - Path=/api/caller/**
          filters:
            - StripPrefix=1
            - name: RequestRateLimiter
              args:
                redis-rate-limiter.replenishRate: 20
                redis-rate-limiter.burstCapacity: 40
    loadbalancer:
      ribbon:
        enabled: false
  redis:
    host: 192.168.99.100

management:
  endpoints.web.exposure.include: '*'
  endpoint:
    health:
      show-details: always

Some settings still need to be configured in the code. It is a configuration of the circuit breaker, that is based on Resilience4J project, where we need to register the bean Customizer. We also have to define a key for rate-limiting responsible setting a strategy of choosing requests for counting the limits.

@SpringBootApplication
class ApiGatewayApplication {

  @Bean
  fun keyResolver(): KeyResolver = KeyResolver { _ -> Mono.just("1") }

  @Bean
  fun defaultCustomizer(): Customizer<ReactiveResilience4JCircuitBreakerFactory> {
    return Customizer { factory: ReactiveResilience4JCircuitBreakerFactory ->
      factory.configureDefault { id: String? ->
        Resilience4JConfigBuilder(id)
          .timeLimiterConfig(TimeLimiterConfig.custom()
            .timeoutDuration(Duration.ofMillis(500))
            .build())
          .circuitBreakerConfig(CircuitBreakerConfig.custom()
            .slidingWindowSize(10)
            .failureRateThreshold(33.3F)
            .slowCallRateThreshold(33.3F)
            .build())
          .build()
      }
    }
  }

}

fun main(args: Array<String>) {
   runApplication<ApiGatewayApplication>(*args)
}

Conclusion

In this article, you may take a quick introduction to using the latest Spring Cloud components for building microservices architecture. For more details, you may refer to my video course published on YouTube.

The post A New Era Of Spring Cloud appeared first on Piotr's TechBlog.

• Rate Limiting In Spring Cloud Gateway with Redis
• Circuit Breaking in Spring Cloud Gateway with Resilience4J

Example

We use the same repository as for two previous articles about Spring Cloud Gateway. The address of repository is https://github.com/piomin/sample-spring-cloud-gateway.git. The test class dedicated for the current article is GatewayRetryTest.

Implementation and testing

As you probably know most of the operations in Spring Cloud Gateway are realized using filter pattern, which is an implementation of Spring Framework GatewayFilter. Here, we can modify incoming requests and outgoing responses before or after sending the downstream request.
The same as for examples described in my two previous articles about Spring Cloud Gateway we will build JUnit test class. It leverages Testcontainers MockServer for running mock exposing REST endpoints.
Before running the test we need to prepare a sample route containing Retry filter. When defining this type of GatewayFilter we may set multiple parameters. Typically you will use the following three of them:

• retries – the number of retries that should be attempted for a single incoming request. The default value of this property is 3
• statuses – the list of HTTP status codes that should be retried, represented by using org.springframework.http.HttpStatus enum name.
• backoff – the policy used for calculating Spring Cloud Gateway timeout between subsequent retry attempts. By default this property is disabled.

Let’s start from the simplest scenario – using default values of parameters. In that case we just need to set a name of GatewayFilter for a route – Retry.

@ClassRule
public static MockServerContainer mockServer = new MockServerContainer();

@BeforeClass
public static void init() {
   System.setProperty("spring.cloud.gateway.routes[0].id", "account-service");
   System.setProperty("spring.cloud.gateway.routes[0].uri", "http://192.168.99.100:" + mockServer.getServerPort());
   System.setProperty("spring.cloud.gateway.routes[0].predicates[0]", "Path=/account/**");
   System.setProperty("spring.cloud.gateway.routes[0].filters[0]", "RewritePath=/account/(?<path>.*), /$\\{path}");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].name", "Retry");
   MockServerClient client = new MockServerClient(mockServer.getContainerIpAddress(), mockServer.getServerPort());
   client.when(HttpRequest.request()
      .withPath("/1"), Times.exactly(3))
      .respond(response()
         .withStatusCode(500)
         .withBody("{\"errorCode\":\"5.01\"}")
         .withHeader("Content-Type", "application/json"));
   client.when(HttpRequest.request()
      .withPath("/1"))
      .respond(response()
         .withBody("{\"id\":1,\"number\":\"1234567891\"}")
         .withHeader("Content-Type", "application/json"));
   // OTHER RULES
}

Our test method is very simple. It is just using Spring Framework TestRestTemplate to perform a single call to the test endpoint.

@Autowired
TestRestTemplate template;

@Test
public void testAccountService() {
   LOGGER.info("Sending /1...");
   ResponseEntity r = template.exchange("/account/{id}", HttpMethod.GET, null, Account.class, 1);
   LOGGER.info("Received: status->{}, payload->{}", r.getStatusCodeValue(), r.getBody());
   Assert.assertEquals(200, r.getStatusCodeValue());
}

Before running the test we will change a logging level for Spring Cloud Gateway logs, to see the additional information about the retrying process.

logging.level.org.springframework.cloud.gateway.filter.factory: TRACE

Since we didn’t set any backoff policy the subsequent attempts were replied without any delay. As you see on the picture below, a default number of retries is 3, and the filter is trying to retry all HTTP 5XX codes (SERVER_ERROR).

Now, let’s provide a little more advanced configuration. We can change the number of retries and set the exact HTTP status code for retrying instead of the series of codes. In our case a retried status code is HTTP 500, since it is returned by our mock endpoint. We can also enable backoff retrying policy beginning from 50ms to max 500ms. The factor is 2 what means that the backoff is calculated by using formula prevBackoff * factor. A formula is becoming slightly different when you set property basedOnPreviousValue to false – firstBackoff * (factor ^ n). Here’s the appropriate configuration for our current test.

@ClassRule
public static MockServerContainer mockServer = new MockServerContainer();

@BeforeClass
public static void init() {
   System.setProperty("spring.cloud.gateway.routes[0].id", "account-service");
   System.setProperty("spring.cloud.gateway.routes[0].uri", "http://192.168.99.100:" + mockServer.getServerPort());
   System.setProperty("spring.cloud.gateway.routes[0].predicates[0]", "Path=/account/**");
   System.setProperty("spring.cloud.gateway.routes[0].filters[0]", "RewritePath=/account/(?<path>.*), /$\\{path}");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].name", "Retry");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.retries", "10");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.statuses", "INTERNAL_SERVER_ERROR");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.backoff.firstBackoff", "50ms");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.backoff.maxBackoff", "500ms");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.backoff.factor", "2");
   System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.backoff.basedOnPreviousValue", "true");
   MockServerClient client = new MockServerClient(mockServer.getContainerIpAddress(), mockServer.getServerPort());
   client.when(HttpRequest.request()
      .withPath("/1"), Times.exactly(3))
      .respond(response()
         .withStatusCode(500)
         .withBody("{\"errorCode\":\"5.01\"}")
         .withHeader("Content-Type", "application/json"));
   client.when(HttpRequest.request()
      .withPath("/1"))
      .respond(response()
         .withBody("{\"id\":1,\"number\":\"1234567891\"}")
         .withHeader("Content-Type", "application/json"));
   // OTHER RULES
}

If you run the same test one more time with a new configuration the logs look a little different. I have highlighted the most important differences in the picture below. As you see the current number of retries 10 only for HTTP 500 status. After setting a backoff policy the first retry attempt is performed after 50ms, the second after 100ms, the third after 200ms etc.

We have already analyzed the retry mechanism in Spring Cloud Gateway. Timeouts is another important aspect of request routing. With Spring Cloud Gateway we may easily set a global read and connect timeout. Alternatively, we may also define them for each route separately. Let’s add the following property to our test route definition. It sets a global timeout on 100ms. Now, our test route contains a test Retry filter with newly added global read timeout on 100ms.

System.setProperty("spring.cloud.gateway.httpclient.response-timeout", "100ms");

Alternatively, we may set timeout per single route. If we would prefer such a solution here a line we should add to our sample test.

System.setProperty("spring.cloud.gateway.routes[1].metadata.response-timeout", "100");

Then we define another test endpoint available under context path /2 with 200ms delay. Our current test method is pretty similar to the previous one, except that we are expecting HTTP 504 as a result.

@Test
public void testAccountServiceFail() {
   LOGGER.info("Sending /2...");
   ResponseEntity<Account> r = template.exchange("/account/{id}", HttpMethod.GET, null, Account.class, 2);
   LOGGER.info("Received: status->{}, payload->{}", r.getStatusCodeValue(), r.getBody());
   Assert.assertEquals(504, r.getStatusCodeValue());
}

Let’s run our test. The result is visible in the picture below. I have also highlighted the most important parts of the logs. After several failed retry attempts the delay between subsequent attempts has been set to the maximum backoff time – 500ms. Since the downstream service is delayed 100ms, the visible interval between retry attempts is around 600ms. Moreover, Retry filter by default handles IOException and TimeoutException, what is visible in the logs (exceptions parameter).

Summary

The current article is the last in series about traffic management in Spring Cloud Gateway. I have already described the following patterns: rate limiting, circuit breaker, fallback, failure retries and timeouts handling. That is only a part of Spring Cloud Gateway features. I hope that my articles help you in building API gateway for your microservices in an optimal way.

The post Timeouts and Retries In Spring Cloud Gateway appeared first on Piotr's TechBlog.

Example

As a code example in this article we will use a typical microservices architecture built with Spring Cloud. It consists of Spring Cloud Config Server, Eureka discovery, and Spring Cloud Gateway as API gateway. We also have three microservices, which expose the REST API and are hidden behind the gateway for an external client. Each of them is exposing OpenAPI documentation that may be accessed on the gateway using Swagger UI. The repository with source code is available on GitHub: https://github.com/piomin/sample-spring-microservices-new.git. This repository has been used as an example in another article, so it contains code not only for Springdoc library demo. The following picture shows the architecture of our system.

Implementating microservice with Springdoc OpenAPI

The first good news related to the Springdoc OpenAPI library is that it may exist together with the SpringFox library without any conflicts. This may simplify your migration into a new tool if anybody is using your Swagger documentation, for example for code generation of contract tests. To enable Springdoc for standard Spring MVC based application you need to include the following dependency into Maven pom.xml.

<dependency>
   <groupId>org.springdoc</groupId>
   <artifactId>springdoc-openapi-webmvc-core</artifactId>
   <version>1.2.32</version>
</dependency>

Each of our Spring Boot microservices is built on top of Spring MVC and provides endpoints for standard synchronous REST communication. However, the API gateway, which is built on top of Spring Cloud Gateway uses Netty as an embedded server and is based on reactive Spring WebFlux. It is also providing Swagger UI for accessing documentation exposed by all the microservices, so it must include a library that enables UI. The following two libraries must be included to enable Springdoc support for a reactive application based on Spring WebFlux.

<dependency>
   <groupId>org.springdoc</groupId>
   <artifactId>springdoc-openapi-webflux-core</artifactId>
   <version>1.2.31</version>
</dependency>
<dependency>
   <groupId>org.springdoc</groupId>
   <artifactId>springdoc-openapi-webflux-ui</artifactId>
   <version>1.2.31</version>
</dependency>

We can customize the default behavior of this library by setting properties in the Spring Boot configuration file or using @Beans. For example, we don’t want to generate OpenAPI manifests for all HTTP endpoints exposed by the application like Spring specific endpoints, so we may define a base package property for scanning as shown below. In our source code example each application YAML configuration file is located inside the config-service module.

springdoc:
  packagesToScan: pl.piomin.services.department

Here’s the main class of employee-service. We use @OpenAPIDefinition annotation to define a description for the application displayed on the Swagger site. As you see we can still have SpringFox enabled with @EnableSwagger2.

@SpringBootApplication
@EnableDiscoveryClient
@EnableSwagger2
@OpenAPIDefinition(info =
   @Info(title = "Employee API", version = "1.0", description = "Documentation Employee API v1.0")
)
public class EmployeeApplication {

   public static void main(String[] args) {
      SpringApplication.run(EmployeeApplication.class, args);
   }

}

OpenAPI on Spring Cloud Gateway

Once you start every microservice it will expose endpoint /v3/api-docs. We can customize that context by using property springdoc.api-docs.path in Spring configuration file. Since it is not required we may proceed to the implementation on the Spring Cloud Gateway. Springdoc doesn’t provide a similar class to SpringFox SwaggerResource, which has been used for exposing multiple APIs from different microservices in the previous article. Fortunately, there is a grouping mechanism that allows splitting OpenAPI definitions into different groups with a given name. To use it we need to declare a list of GroupOpenAPI beans.
Here’s the fragment of code inside gateway-service responsible for creating a list of OpenAPI resources handled by the gateway. First, we get all defined routes for services using RouteDefinitionLocator bean. Then we are fetching the id of each route and set it as a group name. As a result we have multiple OpenAPI resources under path /v3/api-docs/{SERVICE_NAME}, for example /v3/api-docs/employee.

@Autowired
RouteDefinitionLocator locator;

@Bean
public List<GroupedOpenApi> apis() {
   List<GroupedOpenApi> groups = new ArrayList<>();
   List<RouteDefinition> definitions = locator.getRouteDefinitions().collectList().block();
   definitions.stream().filter(routeDefinition -> routeDefinition.getId().matches(".*-service")).forEach(routeDefinition -> {
      String name = routeDefinition.getId().replaceAll("-service", "");
      GroupedOpenApi.builder().pathsToMatch("/" + name + "/**").setGroup(name).build();
   });
   return groups;
}

The API path like /v3/api-docs/{SERVICE_NAME} is not exactly what we want to achieve, because our routing to the downstream services is based on the service name fetched from discovery. So if you call address like http://localhost:8060/employee/** it is automatically load balanced between all registered instances of employee-service. Here’s the routes definition in gateway-service configuration.

spring:
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true
      routes:
      - id: employee-service
        uri: lb://employee-service
        predicates:
        - Path=/employee/**
        filters:
        - RewritePath=/employee/(?<path>.*), /$\{path}
      - id: department-service
        uri: lb://department-service
        predicates:
        - Path=/department/**
        filters:
        - RewritePath=/department/(?<path>.*), /$\{path}
      - id: organization-service
        uri: lb://organization-service
        predicates:
        - Path=/organization/**
        filters:
        - RewritePath=/organization/(?<path>.*), /$\{path}

Since Springdoc doesn’t allow us to customize the default behavior of the grouping mechanism to change the generated paths, we need to provide some workaround. My proposition is just to add a new route definition inside gateway configuration dedicated to Open API path handling. It rewrites path /v3/api-docs/{SERVICE_NAME} into /{SERVICE_NAME}/v3/api-docs, which is handled by the another routes responsible for interacting with Eureka discovery.

  - id: openapi
   uri: http://localhost:${server.port}
   predicates:
   - Path=/v3/api-docs/**
   filters:
   - RewritePath=/v3/api-docs/(?<path>.*), /$\{path}/v3/api-docs

Testing

To test our sample simple we need to run all microservice, config server, discovery and gateway. While microservices are available under a dynamically generated port, config server is available under 8888, discovery under 8061, and gateway under 8060. We can access each microservice by calling http://localhost:8060/{SERVICE_PATH}/**, for example http://localhost:8060/employee/**. The Swagger UI is available under address http://localhost:8060/swagger-ui.html. Before let’s take a look on Eureka after running all required Spring Boot applications.

After accessing Swagger UI exposed on the gateway you may see that we can choose between all three microservices registered in the discovery. This is exactly what we wanted to achieve.

Conclusion

Springdoc OpenAPI is compatible with OpenAPI 3, and supports Spring WebFlux, while SpringFox is not. Therefore, it seems that the choice is obvious especially if you are using reactive APIs or Spring Cloud Gateway. In this article I demonstrated you how to use Springdoc in microservices architecture with a gateway pattern.

The post Microservices API Documentation with Springdoc OpenAPI appeared first on Piotr's TechBlog.

1. Dependencies

To succesfully test some scenarios of using a circuit breaker pattern with Spring Cloud Gateway we need to include a reactive version of Spring Cloud Circuit Breaker since gateway is started on reactive Netty server. We will simulate downstream service using MockServer provided within the Testcontainers framework. It is provisioned inside the test by a mock client written in Java.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-cloud-starter-circuitbreaker-reactor-resilience4j</artifactId>
</dependency>
<dependency>
   <groupId>org.projectlombok</groupId>
   <artifactId>lombok</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-test</artifactId>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.testcontainers</groupId>
   <artifactId>mockserver</artifactId>
   <version>1.12.3</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.mock-server</groupId>
   <artifactId>mockserver-client-java</artifactId>
   <version>3.10.8</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>com.carrotsearch</groupId>
   <artifactId>junit-benchmarks</artifactId>
   <version>0.7.2</version>
   <scope>test</scope>
</dependency>

2. Enabling Spring Cloud Gateway Circuit Breaker with Resilience4J

To enable circuit breaker built on top of Resilience4J we need to declare a Customizer bean that is passed a ReactiveResilience4JCircuitBreakerFactory. The very simple configuration contains default circuit breaker settings and and defines timeout duration using TimeLimiterConfig. For the first test I decided to set 200 milliseconds timeout.

@Bean
public Customizer<ReactiveResilience4JCircuitBreakerFactory> defaultCustomizer() {
    return factory -> factory.configureDefault(id -> new Resilience4JConfigBuilder(id)
        .circuitBreakerConfig(CircuitBreakerConfig.ofDefaults())
        .timeLimiterConfig(TimeLimiterConfig.custom().timeoutDuration(Duration.ofMillis(200)).build())
        .build());
}

3. Building Test Class

In the next step we are creating a test class. Before running the test it is starting and provisioning an instance of mock server. We are defining two endpoints. The second of them /2 adds a delay of 200 milliseconds, which exceeds the timeout defined in the circuit breaker configuration.
We are also setting configuration of Spring Cloud Gateway route which is addressed to the currently started instance of mock server. To enable the circuit breaker for our route we have to define a CircuitBreaker filter with a given name. The test is repeated 200 times. It calls the delayed and not delayed endpoint in 50/50 proportion. Here’s the Spring Cloud Gateway test class.

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
@RunWith(SpringRunner.class)
public class GatewayCircuitBreakerTest {

    private static final Logger LOGGER = LoggerFactory.getLogger(GatewayRateLimiterTest.class);

    @Rule
    public TestRule benchmarkRun = new BenchmarkRule();

    @ClassRule
    public static MockServerContainer mockServer = new MockServerContainer();

    @Autowired
    TestRestTemplate template;
    int i = 0;

    @BeforeClass
    public static void init() {
        System.setProperty("spring.cloud.gateway.routes[0].id", "account-service");
        System.setProperty("spring.cloud.gateway.routes[0].uri", "http://192.168.99.100:" + mockServer.getServerPort());
        System.setProperty("spring.cloud.gateway.routes[0].predicates[0]", "Path=/account/**");
        System.setProperty("spring.cloud.gateway.routes[0].filters[0]", "RewritePath=/account/(?.*), /$\\{path}");
        System.setProperty("spring.cloud.gateway.routes[0].filters[1].name", "CircuitBreaker");
        System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.name", "exampleSlowCircuitBreaker");
        MockServerClient client = new MockServerClient(mockServer.getContainerIpAddress(), mockServer.getServerPort());
        client.when(HttpRequest.request()
            .withPath("/1"))
            .respond(response()
                .withBody("{\"id\":1,\"number\":\"1234567890\"}")
                .withHeader("Content-Type", "application/json"));
        client.when(HttpRequest.request()
            .withPath("/2"))
            .respond(response()
                .withBody("{\"id\":2,\"number\":\"1234567891\"}")
                .withDelay(TimeUnit.MILLISECONDS, 200)
                .withHeader("Content-Type", "application/json"));
    }

    @Test
    @BenchmarkOptions(warmupRounds = 0, concurrency = 1, benchmarkRounds = 200)
    public void testAccountService() {
        int gen = 1 + (i++ % 2);
        ResponseEntity r = template.exchange("/account/{id}", HttpMethod.GET, null, Account.class, gen);
        LOGGER.info("{}. Received: status->{}, payload->{}, call->{}", i, r.getStatusCodeValue(), r.getBody(), gen);
    }

}

Here’s the result of the currently discussed test. With default settings it opens the circuit after processing 100 requests with 50% error rate. The logs visible below include a sequence number of requests, HTTP response status code, response body and URL of the called endpoint.

We may change the default settings a little. To do that we should define a custom CircuitBreakerConfig. One of the properties we can customize is slidingWindowSize. The property slidingWindowSize defines how many outcome calls has to be recorded when a circuit breaker is closed. Assuming we have the same test endpoints what will happen if we change this value to 10 as shown below?

@Bean
public Customizer<ReactiveResilience4JCircuitBreakerFactory> defaultCustomizer() {
    return factory -> factory.configureDefault(id -> new Resilience4JConfigBuilder(id)
        .circuitBreakerConfig(CircuitBreakerConfig.custom()
            .slidingWindowSize(10)
            .build())
        .timeLimiterConfig(TimeLimiterConfig.custom().timeoutDuration(Duration.ofMillis(200)).build()).build());
}

Here’s the result. The circuit is open just after processing 10 requests when at least 50% of them are timeouted.

Moreover, we may change failureRateThreshold. This property is responsible for configuring the failure rate threshold in percentage. If the failure rate is equal or greater than the threshold the circuit breaker is switched to open and starts short-circuiting calls. It is not difficult to predict what will happen if we change it for our current scenario to 66.6F. The circuit will never be opened.

@Bean
public Customizer<ReactiveResilience4JCircuitBreakerFactory> defaultCustomizer() {
    return factory -> factory.configureDefault(id -> new Resilience4JConfigBuilder(id)
        .circuitBreakerConfig(CircuitBreakerConfig.custom()
            .slidingWindowSize(10)
            .failureRateThreshold(66.6F)
            .build())
        .timeLimiterConfig(TimeLimiterConfig.custom().timeoutDuration(Duration.ofMillis(200)).build()).build());
}

4. Spring Cloud Gateway circuit breaker Customization

We were starting with really basic samples. Let’s do something more interesting! First, we will set a really small value of sliding window size. It is set to 5. Thanks to that we will be able to observe the full result of the current test scenario after processing only a few requests. The next step is to modify the rules defined on the mock server. Now, we will delay only 5 first requests sent to the /2 endpoint. After receiving 5 requests it starts to work fine without adding any delay. Thanks to that fact our circuit breaker would be able to back from OPEN state to CLOSE after some time. But first things first, here are the code defining mock endpoints for the current test.

MockServerClient client = new MockServerClient(mockServer.getContainerIpAddress(), mockServer.getServerPort());
client.when(HttpRequest.request()
   .withPath("/1"))
   .respond(response()
      .withBody("{\"id\":1,\"number\":\"1234567890\"}")
      .withHeader("Content-Type", "application/json"));
client.when(HttpRequest.request()
   .withPath("/2"), Times.exactly(5))
   .respond(response()
      .withBody("{\"id\":2,\"number\":\"1234567891\"}")
      .withDelay(TimeUnit.MILLISECONDS, 200)
      .withHeader("Content-Type", "application/json"));
client.when(HttpRequest.request()
   .withPath("/2"))
   .respond(response()
      .withBody("{\"id\":2,\"number\":\"1234567891\"}")
      .withHeader("Content-Type", "application/json"));

As I mentioned before the slidingWindowSize is now equal to 5. If there are 3 timeouts during the last 5 calls the circuit is switched to OPEN state. We can configure how long the circuit should stay in the OPEN state without trying to process any request. The parameter waitDurationInOpenState, which is responsible for that, has been set to 30 milliseconds. Therefore, after 30 milliseconds the circuit is switched to HALF_OPEN state, which means that the incoming requests are processed again. We can also configure a number of permitted calls in the HALF_OPEN state. The property permittedNumberOfCallsInHalfOpenState is set to 5 instead of default value 10. In these five attempts, we get only 2 timeouts, since we set 5 repeats for delayed service on the mock server and the first 3 timeouts have been in the beginning before opening a circuit. Here’s our current configuration of Spring Cloud Circuit Breaker.

@Bean
public Customizer<ReactiveResilience4JCircuitBreakerFactory> defaultCustomizer() {
   return factory -> factory.configureDefault(id -> new Resilience4JConfigBuilder(id)
      .circuitBreakerConfig(CircuitBreakerConfig.custom()
         .slidingWindowSize(5)
         .permittedNumberOfCallsInHalfOpenState(5)
         .failureRateThreshold(50.0F)
         .waitDurationInOpenState(Duration.ofMillis(30))
         .build())
      .timeLimiterConfig(TimeLimiterConfig.custom().timeoutDuration(Duration.ofMillis(200)).build()).build());
}

The following diagram illustrates our scenario.

And here’s the result of our current test. The circuit has been opened after processing 6 requests. There were 3 incoming requests that had not been processed during 30 milliseconds of being in open state. After that time it has been switched to half open state and finally it moved back to close state.

What will happen if we increase the number of delayed requests in this scenario to 20?

client.when(HttpRequest.request()
   .withPath("/2"), Times.exactly(20))
   .respond(response()
      .withBody("{\"id\":2,\"number\":\"1234567891\"}")
      .withDelay(TimeUnit.MILLISECONDS, 200)
      .withHeader("Content-Type", "application/json"));

The circuit will be switched between OPEN and HALF_OPEN state until the downstream service is delaying the responses.

5. Adding Fallback

As you probably noticed, if the request to the downstream service has been finished with timeout the gateway returns HTTP status HTTP 504 - Gateway Timeout. Moreover, if a circuit is open the gateway is returning HTTP Status HTTP 503 - Service Unavailable. To prevent from returning error status code on the gateway we may enable fallback endpoint for our route. To do that we have to set property fallbackUri using forward: scheme. Here’s the current configuration of the test route. I included the endpoint /fallback/account as fallback URI.

System.setProperty("spring.cloud.gateway.routes[0].id", "account-service");
System.setProperty("spring.cloud.gateway.routes[0].uri", "http://192.168.99.100:" + mockServer.getServerPort());
System.setProperty("spring.cloud.gateway.routes[0].predicates[0]", "Path=/account/**");
System.setProperty("spring.cloud.gateway.routes[0].filters[0]", "RewritePath=/account/(?.*), /$\\{path}");
System.setProperty("spring.cloud.gateway.routes[0].filters[1].name", "CircuitBreaker");
System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.name", "exampleSlowCircuitBreaker");
System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.fallbackUri", "forward:/fallback/account");

The fallback endpoint is exposed on the gateway. I defined a simple controller class that implements a single fallback method.

@RestController
@RequestMapping("/fallback")
public class GatewayFallback {

    @GetMapping("/account")
    public Account getAccount() {
        Account a = new Account();
        a.setId(2);
        a.setNumber("123456");
        return a;
    }

}

Assuming we have exactly the same scenario as in the previous section the current test is returning only HTTP 200 instead of responses with HTTP 5xx as shown below.

6. Handling Slow Responses

In all previous examples we have set a short timeout on response, what results in HTTP 504 - Gateway Timeout or fallback. However, we don’t have to timeout the requests, but we can just set a threshold and failure rate for indicating slow responses. There are two parameters responsible for that: slowCallDurationThreshold and slowCallRateThreshold.

@Bean
public Customizer<ReactiveResilience4JCircuitBreakerFactory> defaultCustomizer() {
    return factory -> factory.configureDefault(id -> new Resilience4JConfigBuilder(id)
        .circuitBreakerConfig(CircuitBreakerConfig.custom()
           .slidingWindowSize(5)
           .permittedNumberOfCallsInHalfOpenState(5)
           .failureRateThreshold(50.0F)
           .waitDurationInOpenState(Duration.ofMillis(50))
           .slowCallDurationThreshold(Duration.ofMillis(200))
           .slowCallRateThreshold(50.0F)
           .build())
        .build());
}

Now, the delayed responses are not finished with timeout, however the circuit breaker is still recording these records. When the threshold is exceeded the circuit breaker is open as shown below.

The post Circuit Breaking In Spring Cloud Gateway With Resilience4J appeared first on Piotr's TechBlog.

1. Dependencies

We will test our sample application against Spring Cloud Gateway rate limiting under higher traffic. First, we need to include some dependencies. Of course, Spring Cloud Gateway starter is required. For handling rate limiter with Redis we also need to add dependency to spring-boot-starter-data-redis-reactive starter. Other dependencies are used for test purposes. Module mockserver provided within Testcontainers. It is responsible for mocking a target service. In turn, the library mockserver-client-java is used for integration with mockserver container during the test. And the last library junit-benchmarks is used for a benchmarking test method and running the test concurrently.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-data-redis-reactive</artifactId>
</dependency>
<dependency>
   <groupId>org.projectlombok</groupId>
   <artifactId>lombok</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-test</artifactId>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.testcontainers</groupId>
   <artifactId>mockserver</artifactId>
   <version>1.12.3</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.mock-server</groupId>
   <artifactId>mockserver-client-java</artifactId>
   <version>3.10.8</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>com.carrotsearch</groupId>
   <artifactId>junit-benchmarks</artifactId>
   <version>0.7.2</version>
   <scope>test</scope>
</dependency>

The sample application is built on top of Spring Boot 2.2.1.RELEASE and uses Spring Cloud Hoxton.RC2 Release Train.

<parent>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-parent</artifactId>
   <version>2.2.1.RELEASE</version>
</parent>

<properties>
   <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
   <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
   <java.version>11</java.version>
   <spring-cloud.version>Hoxton.RC2</spring-cloud.version>
</properties>

<dependencyManagement>
   <dependencies>
      <dependency>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-dependencies</artifactId>
         <version>${spring-cloud.version}</version>
         <type>pom</type>
         <scope>import</scope>
      </dependency>
   </dependencies>
</dependencyManagement>

2. Implementation

Request rate limiting is realized using a Spring Cloud Gateway component called GatewayFilter. Each instance of this filter is constructed in a specific factory. Filter is of course responsible for modifying requests and responses before or after sending the downstream request. Currently, there are 30 available built-in gateway filter factories.
The GatewayFilter takes an optional keyResolver parameter and parameters specific to the rate limiter implementation (in that case an implementation using Redis). Parameter keyResolver is a bean that implements the KeyResolver interface. It allows you to apply different strategies to derive the key for limiting requests. Parameter keyResolver is a bean that implements the KeyResolver interface. It allows you to apply different strategies to derive the key for limiting requests. Following Spring Cloud Gateway documentation:

The default implementation of KeyResolver is the PrincipalNameKeyResolver which retrieves the Principal from the ServerWebExchange and calls Principal.getName(). By default, if the KeyResolver does not find a key, requests will be denied. This behavior can be adjusted with the spring.cloud.gateway.filter.request-rate-limiter.deny-empty-key (true or false) and spring.cloud.gateway.filter.request-rate-limiter.empty-key-status-code properties.

Since, we have discussed some theoretical aspects of Spring Cloud Gateway rate limiting we may proceed to the implementation. First, let’s define the main class and very simple KeyResolver bean, that is always equal to one.

@SpringBootApplication
public class GatewayApplication {

   public static void main(String[] args) {
      SpringApplication.run(GatewayApplication.class, args);
   }

   @Bean
   KeyResolver userKeyResolver() {
      return exchange -> Mono.just("1");
   }
}

Assuming we have the following configuration and a target application running on port 8091 we may perform some test calls. You may set two properties for customizing the process. The redis-rate-limiter.replenishRate decides how many requests per second a user is allowed to send without any dropped requests. This is the rate that the token bucket is filled. The second property redis-rate-limiter.burstCapacity is the maximum number of requests a user is allowed to do in a single second. This is the number of tokens the token bucket can hold. Setting this value to zero will block all requests.

server:
  port: ${PORT:8085}

spring:
  application:
    name: gateway-service
  redis:
    host: 192.168.99.100
    port: 6379
  cloud:
    gateway:
      routes:
      - id: account-service
        uri: http://localhost:8091
        predicates:
        - Path=/account/**
        filters:
        - RewritePath=/account/(?.*), /$\{path}
      - name: RequestRateLimiter
          args:
            redis-rate-limiter.replenishRate: 10
            redis-rate-limiter.burstCapacity: 20

Now, if you call the endpoint exposed by the gateway you get the following response. It includes some specific headers, which are prefixed by x-ratelimit. Header x-ratelimit-burst-capacity indicates to burstCapacity value, x-ratelimit-replenish-rate indicates to replenishRate value, and the most important x-ratelimit-remaining, which shows you the number of requests you may send in the next second.

If you exceed the number of allowed requests Spring Cloud Gateway return response with code HTTP 429 - Too Many Requests, and will not process the incoming request.

3. Testing Spring Cloud Gateway rate limiting

We have the Spring Boot test that uses two Docker containers provided by Testcontainers: MockServer and Redis. Because the exposed port is generated dynamically we need to set gateway properties in @BeforeClass method before running the test. Inside the init method we also use MockServerClient to define mock service on the mock server container. Our test method is running concurrently in six threads and is repeated 600 times.

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.DEFINED_PORT)
@RunWith(SpringRunner.class)
public class GatewayRateLimiterTest {

    private static final Logger LOGGER = LoggerFactory.getLogger(GatewayRateLimiterTest.class);

    @Rule
    public TestRule benchmarkRun = new BenchmarkRule();

    @ClassRule
    public static MockServerContainer mockServer = new MockServerContainer();
    @ClassRule
    public static GenericContainer redis = new GenericContainer("redis:5.0.6").withExposedPorts(6379);

    @Autowired
    TestRestTemplate template;

    @BeforeClass
    public static void init() {
        System.setProperty("spring.cloud.gateway.routes[0].id", "account-service");
        System.setProperty("spring.cloud.gateway.routes[0].uri", "http://192.168.99.100:" + mockServer.getServerPort());
        System.setProperty("spring.cloud.gateway.routes[0].predicates[0]", "Path=/account/**");
        System.setProperty("spring.cloud.gateway.routes[0].filters[0]", "RewritePath=/account/(?<path>.*), /$\\{path}");
        System.setProperty("spring.cloud.gateway.routes[0].filters[1].name", "RequestRateLimiter");
        System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.redis-rate-limiter.replenishRate", "10");
        System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.redis-rate-limiter.burstCapacity", "20");
        System.setProperty("spring.redis.host", "192.168.99.100");
        System.setProperty("spring.redis.port", "" + redis.getMappedPort(6379));
        new MockServerClient(mockServer.getContainerIpAddress(), mockServer.getServerPort())
                .when(HttpRequest.request()
                        .withPath("/1"))
                .respond(response()
                        .withBody("{\"id\":1,\"number\":\"1234567890\"}")
                        .withHeader("Content-Type", "application/json"));
    }

    @Test
    @BenchmarkOptions(warmupRounds = 0, concurrency = 6, benchmarkRounds = 600)
    public void testAccountService() {
        ResponseEntity<Account> r = template.exchange("/account/{id}", HttpMethod.GET, null, Account.class, 1);
        LOGGER.info("Received: status->{}, payload->{}, remaining->{}", r.getStatusCodeValue(), r.getBody(), r.getHeaders().get("X-RateLimit-Remaining"));
    }

}

Let’s take a look at the test result. After starting gateway allows a user to send max 20 requests in a single second. After exceeding this value it starts to return HTTP 429.

After dropping some incoming requests gateway starts to accept them in the next second. But this time it allows to process only 10 requests, which is equal to replenishRate parameter value.

The source code is available in GitHub repository: https://github.com/piomin/sample-spring-cloud-gateway.git.

The post Rate Limiting In Spring Cloud Gateway With Redis appeared first on Piotr's TechBlog.