In general, we will cover the following topics in this article:

• Using Spring Boot 3 in cloud-native development
• Provide service discovery for all microservices with Spring Cloud Netflix Eureka. Anticipating your questions – yes, Eureka is still there. It’s the last of Netflix microservices components still available in Spring Cloud
• Spring Cloud OpenFeign in inter-service communication
• Distributed configuration with Spring Cloud Config
• API Gateway pattern with Spring Cloud Gateway including a global OpenAPI documentation with the Springdoc project
• Collecting traces with Micrometer OpenTelemetry and Zipkin

Fortunately, the migration from Spring Boot 2 to 3 is not a painful process. You can even check it out in my example repository, which was originally written in Spring Boot 2. The list of changes is not large. However, times have changed during the last five years… And we will begin our considerations from that point.

Running Environment

Here are the results of my quick 1-day voting poll run on Twitter. I assume that those results are meaningful since around 900 people voted. As you probably expect, currently, the first-choice platform for running your Spring Boot microservices is Kubernetes. I don’t have a survey conducted five years ago, but the results would probably be significantly different. Even if you had Kubernetes in your organization 5 years ago, you were probably starting a migration of your apps or at least it was in progress. Of course, there might be some exceptions, but I’m thinking about the vast majority.

You could migrate to Kubernetes during that time, but also Kubernetes ecosystem has changed a lot. There are many useful tools and platform services you may easily integrate with your apps. We can at least mention Kubernetes native solutions like service mesh (e.g. Istio) or serverless (e.g. Knative). The main question here is: if I’m running microservices on Kubernetes are Spring Cloud components still relevant? The answer is: in most cases no. Of course, you can still use Eureka for service discovery, Spring Cloud Config for a distributed configuration, or Spring Cloud Gateway for the API gateway pattern. However, you can easily replace them with Kubernetes built-in mechanisms and additional platform services.

To conclude, this article is not aimed at Kubernetes users. It shows how to easily run microservices architecture anywhere. If you are looking for staff mainly related to Kubernetes you can read my articles about the best practices for Java apps and microservices there.

Source Code

If you would like to try it by yourself, you may always take a look at my source code. In order to do that you need to clone my GitHub repository. Then you should just follow my instructions.

Before we proceed to the source code, let’s take a look at the following diagram. It illustrates the architecture of our sample system. We have three independent Spring Boot 3 microservices, which register themself in service discovery, fetch properties from the configuration service, and communicate with each other. The whole system is hidden behind the API gateway. Our Spring Boot 3 microservices send traces to the Zipkin instance using the Micrometer OTEL project.

Currently, the newest version of Spring Cloud is 2022.0.1. This version of spring-cloud-dependencies should be declared as a BOM for dependency management.

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-dependencies</artifactId>
      <version>2022.0.1</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

Step 1: Configuration Server with Spring Cloud Config

To enable Spring Cloud Config feature for an application, we should first include spring-cloud-config-server to your project dependencies.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-config-server</artifactId>
</dependency>

Then enable running the embedded configuration server during application boot use @EnableConfigServer annotation.

@SpringBootApplication
@EnableConfigServer
public class ConfigApplication {

   public static void main(String[] args) {
      new SpringApplicationBuilder(ConfigApplication.class).run(args);
   }

}

By default Spring Cloud Config Server stores the configuration data inside the Git repository. We will change that behavior by activating the native mode. In this mode, Spring Cloud Config Server reads property sources from the classpath. We place all the YAML property files inside src/main/resources/config. Here’s the config server application.yml file. It activates the native mode and overrides a default port to 8088.

server:
  port: 8088
spring:
  profiles:
    active: native

The YAML filename will be the same as the name of the service. For example, the YAML file of discovery-service is located here: src/main/resources/config/discovery-service.yml. Besides a default profile, we will also define the custom docker profile. Therefore the name of the config file will contain the docker suffix. On the default profile, we are connecting services through localhost with dynamically assigned ports. So, the typical configuration file for the default profile will look like that:

server:
  port: 0

eureka:
  client:
    serviceUrl:
      defaultZone: http://localhost:8061/eureka/

Here’s the typical configuration file for the default profile:

server:
  port: 8080

eureka:
  client:
    serviceUrl:
      defaultZone: http://discovery-service:8061/eureka/

In order to connect the config server on the client side we need to include the following module in Maven dependencies:

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-config</artifactId>
</dependency>

Depending on the running environment (localhost or docker) we need to provide different addresses for the config server:

spring:
  config:
    import: "optional:configserver:http://config-service:8088"
    activate:
      on-profile: docker
---
spring:
  application:
    name: discovery-service
  config:
    import: "optional:configserver:http://localhost:8088"

Step 2: Discovery Server with Spring Cloud Netflix Eureka

Of course, you can replace Eureka with any other discovery server supported by Spring Cloud. It can be Consul, Alibaba Nacos, or Zookeeper. The best way to run the Eureka server is just to embed it into the Spring Boot app. In order to do that, we first need to include the following Maven dependency:

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-netflix-eureka-server</artifactId>
</dependency>

Then we need to set the @EnableEurekaServer annotation on the main class.

@SpringBootApplication
@EnableEurekaServer
public class DiscoveryApplication {

   public static void main(String[] args) {
      new SpringApplicationBuilder(DiscoveryApplication.class).run(args);
   }

}

There is nothing new with that. As I already mentioned, the configuration files, discovery-service.yml or discovery-service-docker.yml, should be placed inside config-service module. We have changed Eureka’s running port from the default value (8761) to 8061. For the standalone Eureka instance, we have to disable registration and omit to fetch the registry. We just want to activate a single-node, demo discovery server.

server:
  port: 8061

eureka:
  instance:
    hostname: localhost
  client:
    registerWithEureka: false
    fetchRegistry: false
    serviceUrl:
      defaultZone: http://${eureka.instance.hostname}:${server.port}/eureka/

Once you have successfully started the application you may visit Eureka Dashboard available under the address http://localhost:8061/.

Step 3: Build Apps with Spring Boot 3 and Spring Cloud

Let’s take a look at a list of required Maven modules for our microservices. Each app has to get a configuration from the config-service and needs to register itself in the discovery-service. It also exposes REST API, automatically generates API documentation, and export tracing info to the Zipkin instance. We use the springdoc-openapi v2 library dedicated to Spring Boot 3. It generates documentation in both JSON and YAML formats available under the v3/api-docs path (or /v3/api-docs.yaml for the YAML format). In order to export traces to the Zipkin server, we will include the opentelemetry-exporter-zipkin module.

<dependencies>
  <dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-config</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
  </dependency>
  <dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-tracing-bridge-otel</artifactId>
  </dependency>
  <dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-exporter-zipkin</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springdoc</groupId>
    <artifactId>springdoc-openapi-starter-webmvc-api</artifactId>
    <version>2.0.2</version>
  </dependency>
</dependencies>

For the apps that call other services, we also need to include a declarative REST client. We will use Spring Cloud OpenFeign.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>

OpenFeign client automatically integrates with the service discovery. We just to set the name under which it is registered in Eureka inside the @FeingClient annotation. In order to create a client, we need to define an interface containing all the endpoints it has to call.

@FeignClient(name = "employee-service")
public interface EmployeeClient {

   @GetMapping("/organization/{organizationId}")
   List<Employee> findByOrganization(@PathVariable("organizationId") Long organizationId);
	
}

During the demo, we will send all the traces to Zipkin. It requires setting the value of the probability parameter to 1.0. In order to override the default URL of Zipkin we need to use the management.zipkin.tracing.endpoint property.

management:
  tracing:
    sampling:
      probability: 1.0
  zipkin:
    tracing:
      endpoint: http://zipkin:9411/api/v2/spans

Here’s the implementation of the @RestController in department-service. It injects the repository bean to interact with the database, and the Feign client bean to communicate with employee-service. The rest of the code is pretty simple.

@RestController
public class DepartmentController {

  private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);

  DepartmentRepository repository;
  EmployeeClient employeeClient;

  public DepartmentController(DepartmentRepository repository, EmployeeClient employeeClient) {
    this.repository = repository;
    this.employeeClient = employeeClient;
  }

  @PostMapping("/")
  public Department add(@RequestBody Department department) {
    LOGGER.info("Department add: {}", department);
    return repository.add(department);
  }
	
  @GetMapping("/{id}")
  public Department findById(@PathVariable("id") Long id) {
    LOGGER.info("Department find: id={}", id);
    return repository.findById(id);
  }
	
  @GetMapping("/")
  public List<Department> findAll() {
    LOGGER.info("Department find");
    return repository.findAll();
  }
	
  @GetMapping("/organization/{organizationId}")
  public List<Department> findByOrganization(@PathVariable("organizationId") Long organizationId) {
    LOGGER.info("Department find: organizationId={}", organizationId);
    return repository.findByOrganization(organizationId);
  }
	
  @GetMapping("/organization/{organizationId}/with-employees")
  public List<Department> findByOrganizationWithEmployees(@PathVariable("organizationId") Long organizationId) {
    LOGGER.info("Department find: organizationId={}", organizationId);
    List<Department> departments = repository.findByOrganization(organizationId);
    departments.forEach(d -> d.setEmployees(employeeClient.findByDepartment(d.getId())));
    return departments;
  }
	
}

As you see there are almost no differences in the app implementation between Spring Boot 2 and 3. The only thing you would have to do is to change all the javax.persistence to the jakarta.persistance.

Step 4: API Gateway with Spring Cloud Gateway

A gateway-service is the last app in our microservices architecture with Spring Boot 3. Beginning from Spring Boot 2 Spring Cloud Gateway replaced Netflix Zuul. We can also install it on Kubernetes using, for example, the Helm chart provided by VMWare Tanzu.

We will create a separate application with the embedded gateway. In order to do that we need to include Spring Cloud Gateway Starter in the Maven dependencies. Since our gateway has to interact with discovery and config services, it also includes Eureka Client Starter and Spring Cloud Config Starter. We don’t want to use it just as a proxy to the downstream services, but also we expose there OpenAPI documentation generated by all the apps. Since Spring Cloud Gateway is built on top of Spring WebFlux, we need to include Springdoc starters dedicated to that project.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-config</artifactId>
</dependency>
<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-tracing-bridge-otel</artifactId>
</dependency>
<dependency>
  <groupId>io.opentelemetry</groupId>
<artifactId>opentelemetry-exporter-zipkin</artifactId>
</dependency>
<dependency>
  <groupId>org.springdoc</groupId>
  <artifactId>springdoc-openapi-starter-webflux-api</artifactId>
  <version>2.0.2</version>
</dependency>
<dependency>
  <groupId>org.springdoc</groupId>
  <artifactId>springdoc-openapi-starter-webflux-ui</artifactId>
  <version>2.0.2</version>
</dependency>

In order to expose OpenAPI documentation from multiple v3/api-docs endpoints we need to use the GroupedOpenApi object. It should provide a way to switch between documentation generated by employee-service, department-service and organization-service. Those services run on dynamic addresses (or at least random ports). In that case, we will use the RouteDefinitionLocator bean to grab the current URL of each service. Then we just need to filter a list of routes to find only those related to our three microservices. Finally, we create the GroupedOpenApi containing a service name and path.

@SpringBootApplication
public class GatewayApplication {

   private static final Logger LOGGER = LoggerFactory
      .getLogger(GatewayApplication.class);

   public static void main(String[] args) {
      SpringApplication.run(GatewayApplication.class, args);
   }

   @Autowired
   RouteDefinitionLocator locator;

   @Bean
   public List<GroupedOpenApi> apis() {
      List<GroupedOpenApi> groups = new ArrayList<>();
      List<RouteDefinition> definitions = locator
         .getRouteDefinitions().collectList().block();
      assert definitions != null;
      definitions.stream().filter(routeDefinition -> routeDefinition
         .getId()
         .matches(".*-service"))
         .forEach(routeDefinition -> {
            String name = routeDefinition.getId()
               .replaceAll("-service", "");
            groups.add(GroupedOpenApi.builder()
               .pathsToMatch("/" + name + "/**").group(name).build());
         });
      return groups;
   }

}

Here’s the configuration of gateway-service. We should enable integration with the discovery server by setting the property spring.cloud.gateway.discovery.locator.enabled to true. Then we may proceed to define the route rules. We use the Path Route Predicate Factory for matching the incoming requests, and the RewritePath GatewayFilter Factory for modifying the requested path to adapt it to the format exposed by downstream services. The uri parameter specifies the name of the target service registered in the discovery server. For example, organization-service is available on the gateway under the /organization/** path thanks to the predicate Path=/organization/**, and the rewrite path from /organization/** to the /**.

spring:
  output:
    ansi:
      enabled: always
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true
      routes:
      - id: employee-service
        uri: lb://employee-service
        predicates:
        - Path=/employee/**
        filters:
        - RewritePath=/employee/(?<path>.*), /$\{path}
      - id: department-service
        uri: lb://department-service
        predicates:
        - Path=/department/**
        filters:
        - RewritePath=/department/(?<path>.*), /$\{path}
      - id: organization-service
        uri: lb://organization-service
        predicates:
        - Path=/organization/**
        filters:
        - RewritePath=/organization/(?<path>.*), /$\{path}
      - id: openapi
        uri: http://localhost:${server.port}
        predicates:
        - Path=/v3/api-docs/**
        filters:
        - RewritePath=/v3/api-docs/(?<path>.*), /$\{path}/v3/api-docs

springdoc:
  swagger-ui:
    urls:
      - name: employee
        url: /v3/api-docs/employee
      - name: department
        url: /v3/api-docs/department
      - name: organization
        url: /v3/api-docs/organization

As you see above, we are also creating a dedicated route for Springdoc OpenAPI. It rewrites the path for the /v3/api-docs context to serve it properly in the Swagger UI.

Step 5: Running Spring Boot 3 Microservices

Finally, we can run all our microservices. With the current configuration in the repository, you can start them directly on your laptop or with Docker containers.

Option 1: Starting directly on the laptop

In total, we have 6 apps to run: 3 microservices, a discovery server, a config server, and a gateway. We also need to run Zipkin to collect and store traces from communication between the services. In the first step, we should start the config-service. We can use Spring Boot Maven plugin for that. Just go to the config-service directory and the following command. It is exposed on the 8088 port.

$ mvn spring-boot:run

We should repeat the same step for all the other apps. The discovery-service is listening on the 8061 port, while the gateway-service on the 8060 port. Microservices will start on the dynamically generated port number thanks to the server.port=0 property in config. In the final step, we can run Zipkin using its Docker container with the following command:

$ docker run -d --name zipkin -p 9411:9411 openzipkin/zipkin

Option 2: Build images and run them with Docker Compose

In the first step, we will build the whole Maven project and Docker images for all the apps. I created a profile build-image that needs to be activated to build images. It mostly uses the build-image step provided by the Spring Boot Maven Plugin. However, for config-service and discovery-service I’m using Jib because it is built on top of the base image with curl installed. For both these services Docker compose needs to verify health checks before starting other containers.

$ mvn clean package -Pbuild-image

The docker-compose.yml is available in the repository root directory. The whole file is visible below. We need to run config-service before all other apps since it provides property sources. Secondly, we should start discovery-service. In both these cases, we are defining a health check that tests the HTTP endpoint using curl inside the container. Once we start and verify config-service and discovery-service we may run gateway-service and all the microservices. All the apps are running with the docker Spring profile activated thanks to the SPRING_PROFILES_ACTIVE environment variable. It corresponds to the spring.profiles.active param that may be defined in configuration properties.

version: "3.7"
services:
  zipkin:
    container_name: zipkin
    image: openzipkin/zipkin
    ports:
      - "9411:9411"
  config-service:
    image: piomin/config-service:1.1-SNAPSHOT
    ports:
      - "8088:8088"
    healthcheck:
      test: curl --fail http://localhost:8088/employee/docker || exit 1
      interval: 5s
      timeout: 2s
      retries: 3
  discovery-service:
    image: piomin/discovery-service:1.1-SNAPSHOT
    ports:
      - "8061:8061"
    depends_on:
      config-service:
        condition: service_healthy
    links:
      - config-service
    healthcheck:
      test: curl --fail http://localhost:8061/eureka/v2/apps || exit 1
      interval: 4s
      timeout: 2s
      retries: 3
    environment:
      SPRING_PROFILES_ACTIVE: docker
  employee-service:
    image: piomin/employee-service:1.2-SNAPSHOT
    ports:
      - "8080"
    depends_on:
      discovery-service:
        condition: service_healthy
    links:
      - config-service
      - discovery-service
      - zipkin
    environment:
      SPRING_PROFILES_ACTIVE: docker
  department-service:
    image: piomin/department-service:1.2-SNAPSHOT
    ports:
      - "8080"
    depends_on:
      discovery-service:
        condition: service_healthy
    links:
      - config-service
      - discovery-service
      - employee-service
      - zipkin
    environment:
      SPRING_PROFILES_ACTIVE: docker
  organization-service:
    image: piomin/organization-service:1.2-SNAPSHOT
    ports:
      - "8080"
    depends_on:
      discovery-service:
        condition: service_healthy
    links:
      - config-service
      - discovery-service
      - employee-service
      - department-service
      - zipkin
    environment:
      SPRING_PROFILES_ACTIVE: docker
  gateway-service:
    image: piomin/gateway-service:1.1-SNAPSHOT
    ports:
      - "8060:8060"
    depends_on:
      discovery-service:
        condition: service_healthy
    environment:
      SPRING_PROFILES_ACTIVE: docker
    links:
      - config-service
      - discovery-service
      - employee-service
      - department-service
      - organization-service
      - zipkin

Finally, let’s run all the apps using Docker Compose:

$ docker-compose up

Try it out

Once you start all the apps you can perform some test calls to the services through the gateway-service. It listening on the 8060 port. There is some test data automatically generated during startup. You can call the following endpoint to test all the services and communication between them:

$ curl http://localhost:8060/employee/
$ curl http://localhost:8060/department/organization/1
$ curl http://localhost:8060/department/organization/1/with-employees
$ curl http://localhost:8060/organization/
$ curl http://localhost:8060/organization/1/with-departments

Here are the logs generated by the apps during the calls visible above:

Let’s display Swagger UI exposed on the gateway. You can easily switch between contexts for all three microservices as you see below:

We can go to the Zipkin dashboard to verify the collected traces:

Final Thoughts

Treat this article as a quick guide to the most common components related to microservices with Spring Boot 3. I focused on showing you some new features since my last article on this topic. You could read how to implement tracing with Micrometer OpenTelemetry, generate API docs with Springdoc, or build Docker images with Spring Boot Maven Plugin.

The post Microservices with Spring Boot 3 and Spring Cloud appeared first on Piotr's TechBlog.

We will use KrakenD. Why? Although it is not the most popular API gateway tool, it seems to be very interesting. First of all, it is very fast and lightweight. Also, we can easily integrate it with Zipkin and Consul – which obviously is our goal in this article.

Source Code

In this article, I will use the source code from my previous article Guide to Microservices with Micronaut and Consul. Since it has been written two years ago, I had to update a version of the Micronaut framework. Fortunately, the only thing I also had to change was groupId of the micronaut-openapi artifact. After that change, everything was working perfectly fine.

If you would like to try it by yourself, you may always take a look at my source code. In order to do that you need to clone my GitHub repository. Then you should just follow my instructions

Architecture

Firstly, let’s take a look at the architecture of our sample system. We have three microservices: employee-service, department-service and organization-service. All of them are simple REST applications. The organization-service calls API exposed by the department-service, while the department-service calls API from the employee-service. They use Consul discovery to locate a network address of target microservices. They also send traces to Zipkin. Each application may be started in multiple instances. At the edge of our system, there is an API gateway – KrakenD. Krakend is integrating with Consul discovery through DNS. It also sends traces to Zipkin. The architecture is visible in the picture below.

Running Consul, Zipkin and microservices

In the first step, we are going to run Consul and Zipkin on Docker containers. The simplest way to start Consul is to run it in the development mode. To do that you should execute the following command. It is important to expose two ports 8500 and 8600. The first of them is responsible for the discovery, while the second for DNS.

$ docker run -d --name=consul \
   -p 8500:8500 -p 8600:8600/udp \
   -e CONSUL_BIND_INTERFACE=eth0 consul

Then, we need to run Zipkin. Don’t forget to expose port 9411.

$ docker run -d --name=zipkin -p 9411:9411 openzipkin/zipkin

Finally, we can run each of our application. They register themselves in Consul on startup. They are listening on the randomly generated port number. Here’s the common. configuration for every single Micronaut application.

micronaut:
  server:
    port: -1
  router:
    static-resources:
      swagger:
        paths: classpath:META-INF/swagger
        mapping: /swagger/**
endpoints:
  info:
    enabled: true
    sensitive: false
consul:
  client:
    registration:
      enabled: true
tracing:
  zipkin:
    enabled: true
    http:
      url: http://localhost:9411
    sampler:
      probability: 1

Consul acts as a configuration server for the applications. We use Micronaut Config Client for fetching property sources on startup.

micronaut:
  application:
    name: employee-service
  config-client:
    enabled: true
consul:
  client:
    defaultZone: "localhost:8500"
    config:
      format: YAML

Using Micronaut framework

In order to expose REST API, integrate with Consul and Zipkin we need to include the following dependencies.

<dependencies>
        <dependency>
            <groupId>io.micronaut</groupId>
            <artifactId>micronaut-http-server-netty</artifactId>
        </dependency>
        <dependency>
            <groupId>io.micronaut</groupId>
            <artifactId>micronaut-tracing</artifactId>
        </dependency>
        <dependency>
            <groupId>io.zipkin.brave</groupId>
            <artifactId>brave-instrumentation-http</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>io.zipkin.reporter2</groupId>
            <artifactId>zipkin-reporter</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>io.opentracing.brave</groupId>
            <artifactId>brave-opentracing</artifactId>
        </dependency>
        <dependency>
            <groupId>io.micronaut</groupId>
            <artifactId>micronaut-discovery-client</artifactId>
        </dependency>
<dependencies>

The tracing headers (spans) are propagated across applications. Here’s the endpoint in department-service. It calls endpoint GET /employees/department/{departmentId} exposed by employee-service.

@Get("/organization/{organizationId}/with-employees")
@ContinueSpan
public List<Department> findByOrganizationWithEmployees(@SpanTag("organizationId") Long organizationId) {
   LOGGER.info("Department find: organizationId={}", organizationId);
   List<Department> departments = repository.findByOrganization(organizationId);
   departments.forEach(d -> d.setEmployees(employeeClient.findByDepartment(d.getId())));
   return departments;
}

In order to call employee-service endpoint we use Micronaut declarative REST client.

@Client(id = "employee-service", path = "/employees")
public interface EmployeeClient {
   @Get("/department/{departmentId}")
   List<Employee> findByDepartment(Long departmentId);	
}

Here’s the implementation of the GET /employees/department/{departmentId} endpoint inside employee-service. Micronaut propagates tracing spans between subsequent requests using the @ContinueSpan annotation.

@Get("/department/{departmentId}")
@ContinueSpan
public List<Employee> findByDepartment(@SpanTag("departmentId") Long departmentId) {
    LOGGER.info("Employees find: departmentId={}", departmentId);
    return repository.findByDepartment(departmentId);
}

Configure KrakenD Gateway and Consul DNS

We can configure KrakenD using JSON notation. Firstly, we need to define endpoints. The integration with Consul discovery needs to be configured in the backend section. The host has to be the same as the DNS name of a downstream service in Consul. We also set a target URL (url_pattern) and service discovery type (sd). Let’s take a look at the list of endpoints for department-service. We expose methods for searching by id (GET /department/{id}), adding a new department (POST /department), and searching all departments with a list of employees within a single organization (GET /department-with-employees/{organizationId}).

    {
      "endpoint": "/department/{id}",
      "method": "GET",
      "backend": [
        {
          "url_pattern": "/departments/{id}",
          "sd": "dns",
          "host": [
            "department-service.service.consul"
          ],
          "disable_host_sanitize": true
        }
      ]
    },
    {
      "endpoint": "/department-with-employees/{organizationId}",
      "method": "GET",
      "backend": [
        {
          "url_pattern": "/departments/organization/{organizationId}/with-employees",
          "sd": "dns",
          "host": [
            "department-service.service.consul"
          ],
          "disable_host_sanitize": true
        }
      ]
    },
    {
      "endpoint": "/department",
      "method": "POST",
      "backend": [
        {
          "url_pattern": "/departments",
          "sd": "dns",
          "host": [
            "department-service.service.consul"
          ],
          "disable_host_sanitize": true
        }
      ]
    }

It is also worth mentioning that we cannot create conflicting routes with Krakend. For example, I couldn’t define endpoint GET /department/organization/{organizationId}/with-employees, because it would conflict with the already existing endpoint GET /department/{id}. To avoid it I created a new context path /department-with-employees for my endpoint.

Similarly, I created a following configuration for employee-service endpoints.

    {
      "endpoint": "/employee/{id}",
      "method": "GET",
      "backend": [
        {
          "url_pattern": "/employees/{id}",
          "sd": "dns",
          "host": [
            "employee-service.service.consul"
          ],
          "disable_host_sanitize": true
        }
      ]
    },
    {
      "endpoint": "/employee",
      "method": "POST",
      "backend": [
        {
          "url_pattern": "/employees",
          "sd": "dns",
          "host": [
            "employee-service.service.consul"
          ],
          "disable_host_sanitize": true
        }
      ]
    }

In order to integrate KrakenD with Consul, we need to configure local DNS properly on our machine. It was a quite challenging task for me since I’m not very familiar with network topics. By default, Consul will listen on port 8600 for DNS queries in the consul domain. But DNS is served from port 53. Therefore, we need to configure DNS forwarding for Consul service discovery. There are several ways to do that, and you may read more about it here. I chose the dnsmasq tool for that. Following the guide, we need to create a file e.g. /etc/dnsmasq.d/10-consul with the following single line.

server=/consul/127.0.0.1#8600

Finally we need to start dnsmasq service and add 127.0.0.1 to the list of nameservers. Here’s my configuration of DNS servers.

Testing Consul DNS

Firstly, let’s run all our sample microservices. They are registered in Consul under the following names.

I run two instances of employee-service. Of course, all the applications are listening on a randomly generated port.

Finally, if you run the dig command with the DNS name of service you should have a similar response to the following. It means we may proceed to the last part of our exercise!

Running KrakenD API Gateway

Before we run KrakenD API gateway let’s configure one additional thing – integration with Zipkin. To do that we need to create section extra_config. Enabling Zipkin only requires us to add the zipkin exporter in the opencensus module. We need to URL (including port and path) where Zipkin is accepting the spans, and a service name for Krakend spans. I have also enabled metrics. Here’s the currently described part of the KrakenD configuration.

  "extra_config": {
    "github_com/devopsfaith/krakend-opencensus": {
      "sample_rate": 100,
      "reporting_period": 1,
      "exporters": {
        "zipkin": {
          "collector_url": "http://localhost:9411/api/v2/spans",
          "service_name": "api-gateway"
        }
      }
    },
    "github_com/devopsfaith/krakend-metrics": {
      "collection_time": "30s",
      "proxy_disabled": false,
      "listen_address": ":8090"
    }
  }

Finally, we can run KrakenD. The only parameter we need to pass is the location of the krakend.json configuration file. You may find a full version of that file in my GitHub repository inside the config directory.

$ krakend run -c krakend.json -d

Testing KrakenD with Consul and Micronaut

Once we started all our microservices, Consul, Zipkin, and KrakenD we may proceed to the tests. So first, let’s add some employees and departments by sending requests through an API gateway. KrakenD is listening on port 8080.

$ curl http://localhost:8080/employee -d '{"name":"John Smith","age":30,"position":"Architect","departmentId":1,"organizationId":1}' -H "Content-Type: application/json"
{"age":30,"departmentId":1,"id":1,"name":"John Smith","organizationId":1,"position":"Architect"}

$ curl http://localhost:8080/employee -d '{"name":"Paul Walker","age":22,"position":"Developer","departmentId":1,"organizationId":1}' -H "Content-Type: application/json"
{"age":22,"departmentId":1,"id":2,"name":"Paul Walker","organizationId":1,"position":"Developer"}

$ curl http://localhost:8080/employee -d '{"name":"Anna Hamilton","age":26,"position":"Developer","departmentId":2,"organizationId":1}' -H "Content-Type: application/json"
{"age":26,"departmentId":2,"id":3,"name":"Anna Hamilton","organizationId":1,"position":"Developer"}

$ curl http://localhost:8080/department -d '{"name":"Test1","organizationId":1}' -H "Content-Type:application/json"
{"id":1,"name":"Test1","organizationId":1}

$ curl http://localhost:8080/department -d '{"name":"Test2","organizationId":1}' -H "Content-Type:application/json"
{"id":2,"name":"Test2","organizationId":1}

Then let’s call a little bit more complex endpoint GET /department-with-employees/{organizationId}. As you probably remember, it is exposed by department-service and calls employee-service to fetch all employees assigned to the particular department.

$ curl http://localhost:8080/department-with-employees/1

However, we received a response with the HTTP 500 error code. We can find more details in the Krakend logs.

Krakend is unable to parse JSON array returned as a response by department-service. Therefore, we need to declare it explicitly with "is_collection": true so that KrakenD can convert it to an object for further manipulation. Here’s our current configuration for that endpoint.

    {
      "endpoint": "/department-with-employees/{organizationId}",
      "method": "GET",
      "backend": [
        {
          "url_pattern": "/departments/organization/{organizationId}/with-employees",
          "sd": "dns",
          "host": [
            "department-service.service.consul"
          ],
          "disable_host_sanitize": true,
          "is_collection": true
        }
      ]
    }

Now, let’s call the same endpoint once again. It works perfectly fine!

$ curl http://localhost:8080/department-with-employees/1   
{"collection":[{"id":1,"name":"Test1","organizationId":1},{"employees":[{"age":26,"id":3,"name":"Anna Hamilton","position":"Developer"}],"id":2,"name":"Test2","organizationId":1}]}

The last thing we can do here is to check out traces collected by Zipkin. Thanks to Micronaut support for spans propagation (@ContinueSpan), all the subsequent requests are grouped altogether.

The picture visible below shows Zipkin timeline for the latest request.

Conclusion

If you are looking for a gateway for your microservices – KrakenD is an excellent choice. Moreover, if you use Consul as a discovery server, and Zipkin (or Jaeger) as a tracing tool, it is easy to start with KrakenD. It also offers support for service discovery with Netflix Eureka, but it is a little bit more complicated to configure. Of course, you may also run KrakenD on Kubernetes (and integrate with its discovery), which is an absolute “must-have” for the modern API gateway.

The post Microservices with Micronaut, KrakenD and Consul appeared first on Piotr's TechBlog.

A primary goal of our system is to arrange a trip for customers. The order-service application also acts as a gateway. It is receiving requests from customers, saving history, and sending events to orders topic. All the other microservices are listening on this topic and processing orders sent by order-service. Each microservice has its own dedicated topic, where it sends events with information about changes. Such events are received by some other microservices. The architecture is presented in the picture below.

Before reading this article it is worth familiarizing yourself with Micronaut Framework. You may read one of my previous articles describing a process of building microservices communicating via REST API: Quick Guide to Microservices with Micronaut Framework

1. Running Kafka

To run Apache Kafka on the local machine we may use its Docker image. It seems that the most up-to-date image is shared by https://hub.docker.com/u/wurstmeister. Before starting Kafka containers we have to start the ZooKeeper server, which is used by Kafka. If you run Docker on Windows the default address of its virtual machine is 192.168.99.100. It also has to be set as an environment for a Kafka container.
Both Zookeeper and Kafka containers will be started in the same network kafka. Zookeeper is available under the name zookeeper, and is exposed on port 2181. Kafka container requires that address under env variable KAFKA_ZOOKEEPER_CONNECT.

$ docker network create kafka
$ docker run -d --name zookeeper --network kafka -p 2181:2181 wurstmeister/zookeeper
$ docker run -d --name kafka -p 9092:9092 --network kafka --env KAFKA_ADVERTISED_HOST_NAME=192.168.99.100 --env KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 wurstmeister/kafka

2. Including Micronaut Kafka

Micronaut example applications built with Kafka can be started with or without the presence of an HTTP server. To enable Micronaut Kafka you need to include the micronaut-kafka library to your dependencies. In case you would like to expose HTTP API you should also include micronaut-http-server-netty:

<dependency>
   <groupId>io.micronaut.configuration</groupId>
   <artifactId>micronaut-kafka</artifactId>
</dependency>
<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-http-server-netty</artifactId>
</dependency>

3. Building microservice order-service

The application order-service as the only one starts embedded HTTP server and exposes REST API. That’s why we may enable built-in Micronaut health checks for Kafka. To do that we should first include micronaut-management dependency:

<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-management</artifactId>
</dependency>

For convenience, we will enable all management endpoints and disable HTTP authentication for them by defining the following configuration inside application.yml:

endpoints:
  all:
    enabled: true
    sensitive: false

Now, a health check is available under address http://localhost:8080/health. Our sample application will also expose a simple REST API for adding new orders and listing all previously created orders. Here’s the Micronaut controller implementation responsible for exposing those endpoints:

@Controller("orders")
public class OrderController {

    @Inject
    OrderInMemoryRepository repository;
    @Inject
    OrderClient client;

    @Post
    public Order add(@Body Order order) {
        order = repository.add(order);
        client.send(order);
        return order;
    }

    @Get
    public Set<Order> findAll() {
        return repository.findAll();
    }

}

Each microservice uses an in-memory repository implementation. Here’s repository implementation inside order-service:

@Singleton
public class OrderInMemoryRepository {

    private Set<Order> orders = new HashSet<>();

    public Order add(Order order) {
        order.setId((long) (orders.size() + 1));
        orders.add(order);
        return order;
    }

    public void update(Order order) {
        orders.remove(order);
        orders.add(order);
    }

    public Optional<Order> findByTripIdAndType(Long tripId, OrderType type) {
        return orders.stream().filter(order -> order.getTripId().equals(tripId) && order.getType() == type).findAny();
    }

    public Optional<Order> findNewestByUserIdAndType(Long userId, OrderType type) {
        return orders.stream().filter(order -> order.getUserId().equals(userId) && order.getType() == type)
                .max(Comparator.comparing(Order::getId));
    }

    public Set<Order> findAll() {
        return orders;
    }

}

In-memory repository stores Order object instances. Order object is also sent to Kafka topic named orders. Here’s an implementation of Order class:

public class Order {

    private Long id;
    private LocalDateTime createdAt;
    private OrderType type;
    private Long userId;
    private Long tripId;
    private float currentLocationX;
    private float currentLocationY;
    private OrderStatus status;
   
    // ... GETTERS AND SETTERS
}

4. Example of asynchronous communication with Kafka and Micronaut

Now, let’s consider one of the use cases possible to realize by our sample system – adding a new trip. In the first step (1) we are adding a new order of type OrderType.NEW_TRIP. After that order-service creates an order and send it to the orders topic. The order is received by three microservices: driver-service, passenger-service and order-service (2). A new order is processed by all these applications. The passenger-service application checks if there are sufficient funds on the passenger account. If not it cancels the trip, otherwise it does not do anything. The driver-service is looking for the nearest available driver, while trip-service creates and stores new trips. Both driver-service and trip-service sends events to their topics (drivers, trips) with information about changes (3) Every event can be accessed by other microservices, for example trip-service listen for event from driver-service in order to assign a new driver to the trip (4). The following picture illustrates the communication between our microservices when adding a new trip.

Now, let’s proceed to the implementation details.

Step 1: Sending order

First we need to create a Kafka client responsible for sending messages to a topic. To achieve that we should create an interface annotated with @KafkaClient and declare one or more methods for sending messages. Every method should have a target topic name set through @Topic annotation. For method parameters we may use three annotations @KafkaKey, @Body or @Header. @KafkaKey is used for partitioning, which is required by our sample applications. In the client implementation visible below we just use @Body annotation.

@KafkaClient
public interface OrderClient {

    @Topic("orders")
    void send(@Body Order order);

}

Step 2: Receiving order

Once an order has been sent by the client it is received by all other microservices listening on the orders topic. Here’s a listener implementation in the driver-service. A listener class should be annotated with @KafkaListener. We may declare groupId as an annotation field to prevent from receiving the same message by more than one instance of a single application. Then we are declaring a method for processing incoming messages. The same as a client method it should be annotated with @Topic, to set the name of a target topic. Because we are listening for Order objects it should be annotated with @Body – the same as the corresponding client method.

@KafkaListener(groupId = "driver")
public class OrderListener {

    private static final Logger LOGGER = LoggerFactory.getLogger(OrderListener.class);

    private DriverService service;

    public OrderListener(DriverService service) {
        this.service = service;
    }

    @Topic("orders")
    public void receive(@Body Order order) {
        LOGGER.info("Received: {}", order);
        switch (order.getType()) {
            case NEW_TRIP -> service.processNewTripOrder(order);
        }
    }

}

Step 3: Sending to other Kafka topic

Now, let’s take a look on the processNewTripOrder method inside driver-service. DriverService injects two different Kafka client beans: OrderClient and DriverClient. When processing a new order it tries to find the available driver, which is the closest to the customer who sent the order. After finding him it changes the status to UNAVAILABLE and sends the message with Driver object to the drivers topic.

@Singleton
public class DriverService {

    private static final Logger LOGGER = LoggerFactory.getLogger(DriverService.class);

    private DriverClient client;
    private OrderClient orderClient;
    private DriverInMemoryRepository repository;

    public DriverService(DriverClient client, OrderClient orderClient, DriverInMemoryRepository repository) {
        this.client = client;
        this.orderClient = orderClient;
        this.repository = repository;
    }

    public void processNewTripOrder(Order order) {
        LOGGER.info("Processing: {}", order);
        Optional<Driver> driver = repository.findNearestDriver(order.getCurrentLocationX(), order.getCurrentLocationY());
        driver.ifPresent(driverLocal -> {
            driverLocal.setStatus(DriverStatus.UNAVAILABLE);
            repository.updateDriver(driverLocal);
            client.send(driverLocal, String.valueOf(order.getId()));
            LOGGER.info("Message sent: {}", driverLocal);
        });
    }
   
    // ...
}

Here’s an implementation of Kafka client inside driver-service used for sending messages to the drivers topic. Because we need to link the instance of Driver with order we annotate orderId parameter with @Header. There is no sense to include it to Driver class just to assign it to the right trip on the listener side.

@KafkaClient
public interface DriverClient {

    @Topic("drivers")
    void send(@Body Driver driver, @Header("Order-Id") String orderId);

}

Step 4: Inter-service communication example with Micronaut Kafka

The message sent by DriverClient is received by @Listener declared inside trip-service. It listens for messages incoming to the trips topic. The signature of receiving method is pretty similar to the client sending method as shown below:

@KafkaListener(groupId = "trip")
public class DriverListener {

    private static final Logger LOGGER = LoggerFactory.getLogger(OrderListener.class);

    private TripService service;

    public DriverListener(TripService service) {
        this.service = service;
    }

    @Topic("drivers")
    public void receive(@Body Driver driver, @Header("Order-Id") String orderId) {
        LOGGER.info("Received: driver->{}, header->{}", driver, orderId);
        service.processNewDriver(driver);
    }

}

A new driver with given id is being assigned to the trip searched by orderId. That’s a final step of our communication process when adding a new trip.

@Singleton
public class TripService {

    private static final Logger LOGGER = LoggerFactory.getLogger(TripService.class);

    private TripInMemoryRepository repository;
    private TripClient client;

    public TripService(TripInMemoryRepository repository, TripClient client) {
        this.repository = repository;
        this.client = client;
    }


    public void processNewDriver(Driver driver, String orderId) {
        LOGGER.info("Processing: {}", driver);
        Optional<Trip> trip = repository.findByOrderId(Long.valueOf(orderId));
        trip.ifPresent(tripLocal -> {
            tripLocal.setDriverId(driver.getId());
            repository.update(tripLocal);
        });
    }
   
   // ... OTHER METHODS

}

5. Tracing

We may easily enable distributed tracing with Micronaut Kafka. First, we need to enable and configure Micronaut Tracing. To do that you should first add some dependencies:

<dependency>
    <groupId>io.micronaut</groupId>
    <artifactId>micronaut-tracing</artifactId>
</dependency>
<dependency>
    <groupId>io.zipkin.brave</groupId>
    <artifactId>brave-instrumentation-http</artifactId>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>io.zipkin.reporter2</groupId>
    <artifactId>zipkin-reporter</artifactId>
    <scope>runtime</scope>
</dependency>
<dependency>
    <groupId>io.opentracing.brave</groupId>
    <artifactId>brave-opentracing</artifactId>
</dependency>
<dependency>
    <groupId>io.opentracing.contrib</groupId>
    <artifactId>opentracing-kafka-client</artifactId>
    <version>0.0.16</version>
    <scope>runtime</scope>
</dependency>

We also need to configure some application settings inside application.yml including an address of our tracing tool. In that case, it is Zipkin.

tracing:
  zipkin:
    enabled: true
    http:
      url: http://192.168.99.100:9411
    sampler:
      probability: 1

Before starting our application we have to run Zipkin container:

$ docker run -d --name zipkin -p 9411:9411 openzipkin/zipkin

Conclusion

In this article you were guided through the process of building microservice architecture using asynchronous communication via Apache Kafka. I have shown you ea example with the most important features of the Micronaut Kafka library that allows you to easily declare producer and consumer of Kafka topics, enable health checks, and distributed tracing for your microservices. I have described an implementation of a single scenario for our system, that covers adding a new trip at the customer’s request. In order to see the full implementation of the sample system described in this article please check out the source code available on GitHub: https://github.com/piomin/sample-kafka-micronaut-microservices.git.

The post Kafka In Microservices With Micronaut appeared first on Piotr's TechBlog.

Our sample system consists of three independent microservices that communicate with each other. All of them integrate with Consul in order to fetch shared configuration. After startup, every single service will register itself in Consul. Applications organization-service and department-service call endpoints exposed by other microservices using Micronaut declarative HTTP client. The traces from communication are sent to Zipkin. The source code of sample applications is available on GitHub in repository sample-micronaut-microservices.

Step 1. Creating Micronaut application

We need to start by including some dependencies to our Maven pom.xml. First, let’s define BOM with the newest stable Micronaut version.

<properties>
   <exec.mainClass>pl.piomin.services.employee.EmployeeApplication</exec.mainClass>
   <micronaut.version>1.0.3</micronaut.version>
   <jdk.version>1.8</jdk.version>
</properties>
<dependencyManagement>
   <dependencies>
      <dependency>
         <groupId>io.micronaut</groupId>
         <artifactId>micronaut-bom</artifactId>
         <version>${micronaut.version}</version>
         <type>pom</type>
         <scope>import</scope>
      </dependency>
   </dependencies>
</dependencyManagement>

The list of required dependencies isn’t very long. Also not all of them are required, but they will be useful in our demo. For example micronaut-management need to be included in case we would like to expose some built-in management and monitoring endpoints.

<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-http-server-netty</artifactId>
</dependency>
<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-inject</artifactId>
</dependency>
<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-runtime</artifactId>
</dependency>
<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-management</artifactId>
</dependency>
<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-inject-java</artifactId>
   <scope>provided</scope>
</dependency>

To build an application uber-jar we need a configure plugin responsible for packaging a JAR file with dependencies. It can be for example maven-shade-plugin. When building a new application it is also worth to expose basic information about it under /info endpoint. As I have already mentioned Micronaut adds support for monitoring your app via HTTP endpoints after including artifact micronaut-management. Management endpoints are integrated with the Micronaut security module, which means that you need to authenticate yourself to be able to access them. To simplify we can disable authentication for /info endpoint.

endpoints:
  info:
    enabled: true
    sensitive: false

We can customize /info endpoint by adding some supported info sources. This mechanism is very similar to the Spring Boot Actuator approach. If git.properties file is available on the classpath, all the values inside file will be exposed by /info endpoint. The same situation applies to build-info.properties file, that needs to be placed inside META-INF directory. However, in comparison with Spring Boot we need to provide more configuration in pom.xml to generate and package those to application JAR. The following Maven plugins are responsible for generating required properties files.

<plugin>
   <groupId>pl.project13.maven</groupId>
   <artifactId>git-commit-id-plugin</artifactId>
   <version>2.2.6</version>
   <executions>
      <execution>
         <id>get-the-git-infos</id>
         <goals>
            <goal>revision</goal>
         </goals>
      </execution>
   </executions>
   <configuration>
      <verbose>true</verbose>
      <dotGitDirectory>${project.basedir}/.git</dotGitDirectory>
      <dateFormat>MM-dd-yyyy '@' HH:mm:ss Z</dateFormat>
      <generateGitPropertiesFile>true</generateGitPropertiesFile>
      <generateGitPropertiesFilename>src/main/resources/git.properties</generateGitPropertiesFilename>
      <failOnNoGitDirectory>true</failOnNoGitDirectory>
   </configuration>
</plugin>
<plugin>
   <groupId>com.rodiontsev.maven.plugins</groupId>
   <artifactId>build-info-maven-plugin</artifactId>
   <version>1.2</version>
   <configuration>
      <filename>classes/META-INF/build-info.properties</filename>
      <projectProperties>
         <projectProperty>project.groupId</projectProperty>
         <projectProperty>project.artifactId</projectProperty>
         <projectProperty>project.version</projectProperty>
      </projectProperties>
   </configuration>
   <executions>
      <execution>
         <phase>prepare-package</phase>
         <goals>
            <goal>extract</goal>
         </goals>
      </execution>
   </executions>
</plugin>
</plugins>

Now, our /info endpoint is able to print the most important information about our app including Maven artifact name, version, and last Git commit id.

Step 2. Exposing HTTP endpoints

Micronaut provides their own annotations for pointing out HTTP endpoints and methods. As I have mentioned in the preface it also uses JSR-330 (java.inject) for dependency injection. Our controller class should be annotated with @Controller. We also have annotations for every HTTP method type. The path parameter is automatically mapped to the class method parameter by its name, what is a nice simplification in comparison to Spring MVC where we need to use @PathVariable annotation. The repository bean used for CRUD operations is injected into the controller using @Inject annotation.

@Controller("/employees")
public class EmployeeController {

    private static final Logger LOGGER = LoggerFactory.getLogger(EmployeeController.class);

    @Inject
    EmployeeRepository repository;

    @Post
    public Employee add(@Body Employee employee) {
        LOGGER.info("Employee add: {}", employee);
        return repository.add(employee);
    }

    @Get("/{id}")
    public Employee findById(Long id) {
        LOGGER.info("Employee find: id={}", id);
        return repository.findById(id);
    }

    @Get
    public List<Employee> findAll() {
        LOGGER.info("Employees find");
        return repository.findAll();
    }

    @Get("/department/{departmentId}")
    @ContinueSpan
    public List<Employee> findByDepartment(@SpanTag("departmentId") Long departmentId) {
        LOGGER.info("Employees find: departmentId={}", departmentId);
        return repository.findByDepartment(departmentId);
    }

    @Get("/organization/{organizationId}")
    @ContinueSpan
    public List<Employee> findByOrganization(@SpanTag("organizationId") Long organizationId) {
        LOGGER.info("Employees find: organizationId={}", organizationId);
        return repository.findByOrganization(organizationId);
    }

}

Our repository bean is pretty simple. It just provides an in-memory store for Employee instances. We will mark it with @Singleton annotation.

@Singleton
public class EmployeeRepository {

   private List<Employee> employees = new ArrayList<>();
   
   public Employee add(Employee employee) {
      employee.setId((long) (employees.size()+1));
      employees.add(employee);
      return employee;
   }
   
   public Employee findById(Long id) {
      Optional<Employee> employee = employees.stream().filter(a -> a.getId().equals(id)).findFirst();
      if (employee.isPresent())
         return employee.get();
      else
         return null;
   }
   
   public List<Employee> findAll() {
      return employees;
   }
   
   public List<Employee> findByDepartment(Long departmentId) {
      return employees.stream().filter(a -> a.getDepartmentId().equals(departmentId)).collect(Collectors.toList());
   }
   
   public List<Employee> findByOrganization(Long organizationId) {
      return employees.stream().filter(a -> a.getOrganizationId().equals(organizationId)).collect(Collectors.toList());
   }
   
}

Micronaut is able to automatically generate Swagger YAML definition from our controller and methods based on annotations. To achieve this, we first need to include the following dependency to our pom.xml.

<dependency>
   <groupId>io.swagger.core.v3</groupId>
   <artifactId>swagger-annotations</artifactId>
</dependency>

Then we should annotate the application main class with @OpenAPIDefinition and provide some basic information like title or version number. Here’s the employee application main class.

@OpenAPIDefinition(
    info = @Info(
        title = "Employees Management",
        version = "1.0",
        description = "Employee API",
        contact = @Contact(url = "https://piotrminkowski.wordpress.com", name = "Piotr Mińkowski", email = "piotr.minkowski@gmail.com")
    )
)
public class EmployeeApplication {

    public static void main(String[] args) {
        Micronaut.run(EmployeeApplication.class);
    }

}

Micronaut generates Swagger file basing on title and version fields inside @Info annotation. In that case our YAML definition file is available under name employees-management-1.0.yml, and will be generated to the META-INF/swagger directory. We can expose it outside the application using HTTP endpoint. Here’s the appropriate configuration provided inside application.yml file.

micronaut:
  router:
    static-resources:
      swagger:
        paths: classpath:META-INF/swagger
        mapping: /swagger/**

Now, our file is available under path http://localhost:8080/swagger/employees-management-1.0.yml if run it on default 8080 port (we won’t do that, what I’m going to describe in the next part of this article). In comparison to Spring Boot, we don’t have such a project like Swagger SpringFox for Micronaut, so we need to copy the content to an online editor in order to see the graphical representation of Swagger YAML. Here’s it.

Ok, since we have finished implementation of single microservice we may proceed to cloud-native features provided by Micronaut.

Step 3. Distributed configuration with Consul

Micronaut comes with built in APIs for doing distributed configuration. In fact, the only one available solution for now is microservices distributed configuration based on Micronaut integration with HashiCorp’s Consul. Micronaut features for externalizing and adapting configuration to the environment are very similar to the Spring Boot approach. We also have application.yml and bootstrap.yml files, which can be used for application environment configuration. When using distributed configuration we first need to provide a bootstrap.yml file on the classpath. It should contain an address of remote configuration server and preferred configuration store format. Of course, we first need to enable distributed configuration clients by setting property micronaut.config-client.enabled to true. Here’s bootstrap.yml file for department-service.

micronaut:
  application:
    name: department-service
  config-client:
    enabled: true
consul:
  client:
    defaultZone: "192.168.99.100:8500"
    config:
      format: YAML

We can choose between properties, JSON, YAML and FILES (git2consul) configuration formats. I decided to use YAML. To apply this configuration on Consul we first need to start it locally in development mode. Because I’m using Docker Toolbox the default address of Consul is 192.168.99.100. The following Docker command will start a single-node Consul instance and expose it on port 8500.

$ docker run -d --name consul -p 8500:8500 consul

Now, you can navigate to the tab Key/Value in the Consul web console and create a new file in YAML format /config/application.yml as shown below. Besides configuration for Swagger and /info management endpoint it also enables dynamic HTTP generation on startup by setting property micronaut.server.port to -1. Because the name of the file is application.yml it is by default shared between all Micronaut microservices that use the Consul config client.

Step 4. Service discovery with Consul

Micronaut gives you more options when configuring service discovery, than for distributed configuration. You can use Eureka, Consul, Kubernetes or just manually configure a list of available services. However, I have observed that using the Eureka discovery client together with the Consul config client causes some errors on startup. In this example we will use Consul discovery for our Micronaut microservices. Because Consul address has been already provided in bootstrap.yml for all Micronaut microservices, we just need to enable service discovery by adding the following lines to application.yml stored in Consul KV.

consul:
  client:
    registration:
      enabled: true

We should also include the following dependency to Maven pom.xml of every single application.

<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-discovery-client</artifactId>
</dependency>

Finally, you can just run every microservice (you may run more than one instance locally, since HTTP port is generated dynamically). Here’s my list of running Micronaut microservices registered in Consul.

I have run two instances of employee-service as shown below.

Step 5. Inter-service communication

Micronaut uses a built-in HTTP client for load balancing between multiple instances of a single microservice. By default it leverages the Round Robin algorithm. We may choose between low-level HTTP client and declarative HTTP client with @Client. Micronaut declarative HTTP client concept is very similar to Spring Cloud OpenFeign. To use a built-in client we first need to include the following dependency to project pom.xml.

<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-http-client</artifactId>
</dependency>

Declarative client automatically integrates with a discovery client. It tries to find the service registered in Consul under the same name as the value provided inside id field.

@Client(id = "employee-service", path = "/employees")
public interface EmployeeClient {

   @Get("/department/{departmentId}")
   List<Employee> findByDepartment(Long departmentId);
   
}

Now, the client bean needs to be injected into the controller.

@Controller("/departments")
public class DepartmentController {

   private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);
   
   @Inject
   DepartmentRepository repository;
   @Inject
   EmployeeClient employeeClient;
   
   @Post
   public Department add(@Body Department department) {
      LOGGER.info("Department add: {}", department);
      return repository.add(department);
   }
   
   @Get("/{id}")
   public Department findById(Long id) {
      LOGGER.info("Department find: id={}", id);
      return repository.findById(id);
   }
   
   @Get
   public List<Department> findAll() {
      LOGGER.info("Department find");
      return repository.findAll();
   }
   
   @Get("/organization/{organizationId}")
   @ContinueSpan
   public List<Department> findByOrganization(@SpanTag("organizationId") Long organizationId) {
      LOGGER.info("Department find: organizationId={}", organizationId);
      return repository.findByOrganization(organizationId);
   }
   
   @Get("/organization/{organizationId}/with-employees")
   @ContinueSpan
   public List<Department> findByOrganizationWithEmployees(@SpanTag("organizationId") Long organizationId) {
      LOGGER.info("Department find: organizationId={}", organizationId);
      List<Department> departments = repository.findByOrganization(organizationId);
      departments.forEach(d -> d.setEmployees(employeeClient.findByDepartment(d.getId())));
      return departments;
   }
   
}

Step 6. Distributed tracing

Micronaut applications can be easily integrated with Zipkin to send traces with HTTP traffic there automatically. To enable this feature we first need to include the following dependencies to pom.xml.

<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-tracing</artifactId>
</dependency>
<dependency>
   <groupId>io.zipkin.brave</groupId>
   <artifactId>brave-instrumentation-http</artifactId>
   <scope>runtime</scope>
</dependency>
<dependency>
   <groupId>io.zipkin.reporter2</groupId>
   <artifactId>zipkin-reporter</artifactId>
   <scope>runtime</scope>
</dependency>
<dependency>
   <groupId>io.opentracing.brave</groupId>
   <artifactId>brave-opentracing</artifactId>
</dependency>

Then, we have to provide some configuration settings inside application.yml including Zipkin URL and sampler options. By setting property tracing.zipkin.sampler.probability to 1 we are forcing micronaut to send traces for every single request. Here’s our final configuration.

During the tests of my application I have observed that using distributed configuration together with Zipkin tracing results in the problems in communication between microservice and Zipkin. The traces just do not appear in Zipkin. So, if you would like to test this feature now you must provide application.yml on the classpath and disable Consul distributed configuration for all your applications.

We can add some tags to the spans by using @ContinueSpan or @NewSpan annotations on methods.

After making some test calls of GET methods exposed by organization-service and department-service we may take a look on Zipkin web console, available under address http://192.168.99.100:9411. The following picture shows the list of all the traces sent to Zipkin by our microservices in 1 hour.

We can check out the details of every trace by clicking on the element from the list. The following picture illustrates the timeline for HTTP method exposed by organization-service GET /organizations/{id}/with-departments-and-employees. This method finds the organization in the in-memory repository, and then calls HTTP method exposed by department-service GET /departments/organization/{organizationId}/with-employees. This method is responsible for finding all departments assigned to the given organization. It also needs to return employees within the department, so it calls method GET /employees/department/{departmentId} from employee-service.

We can also take a look at the details of every single call from the timeline.

Conclusion

In comparison to Spring Boot Micronaut is still in the early stage of development. For example, I was not able to implement any application that could act as an API gateway to our system, which can easily be achieved with Spring using Spring Cloud Gateway or Spring Cloud Netflix Zuul. There are still some bugs that need to be fixed. But above all that, Micronaut is now probably the most interesting micro-framework on the market. It implements most popular microservice patterns, provides integration with several third-party solutions like Consul, Eureka, Zipkin or Swagger, consumes less memory and starts faster than similar Spring Boot apps. I will definitely follow the progress in Micronaut development closely.

The post Guide to Microservices with Micronaut and Consul appeared first on Piotr's TechBlog.

The topics covered in this article are:

• Using Spring Boot 2 in cloud-native development
• Providing service discovery for all microservices using Spring Cloud Kubernetes project
• Injecting configuration settings into application pods using Kubernetes Config Maps and Secrets
• Building application images using Docker and deploying them on Kubernetes using YAML configuration files
• Using Spring Cloud Kubernetes together with Zuul proxy to expose a single Swagger API documentation for all microservices

Spring Cloud and Kubernetes may be threatened as competitive solutions when you build microservices environments. Such components like Eureka, Spring Cloud Config, or Zuul provided by Spring Cloud may be replaced by built-in Kubernetes objects like services, config maps, secrets, or ingresses. But even if you decide to use Kubernetes components instead of Spring Cloud you can take advantage of some interesting features provided throughout the whole Spring Cloud project.

The one really interesting project that helps us in development is Spring Cloud Kubernetes (https://github.com/spring-cloud/spring-cloud-kubernetes). Although it is still in the incubation stage it is definitely worth dedicating some time to it. It integrates Spring Cloud with Kubernetes. I’ll show you how to use the implementation of discovery client, inter-service communication with Ribbon client, and Zipkin discovery using Spring Cloud Kubernetes.

Before we proceed to the source code, let’s take a look at the following diagram. It illustrates the architecture of our sample system. It is quite similar to the architecture presented in the already mentioned article about microservices on Spring Cloud. There are three independent applications (employee-service, department-service, organization-service), which communicate between each other through REST API. These Spring Boot microservices use some build-in mechanisms provided by Kubernetes: config maps and secrets for distributed configuration, etcd for service discovery, and ingresses for API gateway.

Let’s proceed to the implementation. Currently, the newest stable version of Spring Cloud is Finchley.RELEASE. This version of spring-cloud-dependencies should be declared as a BOM for dependency management.

<dependencyManagement>
   <dependencies>
      <dependency>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-dependencies</artifactId>
         <version>Finchley.RELEASE</version>
         <type>pom</type>
         <scope>import</scope>
      </dependency>
   </dependencies>
</dependencyManagement>

Spring Cloud Kubernetes is not released under Spring Cloud Release Trains. So, we need to explicitly define its version. Because we use Spring Boot 2.0 we have to include the newest SNAPSHOT version of spring-cloud-kubernetes artifacts, which is 0.3.0.BUILD-SNAPSHOT.

The source code of sample Spring Boot microservices on Kubernetes presented in this article is available on GitHub in repository https://github.com/piomin/sample-spring-microservices-kubernetes.git.

Pre-requirements

In order to be able to deploy and test our sample microservices we need to prepare a development environment. We can realize that in the following steps:

• You need at least a single node cluster instance of Kubernetes (Minikube) or Openshift (Minishift) running on your local machine. You should start it and expose the embedded Docker client provided by both of them. The detailed instruction for Minishift may be found there: Quick guide to deploying Java apps on OpenShift. You can also use that description to run Minikube – just replace word ‘minishift’ with ‘minikube’. In fact, it does not matter if you choose Kubernetes or Openshift – the next part of this tutorial would be applicable for both of them
• Spring Cloud Kubernetes requires access to Kubernetes API in order to be able to retrieve a list of addresses of pods running for a single service. If you use Kubernetes you should just execute the following command:

$ kubectl create clusterrolebinding admin --clusterrole=cluster-admin --serviceaccount=default:default

If you deploy your microservices on Minishift you should first enable admin-user addon, then login as a cluster-admin and grant required permissions.

$ minishift addons enable admin-user
$ oc login -u system:admin
$ oc policy add-role-to-user cluster-reader system:serviceaccount:myproject:default

• All our sample microservices use MongoDB as a backend store. So, you should first run an instance of this database on your node. With Minishift it is quite simple, as you can use predefined templates just by selecting service Mongo on the Catalog list. With Kubernetes, the task is more difficult. You have to prepare deployment configuration files by yourself and apply it to the cluster. All the configuration files are available under kubernetes directory inside sample Git repository. To apply the following YAML definition to the cluster you should execute command kubectl apply -f kubernetes\mongo-deployment.yaml. After it, Mongo database would be available under the name mongodb inside Kubernetes cluster.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongodb
  labels:
    app: mongodb
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongodb
  template:
    metadata:
      labels:
        app: mongodb
    spec:
      containers:
      - name: mongodb
        image: mongo:latest
        ports:
        - containerPort: 27017
        env:
        - name: MONGO_INITDB_DATABASE
          valueFrom:
            configMapKeyRef:
              name: mongodb
              key: database-name
        - name: MONGO_INITDB_ROOT_USERNAME
          valueFrom:
            secretKeyRef:
              name: mongodb
              key: database-user
        - name: MONGO_INITDB_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mongodb
              key: database-password
---
apiVersion: v1
kind: Service
metadata:
  name: mongodb
  labels:
    app: mongodb
spec:
  ports:
  - port: 27017
    protocol: TCP
  selector:
    app: mongodb

1. Inject configuration with Config Maps and Secrets

When using Spring Cloud the most obvious choice for realizing distributed configuration in your system is Spring Cloud Config. With Kubernetes, you can use the Config Map. It holds key-value pairs of configuration data that can be consumed in pods or used to store configuration data. It is used for storing and sharing non-sensitive, unencrypted configuration information. To use sensitive information in your clusters, you must use Secrets. Any usage of both these Kubernetes objects can be perfectly demonstrated based on the example of MongoDB connection settings. Inside the Spring Boot application, we can easily inject it using environment variables. Here’s a fragment of application.yml file with URI configuration.

spring:
  data:
    mongodb:
      uri: mongodb://${MONGO_USERNAME}:${MONGO_PASSWORD}@mongodb/${MONGO_DATABASE}

While username or password are sensitive fields, a database name is not. So we can place it inside a config map.

apiVersion: v1
kind: ConfigMap
metadata:
  name: mongodb
data:
  database-name: microservices

Of course, username and password are defined as secrets.

apiVersion: v1
kind: Secret
metadata:
  name: mongodb
type: Opaque
data:
  database-password: MTIzNDU2
  database-user: cGlvdHI=

To apply the configuration to the Kubernetes cluster we run the following commands.

$ kubectl apply -f kubernetes/mongodb-configmap.yaml
$ kubectl apply -f kubernetes/mongodb-secret.yaml

After it we should inject the configuration properties into application’s pods. When defining container configuration inside Deployment YAML file we have to include references to environment variables and secrets as shown below

apiVersion: apps/v1
kind: Deployment
metadata:
  name: employee
  labels:
    app: employee
spec:
  replicas: 1
  selector:
    matchLabels:
      app: employee
  template:
    metadata:
      labels:
        app: employee
    spec:
      containers:
      - name: employee
        image: piomin/employee:1.0
        ports:
        - containerPort: 8080
        env:
        - name: MONGO_DATABASE
          valueFrom:
            configMapKeyRef:
              name: mongodb
              key: database-name
        - name: MONGO_USERNAME
          valueFrom:
            secretKeyRef:
              name: mongodb
              key: database-user
        - name: MONGO_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mongodb
              key: database-password

2. Building Spring Boot microservices discovery with Kubernetes

We usually run microservices on Kubernetes using Docker containers. One or more containers are grouped by pods, which are the smallest deployable units created and managed in Kubernetes. A good practice is to run only one container inside a single pod. If you would like to scale up your microservice you would just have to increase a number of running pods. All running pods that belong to a single microservice are logically grouped by Kubernetes Service. This service may be visible outside the cluster, and is able to load balance incoming requests between all running pods. The following service definition groups all pods labelled with field app equaled to employee.

apiVersion: v1
kind: Service
metadata:
  name: employee
  labels:
    app: employee
spec:
  ports:
  - port: 8080
    protocol: TCP
  selector:
    app: employee

Service can be used for accessing applications outside a Kubernetes cluster or for inter-service communication inside a cluster. However, the communication between microservices can be implemented more comfortably with Spring Cloud Kubernetes. First, we need to include the following dependency to project pom.xml.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-kubernetes</artifactId>
   <version>0.3.0.BUILD-SNAPSHOT</version>
</dependency>

Then we should enable a discovery client for an application – the same as we have always done for discovery Spring Cloud Netflix Eureka. This allows you to query Kubernetes endpoints (services) by name. This discovery feature is also used by the Spring Cloud Kubernetes Ribbon or Zipkin projects to fetch respectively the list of the pods defined for a microservice to be load balanced or the Zipkin servers available to send the traces or spans.

@SpringBootApplication
@EnableDiscoveryClient
@EnableMongoRepositories
@EnableSwagger2
public class EmployeeApplication {

   public static void main(String[] args) {
      SpringApplication.run(EmployeeApplication.class, args);
   }
   
   // ...
}

The last important thing in this section is to guarantee that Spring Boot microservices name would be exactly the same as the Kubernetes service name for the application. For application employee-service it is employee.

spring:
  application:
    name: employee

3. Building microservice using Docker and deploying on Kubernetes

There is nothing unusual in our sample microservices. We have included some standard Spring dependencies for building REST-based microservices, integrating with MongoDB and generating API documentation using Swagger2.

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
   <groupId>io.springfox</groupId>
   <artifactId>springfox-swagger2</artifactId>
   <version>2.9.2</version>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>

In order to integrate with MongoDB we should create interface that extends standard Spring Data CrudRepository.

public interface EmployeeRepository extends CrudRepository<Employee, String> {
   
   List<Employee> findByDepartmentId(Long departmentId);
   List<Employee> findByOrganizationId(Long organizationId);
   
}

Entity class should be annotated with Mongo @Document and a primary key field with @Id.

@Document(collection = "employee")
public class Employee {

   @Id
   private String id;
   private Long organizationId;
   private Long departmentId;
   private String name;
   private int age;
   private String position;
   
   // ...
   
}

The repository bean has been injected to the controller class. Here’s the full implementation of our REST API inside employee-service.

@RestController
public class EmployeeController {

   private static final Logger LOGGER = LoggerFactory.getLogger(EmployeeController.class);
   
   @Autowired
   EmployeeRepository repository;
   
   @PostMapping("/")
   public Employee add(@RequestBody Employee employee) {
      LOGGER.info("Employee add: {}", employee);
      return repository.save(employee);
   }
   
   @GetMapping("/{id}")
   public Employee findById(@PathVariable("id") String id) {
      LOGGER.info("Employee find: id={}", id);
      return repository.findById(id).get();
   }
   
   @GetMapping("/")
   public Iterable<Employee> findAll() {
      LOGGER.info("Employee find");
      return repository.findAll();
   }
   
   @GetMapping("/department/{departmentId}")
   public List<Employee> findByDepartment(@PathVariable("departmentId") Long departmentId) {
      LOGGER.info("Employee find: departmentId={}", departmentId);
      return repository.findByDepartmentId(departmentId);
   }
   
   @GetMapping("/organization/{organizationId}")
   public List<Employee> findByOrganization(@PathVariable("organizationId") Long organizationId) {
      LOGGER.info("Employee find: organizationId={}", organizationId);
      return repository.findByOrganizationId(organizationId);
   }
   
}

In order to run our microservices on Kubernetes we should first build the whole Maven project with mvn clean install command. Each microservice has Dockerfile placed in the root directory. Here’s Dockerfile definition for employee-service.

FROM openjdk:8-jre-alpine
ENV APP_FILE employee-service-1.0-SNAPSHOT.jar
ENV APP_HOME /usr/apps
EXPOSE 8080
COPY target/$APP_FILE $APP_HOME/
WORKDIR $APP_HOME
ENTRYPOINT ["sh", "-c"]
CMD ["exec java -jar $APP_FILE"]

Let’s build Docker images for all three sample microservices.

$ cd employee-service
$ docker build -t piomin/employee:1.0 .
$ cd department-service
$ docker build -t piomin/department:1.0 .
$ cd organization-service
$ docker build -t piomin/organization:1.0 .

The last step is to deploy Docker containers with applications on Kubernetes. To do that just execute commands kubectl apply on YAML configuration files. The sample deployment file for employee-service has been demonstrated in step 1. All required deployment fields are available inside the project repository in kubernetes directory.

$ kubectl apply -f kubernetes\employee-deployment.yaml
$ kubectl apply -f kubernetes\department-deployment.yaml
$ kubectl apply -f kubernetes\organization-deployment.yaml

4. Communication between microservices with Spring Cloud Kubernetes Ribbon

All the microservice are deployed on Kubernetes. Now, it’s worth discussing some aspects related to inter-service communication. Application employee-service in contrast to other microservices did not invoke any other microservices. Let’s take a look at other microservices that call API exposed by employee-service and communicate between each other (organization-service calls department-service API).
First we need to include some additional dependencies to the project. We use Spring Cloud Ribbon and OpenFeign. Alternatively you can also use Spring @LoadBalanced RestTemplate.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-kubernetes-ribbon</artifactId>
   <version>0.3.0.BUILD-SNAPSHOT</version>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>

Here’s the main class of department-service. It enables Feign client using @EnableFeignClients annotation. It works the same as with discovery based on Spring Cloud Netflix Eureka. OpenFeign uses Ribbon for client-side load balancing. Spring Cloud Kubernetes Ribbon provides some beans that forces Ribbon to communicate with Kubernetes API through Fabric8 KubernetesClient.

@SpringBootApplication
@EnableDiscoveryClient
@EnableFeignClients
@EnableMongoRepositories
@EnableSwagger2
public class DepartmentApplication {
   
   public static void main(String[] args) {
      SpringApplication.run(DepartmentApplication.class, args);
   }
   
   // ...
   
}

Here’s implementation of Feign client for calling method exposed by employee-service.

@FeignClient(name = "employee")
public interface EmployeeClient {

   @GetMapping("/department/{departmentId}")
   List<Employee> findByDepartment(@PathVariable("departmentId") String departmentId);
   
}

Finally, we have to inject Feign client’s beans to the REST controller. Now, we may call the method defined inside EmployeeClient, which is equivalent to calling REST endpoints.

@RestController
public class DepartmentController {

   private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);
   
   @Autowired
   DepartmentRepository repository;
   @Autowired
   EmployeeClient employeeClient;
   
   // ...
   
   @GetMapping("/organization/{organizationId}/with-employees")
   public List<Department> findByOrganizationWithEmployees(@PathVariable("organizationId") Long organizationId) {
      LOGGER.info("Department find: organizationId={}", organizationId);
      List<Department> departments = repository.findByOrganizationId(organizationId);
      departments.forEach(d -> d.setEmployees(employeeClient.findByDepartment(d.getId())));
      return departments;
   }
   
}

5. Building API gateway using Kubernetes Ingress

An Ingress is a collection of rules that allow incoming requests to reach the downstream services. In our microservices architecture ingress is playing a role of an API gateway. To create it we should first prepare a YAML description file. The descriptor file should contain the hostname under which the gateway will be available and mapping rules to the downstream services.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gateway-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  backend:
    serviceName: default-http-backend
    servicePort: 80
  rules:
  - host: microservices.info
    http:
      paths:
      - path: /employee
        backend:
          serviceName: employee
          servicePort: 8080
      - path: /department
        backend:
          serviceName: department
          servicePort: 8080
      - path: /organization
        backend:
          serviceName: organization
          servicePort: 8080

You have to execute the following command to apply the configuration visible above to the Kubernetes cluster.

$ kubectl apply -f kubernetes\ingress.yaml

For testing this solution locally we have to insert the mapping between IP address and hostname set in ingress definition inside hosts file as shown below. After it we can access services through ingress using a defined hostname just like that: http://microservices.info/employee.

192.168.99.100 microservices.info

You can check the details of created ingress just by executing command kubectl describe ing gateway-ingress.

6. Enabling API specification on gateway using Swagger2

Ok, what if we would like to expose single swagger documentation for all microservices deployed on Kubernetes? Well, here the things are getting complicated… We can run a container with Swagger UI, and map all paths exposed by the ingress manually, but it is rather not a good solution…
In that case we can use Spring Cloud Kubernetes Ribbon one more time – this time together with Spring Cloud Netflix Zuul. Zuul will act as gateway only for serving Swagger API.
Here’s the list of dependencies used in my gateway-service project.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-netflix-zuul</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-kubernetes</artifactId>
   <version>0.3.0.BUILD-SNAPSHOT</version>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-kubernetes-ribbon</artifactId>
   <version>0.3.0.BUILD-SNAPSHOT</version>
</dependency>
<dependency>
   <groupId>io.springfox</groupId>
   <artifactId>springfox-swagger-ui</artifactId>
   <version>2.9.2</version>
</dependency>
<dependency>
   <groupId>io.springfox</groupId>
   <artifactId>springfox-swagger2</artifactId>
   <version>2.9.2</version>
</dependency>

A Kubernetes discovery client will detect all services exposed on the cluster. We would like to display documentation only for our three microservices. That’s why I defined the following routes for Zuul.

zuul:
  routes:
    department:
      path: /department/**
    employee:
      path: /employee/**
    organization:
      path: /organization/**

Now we can use ZuulProperties bean to get route’s addresses from Kubernetes discovery, and configure them as Swagger resources as shown below.

@Configuration
public class GatewayApi {

   @Autowired
   ZuulProperties properties;

   @Primary
   @Bean
   public SwaggerResourcesProvider swaggerResourcesProvider() {
      return () -> {
         List<SwaggerResource> resources = new ArrayList<>();
         properties.getRoutes().values().stream()
               .forEach(route -> resources.add(createResource(route.getId(), "2.0")));
         return resources;
      };
   }

   private SwaggerResource createResource(String location, String version) {
      SwaggerResource swaggerResource = new SwaggerResource();
      swaggerResource.setName(location);
      swaggerResource.setLocation("/" + location + "/v2/api-docs");
      swaggerResource.setSwaggerVersion(version);
      return swaggerResource;
   }

}

Application gateway-service should be deployed on a cluster the same as other applications. You can list the running services by executing command kubectl get svc. Swagger documentation is available under address http://192.168.99.100:31237/swagger-ui.html.

Conclusion

I’m actually rooting for the Spring Cloud Kubernetes project, which is still at the incubation stage. Kubernetes’ popularity as a platform is rapidly growing during the last months, but it still has some weaknesses. One of them is inter-service communication. Kubernetes doesn’t give us many mechanisms out-of-the-box, which allows us to configure more advanced rules. This is a reason for creating frameworks for service mesh on Kubernetes like Istio or Linkerd. While these projects are still relatively new solutions, Spring Cloud is a stable, opinionated framework. Why not to provide service discovery, inter-service communication or load balancing? Thanks to Spring Cloud Kubernetes it is possible.

The post Quick Guide to Microservices with Kubernetes, Spring Boot 2 and Docker appeared first on Piotr's TechBlog.

I came across Envoy Proxy for the first time a couple of weeks ago, when one of my blog readers suggested that I write an article about it. I had never heard about it before and my first thought was that it is not my area of experience. In fact, this tool is not as popular as its competition like Nginx or HAProxy, but it provides some interesting features among which we can distinguish out-of-the-box support for MongoDB, Amazon RDS, flexibility around discovery and load balancing or generating a lot of useful traffic statistics. Ok, we know a little about its advantages but what exactly is Envoy proxy? ‘Envoy is an open-source edge and service proxy, designed for cloud-native applications’. It was originally developed by Lift as a high-performance C++ distributed proxy designed for standalone services and applications, as well as for large microservices service mesh. It sounds really good right now. That’s why I decided to take a closer look at it and prepare a sample of service discovery and distributed tracing realized with Envoy and microservices-based on Spring Boot.

Envoy Proxy Configuration

In most of the previous samples based on Spring Cloud we have used Zuul as edge and proxy. Zuul is a popular Netflix OSS tool acting as API Gateway in your microservices architecture. As it turns out, it can be successfully replaced by Envoy proxy. One of the things I really like in Envoy is the way to create configuration. The default format is JSON and is validated against JSON schema. This JSON properties and schema are documented well and can be easily understood. Just what you’d expect from a modern solution the recommended way to get started with it is by using the pre-built Docker images. So, in the beginning we have to create a Dockerfile for building a Docker image with Envoy and provide configuration files in JSON format. Here’s my Dockerfile. Parameters service-cluster and service-node are optional and has to do with provided configuration for service discovery, which I’ll say more about in a minute.

FROM lyft/envoy:latest
RUN apt-get update
COPY envoy.json /etc/envoy.json
CMD /usr/local/bin/envoy -c /etc/envoy.json --service-cluster samplecluster --service-node sample1

I assume you have a basic knowledge about Docker and its commands, which is mandatory at this point. After providing envoy.json configuration file we can proceed with building a Docker image.

$ docker build -t envoy:v1 .

Then just run it using docker run command. Useful ports should be exposed outside.

$ docker run -d --name envoy -p 9901:9901 -p 10000:10000 envoy:v1

The first pretty helpful feature is the local HTTP administrator server. It can be configured in a JSON file inside admin property. For the example purpose I selected port 9901 and as you probably noticed I also had exposed that port outside the Envoy Docker container. Now, the admin console is available under http://192.168.99.100:9901/. If you invoke that address it prints all available commands. For me the most helpful were stats, which print all important statistics related to proxy and logging, where I could change logging level dynamically for some of the defined categories. So, first if you had any problems with Envoy try to change logging level by calling /logging?name=level and watch them on Docker container after running docker logs envoy command.

"admin": {
  "access_log_path": "/tmp/admin_access.log",
  "address": "tcp://0.0.0.0:9901"
}

The next required configuration property is listeners. There we define routing settings and the address on which Envoy will listen for incoming TCP connections. The notation tcp://0.0.0.0:10000 is the wild card match for any IPv4 address with port 10000. This port is also exposed outside the Envoy Docker container. In this case it will therefore be our API gateway available under http://192.168.99.100:10000/ address. We will come back to the proxy configuration details at a next stage and now let’s take a closer look at the architecture of the presented example.

"listeners": [{
  "address": "tcp://0.0.0.0:10000",
  ...
}]

Architecture: Envoy proxy, Zipkin and Spring Boot

The architecture of the described solution is visible on the figure below. We have Envoy proxy as API Gateway, which is an entry point to our system. Envoy integrates with Zipkin and sends tracing messages with information about incoming HTTP requests and responses sent back. Two sample microservices Person and Product register itself in service discovery on startup and deregister on shutdown. They are hidden from external clients behind API Gateway. Envoy has to fetch actual configuration with addresses of registered services and route incoming HTTP requests properly. If there are multiple instances of each service available it should perform load balancing.

As it turns out Envoy does not support well known discovery servers like Consul or Zookeeper, but defines its own generic REST based API, which needs to be implemented to enable cluster members fetching. The main method of this API is GET /v1/registration/:service used for fetching the list of currently registered instances of service. Lyft’s provides its default implementation in Python, but for the example purpose we develop our own solution using Java and Spring Boot. Sample application source code is available on GitHub. In addition to service discovery implementation you would also find there two sample microservices.

Service Discovery

Our custom discovery implementation does nothing more than exposing REST based API with methods for registration, unregistration and fetching service’s instances. GET method needs to return a specific JSON structure which matches the following schema.

{
  "hosts": [{
    "ip_address": "...",
    "port": "...",
    ...
  }]
}

Here’s a REST controller class with discovery API implementation.

@RestController
public class EnvoyDiscoveryController {

   private static final Logger LOGGER = LoggerFactory.getLogger(EnvoyDiscoveryController.class);

   private Map<String, List<DiscoveryHost>> hosts = new HashMap<>();

   @GetMapping(value = "/v1/registration/{serviceName}")
   public DiscoveryHosts getHostsByServiceName(@PathVariable("serviceName") String serviceName) {
      LOGGER.info("getHostsByServiceName: service={}", serviceName);
      DiscoveryHosts hostsList = new DiscoveryHosts();
      hostsList.setHosts(hosts.get(serviceName));
      LOGGER.info("getHostsByServiceName: hosts={}", hostsList);
      return hostsList;
   }

   @PostMapping("/v1/registration/{serviceName}")
   public void addHost(@PathVariable("serviceName") String serviceName, @RequestBody DiscoveryHost host) {
      LOGGER.info("addHost: service={}, body={}", serviceName, host);
      List<DiscoveryHost> tmp = hosts.get(serviceName);
      if (tmp == null)
         tmp = new ArrayList<>();
      tmp.add(host);
      hosts.put(serviceName, tmp);
   }

   @DeleteMapping("/v1/registration/{serviceName}/{ipAddress}")
   public void deleteHost(@PathVariable("serviceName") String serviceName, @PathVariable("ipAddress") String ipAddress) {
      LOGGER.info("deleteHost: service={}, ip={}", serviceName, ipAddress);
      List<DiscoveryHost> tmp = hosts.get(serviceName);
      if (tmp != null) {
         Optional<DiscoveryHost> optHost = tmp.stream().filter(it -> it.getIpAddress().equals(ipAddress)).findFirst();
         if (optHost.isPresent())
            tmp.remove(optHost.get());
         hosts.put(serviceName, tmp);
      }
   }
}

Let’s get back to the Envoy configuration settings. Assuming we have built an image from Dockerfile visible below and then run the container on the default port we can invoke it under address http://192.168.99.100:9200. That address should be placed in envoy.json configuration file. Service discovery connection settings should be provided inside the Cluster Manager section.

FROM openjdk:alpine
MAINTAINER Piotr Minkowski <piotr.minkowski@gmail.com>
ADD target/envoy-discovery.jar envoy-discovery.jar
ENTRYPOINT ["java", "-jar", "/envoy-discovery.jar"]
EXPOSE 9200

Here’s a fragment from envoy.json file. Cluster for service discovery should be defined as a global SDS configuration, which must be specified inside sds property (1). The most important thing is to provide a correct URL (2) and on the basis of that Envoy automatically tries to call endpoint GET /v1/registration/{service_name}. The last interesting configuration field for that section is refresh_delay_ms, which is responsible for setting a delay between fetches a list of services registered in a discovery server. That’s not all. We also have to define cluster members. They are identified by the name (4). Their type is sds (5), what means that this cluster uses a service discovery server for locating network addresses of calling microservice with the name defined in the service-name property.

"cluster_manager": {
  "clusters": [{
    "name": "service1", (4)
    "type": "sds", // (5)
    "connect_timeout_ms": 5000,
    "lb_type": "round_robin",
    "service_name": "person-service" // (6)
  }, {
    "name": "service2",
    "type": "sds",
    "connect_timeout_ms": 5000,
    "lb_type": "round_robin",
    "service_name": "product-service"
  }],
  "sds": { // (1)
    "cluster": {
      "name": "service_discovery",
      "type": "strict_dns",
      "connect_timeout_ms": 5000,
      "lb_type": "round_robin",
      "hosts": [{
        "url": "tcp://192.168.99.100:9200" // (2)
      }]
    },
    "refresh_delay_ms": 3000 // (3)
  }
}

Routing configuration is defined for every single listener inside route_config property (1). The first route is configured for person-service, which is processing by cluster service1 (2), second for product-service processing by service2 cluster. So, our services are available under http://192.168.99.100:10000/person and http://192.168.99.100:10000/product addresses.

{
  "name": "http_connection_manager",
  "config": {
    "codec_type": "auto",
    "stat_prefix": "ingress_http",
    "route_config": { // (1)
      "virtual_hosts": [{
        "name": "service",
        "domains": ["*"],
        "routes": [{
          "prefix": "/person", // (2)
          "cluster": "service1"
        }, {
          "prefix": "/product", // (3)
          "cluster": "service2"
        }]
      }]
    },
    "filters": [{
      "name": "router",
      "config": {}
    }]
  }
}

Building Microservices

The routing on Envoy proxy has been already configured. We still don’t have running microservices. Their implementation is based on the Spring Boot framework and does nothing more than expose REST API providing simple operations on the object’s list and registering/unregistering service on the discovery server. Here’s @Service bean responsible for that registration. The onApplicationEvent method is fired after application startup and destroy method just before gracefully shutdown.

@Service
public class PersonRegister implements ApplicationListener<ApplicationReadyEvent> {

   private static final Logger LOGGER = LoggerFactory.getLogger(PersonRegister.class);

   private String ip;
   @Value("${server.port}")
   private int port;
   @Value("${spring.application.name}")
   private String appName;
   @Value("${envoy.discovery.url}")
   private String discoveryUrl;
   
   @Autowired
   RestTemplate template;

   @Override
   public void onApplicationEvent(ApplicationReadyEvent event) {
      LOGGER.info("PersonRegistration.register");
      try {
         ip = InetAddress.getLocalHost().getHostAddress();
         DiscoveryHost host = new DiscoveryHost();
         host.setPort(port);
         host.setIpAddress(ip);
         template.postForObject(discoveryUrl + "/v1/registration/{service}", host, DiscoveryHosts.class, appName);
      } catch (Exception e) {
         LOGGER.error("Error during registration", e);
      }
   }

   @PreDestroy
   public void destroy() {
      try {
         template.delete(discoveryUrl + "/v1/registration/{service}/{ip}/", appName, ip);
         LOGGER.info("PersonRegister.unregistered: service={}, ip={}", appName, ip);
      } catch (Exception e) {
         LOGGER.error("Error during unregistration", e);
      }
   }

}

The best way to shutdown Spring Boot application gracefully is by its Actuator endpoint. To enable such endpoints for the service include spring-boot-starter-actuator to your project dependencies. Shutdown is disabled by default, so we should add the following properties to application.yml to enable it and additionally disable default security (endpoints.shutdown.sensitive=false). Now, just by calling POST /shutdown we can stop our Spring Boot application and test unregister method.

endpoints:
  shutdown:
    enabled: true
    sensitive: false

Same as before for microservices we also build docker images. Here’s person-service Dockerfile, which allows you to override default service and SDS port.

FROM openjdk:alpine
MAINTAINER Piotr Minkowski <piotr.minkowski@gmail.com>
ADD target/person-service.jar person-service.jar
ENV DISCOVERY_URL http://192.168.99.100:9200
ENTRYPOINT ["java", "-jar", "/person-service.jar"]
EXPOSE 9300

To build an image and run a container of the service with custom listen port type you need to execute the following docker commands.

$ docker build -t piomin/person-service .
$ docker run -d --name person-service -p 9301:9300 piomin/person-service

Distributed Tracing

It is time for the last piece of the puzzle – Zipkin tracing. Statistics related to all incoming requests should be sent there. The first part of configuration in Envoy proxy is inside tracing property which specifies global settings for the HTTP tracer.

"tracing": {
  "http": {
    "driver": {
      "type": "zipkin",
      "config": {
        "collector_cluster": "zipkin",
        "collector_endpoint": "/api/v1/spans"
      }
    }
  }
}

Network location and settings for Zipkin connection should be defined as a cluster member.

"clusters": [{
  "name": "zipkin",
  "connect_timeout_ms": 5000,
  "type": "strict_dns",
  "lb_type": "round_robin",
  "hosts": [
    {
      "url": "tcp://192.168.99.100:9411"
    }
  ]
}]

We should also add a new section tracing in HTTP connection manager configuration (1). Field operation_name is required and sets a span name. Only ‘ingress’ and ‘egress’ values are supported.

"listeners": [{
  "filters": [{
    "name": "http_connection_manager",
    "config": {
      "tracing": { // (1)
        "operation_name": "ingress" // (2)
      }
      // ...
    }
  }]
}]

Zipkin server can be started using its Docker image.

$ docker run -d --name zipkin -p 9411:9411 openzipkin/zipkin

Summary

Here’s a list of running Docker containers for the test purpose. As you probably remember we have Zipkin, Envoy, custom discovery, two instances of person-service and one of product-service. You can add some person objects by calling POST /person and that display a list of all persons by calling GET /person. The requests should be load balanced between two instances basing on entries in the service discovery.

Information about every request is sent to Zipkin with a service name taken –service-cluster Envoy proxy running parameter.

The post Envoy Proxy with Microservices appeared first on Piotr's TechBlog.

Spring Cloud Sleuth – library available as a part of Spring Cloud project. Lets you track the progress of subsequent microservices by adding the appropriate headers to the HTTP requests. The library is based on the MDC (Mapped Diagnostic Context) concept, where you can easily extract values put to context and display them in the logs.

Zipkin – distributed tracing system that helps to gather timing data for every request propagated between independent services. It has a simple management console where we can find a visualization of the time statistics generated by subsequent services.

ELK – Elasticsearch, Logstash, Kibana: three different tools usually used together. They are used for searching, analyzing, and visualizing log data in real-time.

Probably many of you, even if you have not had a touch with Java or microservices before, heard about Logstash and Kibana. For example, if you look at the hub.docker.com among the most popular images you will find the ones for the above tools. In our example, we will just use those images. Let’s begin by running the container with Elasticsearch.

docker run -d -it --name es -p 9200:9200 -p 9300:9300 elasticsearch

Then we can run the Kibana container and link it to the Elasticsearch.

docker run -d -it --name kibana --link es:elasticsearch -p 5601:5601 kibana

In the end, we will start Logstash with input and output declared. As an input, we declare TCP which is compatible with LogstashTcpSocketAppender used as a logging appender in our sample application. As an output elasticsearch has been declared. Each microservice will be indexed on its name with micro prefix.

docker run -d -it --name logstash -p 5000:5000 logstash -e 'input { tcp { port => 5000 codec => "json" } } output { elasticsearch { hosts => ["192.168.99.100"] index => "micro-%{serviceName}"} }'

Now we can take a look on sample microservices. This post is a continuation of my previous article Part 1: Creating microservice using Spring Cloud, Eureka and Zuul. Architecture and exposed services are the same as in the previous sample. Source code is available on GitHub (branch logstash). Like a mentioned before we will use Logback library for sending log data to Logstash. In addition to the three Logback dependencies we also add libraries for Zipkin integration and Spring Cloud Sleuth starter. Here’s fragment of pom.xml for microservice.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-sleuth</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-sleuth-zipkin</artifactId>
</dependency>
<dependency>
   <groupId>net.logstash.logback</groupId>
   <artifactId>logstash-logback-encoder</artifactId>
   <version>4.9</version>
</dependency>
<dependency>
   <groupId>ch.qos.logback</groupId>
   <artifactId>logback-classic</artifactId>
   <version>1.2.3</version>
</dependency>
<dependency>
   <groupId>ch.qos.logback</groupId>
   <artifactId>logback-core</artifactId>
   <version>1.2.3</version>
</dependency>
 

There is also Logback configuration file in src/main/resources directory. Here’s logback.xml fragment. We can configure which logging field are sending to Logstash by declaring tags mdc, logLevel, message etc. We are also appending service name field for elasticsearch index creation.

<appender name="STASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
   <destination>192.168.99.100:5000</destination>

   <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
      <providers>
         <mdc />
         <context />
         <logLevel />
         <loggerName />

         <patterns>
            <pattern>
               {
                  "serviceName": "account-service"
               }
            </pattern>
         </patterns>

         <threadName />
         <message />
         <logstashMarkers />
         <stackTrace />
      </providers>
   </encoder>
</appender>
 

The configuration of Spring Cloud Sleuth is very simple. We only have to add spring-cloud-starter-sleuth dependency to pom.xml and declare sampler @Bean . In the sample I declared AlwaysSampler that exports every span, but there is also an other other option – PercentageBasedSampler that samples a fixed fraction of spans.

@Bean
public AlwaysSampler defaultSampler() {
   return new AlwaysSampler();
}
 

After starting ELK docker containers we need to run our microservices. There are 5 Spring Boot applications which need to be run discovery-service, account-service, customer-service, gateway-service and zipkin-service. After launching all of them we can try call some services, for example http://localhost:8765/api/customer/customers/{id}, which causes calling of both customer and account service. All logs will be stored in elasticsearch with micro-%{serviceName} index. They can be searched in Kibana with micro-* index pattern. Index patterns are created in Kibana under section Management -> Index patterns. Kibana is available under address http://192.168.99.100:5601. After first running we will be prompt for index pattern, so let’s type micro-*. Under Discover section we can take o look on all logs matched typed pattern with timeline visualization.

Kibana is an intuitive and user-friendly tool. I will not describe in the details how to use Kibana, because you can easily find it out by yourself reading documentation or just clicking UI. The most important thing is to be able to search for logs by filtering criteria. In the picture below there is an example of searching logs by X-B3-TraceId field, which is added to the request header by Spring Cloud Sleuth. Sleuth also adds X-B3-SpanId for marking request for single microservice. We can select which field is displayed in the result list – in this sample I selected message and serviceName as you see in the left pane of the picture.

Here’s a picture with single request details. It is visible after expanding each log row.

Spring Cloud Sleuth also sends statistics to Zipkin. That is another kind of data that is stored in Logstash. These are timing statistics for each request. Zipkin UI is really simple. You can filter the requests by some criteria like time, service name, endpoint name. Here’s a picture with the same requests which were visualized with Kibana: http://localhost:8765/api/customer/customers/{id}.

We can always see the details of each request by clicking on it. Then you see the picture similar to visible below. In the beginning, the request has been processed on the API gateway. Then gateway discovered customer service on the Eureka server and called that service. Customer service also has to discover account service and then call it. In this view, you can easily find out which operation is the most time-consuming.

The post Part 2: Creating microservices – monitoring with Spring Cloud Sleuth, ELK and Zipkin appeared first on Piotr's TechBlog.