If you are looking for other interesting articles about Quarkus, you will find them in my blog. For example, you will read more about testing strategies with Quarkus and Pact here.

Source Code

If you would like to try it by yourself, you may always take a look at my source code. To do that, you must clone my sample GitHub repository. Then you should only follow my instructions

Architecture

Before proceeding to the implementation, let’s take a look at the diagram of our system architecture. There are three microservices: employee-service, departament-service, and organization-service. They are communicating with each other through a REST API. They use the Consul Key-Value store as a distributed configuration backend. Every instance of service is registering itself in Consul. A load balancer is included in the application. It reads a list of registered instances of a target service from the Consul using the Quarkus Stork extension. Then it chooses an instance using a provided algorithm.

Running Consul Instance

We will run a single-node Consul instance as a Docker container. By default, Consul exposes HTTP API and a UI console on the 8500 port. Let’s expose that port outside the container.

docker run -d --name=consul \
   -e CONSUL_BIND_INTERFACE=eth0 \
   -p 8500:8500 \
   consul

Dependencies

Let’s analyze a list of the most important Maven dependencies using the department-service application as an example. Our application exposes REST endpoints and connects to the in-memory H2 database. We use the Quarkus REST client and the SmallRye Stork Service Discovery library to implement communication between the microservices. On the other hand, the io.quarkiverse.config:quarkus-config-consul is responsible for reading configuration properties the Consul Key-Value store. With the smallrye-mutiny-vertx-consul-client library the application is able to interact directly with the Consul HTTP API. This may not be necessary in the future, once the Stork project will implement the registration and deregistration mechanism. Currently it is not ready. Finally, we will Testcontainers to run Consul and tests our apps against it with the Quarkus JUnit support.

	<dependencies>
		<dependency>
			<groupId>io.quarkus</groupId>
			<artifactId>quarkus-rest-jackson</artifactId>
		</dependency>
		<dependency>
			<groupId>io.quarkus</groupId>
			<artifactId>quarkus-rest-client-jackson</artifactId>
		</dependency>
		<dependency>
			<groupId>io.quarkus</groupId>
			<artifactId>quarkus-hibernate-orm-panache</artifactId>
		</dependency>
		<dependency>
			<groupId>io.quarkus</groupId>
			<artifactId>quarkus-jdbc-h2</artifactId>
		</dependency>
		<dependency>
			<groupId>com.h2database</groupId>
			<artifactId>h2</artifactId>
			<scope>runtime</scope>
		</dependency>
		<dependency>
			<groupId>io.quarkus</groupId>
			<artifactId>quarkus-smallrye-stork</artifactId>
		</dependency>
		<dependency>
			<groupId>io.smallrye.reactive</groupId>
			<artifactId>smallrye-mutiny-vertx-consul-client</artifactId>
		</dependency>
		<dependency>
			<groupId>io.smallrye.stork</groupId>
			<artifactId>stork-service-discovery-consul</artifactId>
		</dependency>
		<dependency>
			<groupId>io.smallrye.stork</groupId>
			<artifactId>stork-service-registration-consul</artifactId>
		</dependency>
		<dependency>
			<groupId>io.quarkus</groupId>
			<artifactId>quarkus-scheduler</artifactId>
		</dependency>
		<dependency>
			<groupId>io.quarkiverse.config</groupId>
			<artifactId>quarkus-config-consul</artifactId>
			<version>${quarkus-consul.version}</version>
		</dependency>
		<dependency>
			<groupId>io.rest-assured</groupId>
			<artifactId>rest-assured</artifactId>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>io.quarkus</groupId>
			<artifactId>quarkus-junit5</artifactId>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.testcontainers</groupId>
			<artifactId>consul</artifactId>
			<version>1.20.3</version>
			<scope>test</scope>
		</dependency>
		<dependency>
			<groupId>org.testcontainers</groupId>
			<artifactId>junit-jupiter</artifactId>
			<version>1.20.3</version>
			<scope>test</scope>
		</dependency>
	</dependencies>

Discovery and Load Balancing with Quarkus Stork for Consul

Let’s begin with the Quarkus Stork part. In the previous section, we included libraries required to provide service discovery and load balancing with Stork: quarkus-smallrye-stork and stork-service-discovery-consul. Now, we can proceed to the implementation. Here’s the EmployeeClient interface from the department-service responsible for calling the GET /employees/department/{departmentId} endpoint exposed by the employee-service. Instead of setting the target URL inside the @RegisterRestClient annotation we should refer to the name of the service registered in Consul.

@Path("/employees")
@RegisterRestClient(baseUri = "stork://employee-service")
public interface EmployeeClient {

    @GET
    @Path("/department/{departmentId}")
    @Produces(MediaType.APPLICATION_JSON)
    List<Employee> findByDepartment(@PathParam("departmentId") Long departmentId);

}

That service name should also be used in the configuration properties. The following property indicates that Stork will use Consul as a discovery server for the employee-service name.

quarkus.stork.employee-service.service-discovery.type = consul

Once we create a REST client with the additional annotations, we must inject it into the DepartmentResource class using the @RestClient annotation. Afterward, we can use that client to interact with the employee-service while calling the GET /departments/organization/{organizationId}/with-employees from the department-service.

@Path("/departments")
@Produces(MediaType.APPLICATION_JSON)
public class DepartmentResource {

    private Logger logger;
    private DepartmentRepository repository;
    private EmployeeClient employeeClient;

    public DepartmentResource(Logger logger,
                              DepartmentRepository repository,
                              @RestClient EmployeeClient employeeClient) {
        this.logger = logger;
        this.repository = repository;
        this.employeeClient = employeeClient;
    }

    // ... other methods for REST endpoints 

    @Path("/organization/{organizationId}")
    @GET
    public List<Department> findByOrganization(@PathParam("organizationId") Long organizationId) {
        logger.infof("Department find: organizationId=%d", organizationId);
        return repository.findByOrganization(organizationId);
    }

    @Path("/organization/{organizationId}/with-employees")
    @GET
    public List<Department> findByOrganizationWithEmployees(@PathParam("organizationId") Long organizationId) {
        logger.infof("Department find with employees: organizationId=%d", organizationId);
        List<Department> departments = repository.findByOrganization(organizationId);
        departments.forEach(d -> d.setEmployees(employeeClient.findByDepartment(d.getId())));
        return departments;
    }

}

Let’s take a look at the implementation of the GET /employees/department/{departmentId} in the employee-service called by the EmployeeClient in the department-service.

@Path("/employees")
@Produces(MediaType.APPLICATION_JSON)
public class EmployeeResource {

    private Logger logger;
    private EmployeeRepository repository;

    public EmployeeResource(Logger logger,
                            EmployeeRepository repository) {
        this.logger = logger;
        this.repository = repository;
    }

    @Path("/department/{departmentId}")
    @GET
    public List<Employee> findByDepartment(@PathParam("departmentId") Long departmentId) {
        logger.infof("Employee find: departmentId=%s", departmentId);
        return repository.findByDepartment(departmentId);
    }

    @Path("/organization/{organizationId}")
    @GET
    public List<Employee> findByOrganization(@PathParam("organizationId") Long organizationId) {
        logger.infof("Employee find: organizationId=%s", organizationId);
        return repository.findByOrganization(organizationId);
    }
    
    // ... other methods for REST endpoints

}

Similarly in the organization-service, we define two REST clients for interacting with employee-service and department-service.

@Path("/departments")
@RegisterRestClient(baseUri = "stork://department-service")
public interface DepartmentClient {

    @GET
    @Path("/organization/{organizationId}")
    @Produces(MediaType.APPLICATION_JSON)
    List<Department> findByOrganization(@PathParam("organizationId") Long organizationId);

    @GET
    @Path("/organization/{organizationId}/with-employees")
    @Produces(MediaType.APPLICATION_JSON)
    List<Department> findByOrganizationWithEmployees(@PathParam("organizationId") Long organizationId);

}

@Path("/employees")
@RegisterRestClient(baseUri = "stork://employee-service")
public interface EmployeeClient {

    @GET
    @Path("/organization/{organizationId}")
    @Produces(MediaType.APPLICATION_JSON)
    List<Employee> findByOrganization(@PathParam("organizationId") Long organizationId);

}

It involves the need to include the following two configuration properties that set the discovery service type for the target services.

quarkus.stork.employee-service.service-discovery.type = consul
quarkus.stork.department-service.service-discovery.type = consul

The OrganizationResource class injects and uses both previously created clients.

@Path("/organizations")
@Produces(MediaType.APPLICATION_JSON)
public class OrganizationResource {

    private Logger logger;
    private OrganizationRepository repository;
    private DepartmentClient departmentClient;
    private EmployeeClient employeeClient;

    public OrganizationResource(Logger logger,
                                OrganizationRepository repository,
                                @RestClient DepartmentClient departmentClient,
                                @RestClient EmployeeClient employeeClient) {
        this.logger = logger;
        this.repository = repository;
        this.departmentClient = departmentClient;
        this.employeeClient = employeeClient;
    }

    // ... other methods for REST endpoints

    @Path("/{id}/with-departments")
    @GET
    public Organization findByIdWithDepartments(@PathParam("id") Long id) {
        logger.infof("Organization find with departments: id={}", id);
        Organization organization = repository.findById(id);
        organization.setDepartments(departmentClient.findByOrganization(organization.getId()));
        return organization;
    }

    @Path("/{id}/with-departments-and-employees")
    @GET
    public Organization findByIdWithDepartmentsAndEmployees(@PathParam("id") Long id) {
        logger.infof("Organization find with departments and employees: id={}", id);
        Organization organization = repository.findById(id);
        organization.setDepartments(departmentClient.findByOrganizationWithEmployees(organization.getId()));
        return organization;
    }

    @Path("/{id}/with-employees")
    @GET
    public Organization findByIdWithEmployees(@PathParam("id") Long id) {
        logger.infof("Organization find with employees: id={}", id);
        Organization organization = repository.findById(id);
        organization.setEmployees(employeeClient.findByOrganization(organization.getId()));
        return organization;
    }

}

Registration in Consul with Quarkus

After including Stork, the Quarkus REST client automatically splits traffic between all the instances of the application existing in the discovery server. However, each application must register itself in the discovery server. Quarkus Stork won’t do that. Theoretically, there is the stork-service-registration-consul module that should register the application instance on startup. As far as I know, this feature is still under active development. For now, we will include a mentioned library and use the same property for enabling the registrar feature.

quarkus.stork.employee-service.service-registrar.type = consul

Our sample applications will interact directly with the Consul server using the SmallRye Mutiny reactive client. Let’s define the ClientConsul bean. It is registered only if the quarkus.stork.employee-service.service-registrar.type property with the consul value exists.

@ApplicationScoped
public class EmployeeBeanProducer {

    @ConfigProperty(name = "consul.host", defaultValue = "localhost")  String host;
    @ConfigProperty(name = "consul.port", defaultValue = "8500") int port;

    @Produces
    @LookupIfProperty(name = "quarkus.stork.employee-service.service-registrar.type", 
                      stringValue = "consul")
    public ConsulClient consulClient(Vertx vertx) {
        return ConsulClient.create(vertx, new ConsulClientOptions()
                .setHost(host)
                .setPort(port));
    }

}

The bean responsible for catching the startup and shutdown events is annotated with @ApplicationScoped. It defines two methods: onStart and onStop. It also injects the ConsulClient bean. Quarkus dynamically generates the HTTP listen port number on startup and saves it in the quarkus.http.port property. Therefore, the startup task needs to wait a moment to ensure that the application is running. We will run it 3 seconds after receiving the startup event. Every instance of the application needs to have a unique id in Consul. Therefore, we retrieve the number of running port and use that number as the id suffix. The name of the service is taken from the quarkus.application.name property. The instance of the application should save id in order to be able to deregister itself on shutdown.

@ApplicationScoped
public class EmployeeLifecycle {

    @ConfigProperty(name = "quarkus.application.name")
    private String appName;
    private int port;

    private Logger logger;
    private Instance<ConsulClient> consulClient;
    private ScheduledExecutorService executor;

    public EmployeeLifecycle(Logger logger,
                             Instance<ConsulClient> consulClient,
                             ScheduledExecutorService executor) {
        this.logger = logger;
        this.consulClient = consulClient;
        this.executor = executor;
    }

    void onStart(@Observes StartupEvent ev) {
        if (consulClient.isResolvable()) {
            executor.schedule(() -> {
                port = ConfigProvider.getConfig().getValue("quarkus.http.port", Integer.class);
                consulClient.get().registerService(new ServiceOptions()
                                .setPort(port)
                                .setAddress("localhost")
                                .setName(appName)
                                .setId(appName + "-" + port),
                        result -> logger.infof("Service %s-%d registered", appName, port));
            }, 3000, TimeUnit.MILLISECONDS);
        }
    }

    void onStop(@Observes ShutdownEvent ev) {
        if (consulClient.isResolvable()) {
            consulClient.get().deregisterService(appName + "-" + port,
                    result -> logger.infof("Service %s-%d deregistered", appName, port));
        }
    }
}

Read Configuration Properties from Consul

The io.quarkiverse.config:quarkus-config-consul is already included in dependencies. Once the quarkus.consul-config.enabled property is set to true, the Quarkus application tries to read properties from the Consul Key-Value store. The quarkus.consul-config.properties-value-keys property indicates the location of the properties file stored in Consul. Here are the properties that exists in the classpath application.properties. For example, the default config location for the department-service is config/department-service.

quarkus.application.name = department-service
quarkus.application.version = 1.1
quarkus.consul-config.enabled = true
quarkus.consul-config.properties-value-keys = config/${quarkus.application.name}

Let’s switch to the Consul UI. It is available under the same 8500 port as the API. In the “Key/Value” section we create configuration for all three sample applications.

These are configuration properties for department-service. They are targeted for the development mode. We enable the dynamically generated port number to run several instances on the same workstation. Our application use an in-memory H2 database. It loads the import.sql script on startup to initialize a demo data store. We also enable Quarkus Stork service discovery for the employee-service REST client and registration in Consul.

quarkus.http.port = 0
quarkus.datasource.db-kind = h2
quarkus.hibernate-orm.database.generation = drop-and-create
quarkus.hibernate-orm.sql-load-script = src/main/resources/import.sql
quarkus.stork.employee-service.service-discovery.type = consul
quarkus.stork.department-service.service-registrar.type = consul

Here are the configuration properties for the employee-service.

Finally, let’s take a look at the organization-service configuration in Consul.

Run Applications in the Development Mode

Let’s run our three sample Quarkus applications in the development mode. Both employee-service and department-service should have two instances running. We don’t have to take care about port conflicts, since they are aqutomatically generated on startup.

$ cd employee-service
$ mvn quarkus:dev
$ mvn quarkus:dev

$ cd department-service
$ mvn quarkus:dev
$ mvn quarkus:dev

$ cd organization-service
$ mvn quarkus:dev

Once we start all the instances we can switch to the Consul UI. You should see exactly the same services in your web console.

There are two instances of the employee-service and deparment-service. We can check out the list of registered instances for the selected application.

This step is optional. To simplify tests I also included API gateway that integrates with Consul discovery. It listens on the static 8080 port and forwards requests to the downstream services, which listen on the dynamic ports. Since Quarkus does not provide a module dedicates for the API gateway, I used Spring Cloud Gateway with Spring Cloud Consul for that. Therefore, you need to use the following command to run the application:

$ cd gateway-service
$ mvn spring-boot:run

Afterward, we can make some API tests with or without the gateway. With the gateway-service, we can use the 8080 port with the /api base context path. Let’s call the following three endpoints. The first one is exposed by the department-service, while the another two by the organization-service.

$ curl http://localhost:8080/api/departments/organization/1/with-employees
$ curl http://localhost:8080/api/organizations/1/with-departments
$ curl http://localhost:8080/api/organizations/1/with-departments-and-employees

Each Quarkus service listens on the dynamic port and register itself in Consul using that port number. Here’s the department-service logs from startup and during test communication.

After including the quarkus-micrometer-registry-prometheus module each application instance exposes metrics under the GET /q/metrics endpoint. There are several metrics related to service discovery published by the Quarkus Stork extension.

$ curl http://localhost:51867/q/metrics | grep stork
# TYPE stork_service_discovery_instances_count counter
# HELP stork_service_discovery_instances_count The number of service instances discovered
stork_service_discovery_instances_count_total{service_name="employee-service"} 12.0
# TYPE stork_service_selection_duration_seconds summary
# HELP stork_service_selection_duration_seconds The duration of the selection operation
stork_service_selection_duration_seconds_count{service_name="employee-service"} 6.0
stork_service_selection_duration_seconds_sum{service_name="employee-service"} 9.93934E-4
# TYPE stork_service_selection_duration_seconds_max gauge
# HELP stork_service_selection_duration_seconds_max The duration of the selection operation
stork_service_selection_duration_seconds_max{service_name="employee-service"} 0.0
# TYPE stork_service_discovery_failures counter
# HELP stork_service_discovery_failures The number of failures during service discovery
stork_service_discovery_failures_total{service_name="employee-service"} 0.0
# TYPE stork_service_discovery_duration_seconds_max gauge
# HELP stork_service_discovery_duration_seconds_max The duration of the discovery operation
stork_service_discovery_duration_seconds_max{service_name="employee-service"} 0.0
# TYPE stork_service_discovery_duration_seconds summary
# HELP stork_service_discovery_duration_seconds The duration of the discovery operation
stork_service_discovery_duration_seconds_count{service_name="employee-service"} 6.0
stork_service_discovery_duration_seconds_sum{service_name="employee-service"} 2.997176541
# TYPE stork_service_selection_failures counter
# HELP stork_service_selection_failures The number of failures during service selection
stork_service_selection_failures_total{service_name="employee-service"} 0.0

Advanced Load Balancing with Quarkus Stork and Consul

Quarkus Stork provides several load balancing strategies to efficiently distribute requests across multiple instances of a application. It can ensure optimal resource usage, better performance, and high availability. By default, Quarkus Stork uses round-robin algorithm. To override the default strategy, we first need to include a library responsible for providing the selected load-balancing algorithm. For example, let’s choose the least-response-time strategy, which collects response times of the calls made with service instances and picks an instance based on this information.

<dependency>
    <groupId>io.smallrye.stork</groupId>
    <artifactId>stork-load-balancer-least-response-time</artifactId>
</dependency>

Then, we have to change the default strategy in configuration properties for the selected client. Let’s add the following property to the config/department-service in Consul Key-Value store.

quarkus.stork.employee-service.load-balancer.type=least-response-time

After that, we can restart the instance of department-service and retest the communication between services.

Testing Integration Between Quarkus and Consul

We have already included the org.testcontainers:consul artifact to the Maven dependencies. Thanks to that, we can create JUnit tests with Quarkus and Testcontainers Consul. Since Quarkus doen’t provide a built-in support for testing Consul container, we need to create the class that implements the QuarkusTestResourceLifecycleManager interface. It is responsible for starting and stopping Consul container during JUnit tests. After starting the container, we add required configuration properties to enable in-memory database creation and a service registration in Consul.

public class ConsulResource implements QuarkusTestResourceLifecycleManager {

    private ConsulContainer consulContainer;

    @Override
    public Map<String, String> start() {
        consulContainer = new ConsulContainer("hashicorp/consul:latest")
                .withConsulCommand(
                """
                kv put config/department-service - <<EOF
                department.name=abc
                quarkus.datasource.db-kind=h2
                quarkus.hibernate-orm.database.generation=drop-and-create
                quarkus.stork.department-service.service-registrar.type=consul
                EOF
                """
                );

        consulContainer.start();

        String url = consulContainer.getHost() + ":" + consulContainer.getFirstMappedPort();

        return ImmutableMap.of(
                "quarkus.consul-config.agent.host-port", url,
                "consul.host", consulContainer.getHost(),
                "consul.port", consulContainer.getFirstMappedPort().toString()
        );
    }

    @Override
    public void stop() {
        consulContainer.stop();
    }
}

To start Consul container during the test, we need to annotate the test class with @QuarkusTestResource(ConsulResource.class). The test loads configuration properties from Consul on startup and registers the service. Then, it verifies that REST endpoints exposed by the department-service work fine and the registered service exists in Consul.

@QuarkusTest
@QuarkusTestResource(ConsulResource.class)
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class DepartmentResourceConsulTests {

    @ConfigProperty(name = "department.name", defaultValue = "")
    private String name;
    @Inject
    ConsulClient consulClient;

    @Test
    @Order(1)
    void add() {
        Department d = new Department();
        d.setOrganizationId(1L);
        d.setName(name);

        given().body(d).contentType(ContentType.JSON)
                .when().post("/departments").then()
                .statusCode(200)
                .body("id", notNullValue())
                .body("name", is(name));
    }

    @Test
    @Order(2)
    void findAll() {
        when().get("/departments").then()
                .statusCode(200)
                .body("size()", is(4));
    }

    @Test
    @Order(3)
    void checkRegister() throws InterruptedException {
        Thread.sleep(5000);
        Uni<ServiceList> uni = Uni.createFrom().completionStage(() -> consulClient.catalogServices().toCompletionStage());
        List<Service> services = uni.await().atMost(Duration.ofSeconds(3)).getList();
        final long count = services.stream()
                .filter(svc -> svc.getName().equals("department-service")).count();
        assertEquals(1 ,count);
    }
}

Final Thoughts

This article introduces Quarkus Stork for Consul discovery and client-side load balancing. It shows how to integrate Quarkus with Consul Key-Value store for distributed configuration. It also covers the topics like integration testing with Testcontainers support, metrics, service registration and advanced load-balancing strategies.

The post Consul with Quarkus and SmallRye Stork appeared first on Piotr's TechBlog.

We will use KrakenD. Why? Although it is not the most popular API gateway tool, it seems to be very interesting. First of all, it is very fast and lightweight. Also, we can easily integrate it with Zipkin and Consul – which obviously is our goal in this article.

Source Code

In this article, I will use the source code from my previous article Guide to Microservices with Micronaut and Consul. Since it has been written two years ago, I had to update a version of the Micronaut framework. Fortunately, the only thing I also had to change was groupId of the micronaut-openapi artifact. After that change, everything was working perfectly fine.

If you would like to try it by yourself, you may always take a look at my source code. In order to do that you need to clone my GitHub repository. Then you should just follow my instructions

Architecture

Firstly, let’s take a look at the architecture of our sample system. We have three microservices: employee-service, department-service and organization-service. All of them are simple REST applications. The organization-service calls API exposed by the department-service, while the department-service calls API from the employee-service. They use Consul discovery to locate a network address of target microservices. They also send traces to Zipkin. Each application may be started in multiple instances. At the edge of our system, there is an API gateway – KrakenD. Krakend is integrating with Consul discovery through DNS. It also sends traces to Zipkin. The architecture is visible in the picture below.

Running Consul, Zipkin and microservices

In the first step, we are going to run Consul and Zipkin on Docker containers. The simplest way to start Consul is to run it in the development mode. To do that you should execute the following command. It is important to expose two ports 8500 and 8600. The first of them is responsible for the discovery, while the second for DNS.

$ docker run -d --name=consul \
   -p 8500:8500 -p 8600:8600/udp \
   -e CONSUL_BIND_INTERFACE=eth0 consul

Then, we need to run Zipkin. Don’t forget to expose port 9411.

$ docker run -d --name=zipkin -p 9411:9411 openzipkin/zipkin

Finally, we can run each of our application. They register themselves in Consul on startup. They are listening on the randomly generated port number. Here’s the common. configuration for every single Micronaut application.

micronaut:
  server:
    port: -1
  router:
    static-resources:
      swagger:
        paths: classpath:META-INF/swagger
        mapping: /swagger/**
endpoints:
  info:
    enabled: true
    sensitive: false
consul:
  client:
    registration:
      enabled: true
tracing:
  zipkin:
    enabled: true
    http:
      url: http://localhost:9411
    sampler:
      probability: 1

Consul acts as a configuration server for the applications. We use Micronaut Config Client for fetching property sources on startup.

micronaut:
  application:
    name: employee-service
  config-client:
    enabled: true
consul:
  client:
    defaultZone: "localhost:8500"
    config:
      format: YAML

Using Micronaut framework

In order to expose REST API, integrate with Consul and Zipkin we need to include the following dependencies.

<dependencies>
        <dependency>
            <groupId>io.micronaut</groupId>
            <artifactId>micronaut-http-server-netty</artifactId>
        </dependency>
        <dependency>
            <groupId>io.micronaut</groupId>
            <artifactId>micronaut-tracing</artifactId>
        </dependency>
        <dependency>
            <groupId>io.zipkin.brave</groupId>
            <artifactId>brave-instrumentation-http</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>io.zipkin.reporter2</groupId>
            <artifactId>zipkin-reporter</artifactId>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>io.opentracing.brave</groupId>
            <artifactId>brave-opentracing</artifactId>
        </dependency>
        <dependency>
            <groupId>io.micronaut</groupId>
            <artifactId>micronaut-discovery-client</artifactId>
        </dependency>
<dependencies>

The tracing headers (spans) are propagated across applications. Here’s the endpoint in department-service. It calls endpoint GET /employees/department/{departmentId} exposed by employee-service.

@Get("/organization/{organizationId}/with-employees")
@ContinueSpan
public List<Department> findByOrganizationWithEmployees(@SpanTag("organizationId") Long organizationId) {
   LOGGER.info("Department find: organizationId={}", organizationId);
   List<Department> departments = repository.findByOrganization(organizationId);
   departments.forEach(d -> d.setEmployees(employeeClient.findByDepartment(d.getId())));
   return departments;
}

In order to call employee-service endpoint we use Micronaut declarative REST client.

@Client(id = "employee-service", path = "/employees")
public interface EmployeeClient {
   @Get("/department/{departmentId}")
   List<Employee> findByDepartment(Long departmentId);	
}

Here’s the implementation of the GET /employees/department/{departmentId} endpoint inside employee-service. Micronaut propagates tracing spans between subsequent requests using the @ContinueSpan annotation.

@Get("/department/{departmentId}")
@ContinueSpan
public List<Employee> findByDepartment(@SpanTag("departmentId") Long departmentId) {
    LOGGER.info("Employees find: departmentId={}", departmentId);
    return repository.findByDepartment(departmentId);
}

Configure KrakenD Gateway and Consul DNS

We can configure KrakenD using JSON notation. Firstly, we need to define endpoints. The integration with Consul discovery needs to be configured in the backend section. The host has to be the same as the DNS name of a downstream service in Consul. We also set a target URL (url_pattern) and service discovery type (sd). Let’s take a look at the list of endpoints for department-service. We expose methods for searching by id (GET /department/{id}), adding a new department (POST /department), and searching all departments with a list of employees within a single organization (GET /department-with-employees/{organizationId}).

    {
      "endpoint": "/department/{id}",
      "method": "GET",
      "backend": [
        {
          "url_pattern": "/departments/{id}",
          "sd": "dns",
          "host": [
            "department-service.service.consul"
          ],
          "disable_host_sanitize": true
        }
      ]
    },
    {
      "endpoint": "/department-with-employees/{organizationId}",
      "method": "GET",
      "backend": [
        {
          "url_pattern": "/departments/organization/{organizationId}/with-employees",
          "sd": "dns",
          "host": [
            "department-service.service.consul"
          ],
          "disable_host_sanitize": true
        }
      ]
    },
    {
      "endpoint": "/department",
      "method": "POST",
      "backend": [
        {
          "url_pattern": "/departments",
          "sd": "dns",
          "host": [
            "department-service.service.consul"
          ],
          "disable_host_sanitize": true
        }
      ]
    }

It is also worth mentioning that we cannot create conflicting routes with Krakend. For example, I couldn’t define endpoint GET /department/organization/{organizationId}/with-employees, because it would conflict with the already existing endpoint GET /department/{id}. To avoid it I created a new context path /department-with-employees for my endpoint.

Similarly, I created a following configuration for employee-service endpoints.

    {
      "endpoint": "/employee/{id}",
      "method": "GET",
      "backend": [
        {
          "url_pattern": "/employees/{id}",
          "sd": "dns",
          "host": [
            "employee-service.service.consul"
          ],
          "disable_host_sanitize": true
        }
      ]
    },
    {
      "endpoint": "/employee",
      "method": "POST",
      "backend": [
        {
          "url_pattern": "/employees",
          "sd": "dns",
          "host": [
            "employee-service.service.consul"
          ],
          "disable_host_sanitize": true
        }
      ]
    }

In order to integrate KrakenD with Consul, we need to configure local DNS properly on our machine. It was a quite challenging task for me since I’m not very familiar with network topics. By default, Consul will listen on port 8600 for DNS queries in the consul domain. But DNS is served from port 53. Therefore, we need to configure DNS forwarding for Consul service discovery. There are several ways to do that, and you may read more about it here. I chose the dnsmasq tool for that. Following the guide, we need to create a file e.g. /etc/dnsmasq.d/10-consul with the following single line.

server=/consul/127.0.0.1#8600

Finally we need to start dnsmasq service and add 127.0.0.1 to the list of nameservers. Here’s my configuration of DNS servers.

Testing Consul DNS

Firstly, let’s run all our sample microservices. They are registered in Consul under the following names.

I run two instances of employee-service. Of course, all the applications are listening on a randomly generated port.

Finally, if you run the dig command with the DNS name of service you should have a similar response to the following. It means we may proceed to the last part of our exercise!

Running KrakenD API Gateway

Before we run KrakenD API gateway let’s configure one additional thing – integration with Zipkin. To do that we need to create section extra_config. Enabling Zipkin only requires us to add the zipkin exporter in the opencensus module. We need to URL (including port and path) where Zipkin is accepting the spans, and a service name for Krakend spans. I have also enabled metrics. Here’s the currently described part of the KrakenD configuration.

  "extra_config": {
    "github_com/devopsfaith/krakend-opencensus": {
      "sample_rate": 100,
      "reporting_period": 1,
      "exporters": {
        "zipkin": {
          "collector_url": "http://localhost:9411/api/v2/spans",
          "service_name": "api-gateway"
        }
      }
    },
    "github_com/devopsfaith/krakend-metrics": {
      "collection_time": "30s",
      "proxy_disabled": false,
      "listen_address": ":8090"
    }
  }

Finally, we can run KrakenD. The only parameter we need to pass is the location of the krakend.json configuration file. You may find a full version of that file in my GitHub repository inside the config directory.

$ krakend run -c krakend.json -d

Testing KrakenD with Consul and Micronaut

Once we started all our microservices, Consul, Zipkin, and KrakenD we may proceed to the tests. So first, let’s add some employees and departments by sending requests through an API gateway. KrakenD is listening on port 8080.

$ curl http://localhost:8080/employee -d '{"name":"John Smith","age":30,"position":"Architect","departmentId":1,"organizationId":1}' -H "Content-Type: application/json"
{"age":30,"departmentId":1,"id":1,"name":"John Smith","organizationId":1,"position":"Architect"}

$ curl http://localhost:8080/employee -d '{"name":"Paul Walker","age":22,"position":"Developer","departmentId":1,"organizationId":1}' -H "Content-Type: application/json"
{"age":22,"departmentId":1,"id":2,"name":"Paul Walker","organizationId":1,"position":"Developer"}

$ curl http://localhost:8080/employee -d '{"name":"Anna Hamilton","age":26,"position":"Developer","departmentId":2,"organizationId":1}' -H "Content-Type: application/json"
{"age":26,"departmentId":2,"id":3,"name":"Anna Hamilton","organizationId":1,"position":"Developer"}

$ curl http://localhost:8080/department -d '{"name":"Test1","organizationId":1}' -H "Content-Type:application/json"
{"id":1,"name":"Test1","organizationId":1}

$ curl http://localhost:8080/department -d '{"name":"Test2","organizationId":1}' -H "Content-Type:application/json"
{"id":2,"name":"Test2","organizationId":1}

Then let’s call a little bit more complex endpoint GET /department-with-employees/{organizationId}. As you probably remember, it is exposed by department-service and calls employee-service to fetch all employees assigned to the particular department.

$ curl http://localhost:8080/department-with-employees/1

However, we received a response with the HTTP 500 error code. We can find more details in the Krakend logs.

Krakend is unable to parse JSON array returned as a response by department-service. Therefore, we need to declare it explicitly with "is_collection": true so that KrakenD can convert it to an object for further manipulation. Here’s our current configuration for that endpoint.

    {
      "endpoint": "/department-with-employees/{organizationId}",
      "method": "GET",
      "backend": [
        {
          "url_pattern": "/departments/organization/{organizationId}/with-employees",
          "sd": "dns",
          "host": [
            "department-service.service.consul"
          ],
          "disable_host_sanitize": true,
          "is_collection": true
        }
      ]
    }

Now, let’s call the same endpoint once again. It works perfectly fine!

$ curl http://localhost:8080/department-with-employees/1   
{"collection":[{"id":1,"name":"Test1","organizationId":1},{"employees":[{"age":26,"id":3,"name":"Anna Hamilton","position":"Developer"}],"id":2,"name":"Test2","organizationId":1}]}

The last thing we can do here is to check out traces collected by Zipkin. Thanks to Micronaut support for spans propagation (@ContinueSpan), all the subsequent requests are grouped altogether.

The picture visible below shows Zipkin timeline for the latest request.

Conclusion

If you are looking for a gateway for your microservices – KrakenD is an excellent choice. Moreover, if you use Consul as a discovery server, and Zipkin (or Jaeger) as a tracing tool, it is easy to start with KrakenD. It also offers support for service discovery with Netflix Eureka, but it is a little bit more complicated to configure. Of course, you may also run KrakenD on Kubernetes (and integrate with its discovery), which is an absolute “must-have” for the modern API gateway.

The post Microservices with Micronaut, KrakenD and Consul appeared first on Piotr's TechBlog.

If you feel you need to enhance your knowledge about the Quarkus framework visit the site with guides. For more advanced information you may read the articles Guide to Quarkus with Kotlin and Guide to Quarkus on Kubernetes.

The Architecture

Before proceeding to the implementation, let’s take a look at the diagram with the architecture of our system. There are three microservices: employee-service, departament-service, and organization-service. They are communicating with each other through a REST API. They use the Consul KV store as a distributed configuration backend. Every single instance of microservice is registering itself in Consul. A load balancer is on the client-side. It reads a list of registered instances of a target service from Consul. Then it chooses a single instance using a round-robin algorithm.

Source code

If you would like to try it by yourself, you may always take a look at my source code. In order to do that, you need to clone my repository sample-quarkus-microservices-consul. Then you should just follow my instructions.

Run the Consul instance

In order to run Consul on the local machine, we use its Docker image. By default, Consul exposes API and a web console on port 8500. We just need to expose that port outside the container.

$ docker run -d --name=consul \
   -e CONSUL_BIND_INTERFACE=eth0 \
   -p 8500:8500 \
   consul

Register Quarkus Microservice in Consul

Our application exposes a REST API on the HTTP server and connects to an in-memory database H2. It also uses the Java Consul client to interact with a Consul API. Therefore, we need to include at least the following dependencies.

<dependencies>
   <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-resteasy-jackson</artifactId>
   </dependency>
   <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-hibernate-orm-panache</artifactId>
   </dependency>
   <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-jdbc-h2</artifactId>
   </dependency>
   <dependency>
      <groupId>com.h2database</groupId>
      <artifactId>h2</artifactId>
      <scope>runtime</scope>
   </dependency>
   <dependency>
      <groupId>com.orbitz.consul</groupId>
      <artifactId>consul-client</artifactId>
      <version>${consul-client.version}</version>
   </dependency>
</dependencies>

Since we will run all our applications locally it is worth enabling an HTTP random port feature. To do that we should set the property quarkus.http.port to 0.

quarkus.http.port=0

Then we create the Consul client bean. By default, it is trying to connect with a server on the localhost and port 8500. So, we don’t need to provide any additional configuration.

@ApplicationScoped
public class EmployeeBeansProducer {

   @Produces
   Consul consulClient = Consul.builder().build();

}

Every single instance of a Quarkus application should register itself in Consul just after startup. Consequently, it needs to be able to deregister itself on shutdown. Therefore, you should first implement a bean responsible for intercepting startup and shutdown events. It is not hard with Quarkus.

The bean responsible for catching the startup and shutdown events is annotated with @ApplicationScoped. It defines two methods: onStart and onStop. It also injects the Consul client bean. Quarkus generates the number of the HTTP listen port on startup and saves it in the quarkus.http.port property. Therefore, the startup task needs to wait a moment to ensure that the application is running. We will run it 5 seconds after receiving the startup event. In order to register an application in Consul, we need to use the ConsulAgent object. Every single instance of the application needs to have a unique id in Consul. Therefore, we retrieve the number of running instances and use that number as the id suffix. The name of the service is taken from the quarkus.application.name property. The instance of the application should save id in order to be able to deregister itself on shutdown.

@ApplicationScoped
public class EmployeeLifecycle {

   private static final Logger LOGGER = LoggerFactory
         .getLogger(EmployeeLifecycle.class);
   private String instanceId;

   @Inject
   Consul consulClient;
   @ConfigProperty(name = "quarkus.application.name")
   String appName;
   @ConfigProperty(name = "quarkus.application.version")
   String appVersion;

   void onStart(@Observes StartupEvent ev) {
      ScheduledExecutorService executorService = Executors
            .newSingleThreadScheduledExecutor();
      executorService.schedule(() -> {
         HealthClient healthClient = consulClient.healthClient();
         List<ServiceHealth> instances = healthClient
               .getHealthyServiceInstances(appName).getResponse();
         instanceId = appName + "-" + instances.size();
         ImmutableRegistration registration = ImmutableRegistration.builder()
               .id(instanceId)
               .name(appName)
               .address("127.0.0.1")
               .port(Integer.parseInt(System.getProperty("quarkus.http.port")))
               .putMeta("version", appVersion)
               .build();
         consulClient.agentClient().register(registration);
         LOGGER.info("Instance registered: id={}", registration.getId());
      }, 5000, TimeUnit.MILLISECONDS);
   }

   void onStop(@Observes ShutdownEvent ev) {
      consulClient.agentClient().deregister(instanceId);
      LOGGER.info("Instance de-registered: id={}", instanceId);
   }

}

Run Quarkus microservices locally

Thanks to the HTTP random port feature we don’t have to care about port conflicts between applications. So, we can run as many instances as we need. To run a single instance of application we should use the quarkus:dev Maven command.

$ mvn compile quarkus:dev

Let’s see at the logs after employee-service startup. The application successfully called the Consul API using a Consul agent. With 5 second delay is sends an instance id and a port number.

Let’s take a look at the list of services registered in Consul.

I run two instances of every microservice. We may take a look at list of instances registered, for example by employee-service.

Integrate Quarkus REST client with Consul discovery

Both departament-service and organization-service applications use the Quarkus REST module to communicate with other microservices.

<dependency>
   <groupId>io.quarkus</groupId>
   <artifactId>quarkus-rest-client</artifactId>
</dependency>

Let’s take a look at the EmployeeClient interface inside the departament-service. We won’t use @RegisterRestClient on it. It is just annotated with @Path and contains a single @GET method.

@Path("/employees")
public interface EmployeeClient {

    @GET
    @Path("/department/{departmentId}")
    @Produces(MediaType.APPLICATION_JSON)
    List<Employee> findByDepartment(@PathParam("departmentId") Long departmentId);

}

We won’t provide a target address of the service, but just its name from the discovery server. The base URI is available in the application.properties file.

client.employee.uri=http://employee

The REST client uses a filter to detect a list of running instances registered in Consul. The filter implements a round-robin load balancer. Consequently, it replaces the name of service in the target URI with a particular IP address and a port number.

public class LoadBalancedFilter implements ClientRequestFilter {

   private static final Logger LOGGER = LoggerFactory
         .getLogger(LoadBalancedFilter.class);

   private Consul consulClient;
   private AtomicInteger counter = new AtomicInteger();

   public LoadBalancedFilter(Consul consulClient) {
      this.consulClient = consulClient;
   }

   @Override
   public void filter(ClientRequestContext ctx) {
      URI uri = ctx.getUri();
      HealthClient healthClient = consulClient.healthClient();
      List<ServiceHealth> instances = healthClient
            .getHealthyServiceInstances(uri.getHost()).getResponse();
      instances.forEach(it ->
            LOGGER.info("Instance: uri={}:{}",
                  it.getService().getAddress(),
                  it.getService().getPort()));
      ServiceHealth instance = instances.get(counter.getAndIncrement());
      URI u = UriBuilder.fromUri(uri)
            .host(instance.getService().getAddress())
            .port(instance.getService().getPort())
            .build();
      ctx.setUri(u);
   }

}

Finally, we need to inject the filter bean into the REST client builder. After that, our Quarkus application is fully integrated with the Consul discovery.

@ApplicationScoped
public class DepartmentBeansProducer {

   @ConfigProperty(name = "client.employee.uri")
   String employeeUri;
   @Produces
   Consul consulClient = Consul.builder().build();

   @Produces
   LoadBalancedFilter filter = new LoadBalancedFilter(consulClient);

   @Produces
   EmployeeClient employeeClient() throws URISyntaxException {
      URIBuilder builder = new URIBuilder(employeeUri);
      return RestClientBuilder.newBuilder()
            .baseUri(builder.build())
            .register(filter)
            .build(EmployeeClient.class);
   }

}

Read configuration properties from Consul

Although Quarkus does not provide built-in integration with a Consul discovery, it is able to read configuration properties from there. Firstly, we need to include the Quarkus Consul Config module to the Maven dependencies.

<dependency>
   <groupId>io.quarkus</groupId>
   <artifactId>quarkus-consul-config</artifactId>
</dependency>

Then, we enable the mechanism with the quarkus.consul-config.enable property.

quarkus.application.name=employee
quarkus.consul-config.enabled=true
quarkus.consul-config.properties-value-keys=config/${quarkus.application.name}

The Quarkus Config client reads properties from a KV store based on the location set in quarkus.consul-config.properties-value-keys property. Let’s create the settings responsible for a database connection and for enabling a random HTTP port feature.

Finally, we can run the application. The effect is the same as they would be stored in the standard application.properties file. The configuration for departament-service and organization-service looks pretty similar, but it also contains URLs used by the HTTP clients to call other microservices. For some reasons the property quarkus.datasource.db-kind=h2 always needs to be set inside application.properties file.

Testing Quarkus Consul discovery with gateway

All the applications listen on the random HTTP port. In order to simplify testing, we should run an API gateway. It will listen on a defined port. Since Quarkus does not provide any implementation of an API gateway, we are going to use Spring Cloud Gateway. We can easily integrate it with Consul using a Spring Cloud discovery client.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-loadbalancer</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-consul-discovery</artifactId>
</dependency>

The configuration of Spring Cloud Gateway contains a list of routes. We need to create three routes for all our sample applications.

spring:
  application:
    name: gateway-service
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true
      routes:
        - id: employee-service
          uri: lb://employee
          predicates:
            - Path=/api/employees/**
          filters:
            - StripPrefix=1
        - id: department-service
          uri: lb://department
          predicates:
            - Path=/api/departments/**
          filters:
            - StripPrefix=1
        - id: organization-service
          uri: lb://organization
          predicates:
            - Path=/api/organizations/**
          filters:
            - StripPrefix=1
    loadbalancer:
      ribbon:
        enabled: false

Now, you may perform some test calls by yourself. The API gateway is available on port 8080. It uses prefix /api. Here are some curl commands to list all available employees, departments and organizations.

$ http://localhost:8080/api/employees
$ http://localhost:8080/api/departments
$ http://localhost:8080/api/organizations

Conclusion

Although Quarkus is a Kubernetes-native framework, we can use it to run microservices outside Kubernetes. The only problem we may encounter is a lack of support for external discovery. This article shows how to solve it. As a result, we created microservices architecture based on our custom discovery mechanism and built-in support for configuration properties in Consul. It is worth saying that Quarkus also provides integration with other third-party configuration solutions like Vault or Spring Cloud Config. If you are interested in a competitive solution based on Spring Boot and Spring Cloud you should read the article Microservices with Spring Boot, Spring Cloud Gateway and Consul Cluster.

The post Quarkus Microservices with Consul Discovery appeared first on Piotr's TechBlog.

• Integrating Spring Boot application with Consul discovery
• Integrating Spring Cloud Gateway with Consul discovery
• Using Consul KV for distributing configuration across Spring Boot applications
• Running Consul in a clustered mode
• Defining virtual zones for microservices and gateway

Microservices Architecture with Spring Cloud Consul cluster

Let’s proceed to the example system built with Spring Cloud Consul cluster support. It consists of four independent microservices. Some of them may call endpoints exposed by the others. The application source code is available on GitHub here: https://github.com/piomin/sample-spring-cloud-consul.git.
In the current example, we will try to develop a simple order system where customers may buy products. If a customer decides to confirm a selected list of products to buy, the POST request is sent to the order-service. It is processed by the Order prepare(@RequestBody Order order) method inside REST controller. This method is responsible for order preparation. First, it calculates the final price, considering the price of each product from the list, customer order history, and their category in the system by calling the proper API method from the customer-service. Then, it verifies if the customer’s account balance is enough to execute the order by calling the account-service, and finally, it returns the calculated price. If the customer confirms the action, the PUT /{id} method is called. The request is processed by the method Order accept(@PathVariable Long id) inside REST controller. It changes the order status and withdraws money from the customer’s account. The system architecture is broken down into the individual microservices hidden behind API gateway as shown here:

The description created above should give you a big picture of our example system. However, business logic plays a supporting role, technically we have four Spring Boot applications using Consul discovery and KV store communicating with each other through REST APIs. The whole system is hidden for the external client behind the API gateway built on top of the Spring Cloud Gateway. Let’s proceed to the implementation.

1. Building Spring Cloud Consul Microservices

Let’s begin from dependencies. We use the currently newest stable version of Spring Boot – 2.2.0.RELEASE together with Spring Cloud Release Train Hoxton.RC1. The minimal set of required dependencies is to have Spring Web, Actuator (optionally), and Spring Cloud Consul (discovery + config).

<parent>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-parent</artifactId>
   <version>2.2.0.RELEASE</version>
</parent>

<dependencyManagement>
   <dependencies>
      <dependency>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-dependencies</artifactId>
         <version>Hoxton.RC1</version>
         <type>pom</type>
         <scope>import</scope>
      </dependency>
   </dependencies>
</dependencyManagement>

<dependencies>
   <dependency>
      <groupId>org.springframework.cloud</groupId>
      <artifactId>spring-cloud-starter-consul-all</artifactId>
   </dependency>
   <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-web</artifactId>
   </dependency>
   <dependency>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-starter-actuator</artifactId>
   </dependency>
</dependencies>

When running the application we will use dynamic listen port number generation feature by setting property server.port to 0. Because we will run more than instance of every service we also need to override default value of spring.cloud.consul.discovery.instance-id which is based on port number that is not applicable when it is set to 0. Here’s our application.yml file for account-service.

spring:  
  cloud:
    consul:
      discovery:
        instance-id: "${spring.cloud.client.hostname}:${spring.application.name}:${random.int[1,999999]}"

server:
  port: 0

The configuration is deployed on Consul, which means we are only having bootstrap.yml file on classpath. If you have both Spring Cloud Consul Discovery and Config dependencies distributed configuration is enabled by default. You only have to override the address of the Consul server if required.

spring:  
  application:
    name: account-service
  cloud:
    consul:
      host: 192.168.99.100
      port: 8500

In the current version of Spring Cloud we don’t have to enable anything, so just need to declare the main class:

@SpringBootApplication
public class AccountApplication {
   
   public static void main(String[] args) {
      SpringApplication.run(AccountApplication.class);
   }
}

2. Running Consul Cluster using Docker

In this section how to set up the local environment similar to the production mode. Therefore, we would like to have a scalable, production-grade service discovery infrastructure, consisting of some nodes working together inside the cluster. Consul provides support for clustering based on a gossip protocol used for communication between members and a Raft consensus protocol for a leadership election. I wouldn’t like to go into the details of that process, but some basics about Consul architecture should be clarified.
The first important element is the Consul agent. An agent is a long-running daemon on every member of the Consul cluster. It may be run in either client or server mode. All agents are responsible for running checks and keeping services registered, in different nodes and in sync, globally. Our main goal in this section is to set up and configure the Consul cluster using its Docker image. First, we will start the container, which acts as a leader of the cluster. There is only one difference in the currently used Docker command than for the standalone Consul server. We have set the environment variable CONSUL_BIND_INTERFACE=eth0 in order to change the network address of the cluster agent from 127.0.0.1 to the one available for the other member containers. My Consul server is now running at the internal address 172.17.0.2. To check out what your address is (it should be the same) you may run the command docker logs consul. The appropriate information is logged just after the container startup. Here’s the command that starts the first Consul node:

$ docker run -d --name consul-1 -p 8500:8500 -e CONSUL_BIND_INTERFACE=eth0 consul

Knowledge of that address is very important since now we have to pass it to every member container startup command as a cluster join parameter. We also bind it to all
interfaces by setting 0.0.0.0 as a client address. Now, we may easily expose the client agent API outside the container using the -p parameter:

$ docker run -d --name consul-2 -e CONSUL_BIND_INTERFACE=eth0 -p 8501:8500 consul agent -dev -join=172.17.0.2
$ docker run -d --name consul-3 -e CONSUL_BIND_INTERFACE=eth0 -p 8502:8500 consul agent -dev -join=172.17.0.2

After running two containers with Consul agent, you may check out the full list of cluster members by executing the following command on the leader’s container:

We can always get the same information using the Consul Web Console.

We may easily change the default Consul node address for the Spring Boot application by changing configuration properties. Spring Cloud Consul cluster support allows you to define only a single host address and port number of Consul agent. It is worth noting that in normal production mode with multiple machines you would install only a Consul agent on every machine, which is connected with a cluster of Consul servers.

spring:
  application:
    name: customer-service
  cloud:
    consul:
      host: 192.168.99.100
      port: 8501

3. Inter-service Communication

An inter-service communication is performed using OpenFeign declarative REST client. We can also include Spring Cloud Sleuth dependency for propagating correlationId between subsequent calls.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-sleuth</artifactId>
</dependency>

The OpenFeign client is auto-integrated with service discovery. To use it we need to declare an interface with required methods for communication. The interface has to be annotated with @FeignClient that points to the service using its discovery name.

@FeignClient(name = "account-service")
public interface AccountClient {

   @GetMapping("/customer/{customerId}")
   List<Account> findByCustomer(@PathVariable("customerId") Long customerId);
   
}

Finally, OpenFeign client needs to be enabled for the whole application.

@SpringBootApplication
@EnableFeignClients
public class CustomerApplication {
   
   public static void main(String[] args) {
      SpringApplication.run(CustomerApplication.class, args);
   }
   
}

4. Enable Zone Affinity Mechanism

When using Spring Cloud Discovery we may take advantage of zones affinity mechanism. If your microservices has been deployed to multiple zones, you may prefer that those services communicate with other services within the same zone before trying to access them in another zone. The same rule applies to API gateway that prefers communication with microservices within the same zone as gateway.
For testing purposes we run two instances of every microservice distributed across two zones: zone1 and zone2. The same with gateway-service. The current architecture of our system looks as shown below.

The whole mechanism is enabled through the configuration. We need to set the default zone name for our microservice using property spring.cloud.consul.discovery.instanceZone. I defined two profiles for each application that may be set during startup with --spring.profiles.active command-line argument.

---
spring:
  profiles: zone1
  cloud:
    consul:
      discovery:
        instanceZone: zone1

---
spring:
  profiles: zone2
  cloud:
    consul:
      discovery:
        instanceZone: zone2

Spring Cloud provides a zone affinity mechanism based on Consul tags. If you set spring.cloud.consul.discovery.instanceZone property, Spring Cloud Consul tags a registered instance of service with zone metadata. The name of that tag may be overridden with spring.cloud.consul.discovery.defaultZoneMetadataName property. Assuming you have run two instances of each microservice divided into two zones using the command, for example java -jar --spring.profiles.active=zone1 target/order-service-1.1.jar, you should see the following list of registered services on your Consul instance.

Here’s a more detailed view is the Nodes section that prints all tags and a listen port number for every instance of microservice.

We can also display all running instances of a single service. In the following picture you can see instances of account-service.

5. Building API Gateway with Spring Cloud

Since now, we have succesfully run all the microservices in two instances distributed across two different zones. Because they are all listening on dynamically generated ports we need an API gateway which is exposed on a static port to an external client. Here’s the list of dependencies used for building gateway-service:

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-consul-all</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Because we would like to run a gateway on a static port the configuration of Maven profiles is slightly larger than for microservices. We also don’t need to register gateway in Consul discovery, because it is not accessed internally. We will run two instances of gateway, first available under port 8080 in zone1, and second available under port 9080 in zone2. Because it is not registered in Consul discovery we have to manually set a value for zone tag.

---
spring:
  profiles: zone1
  cloud:
    consul:
      discovery:
        instanceZone: zone1
        register: false
        registerHealthCheck: false
        tags: zone=zone1
server:  
  port: ${PORT:8080}

---
spring:
  profiles: zone2
  cloud:
    consul:
      discovery:
        instanceZone: zone2
        register: false
        registerHealthCheck: false
        tags: zone=zone2
server:  
  port: ${PORT:9080}

To enable integration with Consul discovery we need to set property spring.cloud.gateway.discovery.locator.enabled to true. In order to expose service under custom path we should define Path predicate and RewritePath filter for each service. In that case account-service is available under address http://localhost:8080/account/, customer-service under http://localhost:8080/customer/ etc.

spring:
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true
      routes:
        - id: account-service
          uri: lb://account-service
          predicates:
            - Path=/account/**
          filters:
            - RewritePath=/account/(?<path>.*), /$\{path}
        - id: customer-service
          uri: lb://customer-service
          predicates:
            - Path=/customer/**
          filters:
            - RewritePath=/customer/(?<path>.*), /$\{path}
        - id: order-service
          uri: lb://order-service
          predicates:
            - Path=/order/**
          filters:
            - RewritePath=/order/(?<path>.*), /$\{path}
        - id: product-service
          uri: lb://product-service
          predicates:
            - Path=/product/**
          filters:
            - RewritePath=/product/(?<path>.*), /$\{path}

Now you can be sure that each request incoming to gateway-service started in zone1 would be forwarded to in the first place to microservice also started in zone1. And the same for zone2.

6. Distributed Configuration

Consul Config is automatically enabled for the application just after including dependency spring-cloud-starter-consul-config. Of course it is included together with spring-cloud-starter-consul-all also. Configuration is stored in the /config folder by default. We can create the configuration per all applications or just for a single application in a dedicated folder. Assuming we have four microservices and API gateway deployed in two zones we would have to define ten configuration folders. We have different options for storing application properties, but I chose YAML format. YAML must be set in the appropriate data key in consul. So the Consul folders structure for all our sample applications looks as shown below.

config/account-service,zone1/data
config/account-service,zone2/data
config/customer-service,zone1/data
config/customer-service,zone2/data
config/order-service,zone1/data
config/order-service,zone2/data
config/product-service,zone1/data
config/product-service,zone2/data
config/gateway-service,zone1/data
config/gateway-service,zone2/data

Here’s the typical configuration for one our sample microservice running in zone1 zone.

spring:  
  cloud:
    consul:
      discovery:
        instanceId: "${spring.cloud.client.hostname}:${spring.application.name}:${random.int[1,999999]}"
        instanceZone: zone1      
server.port: 0

And the same configuration created on Consul for account-service with active zone1 profile.

In case we use Consul Config for our application the only file that should be available on classpath is bootstrap.yml. Except overriding Consul IP address or port if required we have to set the format of configuration properties to YAML. Here’s bootstrap.yml file for account-service.

spring:  
  application:
    name: account-service
  cloud:
    consul:
      host: 192.168.99.100
      port: 8500
        config:
          format: YAML

Summary

In this article I show you how to run microservices using Spring Consul Discovery and Config in the local environment similar to production. The applications use an affinity mechanism for inter-service communication and integrate with a cluster of Consul nodes. The configuration may be stored on the classpath or externalized to be stored on Consul with division into different active profiles. I think that Consul is a future of Spring Cloud microservices in the post-Netflix era, so it is definitely worth to know Spring Cloud Consul cluster support better. Before starting with Spring Cloud Consul cluster it worth knowing the basics: Quick Guide to Microservices with Spring Boot 2.0 and Spring Cloud.

The post Microservices with Spring Boot, Spring Cloud Gateway and Consul Cluster appeared first on Piotr's TechBlog.

Release Train

The Release Train version used in this article is Hoxton.M2. To access it we need to use the Spring Milestone repository.

<repositories>
   <repository>
      <id>spring-milestones</id>
      <name>Spring Milestones</name>
      <url>https://repo.spring.io/milestone</url>
   </repository>
</repositories>
<pluginRepositories>
   <pluginRepository>
      <id>spring-milestones</id>
      <name>Spring Milestones</name>
      <url>https://repo.spring.io/milestone</url>
   </pluginRepository>
</pluginRepositories>

Here’s declaration of release train inside dependency management section:

<parent>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-parent</artifactId>
   <version>2.2.0.RC1</version>
   <relativePath/>
</parent>
<properties>
   <java.version>1.8</java.version>
   <spring-cloud.version>Hoxton.M2</spring-cloud.version>
</properties>
<dependencyManagement>
   <dependencies>
      <dependency>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-dependencies</artifactId>
         <version>${spring-cloud.version}</version>
         <type>pom</type>
         <scope>import</scope>
      </dependency>
   </dependencies>
</dependencyManagement>

Dependencies

Besides standard Spring Boot Web Starter we need to include starters for Consul discovery and config client, and of course spring-cloud-loadbalancer library. We should remember about excluding Ribbon and Hystrix artifacts from spring-cloud-starter-consul-discovery starter.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-consul-discovery</artifactId>
   <exclusions>
      <exclusion>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-netflix-hystrix</artifactId>
      </exclusion>
      <exclusion>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-starter-netflix-ribbon</artifactId>
      </exclusion>
   </exclusions>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-consul-config</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-loadbalancer</artifactId>
</dependency>

Sample Applications

We have three sample applications. First callme-service exposes some REST endpoints, second caller-service communicates with the first using Consul discovery, RestTemplate and Spring Cloud Loadbalancer, and third gateway-service acts as API gateway in our architecture.

The repository with source code snippets is available on GitHub: https://github.com/piomin/sample-spring-cloud-microservices-future.git. For some configuration details related to Consul discovery please refer to my previous article. In fact we don’t have to provide any additional code to make it working. We just have to declare RestTemplate bean annotated with @LoadBalanced:

@SpringBootApplication
public class CallerApplication {

   public static void main(String[] args) {
      SpringApplication.run(CallerApplication.class, args);
   }

   @LoadBalanced
   @Bean
   RestTemplate template() {
      return new RestTemplate();
   }
}

Then we may call the target service using its name taken from the discovery server:

@RestController
@RequestMapping("/caller")
public class CallerController {

   private static final Logger LOGGER = LoggerFactory.getLogger(CallerController.class);

   @Autowired
   Environment environment;
   @Autowired
   RestTemplate template;

   @GetMapping
   public String call() {
      String url = "http://callme-service/callme";
      String callmeResponse = template.getForObject(url, String.class);
      LOGGER.info("Response: {}", callmeResponse);
      return "I'm Caller running on port " + environment.getProperty("local.server.port")
            + " calling-> " + callmeResponse;
   }
   
}

Summary

In this article, I show you a working example of communication between microservices using the new Spring Cloud Load Balancer. I also tried to use that load balancer instead of Spring Cloud Netflix Ribbon inside my application with Spring Cloud Gateway – with no success. According to the newest documentation of Spring Cloud Circuit Breaker built on top of Resilience4J I tried to include circuit breaker in the communication between caller-service and callme-service – also with no success. These new Spring Cloud features are still under active development. I hope to use them successfully in my applications soon. For more advanced Spring Cloud load balancing usage please refer to the article A Deep Dive Into Spring Cloud Load Balancer.

The post Using Spring Cloud Load Balancer In Microservices Communication appeared first on Piotr's TechBlog.

Service Discovery

Eureka is the only one important Spring Cloud Netflix module that has not been moved to maintenance mode. However, I would not say that it is actively developed. The last commit in the repository maintained by Netflix is from 11th January. Some time ago they have started working on Eureka 2, but it seems these works have been abandoned or they just have postponed open sourcing the newest version code to the future. Here https://github.com/Netflix/eureka/tree/2.x you can find an interesting comment about it: “The 2.x branch is currently frozen as we have had some internal changes w.r.t. to eureka2, and do not have any time lines for open sourcing of the new changes.”. So, we have two possibilities. Maybe, Netflix will decide to open source those internal changes as a version 2 of Eureka server. It is worth remembering that Eureka is a battle proven solution used at Scale by Netflix directly, and probably by many other organizations through Spring Cloud.
The second option is to choose another discovery server. Currently, Spring Cloud supports discovery based on various tools: ZooKeeper, Consul, Alibaba Nacos, Kubernetes. In fact, Kubernetes is based on etcd. Support for etcd is also being developed by Spring Cloud, but it is still in the incubation stage, and it is not known if it will ever be promoted to the official release train. In my opinion, there is one leader amongst these solutions – HashiCorp’s Consul.
Consul is now described as a service mesh solution providing a full featured control plane with service discovery, configuration, and segmentation functionality. It can be used as a discovery server or a key/value store in your microservices-based architecture. The integration with Consul is implemented by the Spring Cloud Consul project. To enable Consul client for your application you just need to include the following dependency to your Maven pom.xml:

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-consul-discovery</artifactId>
</dependency>

By default, Spring tries to connect with Consul on the address localhost:8500. If you need to override this address you should set the appropriate properties inside application.yml:

spring:  
  cloud:
    consul:
      host: 192.168.99.100
      port: 8500

You can easily test this solution with local instance of Consul started as the Docker container:

$ docker run -d --name consul -p 8500:8500 consul

As you see Consul discovery implementation with Spring Cloud is very easy – the same as for Eureka. Consul has one undoubted advantage over Eureka – it is continuously maintained and developed by HashiCorp. Its popularity is growing fast. It is a part of the biggest HashiCorp ecosystem, which includes Vault, Nomad and Terraform. In contrast to Eureka, Consul can be used not only for service discovery, but also as a configuration server in your microservices-based architecture.

Distributed Configuration

Netflix Archaius is an interesting solution for managing externalized configuration in microservices architecture. Although it offers some interesting features like dynamic and typed properties or support for dynamic data sources such as URLs, JDBC or AWS DynamoDB, Spring Cloud has also decided to move it to the maintenance mode. However, the popularity of Spring Cloud Archaius was limited, due to the existence of a similar project fully created by the Pivotal team and community – Spring Cloud Config. Spring Cloud Config supports multiple source repositories including Git, JDBC, Vault or simple files. You can find many examples of using this project for providing distributed configuration for your microservices in my previous posts. Today, I’m not going to talk about it. We will discuss an alternative solution – also supported by Spring Cloud.
As I have mentioned in the end of the previous section Consul can also be used as a configuration server. If you use Eureka as a discovery server, using Spring Cloud Config as a configuration server is a natural choice, because Eureka simply does not provide such features. This is not the case if you decide to use Consul. Now it makes sense to choose between two solutions: Spring Cloud Consul Config and Spring Cloud Config. Of course, both of them have their advantages and disadvantages. For example, you can easily build a cluster with Consul nodes, while with Spring Cloud Config you must rely on external discovery.
Now, let’s see how to use Spring Cloud Consul for managing external configuration in your application. To enable it on the application side you just need to include the following dependency to your Maven pom.xml:

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-consul-config</artifactId>
</dependency>

The same as for service discovery, If you would like to override some default client settings you need to set properties spring.cloud.consul.*. However, such a configuration must be provided inside bootstrap.yml.

spring:  
  application:
    name: callme-service
  cloud:
    consul:
      host: 192.168.99.100
      port: 8500

The name of the property source created on Consul should be the same as the application name provided in bootstrap.yml inside the config folder. You should create key server.port with value 0, to force Spring Boot to generate listening port number randomly. Supposing you need to set the application default listening port you should the following configuration.

When enabling dynamic port number generation you also need to override application instance id to be unique across a single machine. This feature is required if you are running multiple instances of a single service in the same machine. We will do it for callme-service, so we need to override the property spring.cloud.consul.discovery.instance-id with our value as shown below.

Then, you should see the following log on your application startup.

API Gateway

The successor of Spring Cloud Netflix Zuul is Spring Cloud Gateway. This project started around two years ago, and now is the second most popular Spring Cloud project with 1.4k stars on GitHub. It provides an API Gateway built on top of the Spring Ecosystem, including: Spring 5, Spring Boot 2 and Project Reactor. It is running on Netty, and does not work with traditional servlet containers like Tomcat or Jetty. It allows us to define routes, predicates and filters.
API gateway, the same as every Spring Cloud microservice may be easily integrated with service discovery based on Consul. We just need to include the appropriate dependencies inside pom.xml. We will use the latest development version of Spring Cloud libraries – 2.2.0.BUILD-SNAPSHOT. Here’s the list of required dependencies:

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-consul-discovery</artifactId>
   <version>2.2.0.BUILD-SNAPSHOT</version>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-consul-config</artifactId>
   <version>2.2.0.BUILD-SNAPSHOT</version>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-gateway</artifactId>
   <version>2.2.0.BUILD-SNAPSHOT</version>
</dependency>

The gateway configuration will also be served by Consul. Because we have pretty more configuration settings than for sample microservices, we will store it as YAML file. To achieve that we should create a YAML file available under path /config/gateway-service/data on Consul Key/Value. The configuration visible below enables service discovery integration and defines routes to the downstream services. Each route contains the name of the target service under which it is registered in service discovery, matching path and rewrite path used for call endpoint exposed by the downstream service. The following configuration is load on startup by our API gateway:

spring:
  cloud:
    gateway:
      discovery:
        locator:
          enabled: true
      routes:
        - id: caller-service
          uri: lb://caller-service
          predicates:
            - Path=/caller/**
          filters:
            - RewritePath=/caller/(?<path>.*), /$\{path}
        - id: callme-service
          uri: lb://callme-service
          predicates:
            - Path=/callme/**
          filters:
            - RewritePath=/callme/(?<path>.*), /$\{path}

Here’s the same configuration visible on Consul.

The last step is to force gateway-service to read configuration stored as YAML. To do that we need to set property spring.cloud.consul.config.format to YAML. Here’s the full configuration provided inside bootstrap.yml.

spring:
  application:
    name: gateway-service
  cloud:
    consul:
      host: 192.168.99.100
      config:
        format: YAML

Client-side Load Balancer

In version 2.2.0.BUILD-SNAPSHOT of Spring Cloud Commons Ribbon is still the main auto-configured load balancer for HTTP clients. Although the Spring Cloud team has announced that Spring Cloud Load Balancer will be the successor of Ribbon, we currently won’t find much information about that project in documentation and on the web. We may expect that the same as for Netflix Ribbon, any configuration will be transparent for us, especially if we use a discovery client. Currently, spring-cloud-loadbalancer module is a part of Spring Cloud Commons project. You may include it directly to your application by declaring the following dependency in pom.xml:

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-loadbalancer</artifactId>
   <version>2.2.0.BUILD-SNAPSHOT</version>
</dependency>

For the test purposes it is worth to exclude some Netflix modules included together with <code>spring-cloud-starter-consul-discovery</code> starter. Now, we are sure that Ribbon is not used in background as load balancer. Here’s the list of exclusions I set for my sample application:

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-consul-discovery</artifactId>
   <version>2.2.0.BUILD-SNAPSHOT</version>
   <exclusions>
      <exclusion>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-netflix-core</artifactId>
      </exclusion>
      <exclusion>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-starter-netflix-archaius</artifactId>
      </exclusion>
      <exclusion>
         <groupId>com.netflix.ribbon</groupId>
         <artifactId>ribbon</artifactId>
      </exclusion>
      <exclusion>
         <groupId>com.netflix.ribbon</groupId>
         <artifactId>ribbon-core</artifactId>
      </exclusion>
      <exclusion>
         <groupId>com.netflix.ribbon</groupId>
         <artifactId>ribbon-httpclient</artifactId>
      </exclusion>
      <exclusion>
         <groupId>com.netflix.ribbon</groupId>
         <artifactId>ribbon-loadbalancer</artifactId>
      </exclusion>
   </exclusions>
</dependency>

Treat my example just as a playground. Certainly the targeted approach is going to be much easier. First, we should annotate our main or configuration class with @LoadBalancerClient. As always, the name of the client should be the same as the name of the target service registered in the registry. The annotation should also contain the class with client configuration.

@SpringBootApplication
@LoadBalancerClients({
   @LoadBalancerClient(name = "callme-service", configuration = ClientConfiguration.class)
})
public class CallerApplication {

   public static void main(String[] args) {
      SpringApplication.run(CallerApplication.class, args);
   }

   @Bean
   RestTemplate template() {
      return new RestTemplate();
   }

}

Here’s our load balancer configuration class. It contains the declaration of a single @Bean. I have chosen RoundRobinLoadBalancer type.

public class ClientConfiguration {

   @Bean
   public RoundRobinLoadBalancer roundRobinContextLoadBalancer(LoadBalancerClientFactory clientFactory, Environment env) {
      String serviceId = clientFactory.getName(env);
      return new RoundRobinLoadBalancer(serviceId, clientFactory
            .getLazyProvider(serviceId, ServiceInstanceSupplier.class), -1);
   }

}

Finally, here’s the implementation of caller-service controller. It uses LoadBalancerClientFactory directly to find a list of available instances of callme-service. Then it selects a single instance, get its host and port, and sets in as a target URL.

@RestController
@RequestMapping("/caller")
public class CallerController {

   @Autowired
   Environment environment;
   @Autowired
   RestTemplate template;
   @Autowired
   LoadBalancerClientFactory clientFactory;

   @GetMapping
   public String call() {
      RoundRobinLoadBalancer lb = clientFactory.getInstance("callme-service", RoundRobinLoadBalancer.class);
      ServiceInstance instance = lb.choose().block().getServer();
      String url = "http://" + instance.getHost() + ":" + instance.getPort() + "/callme";
      String callmeResponse = template.getForObject(url, String.class);
      return "I'm Caller running on port " + environment.getProperty("local.server.port")
            + " calling-> " + callmeResponse;
   }

}

Summary

The following picture illustrates the architecture of our sample system. We have two instances of callme-service, a single instance of caller-service, which uses Spring Cloud Balancer to find the list of available instances of callme-service. The ports are generated dynamically. The API gateway is hiding the complexity of our system from an external client. It is available on port 8080, and is forwarding requests to the downstream basing on request context path.

After starting, all the microservices you should be registered on your Consul node.

Now, you can try to endpoint exposed by caller-service through gateway: http://localhost:8080/caller. You should something like that:

The sample application source code is available on GitHub in repository https://github.com/piomin/sample-spring-cloud-microservices-future.git.

The post The Future of Spring Cloud Microservices After Netflix Era appeared first on Piotr's TechBlog.

Our sample system consists of three independent microservices that communicate with each other. All of them integrate with Consul in order to fetch shared configuration. After startup, every single service will register itself in Consul. Applications organization-service and department-service call endpoints exposed by other microservices using Micronaut declarative HTTP client. The traces from communication are sent to Zipkin. The source code of sample applications is available on GitHub in repository sample-micronaut-microservices.

Step 1. Creating Micronaut application

We need to start by including some dependencies to our Maven pom.xml. First, let’s define BOM with the newest stable Micronaut version.

<properties>
   <exec.mainClass>pl.piomin.services.employee.EmployeeApplication</exec.mainClass>
   <micronaut.version>1.0.3</micronaut.version>
   <jdk.version>1.8</jdk.version>
</properties>
<dependencyManagement>
   <dependencies>
      <dependency>
         <groupId>io.micronaut</groupId>
         <artifactId>micronaut-bom</artifactId>
         <version>${micronaut.version}</version>
         <type>pom</type>
         <scope>import</scope>
      </dependency>
   </dependencies>
</dependencyManagement>

The list of required dependencies isn’t very long. Also not all of them are required, but they will be useful in our demo. For example micronaut-management need to be included in case we would like to expose some built-in management and monitoring endpoints.

<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-http-server-netty</artifactId>
</dependency>
<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-inject</artifactId>
</dependency>
<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-runtime</artifactId>
</dependency>
<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-management</artifactId>
</dependency>
<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-inject-java</artifactId>
   <scope>provided</scope>
</dependency>

To build an application uber-jar we need a configure plugin responsible for packaging a JAR file with dependencies. It can be for example maven-shade-plugin. When building a new application it is also worth to expose basic information about it under /info endpoint. As I have already mentioned Micronaut adds support for monitoring your app via HTTP endpoints after including artifact micronaut-management. Management endpoints are integrated with the Micronaut security module, which means that you need to authenticate yourself to be able to access them. To simplify we can disable authentication for /info endpoint.

endpoints:
  info:
    enabled: true
    sensitive: false

We can customize /info endpoint by adding some supported info sources. This mechanism is very similar to the Spring Boot Actuator approach. If git.properties file is available on the classpath, all the values inside file will be exposed by /info endpoint. The same situation applies to build-info.properties file, that needs to be placed inside META-INF directory. However, in comparison with Spring Boot we need to provide more configuration in pom.xml to generate and package those to application JAR. The following Maven plugins are responsible for generating required properties files.

<plugin>
   <groupId>pl.project13.maven</groupId>
   <artifactId>git-commit-id-plugin</artifactId>
   <version>2.2.6</version>
   <executions>
      <execution>
         <id>get-the-git-infos</id>
         <goals>
            <goal>revision</goal>
         </goals>
      </execution>
   </executions>
   <configuration>
      <verbose>true</verbose>
      <dotGitDirectory>${project.basedir}/.git</dotGitDirectory>
      <dateFormat>MM-dd-yyyy '@' HH:mm:ss Z</dateFormat>
      <generateGitPropertiesFile>true</generateGitPropertiesFile>
      <generateGitPropertiesFilename>src/main/resources/git.properties</generateGitPropertiesFilename>
      <failOnNoGitDirectory>true</failOnNoGitDirectory>
   </configuration>
</plugin>
<plugin>
   <groupId>com.rodiontsev.maven.plugins</groupId>
   <artifactId>build-info-maven-plugin</artifactId>
   <version>1.2</version>
   <configuration>
      <filename>classes/META-INF/build-info.properties</filename>
      <projectProperties>
         <projectProperty>project.groupId</projectProperty>
         <projectProperty>project.artifactId</projectProperty>
         <projectProperty>project.version</projectProperty>
      </projectProperties>
   </configuration>
   <executions>
      <execution>
         <phase>prepare-package</phase>
         <goals>
            <goal>extract</goal>
         </goals>
      </execution>
   </executions>
</plugin>
</plugins>

Now, our /info endpoint is able to print the most important information about our app including Maven artifact name, version, and last Git commit id.

Step 2. Exposing HTTP endpoints

Micronaut provides their own annotations for pointing out HTTP endpoints and methods. As I have mentioned in the preface it also uses JSR-330 (java.inject) for dependency injection. Our controller class should be annotated with @Controller. We also have annotations for every HTTP method type. The path parameter is automatically mapped to the class method parameter by its name, what is a nice simplification in comparison to Spring MVC where we need to use @PathVariable annotation. The repository bean used for CRUD operations is injected into the controller using @Inject annotation.

@Controller("/employees")
public class EmployeeController {

    private static final Logger LOGGER = LoggerFactory.getLogger(EmployeeController.class);

    @Inject
    EmployeeRepository repository;

    @Post
    public Employee add(@Body Employee employee) {
        LOGGER.info("Employee add: {}", employee);
        return repository.add(employee);
    }

    @Get("/{id}")
    public Employee findById(Long id) {
        LOGGER.info("Employee find: id={}", id);
        return repository.findById(id);
    }

    @Get
    public List<Employee> findAll() {
        LOGGER.info("Employees find");
        return repository.findAll();
    }

    @Get("/department/{departmentId}")
    @ContinueSpan
    public List<Employee> findByDepartment(@SpanTag("departmentId") Long departmentId) {
        LOGGER.info("Employees find: departmentId={}", departmentId);
        return repository.findByDepartment(departmentId);
    }

    @Get("/organization/{organizationId}")
    @ContinueSpan
    public List<Employee> findByOrganization(@SpanTag("organizationId") Long organizationId) {
        LOGGER.info("Employees find: organizationId={}", organizationId);
        return repository.findByOrganization(organizationId);
    }

}

Our repository bean is pretty simple. It just provides an in-memory store for Employee instances. We will mark it with @Singleton annotation.

@Singleton
public class EmployeeRepository {

   private List<Employee> employees = new ArrayList<>();
   
   public Employee add(Employee employee) {
      employee.setId((long) (employees.size()+1));
      employees.add(employee);
      return employee;
   }
   
   public Employee findById(Long id) {
      Optional<Employee> employee = employees.stream().filter(a -> a.getId().equals(id)).findFirst();
      if (employee.isPresent())
         return employee.get();
      else
         return null;
   }
   
   public List<Employee> findAll() {
      return employees;
   }
   
   public List<Employee> findByDepartment(Long departmentId) {
      return employees.stream().filter(a -> a.getDepartmentId().equals(departmentId)).collect(Collectors.toList());
   }
   
   public List<Employee> findByOrganization(Long organizationId) {
      return employees.stream().filter(a -> a.getOrganizationId().equals(organizationId)).collect(Collectors.toList());
   }
   
}

Micronaut is able to automatically generate Swagger YAML definition from our controller and methods based on annotations. To achieve this, we first need to include the following dependency to our pom.xml.

<dependency>
   <groupId>io.swagger.core.v3</groupId>
   <artifactId>swagger-annotations</artifactId>
</dependency>

Then we should annotate the application main class with @OpenAPIDefinition and provide some basic information like title or version number. Here’s the employee application main class.

@OpenAPIDefinition(
    info = @Info(
        title = "Employees Management",
        version = "1.0",
        description = "Employee API",
        contact = @Contact(url = "https://piotrminkowski.wordpress.com", name = "Piotr Mińkowski", email = "piotr.minkowski@gmail.com")
    )
)
public class EmployeeApplication {

    public static void main(String[] args) {
        Micronaut.run(EmployeeApplication.class);
    }

}

Micronaut generates Swagger file basing on title and version fields inside @Info annotation. In that case our YAML definition file is available under name employees-management-1.0.yml, and will be generated to the META-INF/swagger directory. We can expose it outside the application using HTTP endpoint. Here’s the appropriate configuration provided inside application.yml file.

micronaut:
  router:
    static-resources:
      swagger:
        paths: classpath:META-INF/swagger
        mapping: /swagger/**

Now, our file is available under path http://localhost:8080/swagger/employees-management-1.0.yml if run it on default 8080 port (we won’t do that, what I’m going to describe in the next part of this article). In comparison to Spring Boot, we don’t have such a project like Swagger SpringFox for Micronaut, so we need to copy the content to an online editor in order to see the graphical representation of Swagger YAML. Here’s it.

Ok, since we have finished implementation of single microservice we may proceed to cloud-native features provided by Micronaut.

Step 3. Distributed configuration with Consul

Micronaut comes with built in APIs for doing distributed configuration. In fact, the only one available solution for now is microservices distributed configuration based on Micronaut integration with HashiCorp’s Consul. Micronaut features for externalizing and adapting configuration to the environment are very similar to the Spring Boot approach. We also have application.yml and bootstrap.yml files, which can be used for application environment configuration. When using distributed configuration we first need to provide a bootstrap.yml file on the classpath. It should contain an address of remote configuration server and preferred configuration store format. Of course, we first need to enable distributed configuration clients by setting property micronaut.config-client.enabled to true. Here’s bootstrap.yml file for department-service.

micronaut:
  application:
    name: department-service
  config-client:
    enabled: true
consul:
  client:
    defaultZone: "192.168.99.100:8500"
    config:
      format: YAML

We can choose between properties, JSON, YAML and FILES (git2consul) configuration formats. I decided to use YAML. To apply this configuration on Consul we first need to start it locally in development mode. Because I’m using Docker Toolbox the default address of Consul is 192.168.99.100. The following Docker command will start a single-node Consul instance and expose it on port 8500.

$ docker run -d --name consul -p 8500:8500 consul

Now, you can navigate to the tab Key/Value in the Consul web console and create a new file in YAML format /config/application.yml as shown below. Besides configuration for Swagger and /info management endpoint it also enables dynamic HTTP generation on startup by setting property micronaut.server.port to -1. Because the name of the file is application.yml it is by default shared between all Micronaut microservices that use the Consul config client.

Step 4. Service discovery with Consul

Micronaut gives you more options when configuring service discovery, than for distributed configuration. You can use Eureka, Consul, Kubernetes or just manually configure a list of available services. However, I have observed that using the Eureka discovery client together with the Consul config client causes some errors on startup. In this example we will use Consul discovery for our Micronaut microservices. Because Consul address has been already provided in bootstrap.yml for all Micronaut microservices, we just need to enable service discovery by adding the following lines to application.yml stored in Consul KV.

consul:
  client:
    registration:
      enabled: true

We should also include the following dependency to Maven pom.xml of every single application.

<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-discovery-client</artifactId>
</dependency>

Finally, you can just run every microservice (you may run more than one instance locally, since HTTP port is generated dynamically). Here’s my list of running Micronaut microservices registered in Consul.

I have run two instances of employee-service as shown below.

Step 5. Inter-service communication

Micronaut uses a built-in HTTP client for load balancing between multiple instances of a single microservice. By default it leverages the Round Robin algorithm. We may choose between low-level HTTP client and declarative HTTP client with @Client. Micronaut declarative HTTP client concept is very similar to Spring Cloud OpenFeign. To use a built-in client we first need to include the following dependency to project pom.xml.

<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-http-client</artifactId>
</dependency>

Declarative client automatically integrates with a discovery client. It tries to find the service registered in Consul under the same name as the value provided inside id field.

@Client(id = "employee-service", path = "/employees")
public interface EmployeeClient {

   @Get("/department/{departmentId}")
   List<Employee> findByDepartment(Long departmentId);
   
}

Now, the client bean needs to be injected into the controller.

@Controller("/departments")
public class DepartmentController {

   private static final Logger LOGGER = LoggerFactory.getLogger(DepartmentController.class);
   
   @Inject
   DepartmentRepository repository;
   @Inject
   EmployeeClient employeeClient;
   
   @Post
   public Department add(@Body Department department) {
      LOGGER.info("Department add: {}", department);
      return repository.add(department);
   }
   
   @Get("/{id}")
   public Department findById(Long id) {
      LOGGER.info("Department find: id={}", id);
      return repository.findById(id);
   }
   
   @Get
   public List<Department> findAll() {
      LOGGER.info("Department find");
      return repository.findAll();
   }
   
   @Get("/organization/{organizationId}")
   @ContinueSpan
   public List<Department> findByOrganization(@SpanTag("organizationId") Long organizationId) {
      LOGGER.info("Department find: organizationId={}", organizationId);
      return repository.findByOrganization(organizationId);
   }
   
   @Get("/organization/{organizationId}/with-employees")
   @ContinueSpan
   public List<Department> findByOrganizationWithEmployees(@SpanTag("organizationId") Long organizationId) {
      LOGGER.info("Department find: organizationId={}", organizationId);
      List<Department> departments = repository.findByOrganization(organizationId);
      departments.forEach(d -> d.setEmployees(employeeClient.findByDepartment(d.getId())));
      return departments;
   }
   
}

Step 6. Distributed tracing

Micronaut applications can be easily integrated with Zipkin to send traces with HTTP traffic there automatically. To enable this feature we first need to include the following dependencies to pom.xml.

<dependency>
   <groupId>io.micronaut</groupId>
   <artifactId>micronaut-tracing</artifactId>
</dependency>
<dependency>
   <groupId>io.zipkin.brave</groupId>
   <artifactId>brave-instrumentation-http</artifactId>
   <scope>runtime</scope>
</dependency>
<dependency>
   <groupId>io.zipkin.reporter2</groupId>
   <artifactId>zipkin-reporter</artifactId>
   <scope>runtime</scope>
</dependency>
<dependency>
   <groupId>io.opentracing.brave</groupId>
   <artifactId>brave-opentracing</artifactId>
</dependency>

Then, we have to provide some configuration settings inside application.yml including Zipkin URL and sampler options. By setting property tracing.zipkin.sampler.probability to 1 we are forcing micronaut to send traces for every single request. Here’s our final configuration.

During the tests of my application I have observed that using distributed configuration together with Zipkin tracing results in the problems in communication between microservice and Zipkin. The traces just do not appear in Zipkin. So, if you would like to test this feature now you must provide application.yml on the classpath and disable Consul distributed configuration for all your applications.

We can add some tags to the spans by using @ContinueSpan or @NewSpan annotations on methods.

After making some test calls of GET methods exposed by organization-service and department-service we may take a look on Zipkin web console, available under address http://192.168.99.100:9411. The following picture shows the list of all the traces sent to Zipkin by our microservices in 1 hour.

We can check out the details of every trace by clicking on the element from the list. The following picture illustrates the timeline for HTTP method exposed by organization-service GET /organizations/{id}/with-departments-and-employees. This method finds the organization in the in-memory repository, and then calls HTTP method exposed by department-service GET /departments/organization/{organizationId}/with-employees. This method is responsible for finding all departments assigned to the given organization. It also needs to return employees within the department, so it calls method GET /employees/department/{departmentId} from employee-service.

We can also take a look at the details of every single call from the timeline.

Conclusion

In comparison to Spring Boot Micronaut is still in the early stage of development. For example, I was not able to implement any application that could act as an API gateway to our system, which can easily be achieved with Spring using Spring Cloud Gateway or Spring Cloud Netflix Zuul. There are still some bugs that need to be fixed. But above all that, Micronaut is now probably the most interesting micro-framework on the market. It implements most popular microservice patterns, provides integration with several third-party solutions like Consul, Eureka, Zipkin or Swagger, consumes less memory and starts faster than similar Spring Boot apps. I will definitely follow the progress in Micronaut development closely.

The post Guide to Microservices with Micronaut and Consul appeared first on Piotr's TechBlog.

• Manually with rabbitmqctl (as described in my article RabbitMQ in cluster)
• Declaratively by listing cluster nodes in config file
• Using DNS-based discovery
• Using AWS (EC2) instance discovery via a dedicated plugin
• Using Kubernetes discovery via a dedicated plugin
• Using Consul discovery via a dedicated plugin
• Using etcd-based discovery via a dedicated plugin

Today, I’m going to show you how to create RabbitMQ cluster using service discovery based on HashiCorp’s Consul. Additionally, we will include Vault to our architecture in order to use its interesting feature called secrets engine for managing credentials used for accessing RabbitMQ. We will set up this sample on the local machine using Docker images of RabbitMQ, Consul and Vault. Finally, we will test our solution using a simple Spring Boot application that sends and listens for incoming messages to the cluster. That application is available on GitHub repository sample-haclustered-rabbitmq-service in the branch consul.

Architecture

We use Vault as a credentials manager when applications try to authenticate against RabbitMQ nodes or when a user tries to login to RabbitMQ web admin console. Each RabbitMQ node registers itself after startup in Consul and retrieves a list of nodes running inside a cluster. Vault is integrated with RabbitMQ using a dedicated secrets engine. Here’s an architecture of our sample solution.

1. Configure RabbitMQ Consul plugin

The integration between RabbitMQ and Consul is realized via plugin rabbitmq-peer-discovery-consul. This plugin is not enabled by default on the official RabbitMQ Docker container. So, the first step is to build our own Docker image based on the official RabbitMQ image that installs and enables the required plugin. By default, RabbitMQ main configuration file is available under path /etc/rabbitmq/rabbitmq.conf inside a Docker container. To override it we just use the COPY statement as shown below. The following Dockerfile definition takes RabbitMQ with a management web console as base image and enables rabbitmq_peer_discovery_consul plugin.

FROM rabbitmq:3.7.8-management
COPY rabbitmq.conf /etc/rabbitmq
RUN rabbitmq-plugins enable --offline rabbitmq_peer_discovery_consul

Now, let’s take a closer look on our plugin configuration settings. Because I run Docker on Windows Consul is not available under default localhost address, but on 192.168.99.100. So, first we need to set that IP address using property cluster_formation.consul.host. We also need to set Consul as a default peer discovery implementation by setting the name of plugin for property cluster_formation.peer_discovery_backend. Finally, we have to set two additional properties to make it work in our local Docker environment. It is related to the address of RabbitMQ node sent to Consul during the registration process. It is important to compute it properly, and not to send for example localhost. After setting property cluster_formation.consul.svc_addr_use_nodename to false node will register itself using host name instead of node name. We can set the name of the host for the container inside its running command. Here’s my full RabbitMQ configuration file used in the demo for this article.

loopback_users.guest = false
listeners.tcp.default = 5672
hipe_compile = false
management.listener.port = 15672
management.listener.ssl = false
cluster_formation.peer_discovery_backend = rabbit_peer_discovery_consul
cluster_formation.consul.host = 192.168.99.100
cluster_formation.consul.svc_addr_auto = true
cluster_formation.consul.svc_addr_use_nodename = false

After saving the configuration visible above in the file rabbitmq.conf we can proceed to building our custom Docker image with RabbitMQ. This image is available in my Docker repository under alias piomin/rabbitmq, but you can also build it by yourself from Dockerfile by executing the following command.

$ docker build -t piomin/rabbitmq:1.0 .
Sending build context to Docker daemon  3.072kB
Step 1 : FROM rabbitmq:3.7.8-management
 ---> d69a5113ceae
Step 2 : COPY rabbitmq.conf /etc/rabbitmq
 ---> aa306ef88085
Removing intermediate container fda0e21178f9
Step 3 : RUN rabbitmq-plugins enable --offline rabbitmq_peer_discovery_consul
 ---> Running in 0892a42bffef
The following plugins have been configured:
  rabbitmq_management
  rabbitmq_management_agent
  rabbitmq_peer_discovery_common
  rabbitmq_peer_discovery_consul
  rabbitmq_web_dispatch
Applying plugin configuration to rabbit@fda0e21178f9...
The following plugins have been enabled:
  rabbitmq_peer_discovery_common
  rabbitmq_peer_discovery_consul

set 5 plugins.
Offline change; changes will take effect at broker restart.
 ---> cfe73f9d9904
Removing intermediate container 0892a42bffef
Successfully built cfe73f9d9904

2. Running RabbitMQ cluster on Docker

In the previous step we have succesfully created a Docker image of RabbitMQ configured to run in cluster mode using Consul discovery. Before running this image we need to start an instance of Consul. Here’s the command that starts the Docker container with Consul and exposes it on port 8500.

$ docker run -d --name consul -p 8500:8500 consul

We will also create a Docker network to enable communication between containers by hostname. It is required in this scenario, because each RabbitMQ container is registered using container hostname.

$ docker network create rabbitmq

Now, we can run our three clustered RabbitMQ containers. We will set a unique hostname for every single container (using -h option) and set the same Docker network everywhere. We also have to set the environment variable RABBITMQ_ERLANG_COOKIE.

$ docker run -d --name rabbit1 -h rabbit1 --network rabbitmq -p 30000:5672 -p 30010:15672 -e RABBITMQ_ERLANG_COOKIE='rabbitmq' piomin/rabbitmq:1.0
$ docker run -d --name rabbit2 -h rabbit2 --network rabbitmq -p 30001:5672 -p 30011:15672 -e RABBITMQ_ERLANG_COOKIE='rabbitmq' piomin/rabbitmq:1.0
$ docker run -d --name rabbit3 -h rabbit3 --network rabbitmq -p 30002:5672 -p 30012:15672 -e RABBITMQ_ERLANG_COOKIE='rabbitmq' piomin/rabbitmq:1.0

After running all three instances of RabbitMQ we can first take a look on the Consul web console. You should see there the new service called rabbitmq. This value is the default name of a cluster set by RabbitMQ Consul plugin. We can override inside rabbitmq.conf using cluster_formation.consul.svc property.

We can check out if the cluster has been succesfully started using RabbitMQ web management console. Every node is exposing it. I just had to override default port 15672 to avoid port conflicts between three running instances.

3. Integrating RabbitMQ with Vault

In the two previous steps we have succesfully run the cluster of three RabbitMQ nodes based on Consul discovery. Now, we will include Vault to our sample system to dynamically generate user credentials. Let’s begin from running Vault on Docker. You can find detailed information about it in my previous article Secure Spring Cloud Microservices with Vault and Nomad. We will run Vault in development mode using the following command.

$ docker run --cap-add=IPC_LOCK -d --name vault -p 8200:8200 vault

You can copy the root token from container logs using docker logs -f vault command. Then you have to login to the Vault web console available under address http://192.168.99.100:8200 using this token and enable RabbitMQ secret engine as shown below.

And confirm.

You can easily run Vault commands using a terminal provided by the web admin console or do the same thing using HTTP API. The first command visible below is used for writing connection details. We just need to pass RabbitMQ address and admin user credentials. The provided configuration settings points to #1 RabbitMQ node, but the changes are then replicated to the whole cluster.

$ vault write rabbitmq/config/connection connection_uri="http://192.168.99.100:30010" username="guest" password="guest"

The next step is to configure a role that maps a name in Vault to virtual host permissions.

$ vault write rabbitmq/roles/default vhosts='{"/":{"write": ".*", "read": ".*"}}'

We can test our newly created configuration by running command vault read rabbitmq/creds/default as shown below.

4. Sample application

Our sample application is pretty simple. It consists of two modules. First of them sender is responsible for sending messages to RabbitMQ, while second listener for receiving incoming messages. Both of them are Spring Boot applications that integrate with RabbitMQ and Vault using the following dependencies.

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-vault-config-rabbitmq</artifactId>
   <version>2.0.2.RELEASE</version>
</dependency>

We need to provide some configuration settings in bootstrap.yml file to integrate our application with Vault. First, we need to enable plugin for that integration by setting property spring.cloud.vault.rabbitmq.enabled to true. Of course, Vault address and root token are required. It is also important to set property spring.cloud.vault.rabbitmq.role with the name of Vault role configured in step 3. Spring Cloud Vault injects username and password generated by Vault to the application properties spring.rabbitmq.username and spring.rabbitmq.password, so the only thing we need to configure in bootstrap.yml file is the list of available cluster nodes.

spring:
  rabbitmq:
    addresses: 192.168.99.100:30000,192.168.99.100:30001,192.168.99.100:30002
  cloud:
    vault:
      uri: http://192.168.99.100:8200
      token: s.7DaENeiqLmsU5ZhEybBCRJhp
      rabbitmq:
        enabled: true
        role: default
        backend: rabbitmq

For the test purposes you should enable high-available queues on RabbitMQ. For instructions how to configure them using policies you can refer to my article RabbitMQ in cluster. The application works at the level of exchanges. Auto-configured connection factory is injected into the application and set for RabbitTemplate bean.

@SpringBootApplication
public class Sender {
   
   private static final Logger LOGGER = LoggerFactory.getLogger("Sender");
   
   @Autowired
   RabbitTemplate template;

   public static void main(String[] args) {
      SpringApplication.run(Sender.class, args);
   }

   @PostConstruct
   public void send() {
      for (int i = 0; i < 1000; i++) {
         int id = new Random().nextInt(100000);
         template.convertAndSend(new Order(id, "TEST"+id, OrderType.values()[(id%2)]));
      }
      LOGGER.info("Sending completed.");
   }
    
    @Bean
    public RabbitTemplate template(ConnectionFactory connectionFactory) {
        RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
        rabbitTemplate.setExchange("ex.example");
        return rabbitTemplate;
    }
    
}

Our listener app is connected only to the third node of the cluster (spring.rabbitmq.addresses=192.168.99.100:30002). However, the test queue is mirrored between all clustered nodes, so it is able to receive messages sent by sender app. You can easily test using my sample applications.

@SpringBootApplication
@EnableRabbit
public class Listener {

   private static final  Logger LOGGER = LoggerFactory.getLogger("Listener");

   private Long timestamp;

   public static void main(String[] args) {
      SpringApplication.run(Listener.class, args);
   }

   @RabbitListener(queues = "q.example")
   public void onMessage(Order order) {
      if (timestamp == null)
         timestamp = System.currentTimeMillis();
      LOGGER.info((System.currentTimeMillis() - timestamp) + " : " + order.toString());
   }

   @Bean
   public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory connectionFactory) {
      SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
      factory.setConnectionFactory(connectionFactory);
      factory.setConcurrentConsumers(10);
      factory.setMaxConcurrentConsumers(20);
      return factory;
   }
   
}

The post RabbitMQ Cluster with Consul and Vault appeared first on Piotr's TechBlog.

Today I’m going to show you how to integrate your Spring Boot application with HashiCorp’s Vault in order to store your sensitive data properly. The first good news is that you don’t have to create any keys or certificates for encryption and decryption, because Vault will do it in your place. In this article in a few areas I’ll refer to my previous article about HashiCorp’s solutions Deploying Spring Cloud Microservices on HashiCorp’s Nomad. Now, as then, I also deploy my sample applications on Nomad to take advantage of built-in integration between those two very interesting HashiCorp’s tools. We will also use another HashiCorp’s solution for service discovery in inter-service communication – Consul. It’s also worth mentioning that Spring Cloud provides a dedicated project for integration with Vault – Spring Cloud Vault.

Architecture

The sample presented in this article will consist of two applications deployed on HashiCorp’s Nomad callme-service and caller-service. Microservice caller-service is the endpoint exposed by callme-service. An inter-service communication is performed using the name of the target application registered in Consul server. Microservice callme-service will store the history of all interactions triggered by caller-service in the database. The credentials to the database are stored on Vault. Nomad is integrated with Vault and stores the root token, which is not visible by the applications. The architecture of the described solution is visible on the following picture.

The current sample is pretty similar to the sample presented in my article Deploying Spring Cloud Microservices on Hashicorp’s Nomad. It is also available in the same repository on GitHub sample-nomad-java-service, but in the different branch vault. The current sample add an integration with PostgreSQL and Vault server for managing credentials to database.

1. Running Vault

We will run Vault inside the Docker container in a development mode. Server in development mode does not require any further setup, it is ready to use just after startup. It provides in-memory encrypted storage and unsecure (HTTP) connection, which is not a problem for a demo purposes. We can override default server IP address and listening port by setting environment property VAULT_DEV_LISTEN_ADDRESS, but we won’t do that. After startup our instance of Vault is available on port 8200. We can use the admin web console, which is for me available under address http://192.168.99.100:8200. The current version of Vault is 1.0.0.

$ docker run --cap-add=IPC_LOCK -d --name vault -p 8200:8200 vault

It is possible to login using different methods, but the most suitable way for us is through a token. To do that we have to display container logs using command docker logs vault, and then copy Root Token as shown below.

Now you can login to the Vault web console.

2. Integration with Postgres database

In Vault we can create Secret Engine that connects to other services and generates dynamic credentials on demand. Secrets engines are available under path. There is a dedicated engine for the various databases, for example PostgreSQL. Before activating such an engine we should run an instance of Postgres database. This time we will also use a Docker container. It is possible to set login and password to the database using environment variables.

$ docker run -d --name postgres -p 5432:5432 -e POSTGRES_PASSWORD=postgres123456 -e POSTGRES_USER=postgres postgres

After starting the database, we may proceed to the engine configuration in the Vault web console. First, let’s create our first secret engine. We may choose between some different types of engine. The right choice for now is Databases.

You can apply a new configuration to Vault using vault command or by HTTP API. Vault web console provides a terminal for running CLI commands, but it could be problematic in some cases. For example, I have a problem with escaping strings in some SQL commands, and therefore I had to add it using HTTP API. No matter which method you use, the next steps are the same. Following Vault documentation we first need to configure a plugin for PostgreSQL database and then provide connection settings and credentials.

$ vault write database/config/postgres plugin_name=postgresql-database-plugin allowed_roles="default" connection_url="postgresql://{{username}}:{{password}}@192.168.99.100:5432?sslmode=disable" username="postgres" password="postgres123456"

Alternatively, you can perform the same action using the HTTP API method. To authenticate against Vault we need to add header X-Vault-Token with root token. I have disabled SSL for connection with Postgres by setting sslmode=disable. There is only one role allowed to use this plugin: default. Now, let’s configure that role.

$ curl --header "X-Vault-Token: s.44GiacPqbV78fNbmoWK4mdYq" --request POST --data '{"plugin_name": "postgresql-database-plugin","allowed_roles": "default","connection_url": "postgresql://{{username}}:{{password}}@localhost:5432?sslmode=disable","username": "postgres","password": "postgres123456"}' http://192.168.99.100:8200/v1/database/config/postgres

The role can be created either with CLI or with HTTP API. The name of the role should be the same as the name passed in field allowed_roles in the previous step. We also have to set a target database name and SQL statement that creates users with privileges.

$ vault write database/roles/default db_name=postgres creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';GRANT SELECT, UPDATE, INSERT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";GRANT USAGE,  SELECT ON ALL SEQUENCES IN SCHEMA public TO \"{{name}}\";" default_ttl="1h" max_ttl="24h"

Alternatively you can call the following HTTP API endpoint.

$ curl --header "X-Vault-Token: s.44GiacPqbV78fNbmoWK4mdYq" --request POST --data '{"db_name":"postgres", "creation_statements": ["CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';GRANT SELECT, UPDATE, INSERT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO \"{{name}}\";"]}' http://192.168.99.100:8200/v1/database/roles/default

And that’s all. Now, we can test our configuration using command with role’s name vault read database/creds/default as shown below. You can login to the database using returned credentials. By default, they are valid for one hour.

3. Enabling Spring Cloud Vault

We have succesfully configured a secret engine that is responsible for creating users on Postgres. Now, we can proceed to the development and integrate our application with Vault. Fortunately, there is a project Spring Cloud Vault, which provides out-of-the-box integration with Vault database secret engines. The only thing we have to do is to include Spring Cloud Vault to our project and provide some configuration settings. Let’s start from setting Spring Cloud Release Train. We use the newest stable version Finchley.SR2.

<dependencyManagement>
   <dependencies>
      <dependency>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-dependencies</artifactId>
         <version>Finchley.SR2</version>
         <type>pom</type>
         <scope>import</scope>
      </dependency>
   </dependencies>
</dependencyManagement>

We have to include two dependencies to our pom.xml. Starter spring-cloud-starter-vault-config is responsible for loading configuration from Vault and spring-cloud-vault-config-databases responsible for integration with secret engines for databases.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-vault-config</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-vault-config-databases</artifactId>
</dependency>

The sample application also connects to Postgres database, so we will include the following dependencies.

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
   <groupId>org.postgresql</groupId>
   <artifactId>postgresql</artifactId>
   <version>42.2.5</version>
</dependency>

The only thing we have to do is to configure integration with Vault via Spring Cloud Vault. The following configuration settings should be placed in bootstrap.yml (no application.yml). Because we run our application on Nomad server, we use the port number dynamically set by Nomad available under environment property NOMAD_HOST_PORT_http and secret token from Vault available under environment property VAULT_TOKEN.

server:
  port: ${NOMAD_HOST_PORT_http:8091}

spring:
  application:
    name: callme-service
  cloud:
    vault:
      uri: http://192.168.99.100:8200
      token: ${VAULT_TOKEN}
      postgresql:
        enabled: true
        role: default
        backend: database
  datasource:
    url: jdbc:postgresql://192.168.99.100:5432/postgres

The important part of the configuration visible above is under the property spring.cloud.vault.postgresql. Following Spring Cloud documentation “Username and password are stored in spring.datasource.username and spring.datasource.password so using Spring Boot will pick up the generated credentials for your DataSource without further configuration”. Spring Cloud Vault is connecting with Vault, and then using role default (previously created on Vault) to generate new credentials to database. Those credentials are injected into spring.datasource properties. Then, the application is connected to the database using injected credentials. Everything works fine. Now, let’s try to run our applications on Nomad.

4. Deploying apps on Nomad

Before starting Nomad node we should also run Consul using its Docker container. Here’s a Docker command that starts a single node Consul instance.

$ docker run -d --name consul -p 8500:8500 consul

After that we can configure connection settings to Consul and Vault in Nomad configuration. I have created the file nomad.conf. Nomad is authenticating itself against Vault using root token. Connection with Consul is not secured. Sometimes it is also required to set network interface name and total CPU on the machine for the Nomad client. Most clients are able to determine it automatically, but it does not work for me.

client {
  network_interface = "Połączenie lokalne 4"
  cpu_total_compute = 10400
}

consul {
  address = "192.168.99.100:8500"
}

vault {
  enabled = true
  address = "http://192.168.99.100:8200"
  token = "s.6jhQ1WdcYrxpZmpa0RNd0LMw"
}

Let’s run Nomad in development mode passing configuration file location.

$ nomad agent -dev -config=nomad.conf

If everything works fine you should see the similar log on startup.

Once we have succesfully started Nomad agent integrated with Consul and Vault, we can proceed to the application deployment. First build the whole project with mvn clean install command. The next step is to prepare Nomad’s job descriptor file. For more details about the Nomad deployment process and its descriptor file you can refer to my previous article about it (mentioned in the preface of this article). Descriptor file is available inside application GitHub under path callme-service/job.nomad for callme-service, and caller-service/job.nomad for caller-service.

job "callme-service" {
   datacenters = ["dc1"]
   type = "service"
   group "callme" {
      count = 2
      task "api" {
         driver = "java"
         config {
            jar_path    = "C:\\Users\\minkowp\\git\\sample-nomad-java-services-idea\\callme-service\\target\\callme-service-1.0.0-SNAPSHOT.jar"
            jvm_options = ["-Xmx256m", "-Xms128m"]
         }
         resources {
            cpu    = 500 # MHz
            memory = 300 # MB
            network {
               port "http" {}
            }
         }
         service {
            name = "callme-service"
            port = "http"
         }
         vault {
            policies = ["nomad"]
         }
      }
      restart {
         attempts = 1
      }
   }
}

You will have to change the value of jar_path property with your path of application binaries. Before applying this deployment to Nomad we will have to add some additional configuration on Vault. When adding integration with Vault we have to pass the name of policies used for checking permissions. I set the policy with the name nomad, which now has to be created in Vault. Our application requires a permission for reading paths /secret/* and /database/* as shown below.

Finally, we can deploy our application callme-service on Nomad by executing the following command.

$ nomad job run job.nomad

The similar descriptor file is available for caller-service, so we can also deploy it. All the microservice has been registered in Consul as shown below.

Here is the list of registered instances of caller-service. As you can see on the picture below it is available under port 25816.

You can also take a look at the Nomad jobs view.

The post Secure Spring Cloud Microservices with Vault and Nomad appeared first on Piotr's TechBlog.

1. Setting up an environment

We can use Gradle or Maven for setting up our build environment. The first goal is to add some dedicated Maven repositories, because Ktor dependencies are not available in central repo.

<repositories>
    <repository>
        <id>ktor</id>
        <url>http://dl.bintray.com/kotlin/ktor</url>
    </repository>
    <repository>
        <id>kotlinx</id>
        <url>http://dl.bintray.com/kotlin/kotlinx</url>
    </repository>
    <repository>
        <id>jcenter</id>
        <url>http://jcenter.bintray.com</url>
    </repository>
</repositories>

The current version of Ktor framework is 0.9.5.

<properties>
    <ktor.version>0.9.5</ktor.version>
</properties>

Before adding any dependencies we should also configure kotlin-maven-plugin, and the compiler to avoid warnings when using Kotlin coroutines. They are still an experimental feature in Kotlin.

<plugin>
   <groupId>org.jetbrains.kotlin</groupId>
   <artifactId>kotlin-maven-plugin</artifactId>
   <version>${kotlin.version}</version>
   <executions>
      <execution>
         <id>compile</id>
         <phase>compile</phase>
         <goals>
            <goal>compile</goal>
         </goals>
      </execution>
      <execution>
         <id>test-compile</id>
         <phase>test-compile</phase>
         <goals>
            <goal>test-compile</goal>
         </goals>
      </execution>
   </executions>
   <configuration>
      <jvmTarget>1.8</jvmTarget>
      <args>
         <arg>-Xcoroutines=enable</arg>
      </args>
   </configuration>
</plugin>

2. Sample applications

We will create two microservices built on top of Ktor Framework: account-service and customer-service. The application customer-service calls endpoint exposed by account-service for searching all accounts of a given customer. Both microservices expose some other endpoints for adding or finding objects. There is also a third element in our architecture: discovery server. We will use HashiCorp’s Consul for that. Every instance of account-service would register itself in the discovery server, while customer-service would fetch the list of register instances to obtain their addresses and ports. In summary, that is a typical example of communication between microservices presented in some of my previous articles on this blog. The sample system architecture is visualized on the following diagram.

3. The Basics

First, we will add some dependencies required for running Ktor on the server side. Our application will be launched on Netty server, and will use Jackson library for JSON serialization.

<dependency>
   <groupId>io.ktor</groupId>
   <artifactId>ktor-server-core</artifactId>
   <version>${ktor.version}</version>
</dependency>
<dependency>
   <groupId>io.ktor</groupId>
   <artifactId>ktor-server-netty</artifactId>
   <version>${ktor.version}</version>
</dependency>
<dependency>
   <groupId>io.ktor</groupId>
   <artifactId>ktor-jackson</artifactId>
   <version>${ktor.version}</version>
</dependency>

After that we can create the main method that initializes the embedded server based on Netty. Because we would like to set the server listen port dynamically during application startup, we use a method that takes command line arguments as a parameter. Thanks to that we will use parameter -port=TARGET_PORT, when starting every single instance of application.

fun main(args: Array<String>) {
   val server = embeddedServer(Netty, commandLineEnvironment(args))
   server.start(wait = true)
}

It is possible to implement different configuration strategies with Ktor, but the recommended way is through a configuration file, called a HOCON file. Here’s the application.conf file for customer-service.

ktor {
  deployment {
    port: 8095
  }
  application {
    modules = [ pl.piomin.services.CustomerApplicationModuleKt.main ]
  }
}

Each application takes a list of required Ktor modules as a parameter. In fact, modules are the vital part of your application. These are user-defined functions receiving the Application class that is in charge of configuring the server pipeline, install features, registering routes, handling requests, etc. For the configuration visible above, the method main that defines our module is available inside file CustomerApplicationModule.kt located inside package pl.piomin.services.
Inside that method you can find the declarations of features provided by the Ktor framework and used by the application. This is the most important part of our code, because it contains almost all the logic implemented by the application. The most commonly used feature is Routing. It defines all the HTTP API endpoints exposed by our application. In the following fragment of code I have defined four endpoints: a single POST method for adding a new account, and three GET endpoints providing different find methods.

package pl.piomin.services
fun Application.main() {
   val repository = AccountRepository()
   //...
   routing {
      get("/accounts") {
         call.respond(message = repository.accounts)
      }
      get("/accounts/{id}") {
         val id: String? = call.parameters["id"]
         if (id != null)
            call.respond(message = repository.accounts.filter { it.id == id.toInt() })
      }
      get("/accounts/customer/{customerId}") {
         val customerId: String? = call.parameters["customerId"]
         if (customerId != null)
            call.respond(message = repository.accounts.filter { it.customerId == customerId.toInt() })
      }
      post("/accounts") {
         var account: Account = call.receive()
         account.id = repository.accounts.size + 1
         repository.addAccount(account)
         log.info("$account")
         call.respond(message = account)
      }
   }
}

4. Using built-in features

A Ktor application typically consists of a series of features. You can think of features as functionality that is injected into the request and response pipeline. Usually, an application would have a series of features such as DefaultHeaders which add headers to every outgoing response or Routing which allows us to define routes to handle requests, etc. We can create our own custom features, but there is also a set of built-in features implemented as Ktor modules. We can install any of the built-in features just by passing the class name inside the install statement. Here’s the list of features I used in the sample applications:

• ContentNegotation – it provides automatic content conversion according to Content-Type and Accept headers. I have used Jackson library for converting between objects and JSON content
• Metrics – it provides implementation for generating metrics with useful information about the server and the requests. We can use different exporters by selecting from modules provided by Dropwizard Metrics library. I decided to use Slf4jReporter that prints metric values to the log destination. But you could also use, for example InfluxReporter for sending metrics to InfluxDB
• CallLogging – it is used for logging the incoming client requests. It leverages the ApplicationEnvironment.log that uses slf4j, so you can easily configure the output
• CallId – this feature allows identifying a request/call by generating request id and can work along the CallLogging feature. The CallLogging feature add generated value to MDC by calling method callIdMdc with MDC field name as a parameter

install(ContentNegotiation) {
   jackson {
   }
}
install(Metrics) {
   Slf4jReporter.forRegistry(registry).outputTo(log).build().start(10, TimeUnit.SECONDS)
}
install(CallLogging) {
   level = Level.TRACE
   callIdMdc("X-Request-ID")
}
install(CallId) {
   generate(10)
}

To be able to work with the Metrics feature we first need to include the following dependency to pom.xml file.

<dependency>
   <groupId>io.ktor</groupId>
   <artifactId>ktor-metrics</artifactId>
   <version>${ktor.version}</version>
</dependency>

5. Configure logging

The logging configuration for the Ktor framework is based on SLF4J. You just need to include dependency of the logging provider to pom.xml. It can be Logback.

<dependency>
    <groupId>ch.qos.logback</groupId>
    <artifactId>logback-classic</artifactId>
    <version>1.2.3</version>
</dependency>

Then you need to create logback.xml configuration file, and place it in src/main/resources directory. The following configuration forces SLF4J to print logs only to the console. It includes the MDC field X-Request-ID generated using CallId feature.

<configuration>
    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>%d{YYYY-MM-dd HH:mm:ss.SSS} [%thread] [%X{X-Request-ID}] %-5level %logger{36} - %msg%n</pattern>
        </encoder>
    </appender>

    <root level="trace">
        <appender-ref ref="STDOUT"/>
    </root>

    <logger name="org.eclipse.jetty" level="INFO"/>
    <logger name="io.netty" level="INFO"/>
</configuration>

Now, you are able to use a logger in your application just by calling io.ktor.application.log instance.

import io.ktor.application.log
// ...
post("/accounts") {
   var account: Account = call.receive()
   account.id = repository.accounts.size + 1
   repository.addAccount(account)
   log.info("$account")
   call.respond(message = account)
}

6. Running Consul server

The implementation of our applications is almost finished. We just need to add the communication between two sample microservices. To achieve it we first need to run a discovery server. Running Consul on the local machine is pretty easily using a Docker container. Here’s the command that runs Consul in standalone mode, and exposes API on port 8500.

$ docker run -d --name=consul -e CONSUL_BIND_INTERFACE=eth0 -p 8500:8500 -p 8600:8600 consul

Consul provides a web interface, which is available under address http://192.168.99.100:8500.

7. Custom Ktor feature for service discovery with Consul

Ktor Framework does not provide any components that implement typical microservice patterns like service discovery or distributed configuration. This library is in the early stage of development (still the current version is below 1.0), so such features will probably be implemented in the future. However, with Ktor we may easily implement a custom feature that is able to communicate with Consul. There are two types of features available for applications that use Ktor: server-side and client-side features. In that case we will implement a feature on the client side. It is responsible for intercepting an event of sending a message by HTTP client, and including communication with Consul to the pipeline.
Before starting any implementation we need to include a set of Ktor client libraries and also a library providing methods for calling Consul API.

<dependency>
   <groupId>io.ktor</groupId>
   <artifactId>ktor-client-apache</artifactId>
   <version>${ktor.version}</version>
</dependency>
<dependency>
   <groupId>io.ktor</groupId>
   <artifactId>ktor-client-json</artifactId>
   <version>${ktor.version}</version>
</dependency>
<dependency>
   <groupId>io.ktor</groupId>
   <artifactId>ktor-client-jackson</artifactId>
   <version>${ktor.version}</version>
</dependency>
<dependency>
   <groupId>com.orbitz.consul</groupId>
   <artifactId>consul-client</artifactId>
   <version>1.2.3</version>
</dependency>

Let’s take a look on the implementation of ConsulFeature. It uses Consul client provided by consul-client. The default Consul address is http://localhost:8500, but it can be overridden on the calling side. I would not like to get into the implementation details of the Ktor client feature. The most important thing is the code inside install method. It is executed during Render phase, which is fired before Send phase. Our custom feature replaces an address provided as a hostname in the client’s URL by the address and port taken from Consul.

class ConsulFeature(var consulUrl: String) {

    class Config {
        var consulUrl: String = "http://localhost:8500"
        fun build(): ConsulFeature = ConsulFeature(consulUrl)
    }

    companion object Feature : HttpClientFeature<Config, ConsulFeature> {
        var currentNodeIndex: Int = 0

        override val key = AttributeKey<ConsulFeature>("ConsulFeature")

        override fun prepare(block: Config.() -> Unit): ConsulFeature = Config().apply(block).build()

        override fun install(feature: ConsulFeature, scope: HttpClient) {
            scope.requestPipeline.intercept(HttpRequestPipeline.Render) {
                var consulClient = Consul.builder().withUrl(feature.consulUrl).build()
                val nodes = consulClient.healthClient().getHealthyServiceInstances(context.url.host).response
                val selectedNode = nodes[currentNodeIndex]
                context.url.host = selectedNode.service.address
                context.url.port = selectedNode.service.port
                currentNodeIndex = (currentNodeIndex + 1) % nodes.size
                println("Calling ${selectedNode.service.id}: ${context.url.buildString()}")
            }
        }
    }
}

How it works? Everything should be clear for you after looking at that fragment of code containing HTTP client declaration, and usage of that client inside customer-service endpoint implementation. Instead of setting an IP address in the client’s URL I used the name of the calling service – in that case account-service. This name is then replaced by ConsulFeature with an address and port taken for Consul server. If there is more than one instance of account-service registered in Consul server, the feature performs load balancing using the typical Round Robin method.

val client = HttpClient(Apache) {
   install(ConsulFeature) {
      consulUrl = "http://192.168.99.100:8500"
   }
   install(JsonFeature)
}
// ...
routing {
   get("/customers/{id}") {
      val id: String? = call.parameters["id"]
      if (id != null) {
         val accounts = client.get<Accounts>("http://account-service/accounts/customer/$id")
         val customerRet = customer.copy(id = customer.id, name = customer.name)
         customerRet.accounts.addAll(accounts)
         call.respond(message = customerRet)
      }
   }
}

And the last thing. The application needs to register itself in Consul after startup. Here’s the main function of account-service. It uses the register method of Consul client. An unique id is automatically generated using application listen port number.

fun main(args: Array<String>) {
    val server = embeddedServer(Netty, commandLineEnvironment(args))
    val consulClient = Consul.builder().withUrl("http://192.168.99.100:8500").build()
    val service = ImmutableRegistration.builder()
            .id("account-${server.environment.connectors[0].port}")
            .name("account-service")
            .address("localhost")
            .port(server.environment.connectors[0].port)
            .build()
    consulClient.agentClient().register(service)

    server.start(wait = true)
}

8. How it works?

Ok, let’s run two instances of account-service and a single instance of customer-service. When running two instances of account-service we need to override default port number by declaring an application running parameter -port=PORT_NUMBER.

Every instance of microservice should be registered in Consul after startup.

We are running instances of account-service. Here are the details.

Now, let’s add some test data by calling POST endpoints exposed by our microservices. As you can see I added new accounts only on the first instance of account-service. Because all the microservices store data in-memory, those objects will be stored only by instance running on port 8090.

$ curl -d '{"name":"John Smith"}' -H "Content-Type: application/json" -X POST http://localhost:8095/customers
$ curl -d '{"number":"1234567890", "balance":5000, "customerId":1}' -H "Content-Type: application/json" -X POST http://localhost:8090/accounts
$ curl -d '{"number":"1234567891", "balance":10000, "customerId":1}' -H "Content-Type: application/json" -X POST http://localhost:8090/accounts

If you call endpoint http://localhost:8095/customers/1 it tries to connect with running instances of account-service. Once it returns response {"id":1,"name":"John Smith","accounts":[{"id":1,"balance":5000,"number":"1234567890","customerId":1},{"id":2,"balance":10000,"number":"1234567891","customerId":1}]}, while the second time {"id":1,"name":"John Smith","accounts":[]}. That is the expected result. Because ConsulFeature load balances between two instances of account-service, when only the first instance stores data. Here’s the fragment of application logs.

Conclusion

In this article I presented how to use the Ktor framework for building microservices architecture with Consul server. We have used some basic Ktor features like mechanisms for routing, logging, metrics, and some more advanced solutions for building our own feature that interacts with Consul during inter-service communication. Ktor seems to be a very interesting framework. I will definitely follow the progress in development of this framework.

The post Kotlin Microservices with Ktor appeared first on Piotr's TechBlog.