To learn how Spring Boot supports Testcontainers, read my article on the subject. If you’re interested in Quarkus Dev Services, consider this post, which focuses on automated testing support in Quarkus.

Prerequisites

To perform the exercise described in this article, you must have the following on your laptop:

• Docker / Podman
• Java 21+
• Maven 3.9+

Source Code

Feel free to use my source code if you’d like to try it out yourself. To do that, you must clone my sample GitHub repository. Then you should only follow my instructions.

Create Spring Boot Application

For this exercise, we will build a simple application that connects to a Postgres database and returns employee information through a REST interface. In addition to the core logic, we will also implement integration tests to verify endpoint functionality with a live database. Below is a list of required dependencies.

<dependencies>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
  </dependency>
  <dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <scope>runtime</scope>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
  </dependency>
</dependencies>

<dependencies>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
  </dependency>
  <dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <scope>runtime</scope>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
  </dependency>
</dependencies>

Here’s the Employee domain object, which is stored in the employee table:

@Entity
public class Employee {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Integer id;
    private int organizationId;
    private int departmentId;
    private String name;
    private int age;
    private String position;
    
    // GETTERS and SETTERS
    
}

@Entity
public class Employee {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Integer id;
    private int organizationId;
    private int departmentId;
    private String name;
    private int age;
    private String position;
    
    // GETTERS and SETTERS
    
}

Here’s the Spring Data repository interface responsible for interacting with the Postgres database:

public interface EmployeeRepository extends CrudRepository<Employee, Integer> {

    List<Employee> findByDepartmentId(int departmentId);
    List<Employee> findByOrganizationId(int organizationId);

}

public interface EmployeeRepository extends CrudRepository<Employee, Integer> {

    List<Employee> findByDepartmentId(int departmentId);
    List<Employee> findByOrganizationId(int organizationId);

}

This is the EmployeeController code with several REST endpoints that allow us to add and find employees in the database:

@RestController
@RequestMapping("/employees")
public class EmployeeController {

    private static final Logger LOGGER = LoggerFactory
       .getLogger(EmployeeController.class);

    @Autowired
    EmployeeRepository repository;

    @PostMapping
    public Employee add(@RequestBody Employee employee) {
        LOGGER.info("Employee add...: {}", employee);
        return repository.save(employee);
    }

    @GetMapping("/{id}")
    public Employee findById(@PathVariable("id") Integer id) {
        LOGGER.info("Employee find: id={}", id);
        return repository.findById(id).get();
    }

    @GetMapping
    public List<Employee> findAll() {
        LOGGER.info("Employee find");
        return (List<Employee>) repository.findAll();
    }

    @GetMapping("/department/{departmentId}")
    public List<Employee> findByDepartment(@PathVariable("departmentId") int departmentId) {
        LOGGER.info("Employee find: departmentId={}", departmentId);
        return repository.findByDepartmentId(departmentId);
    }

    @GetMapping("/organization/{organizationId}")
    public List<Employee> findByOrganization(@PathVariable("organizationId") int organizationId) {
        LOGGER.info("Employee find: organizationId={}", organizationId);
        return repository.findByOrganizationId(organizationId);
    }

    @GetMapping("/department-with-delay/{departmentId}")
    public List<Employee> findByDepartmentWithDelay(@PathVariable("departmentId") int departmentId) throws InterruptedException {
        LOGGER.info("Employee find with delay: departmentId={}", departmentId);
        Thread.sleep(2000);
        return repository.findByDepartmentId(departmentId);
    }

}

@RestController
@RequestMapping("/employees")
public class EmployeeController {

    private static final Logger LOGGER = LoggerFactory
       .getLogger(EmployeeController.class);

    @Autowired
    EmployeeRepository repository;

    @PostMapping
    public Employee add(@RequestBody Employee employee) {
        LOGGER.info("Employee add...: {}", employee);
        return repository.save(employee);
    }

    @GetMapping("/{id}")
    public Employee findById(@PathVariable("id") Integer id) {
        LOGGER.info("Employee find: id={}", id);
        return repository.findById(id).get();
    }

    @GetMapping
    public List<Employee> findAll() {
        LOGGER.info("Employee find");
        return (List<Employee>) repository.findAll();
    }

    @GetMapping("/department/{departmentId}")
    public List<Employee> findByDepartment(@PathVariable("departmentId") int departmentId) {
        LOGGER.info("Employee find: departmentId={}", departmentId);
        return repository.findByDepartmentId(departmentId);
    }

    @GetMapping("/organization/{organizationId}")
    public List<Employee> findByOrganization(@PathVariable("organizationId") int organizationId) {
        LOGGER.info("Employee find: organizationId={}", organizationId);
        return repository.findByOrganizationId(organizationId);
    }

    @GetMapping("/department-with-delay/{departmentId}")
    public List<Employee> findByDepartmentWithDelay(@PathVariable("departmentId") int departmentId) throws InterruptedException {
        LOGGER.info("Employee find with delay: departmentId={}", departmentId);
        Thread.sleep(2000);
        return repository.findByDepartmentId(departmentId);
    }

}

With the following configuration in the application.yml, we will initialize the database schema on the application or tests startup:

spring:
  application:
    name: sample-spring-web-with-db
  jpa:
    hibernate:
      ddl-auto: create
    properties:
      hibernate:
        show_sql: true
        format_sql: true

spring:
  application:
    name: sample-spring-web-with-db
  jpa:
    hibernate:
      ddl-auto: create
    properties:
      hibernate:
        show_sql: true
        format_sql: true

Finally, here’s the @SpringBootTest that calls and verifies previously implemented REST endpoints:

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class EmployeeControllerTests {

    @Autowired
    TestRestTemplate restTemplate;

    @Test
    @Order(1)
    public void testAdd() {
        Employee employee = new Employee();
        employee.setName("John Doe");
        employee.setAge(30);
        employee.setPosition("Manager");
        employee.setDepartmentId(1);
        employee.setOrganizationId(1);
        employee = restTemplate.postForObject("/employees", employee, Employee.class);
        Assertions.assertNotNull(employee);
        Assertions.assertNotNull(employee.getId());
    }

    @Test
    @Order(2)
    public void testFindById() {
        Employee employee = restTemplate.getForObject("/employees/1", Employee.class);
        Assertions.assertNotNull(employee);
        Assertions.assertEquals(1, employee.getId());
    }

    @Test
    @Order(3)
    public void testFindAll() {
        Employee[] employees = restTemplate.getForObject("/employees", Employee[].class);
        Assertions.assertNotNull(employees);
        Assertions.assertEquals(1, employees.length);
    }

    @Test
    @Order(4)
    public void testFindByDepartment() {
        List<Employee> employees = restTemplate.getForObject("/employees/department/1", List.class);
        Assertions.assertNotNull(employees);
        Assertions.assertEquals(1, employees.size());
    }

}

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class EmployeeControllerTests {

    @Autowired
    TestRestTemplate restTemplate;

    @Test
    @Order(1)
    public void testAdd() {
        Employee employee = new Employee();
        employee.setName("John Doe");
        employee.setAge(30);
        employee.setPosition("Manager");
        employee.setDepartmentId(1);
        employee.setOrganizationId(1);
        employee = restTemplate.postForObject("/employees", employee, Employee.class);
        Assertions.assertNotNull(employee);
        Assertions.assertNotNull(employee.getId());
    }

    @Test
    @Order(2)
    public void testFindById() {
        Employee employee = restTemplate.getForObject("/employees/1", Employee.class);
        Assertions.assertNotNull(employee);
        Assertions.assertEquals(1, employee.getId());
    }

    @Test
    @Order(3)
    public void testFindAll() {
        Employee[] employees = restTemplate.getForObject("/employees", Employee[].class);
        Assertions.assertNotNull(employees);
        Assertions.assertEquals(1, employees.length);
    }

    @Test
    @Order(4)
    public void testFindByDepartment() {
        List<Employee> employees = restTemplate.getForObject("/employees/department/1", List.class);
        Assertions.assertNotNull(employees);
        Assertions.assertEquals(1, employees.size());
    }

}

Spring Boot Dev Services with Arconia

The Arconia framework offers multiple modules to support development services for the most popular databases and event brokers. To add support for the Postgres database, include the following dependency in your Maven pom.xml:

<dependency>
  <groupId>io.arconia</groupId>
  <artifactId>arconia-dev-services-postgresql</artifactId>
  <scope>runtime</scope>
  <optional>true</optional>
</dependency>

<dependency>
  <groupId>io.arconia</groupId>
  <artifactId>arconia-dev-services-postgresql</artifactId>
  <scope>runtime</scope>
  <optional>true</optional>
</dependency>

We will add other Arconia modules later, so let’s include the BOM (Bill of Materials) with the latest version:

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>io.arconia</groupId>
      <artifactId>arconia-bom</artifactId>
      <version>0.18.2</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>io.arconia</groupId>
      <artifactId>arconia-bom</artifactId>
      <version>0.18.2</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

And that’s all we needed to do. Now you can run the application in developer mode using the Maven command or through the arconia CLI. CLI is an add-on here, so for now, let’s stick with the standard mvn command.

mvn spring-boot:run

mvn spring-boot:run

You can also run automated tests with the mvn test command or through the IDE’s graphical interface.

Spring Boot Observability with Arconia

Dev services are just one of the features offered by Arconia. In this article, I will present a simple scenario of integrating with the Grafana observability stack using OpenTelemetry. This time, we will include two dependencies. The first is a special Spring Boot starter provided by Arconia, which automatically configures OpenTelemetry, Micrometer, and Spring Boot Actuator for your app. The second dependency includes the dev services for a Grafana LGTM observability platform, which contains: Loki, Grafana, Prometheus, Tempo, and OpenTelemetry collector.

<dependency>
  <groupId>io.arconia</groupId>
  <artifactId>arconia-opentelemetry-spring-boot-starter</artifactId>
</dependency>
<dependency>
  <groupId>io.arconia</groupId>
  <artifactId>arconia-dev-services-lgtm</artifactId>
  <scope>runtime</scope>
  <optional>true</optional>
</dependency>

<dependency>
  <groupId>io.arconia</groupId>
  <artifactId>arconia-opentelemetry-spring-boot-starter</artifactId>
</dependency>
<dependency>
  <groupId>io.arconia</groupId>
  <artifactId>arconia-dev-services-lgtm</artifactId>
  <scope>runtime</scope>
  <optional>true</optional>
</dependency>

In addition to the standard Arconia Observability settings, we will enable all built-in resource contributors. Below is the required configuration to add to the application settings in the application.yml file.

arconia:
 otel:
   resource:
     contributors:
       build:
         enabled: true
       host:
         enabled: true
       java:
         enabled: true
       os:
         enabled: true
       process:
         enabled: true

arconia:
 otel:
   resource:
     contributors:
       build:
         enabled: true
       host:
         enabled: true
       java:
         enabled: true
       os:
         enabled: true
       process:
         enabled: true

That’s it. Let’s start our application in development mode once again.

mvn spring-boot:run

mvn spring-boot:run

This time, Arconia starts one container more than before. I can access the Grafana dashboard at http://localhost:33383.

Let’s display all the containers running locally:

$ docker ps
CONTAINER ID  IMAGE                                   COMMAND               CREATED        STATUS        PORTS                                                                                                                                                 NAMES
a6a097fb9ebe  docker.io/testcontainers/ryuk:0.12.0    /bin/ryuk             2 minutes ago  Up 2 minutes  0.0.0.0:42583->8080/tcp                                                                                                                               testcontainers-ryuk-dfdea2da-0bbd-43fa-9f50-3e9d966d877f
917d74a5a0ad  docker.io/library/postgres:18.0-alpine  postgres -c fsync...  2 minutes ago  Up 2 minutes  0.0.0.0:38409->5432/tcp                                                                                                                               pensive_mcnulty
090a9434d1fd  docker.io/grafana/otel-lgtm:0.11.16     /otel-lgtm/run-al...  2 minutes ago  Up 2 minutes  0.0.0.0:33383->3000/tcp, 0.0.0.0:39501->3100/tcp, 0.0.0.0:32867->3200/tcp, 0.0.0.0:40389->4317/tcp, 0.0.0.0:46739->4318/tcp, 0.0.0.0:36915->9090/tcp  vigorous_euler

$ docker ps
CONTAINER ID  IMAGE                                   COMMAND               CREATED        STATUS        PORTS                                                                                                                                                 NAMES
a6a097fb9ebe  docker.io/testcontainers/ryuk:0.12.0    /bin/ryuk             2 minutes ago  Up 2 minutes  0.0.0.0:42583->8080/tcp                                                                                                                               testcontainers-ryuk-dfdea2da-0bbd-43fa-9f50-3e9d966d877f
917d74a5a0ad  docker.io/library/postgres:18.0-alpine  postgres -c fsync...  2 minutes ago  Up 2 minutes  0.0.0.0:38409->5432/tcp                                                                                                                               pensive_mcnulty
090a9434d1fd  docker.io/grafana/otel-lgtm:0.11.16     /otel-lgtm/run-al...  2 minutes ago  Up 2 minutes  0.0.0.0:33383->3000/tcp, 0.0.0.0:39501->3100/tcp, 0.0.0.0:32867->3200/tcp, 0.0.0.0:40389->4317/tcp, 0.0.0.0:46739->4318/tcp, 0.0.0.0:36915->9090/tcp  vigorous_euler

And now for the best part. Right after launch, our application is fully integrated with the Grafana stack. For example, logs are sent to the Loki instance, from which we can view them in the Grafana UI.

We can also display a dashboard with Spring Boot metrics.

Right after launch, I sent several test POST and GET requests to the application endpoints. Information about this is available in Grafana Tempo.

We can also verify JVM statistics in a dedicated dashboard.

What more could you want?

Conclusion

Arconia is an exciting and promising project, which I will be watching closely in the future. It is a relatively new initiative that is still undergoing intensive development. Arconia already offers several practical solutions that significantly simplify working with the Spring Boot application. I have shown you how this framework works in a simple scenario: running and integrating our application with the Postgres database and the Grafana observability stack using the Micrometer framework.

The post Arconia for Spring Boot Dev Services and Observability appeared first on Piotr's TechBlog.

It’s not my first article about Devtron. You can read more about the GitOps approach with Devtron in this article. Today, I’m going to focus more on the developer-friendly features around Helm charts support.

Install Devtron on Kubernetes

In the first step, we will install Devtron on Kubernetes. There are two options for installation: with CI/CD module or without it. We won’t build a CI/CD process today, but there are some important features for our scenario included in this module. Firstly, let’s add the Devtron Helm repository:

$ helm repo add devtron https://helm.devtron.ai

Then, we have to execute the following Helm command:

$ helm install devtron devtron/devtron-operator \
    --create-namespace --namespace devtroncd \
    --set installer.modules={cicd}

For detailed installation instructions please refer to the Devtron documentation available here.

Create Kubernetes Cluster with Kind

In order to prepare a multi-cluster environment on the local machine, we will use Kind. Let’s create the second Kubernetes cluster c1 by executing the following command:

$ kind create cluster --name c1

The second cluster is available as the kind-c1 context. It becomes a default context after you create a Kind cluster.

Now, our goal is to add the newly created Kind cluster as a managed cluster in Devtron. A single instance of Devtron can manage multiple Kubernetes clusters. Of course, by default, it just manages a local cluster. Before we add our Kind cluster to the Devtron dashboard, we should first configure privileges on that cluster. The following script will generate a bearer token for authentication purposes so that Devtron is able to communicate with the target cluster:

$ curl -O https://raw.githubusercontent.com/devtron-labs/utilities/main/kubeconfig-exporter/kubernetes_export_sa.sh && bash kubernetes_export_sa.sh cd-user devtroncd https://raw.githubusercontent.com/devtron-labs/utilities/main/kubeconfig-exporter/clusterrole.yaml

The bearer token is printed in the output of that command. Just copy it.

We will also have to provide an URL of the master API of a target cluster. Since I’m running Kubernetes on Kind I need to get an internal address of the Docker container that contains Kind. In order to obtain it we need to run the following command:

$ docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' c1-control-plane

Here’s the IP address of my Kind cluster:

Now, we have all the required data to add a new managed cluster in the Devtron dashboard. In order to do that let’s navigate to the “Global Configuration” section. Then we need to choose the “Clusters and Environments” item and click the “Add cluster” button. We need to put the Kind cluster URL and previously generated bearer token.

If everything works fine, you should see the second cluster on the managed clusters list. Now, you also need to install the Devtron agent on Kind according to the message visible below:

Create Environments

In the next step, we will define three environments. In Devtron environment is assigned to the cluster. We will create a single environment on the local cluster (local), and another two on the Kind cluster (remote-dev, remote-devqa). Each environment has a target namespace. In order to simplify, the name of the namespace is the same as the name environment. Of course, you may set any names you want.

Now, let’s switch to the “Clusters” view.

As you see there are two clusters connected to Devtron:

We can take a look at the details of each cluster. Here you can see a detailed view for the kind-c1 cluster:

Add Custom Helm Repository

One of the most important Devtron features is support for Helm charts. We can deploy charts individually or by creating a group of charts. By default, there are several Helm repositories available in Devtron including bitnami or elastic. It is also possible to add a custom repository. That’s something that we are going to do. We have our own custom Helm repository with a chart for deploying the Spring Boot app. I have already published it on GitHub under the address https://piomin.github.io/helm-charts/. The name of our chart is spring-boot-api-app, and the latest version is 0.3.2.

In order to add the custom repository in Devtron, we need to go to the “Global Configurations” section once again. Then go to the “Chart repositories” menu item, and click the “Add repository” button. As you see below, I added a new repository under the name piomin.

Once you created a repository you can go to the “Chart Store” section to verify if the new chart is available.

Deploy the Spring Boot App with Devtron

Now, we can proceed to the most important part of our exercise – application deployment. Our sample Spring Boot app is available in the following repository on GitHub. It is a simple REST app written in Kotlin. It exposes some HTTP endpoints for adding and returning persons and uses an in-memory store. Here’s our Spring @RestController:

@RestController
@RequestMapping("/persons")
class PersonController(val repository: PersonRepository) {

   val log: Logger = LoggerFactory.getLogger(PersonController::class.java)

   @GetMapping("/{id}")
   fun findById(@PathVariable id: Int): Person? {
      log.info("findById({})", id)
      return repository.findById(id)
   }

   @GetMapping("/age/{age}")
   fun findByAge(@PathVariable age: Int): List<Person> {
      log.info("findByAge({})", age)
      return repository.findByAge(age)
   }

   @GetMapping
   fun findAll(): List<Person> = repository.findAll()

   @PostMapping
   fun add(@RequestBody person: Person): Person = repository.save(person)

   @PutMapping
   fun update(@RequestBody person: Person): Person = repository.update(person)

   @DeleteMapping("/{id}")
   fun remove(@PathVariable id: Int): Boolean = repository.removeById(id)

}

Let’s imagine we are just working on the latest version of that, and we want to deploy it on Kubernetes to perform some development tests. In the first step, we will build the app locally and push the image to the container registry using Jib Maven Plugin. Here’s the required configuration:

<plugin>
  <groupId>com.google.cloud.tools</groupId>
  <artifactId>jib-maven-plugin</artifactId>
  <version>3.3.0</version>
  <configuration>
    <to>
      <image>piomin/sample-spring-kotlin-microservice</image>
      <tags>
        <tag>1.1</tag>
      </tags>
    </to>
    <container>
      <user>999</user>
    </container>
  </configuration>
</plugin>

Let’s build and push the image to the container registry using the following command:

$ mvn clean compile jib:build -Pjib,tomcat

Besides YAML templates our Helm repository also contains a JSON schema for values.yaml validation. Thanks to that schema we would be able to take advantage of Devtron GUI for creating apps from the chart. Let’s see how it works. Once you click on our custom chart you will be redirected to the page with the details. The latest version of the chart is 0.3.2. Just click the Deploy button.

On the next page, we need to provide a configuration of our app. The target environment is local, which exists on the main cluster. Thanks to Devtron support for Helm values.schema.json we define all values using the GUI form. For example, we can increase change the value of the image to the latest – 1.1.

Once we deploy the app we may verify its status:

Let’s make some test calls. Our sample Spring Boot exposes Swagger UI, so we can easily send HTTP requests. To interact with the app running on Kubernetes we should enable port-forwarding for our service kubectl port-forward svc/sample-spring-boot-api 8080:8080. After executing that command you can access the Swagger UI under the address http://localhost:8080/swagger-ui.html.

Devtron allows us to view pod logs. We can “grep” them with our criteria. Let’s display the logs related to our test calls.

Deploy App to the Remote Cluster

Now, we will deploy our sample Spring Boot app to the remote cluster. In order to do that go to the same page as before, but instead of the local environment choose remote-dev. It is related to the kind-c1 cluster.

Now, there are two same applications running on two different clusters. We can do the same thing for the app running on the Kind cluster as for the local cluster, e.g. verify its status or check logs.

Deploy Group of Apps

Let’s assume we would like to deploy the app that connects to the database. We can do it in a single step using the Devtron feature called “Chart Group”. With that feature, we can place our Helm chart for Spring Boot and the chart for e.g. Postgres inside the same logical group. Then, we can just deploy the whole group into the target environment. In order to create a chart group go to the Chart Store menu and then click the “Create Group” button. You should set the name of the group and choose the charts that will be included. For me, these are bitnami/postgresql and my custom Helm chart.

After creating a group you will see it on the main “Chart Store” page. Now, just click on it to deploy the apps.

After you click the tile with the chart group, you will be predicted to the deploy page.

After you click the “Deploy to…” button Devtron will redirect you to the next page. You can set there a target project and environment for all member charts of the group. We will deploy them to the remote-devqa environment from the kind-c1 cluster. We can use the image from my Docker account: piomin/person:1.1. By default, it tries to connect to the database postgres on the postgres host. The only thing we need to inject into the app container is the postgres user password. It is available inside the postgresql Secret generated by the Bitnami Helm chart. To inject envs defined in that secret use the extraEnvVarsSecret parameter in our custom Spring Boot chart. Finally, let’s deploy both Spring Boot and Postgres in the remove-devqa namespace by clicking the “Deploy” button.

Here’s the final list of apps we have already deployed during this exercise:

Final Thoughts

With Devtron you can easily deploy applications across multiple Kubernetes clusters using Helm chart support. Devtron simplifies development on Kubernetes. You can deploy all required applications just with a “single click” with the chart group feature. Then you can manage and monitor them using a GUI dashboard. In general, you can do everything in the dashboard without passing any YAML manifests by yourself or executing kubectl commands.

The post Development on Kubernetes Multicluster with Devtron appeared first on Piotr's TechBlog.

There are a lot of articles about Quarkus on my blog. If you don’t have any experience with that framework and you are looking for something to begin you can read, for example, the following article. Also, there are a lot of simple guides to the Quarkus features available on its website here.

Source Code

If you would like to try this exercise yourself, you may always take a look at my source code. In order to do that, you need to clone my GitHub repository. Then you need to go to the person-virtual-service directory. After that, just follow my instructions.

Prerequisites

Before we start, we need to install several tools on the local machine. First of all, you have to install JDK 19. There are also some other tools I’m using in that exercise:

• Docker – our sample Quarkus app connects to the Postgresql database. We can easily run Postgres on Docker automatically using the Quarkus Dev Services feature
• JProfiler – in order to visualize and observe the thread pool
• k6 – a javascript-based tool for load testing

Use Case

We will build a simple REST application using the Quarkus framework. It will connect to the Postgres database and expose some endpoints for adding data and the basic search operations. Apart from enabling virtual threads support in Quarkus, our main goal is not to block the thread. In the virtual threads nomenclature, this thread is called “carrier thread”. It is the platform thread responsible for executing a virtual thread. It might be blocked by e.g. JDBC client driver. In order to avoid it, we should use non-blocking clients.

Dependencies

We need to include two Quarkus modules into the Maven dependencies. The first of them is Quarkus Resteasy Reactive which provides an implementation of JAX-RS specification and allows us to create reactive REST services. The quarkus-reactive-pg-client module provides an implementation of a reactive driver for the Postgres database.

<dependency>
  <groupId>io.quarkus</groupId>
  <artifactId>quarkus-resteasy-reactive-jackson</artifactId>
</dependency>
<dependency>
  <groupId>io.quarkus</groupId>
  <artifactId>quarkus-reactive-pg-client</artifactId>
</dependency>

We should also set JDK 19 as a default compiler. Since virtual threads are available as a preview feature we need to set --enable-preview as a JVM argument.

<plugin>
  <groupId>io.quarkus</groupId>
  <artifactId>quarkus-maven-plugin</artifactId>
  <version>${quarkus.version}</version>
  <extensions>true</extensions>
  <executions>
    <execution>
      <goals>
        <goal>build</goal>
        <goal>generate-code</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <source>19</source>
    <target>19</target>
    <compilerArgs>
      <arg>--enable-preview</arg>
    </compilerArgs>
    <jvmArgs>--enable-preview --add-opens java.base/java.lang=ALL-UNNAMED</jvmArgs>
  </configuration>
</plugin>

I’m also generating some test data for load tests. There is a really useful library for that – Datafaker.

<dependency>
  <groupId>net.datafaker</groupId>
  <artifactId>datafaker</artifactId>
  <version>1.6.0</version>
</dependency>

Enable Virtual Threads for Reactive Services

The Quarkus Resteasy Reactive module works in a non-blocking, event-loop style way. Instead of using the “event loop”, we need to force the execution of an endpoint handler on a new virtual thread. To do that we need to annotate the REST endpoint with the @RunOnVirtualThread. The rest of the code may be pretty similar to the blocking-style way of building services. Of course, instead of blocking the database driver, we will use its reactive alternative. The implementation is provided inside the PersonRepositoryAsyncAwait class.

@Path("/persons")
public class PersonResource {

   @Inject
   PersonRepositoryAsyncAwait personRepository;
   @Inject
   Logger log;

   @POST
   @RunOnVirtualThread
   public Person addPerson(Person person) {
      person = personRepository.save(person);
      return person;
   }

   @GET
   @RunOnVirtualThread
   public List<Person> getPersons() {
      return personRepository.findAll();
   }

   @GET
   @Path("/name/{name}")
   @RunOnVirtualThread
   public List<Person> getPersonsByName(@PathParam("name") String name) {
      return personRepository.findByName(name);
   }

   @GET
   @Path("/age-greater-than/{age}")
   @RunOnVirtualThread
   public List<Person> getPersonsByName(@PathParam("age") int age) {
      return personRepository.findByAgeGreaterThan(age);
   }

   @GET
   @Path("/{id}")
   @RunOnVirtualThread
   public Person getPersonById(@PathParam("id") Long id) {
      log.infof("(%s) getPersonById(%d)", Thread.currentThread(), id);
      return personRepository.findById(id);
   }

}

Let’s switch to the repository implementation. We use an implementation of the SmallRye Mutiny client for Postgres. The PgPool client allows us to create and execute SQL queries in a non-blocking way. In order to create a query, we should call the preparedQuery method. Finally, we need to invoke the executeAndAwait method to perform the operation asynchronously.

@ApplicationScoped
public class PersonRepositoryAsyncAwait {

   @Inject
   PgPool pgPool;
   @Inject
   Logger log;

   public Person save(Person person) {
      Long id = pgPool
         .preparedQuery("INSERT INTO person(name, age, gender) VALUES ($1, $2, $3) RETURNING id")
         .executeAndAwait()
         .iterator().next().getLong("id");
      person.setId(id);
      return person;
   }

   public List<Person> findAll() {
      log.info("FindAll()" + Thread.currentThread());
      RowSet<Row> rowSet = pgPool
         .preparedQuery("SELECT id, name, age, gender FROM person")
         .executeAndAwait();
      return iterateAndCreate(rowSet);
   }

   public Person findById(Long id) {
      RowSet<Row> rowSet = pgPool
         .preparedQuery("SELECT id, name, age, gender FROM person WHERE id = $1")
         .executeAndAwait(Tuple.of(id));
      List<Person> persons = iterateAndCreate(rowSet);
      return persons.size() == 0 ? null : persons.get(0);
   }

   public List<Person> findByName(String name) {
      RowSet<Row> rowSet = pgPool
         .preparedQuery("SELECT id, name, age, gender FROM person WHERE id = $1")
         .executeAndAwait(Tuple.of(name));
      return iterateAndCreate(rowSet);
   }

   public List<Person> findByAgeGreaterThan(int age) {
      RowSet<Row> rowSet = pgPool
         .preparedQuery("SELECT id, name, age, gender FROM person WHERE age > $1")
         .executeAndAwait(Tuple.of(age));
      return iterateAndCreate(rowSet);
   }

   private List<Person> iterateAndCreate(RowSet<Row> rowSet) {
      List<Person> persons = new ArrayList<>();
      for (Row row : rowSet) {
         persons.add(Person.from(row));
      }
      return persons;
   }

}

Prepare Test Data

Before we run load tests let’s add some test data to the Postgres database. We will use the Datafaker library for generating persons’ names. We will use the same reactive, non-blocking PgPool client as before. The following part of the code generates and inserts 1000 persons into the database on the Quarkus app startup. It is a part of our repository implementation.

@ApplicationScoped
public class PersonRepositoryAsyncAwait {

   // ... methods

   @Inject
   @ConfigProperty(name = "myapp.schema.create", defaultValue = "true")
   boolean schemaCreate;

   void config(@Observes StartupEvent ev) {
      if (schemaCreate) {
         initDb();
      }
   }

   private void initDb() {
      List<Tuple> persons = new ArrayList<>(1000);
      Faker faker = new Faker();
      for (int i = 0; i < 1000; i++) {
         String name = faker.name().fullName();
         String gender = faker.gender().binaryTypes().toUpperCase();
         int age = faker.number().numberBetween(18, 65);
         int externalId = faker.number().numberBetween(100000, 999999);
         persons.add(Tuple.of(name, age, gender, externalId));
      }

      pgPool.query("DROP TABLE IF EXISTS person").execute()
         .flatMap(r -> pgPool.query("""
                  create table person (
                    id serial primary key,
                    name varchar(255),
                    gender varchar(255),
                    age int,
                    external_id int
                  )
                  """).execute())
         .flatMap(r -> pgPool
            .preparedQuery("insert into person(name, age, gender, external_id) values($1, $2, $3, $4)")
            .executeBatch(persons))
         .await().indefinitely();
    }
}

Load Testing with k6 and JProfiler

I’m using the k6 tool for load testing, but you can use any other popular tool as well. In order to install it on macOS just run the following homebrew command:

$ brew install k6

The k6 tool uses Javascript for creating tests. We need to prepare an input with the test definition. My test generates a number between 1 and 1000 and then places it as a path parameter for the GET /persons/{id} endpoint. Finally, it checks if the response status is 200 and the body is not empty.

import http from 'k6/http';
import { sleep, check } from 'k6';

export default function () {
  let r = Math.floor(Math.random() * 1000) + 1;
  const res = http.get(`http://localhost:8080/persons/${r}`);
  check(res, {
    'is status 200': (res) => res.status === 200,
    'body size is > 0': (r) => r.body.length > 0,
  });
  sleep(1);
}

To simplify running the Quarkus app from your IDE you can annotate the main class with @QuarkusMain.

@QuarkusMain
public class PersonVirtualApp {

   public static void main(String... args) {
      Quarkus.run(args);
   }

}

Don’t forget to enable Java preview mode in your IDE. Here’s how it looks in IntelliJ.

Once you run the app and then attach JProfiler to the running JVM process you may execute tests with k6. To do that pass the location of the file with the test definition, and then set the number of concurrent threads (--vus parameter). Finally set the test duration with the --duration parameter. I’m running the test four times using a different number of concurrent threads (20, 50, 100, 200).

$ k6 run --vus 20 --duration 90s k6-test.js

Analyze Test Results

Let’s take a look at the fragment of application logs. You can see that there are many virtual threads running by the same “carrier” thread, for example, on the ForkJoinPool-1-worker-9.

Let’s switch to the JProfiler UI for a moment. Here’s the fragment of a thread pool visualization.

We run the same test four times with a different number of concurrent users (20, 50, 100, 200). Here’s a visualization of total Java threads (non-virtual) running. As you can see there is no significant difference between tests for 20 and 200 users.

During the first test (20 virtual users), we generated ~1.8k requests in 90 seconds.

During the last test (200 virtual users), we generated ~17.8k requests in 90 seconds.

Final Thoughts

One of the most important things I really like in Quarkus is that it is always up-to-date with the latest features. Once, the virtual threads have been released I can refactor my current app and use them instead of the standard platform threads. In this article, I presented how to use virtual threads with the app that connects to the database. If you are interested in other articles about Quarkus you can see the full list here. Enjoy

The post Quarkus with Java Virtual Threads appeared first on Piotr's TechBlog.

Currently, Micronaut Predator provides runtime support for JPA (Hibernate) and SQL (JDBC). Some other implementations are planned in the future. In this article I’m going to show you how to include Micronaut Data in your application and use its main features for providing JPA data access.

1. Dependencies

The snapshot dependency of Micronaut Predator is available at https://oss.sonatype.org/content/repositories/snapshots/, so first we need to include it to the repository list in our pom.xml together with jcenter:

<repositories>
   <repository>
      <id>jcenter.bintray.com</id>
      <url>https://jcenter.bintray.com</url>
   </repository>
   <repository>
      <id>sonatype-snapshots</id>
      <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
   </repository>
</repositories>

In addition to the standard libraries included for building a web application with Micronaut, we have to add the following dependencies: database driver (we will use PostgreSQL as the database for our sample application) and micronaut-predator-hibernate-jpa.

<dependency>
   <groupId>io.micronaut.data</groupId>
   <artifactId>micronaut-predator-hibernate-jpa</artifactId>
   <version>${predator.version}</version>
   <scope>compile</scope>
</dependency>
<dependency>
   <groupId>io.micronaut.configuration</groupId>
   <artifactId>micronaut-jdbc-tomcat</artifactId>
   <scope>runtime</scope>
</dependency>    
<dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <version>42.2.6</version>
</dependency> 

Some Micronaut libraries including micronaut-predator-processor have to be added to the annotation processor path. Such a configuration should be provided inside Maven Compiler Plugin configuration:

<plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-compiler-plugin</artifactId>
   <version>3.7.0</version>
   <configuration>
      <source>${jdk.version}</source>
      <target>${jdk.version}</target>
      <encoding>UTF-8</encoding>
      <compilerArgs>
         <arg>-parameters</arg>
      </compilerArgs>
      <annotationProcessorPaths>
         <path>
            <groupId>io.micronaut</groupId>
            <artifactId>micronaut-inject-java</artifactId>
            <version>${micronaut.version}</version>
         </path>
         <path>
            <groupId>io.micronaut.data</groupId>
            <artifactId>micronaut-predator-processor</artifactId>
            <version>${predator.version}</version>
         </path>
         <path>
            <groupId>io.micronaut</groupId>
            <artifactId>micronaut-validation</artifactId>
            <version>${micronaut.version}</version>
         </path>
      </annotationProcessorPaths>
   </configuration>
</plugin>

The current newest RC version of Micronaut is 1.2.0.RC:

<dependencyManagement>
   <dependencies>
      <dependency>
         <groupId>io.micronaut</groupId>
         <artifactId>micronaut-bom</artifactId>
         <version>1.2.0.RC2</version>
         <type>pom</type>
         <scope>import</scope>
      </dependency>
   </dependencies>
</dependencyManagement>

2. Domain Model

Our database model consists of four tables as shown below. The same database model has been used for some of my previous examples including those for Spring Data usage. We have employee table. Each employee is assigned to the exactly one department and one organization. Each department is assigned to exactly one organization. There is also table employment, which provides a history of employment for every single employee.

Here is the implementation of entity classes corresponding to the database model. Let’s start from Employee class:

@Entity
public class Employee {

   @Id
   @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "employee_id_seq")
   @SequenceGenerator(name = "employee_id_seq", sequenceName = "employee_id_seq", allocationSize = 1)
   private Long id;
   private String name;
   private int age;
   private String position;
   private int salary;
   @ManyToOne
   private Organization organization;
   @ManyToOne
   private Department department;
   @OneToMany
   private Set<Employment> employments;
   
   // ... GETTERS AND SETTERS
}

Here’s the implementation of Department class:

@Entity
public class Department {

   @Id
   @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "department_id_seq")
   @SequenceGenerator(name = "department_id_seq", sequenceName = "department_id_seq", allocationSize = 1)
   private Long id;
   private String name;
   @OneToMany
   private Set<Employee> employees;
   @ManyToOne
   private Organization organization;
   
   // ... GETTERS AND SETTERS
}

And here’s Organization entity:

@Entity
public class Organization {

    @Id
    @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "organization_id_seq")
    @SequenceGenerator(name = "organization_id_seq", sequenceName = "organization_id_seq", allocationSize = 1)
    private Long id;
    private String name;
    private String address;
    @OneToMany
    private Set<Department> departments;
    @OneToMany
    private Set<Employee> employees;
   
   // ... GETTERS AND SETTERS
}

And the last entity Employment:

@Entity
public class Employment {

    @Id
    @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "employment_id_seq")
    @SequenceGenerator(name = "employment_id_seq", sequenceName = "employment_id_seq", allocationSize = 1)
    private Long id;
    @ManyToOne
    private Employee employee;
    @ManyToOne
    private Organization organization;
    @Temporal(TemporalType.DATE)
    private Date start;
    @Temporal(TemporalType.DATE)
    private Date end;
   
   // ... GETTERS AND SETTERS
}

3. Creating JPA repositories with Micronaut Data

If you are familiar with the Spring Data repositories pattern, you won’t have any problems when using Micronaut repositories. The approach to declaring repositories and building queries is the same as in Spring Data. You need to declare an interface (or an abstract class) annotated with @Repository that extends interface CrudRepository. CrudRepository is not the only one interface that can be extended. You can also use GenericRepository, AsyncCrudRepository for asynchronous operations, ReactiveStreamsCrudRepository for reactive CRUD execution or PageableRepository that adds methods for pagination. The typical repository declaration looks like as shown below.

@Repository
public interface EmployeeRepository extends CrudRepository<Employee, Long> {

    Set<EmployeeDTO> findBySalaryGreaterThan(int salary);

    Set<EmployeeDTO> findByOrganization(Organization organization);

    int findAvgSalaryByAge(int age);

    int findAvgSalaryByOrganization(Organization organization);

}

I have declared there some additional find methods. The most common query prefix is found, but you can also use search, query, get, read, or retrieve. The first two queries return all employees with a salary greater than a given value and all employees assigned to a given organization. The Employee entity is in many-to-one relation with Organization, so we may also use relational fields as query parameters. It is noteworthy that both two queries return DTO objects as a result inside the collection. That’s possible because Micronaut Predator supports reflection-free Data Transfer Object (DTO) projections if the return type is annotated with @Introspected. Here’s the declaration of EmployeeDTO.

@Introspected
public class EmployeeDTO {

    private String name;
    private int age;
    private String position;
    private int salary;
   
    // ... GETTERS AND SETTERS
}

The EmployeeRepository contains two other methods using aggregation expressions. Method findAvgSalaryByAge counts average salary by a given age of employees, while findAvgSalaryByOrganization counts avarage salary by a given organization.
For comparison, let’s take a look on another repository implementation EmploymentRepository. We need two additional find methods. First findByEmployeeOrderByStartDesc for searching employment history for a given employee ordered by start date. The second method finds employment without an end date set, which in fact means that’s the employment for a current job.

@Repository
public interface EmploymentRepository extends CrudRepository<Employment, Long> {

    Set<EmploymentDTO> findByEmployeeOrderByStartDesc(Employee employee);

    Employment findByEmployeeAndEndIsNull(Employee employee);

}

Micronaut Predator is able to automatically manage transactions. You just need to annotate your method with @Transactional. In the source code fragment visible below you may see the method used for changing a job by an employee. We are performing a bunch of save operations inside that method. First, we change the target department and organization for a given employee, then we are creating new employment history record for a new job, and also setting end date for the previous employment entity (found using repository method findByEmployeeAndEndIsNull).

@Inject
DepartmentRepository departmentRepository;
@Inject
EmployeeRepository employeeRepository;
@Inject
EmploymentRepository employmentRepository;

@Transactional
public void changeJob(Long employeeId, Long targetDepartmentId) {
   Optional<Employee> employee = employeeRepository.findById(employeeId);
   employee.ifPresent(employee1 -> {
      Optional<Department> department = departmentRepository.findById(targetDepartmentId);
      department.ifPresent(department1 -> {
         employee1.setDepartment(department1);
         employee1.setOrganization(department1.getOrganization());
         Employment employment = new Employment(employee1, department1.getOrganization(), new Date());
         employmentRepository.save(employment);
         Employment previousEmployment = employmentRepository.findByEmployeeAndEndIsNull(employee1);
         previousEmployment.setEnd(new Date());
         employmentRepository.save(previousEmployment);
      });
   });
}

Ok, now let’s move on to the last repository implementation discussed in this section – OrganizationRepository. Since Organization entity is in lazy load one-to-many relation with Employee and Department, we need to fetch data to present dependencies in the output. To achieve that we can use @Join annotation on the repository interface with specifying JOIN FETCH. Since the @Join annotation is repeatable it can be specified multiple times for different associations as shown below.

@Repository
public interface OrganizationRepository extends CrudRepository<Organization, Long> {

    @Join(value = "departments", type = Join.Type.LEFT_FETCH)
    @Join(value = "employees", type = Join.Type.LEFT_FETCH)
    Optional<Organization> findByName(String name);

}

4. Batch operations

Micronaut Predator repositories support batch operations. It can be sometimes useful, for example in automatic tests. Here’s my simple JUnit test that adds multiple employees into a single department inside an organization:

@Test
public void addMultiple() {
   List<Employee> employees = Arrays.asList(
      new Employee("Test1", 20, "Developer", 5000),
      new Employee("Test2", 30, "Analyst", 15000),
      new Employee("Test3", 40, "Manager", 25000),
      new Employee("Test4", 25, "Developer", 9000),
      new Employee("Test5", 23, "Analyst", 8000),
      new Employee("Test6", 50, "Developer", 12000),
      new Employee("Test7", 55, "Architect", 25000),
      new Employee("Test8", 43, "Manager", 15000)
   );

   Organization organization = new Organization("TestWithEmployees", "TestAddress");
   Organization organizationSaved = organizationRepository.save(organization);
   Assertions.assertNotNull(organization.getId());
   Department department = new Department("TestWithEmployees");
   department.setOrganization(organization);
   Department departmentSaved = departmentRepository.save(department);
   Assertions.assertNotNull(department.getId());
   employeeRepository.saveAll(employees.stream().map(employee -> {
      employee.setOrganization(organizationSaved);
      employee.setDepartment(departmentSaved);
      return employee;
   }).collect(Collectors.toList()));
}

5. Controllers

Finally, the last implementation step – building REST controllers. OrganizationController is pretty simple. It injects OrganizationRepository and uses it for saving entities and searching their by name. Here’s the implementation:

@Controller("organizations")
public class OrganizationController {

    @Inject
    OrganizationRepository repository;

    @Post("/organization")
    public Long addOrganization(@Body Organization organization) {
        Organization organization1 = repository.save(organization);
        return organization1.getId();
    }

    @Get("/organization/name/{name}")
    public Optional<Organization> findOrganization(@NotNull String name) {
        return repository.findByName(name);
    }

}

EmployeeController is a little bit more complicated. We have an implementation that exposes four additional find methods defined in EmployeeRepository. There is also a method for adding a new employee and assigning it to the department, and changing the job implemented inside SampleService bean.

@Controller("employees")
public class EmployeeController {

    @Inject
    EmployeeRepository repository;
    @Inject
    OrganizationRepository organizationRepository;
    @Inject
    SampleService service;

    @Get("/salary/{salary}")
    public Set<EmployeeDTO> findEmployeesBySalary(int salary) {
        return repository.findBySalaryGreaterThan(salary);
    }

    @Get("/organization/{organizationId}")
    public Set<EmployeeDTO> findEmployeesByOrganization(Long organizationId) {
        Optional<Organization> organization = organizationRepository.findById(organizationId);
        return repository.findByOrganization(organization.get());
    }

    @Get("/salary-avg/age/{age}")
    public int findAvgSalaryByAge(int age) {
        return repository.findAvgSalaryByAge(age);
    }

    @Get("/salary-avg/organization/{organizationId}")
    public int findAvgSalaryByAge(Long organizationId) {
        Optional<Organization> organization = organizationRepository.findById(organizationId);
        return repository.findAvgSalaryByOrganization(organization.get());
    }

    @Post("/{departmentId}")
    public void addNewEmployee(@Body Employee employee, Long departmentId) {
        service.hireEmployee(employee, departmentId);
    }

    @Put("/change-job")
    public void changeJob(@Body ChangeJobRequest request) {
        service.changeJob(request.getEmployeeId(), request.getTargetOrganizationId());
    }

}

6. Configuring database connection

As usual we use Docker image for running database instances locally. Here’s the command that runs container with Postgres and expose it on port 5432:

$ docker run -d --name postgres -p 5432:5432 -e POSTGRES_USER=predator -e POSTGRES_PASSWORD=predator123 -e POSTGRES_DB=predator postgres

After startup my Postgres instance is available on the virtual address 192.168.99.100, so I have to set it in the Micronaut application.yml. Besides database connection settings we will also set some JPA properties, that enable SQL logging and automatically applies model changes into a database schema. Here’s full configuration of our sample application inside application.yml:

micronaut:
  application:
    name: sample-micronaut-jpa

jackson:
  bean-introspection-module: true

datasources:
  default:
    url: jdbc:postgresql://192.168.99.100:5432/predator?ssl=false
    driverClassName: org.postgresql.Driver
    username: predator
    password: predator123

jpa:
  default:
    properties:
      hibernate:
        hbm2ddl:
          auto: update
        show_sql: true

Conclusion

The support for ORM was one of the most expected features for the Micronaut Framework. Not only it will be available in the release version soon, but it is almost 1.5x faster than Spring Data JPA – following this article https://objectcomputing.com/news/2019/07/18/unleashing-predator-precomputed-data-repositories created by the leader of Micronaut Project Graeme Rocher. In my opinion, the support for ORM via project Predator may be the reason that developers decide to use Micronaut instead of Spring Boot.
In this article, I have demonstrated the most interesting features of Micronaut Data JPA. I think that it will be continuously improved, and we see some new useful features soon. The sample application source code snippet is, as usual, available on GitHub: https://github.com/piomin/sample-micronaut-jpa.git. Before starting with Micronaut Data it is worth reading about the basics: Micronaut Tutorial: Beans and scopes.

The post JPA Data Access with Micronaut Data appeared first on Piotr's TechBlog.

Including dependencies

We use Kotlin for implementation. So first, we include some required Kotlin dependencies.

<dependency>
   <groupId>org.jetbrains.kotlin</groupId>
   <artifactId>kotlin-stdlib</artifactId>
   <version>${kotlin.version}</version>
</dependency>
<dependency>
   <groupId>com.fasterxml.jackson.module</groupId>
   <artifactId>jackson-module-kotlin</artifactId>
</dependency>
<dependency>
   <groupId>org.jetbrains.kotlin</groupId>
   <artifactId>kotlin-reflect</artifactId>
</dependency>
<dependency>
   <groupId>org.jetbrains.kotlin</groupId>
   <artifactId>kotlin-test-junit</artifactId>
   <version>${kotlin.version}</version>
   <scope>test</scope>
</dependency>

We should also add kotlin-maven-plugin with support for Spring.

<plugin>
   <groupId>org.jetbrains.kotlin</groupId>
   <artifactId>kotlin-maven-plugin</artifactId>
   <version>${kotlin.version}</version>
   <executions>
      <execution>
         <id>compile</id>
         <phase>compile</phase>
         <goals>
            <goal>compile</goal>
         </goals>
      </execution>
      <execution>
         <id>test-compile</id>
         <phase>test-compile</phase>
         <goals>
            <goal>test-compile</goal>
         </goals>
      </execution>
   </executions>
   <configuration>
      <args>
         <arg>-Xjsr305=strict</arg>
      </args>
      <compilerPlugins>
         <plugin>spring</plugin>
      </compilerPlugins>
   </configuration>
</plugin>

Then, we may proceed to including frameworks required for the demo implementation. We need to include the special SNAPSHOT version of Spring Data JDBC dedicated for accessing databases using R2DBC. We also have to add some R2DBC libraries and Spring WebFlux. As you may see below only Spring WebFlux is available in stable version (as a part of Spring Boot RELEASE).

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.data</groupId>
   <artifactId>spring-data-jdbc</artifactId>
   <version>1.0.0.r2dbc-SNAPSHOT</version>
</dependency>
<dependency>
   <groupId>io.r2dbc</groupId>
   <artifactId>r2dbc-spi</artifactId>
   <version>1.0.0.M5</version>
</dependency>
<dependency>
   <groupId>io.r2dbc</groupId>
   <artifactId>r2dbc-postgresql</artifactId>
   <version>1.0.0.M5</version>
</dependency>

It is also important to set dependency management for Spring Data project.

<dependencyManagement>
   <dependencies>
      <dependency>
         <groupId>org.springframework.data</groupId>
         <artifactId>spring-data-releasetrain</artifactId>
         <version>Lovelace-RELEASE</version>
         <scope>import</scope>
         <type>pom</type>
      </dependency>
   </dependencies>
</dependencyManagement>

Repositories

We are using the well known Spring Data style of CRUD repository implementation. In that case we need to create an interface that extends ReactiveCrudRepository interface.
Here’s the implementation of a repository for managing Employee objects.

interface EmployeeRepository : ReactiveCrudRepository<Employee, Int> {
    @Query("select id, name, salary, organization_id from employee e where e.organization_id = $1")
    fun findByOrganizationId(organizationId: Int) : Flux<Employee>
}

Here’s another implementation of repository – this time for managing Organization objects.

interface OrganizationRepository : ReactiveCrudRepository<Organization, Int> 

Implementing Entities and DTOs

Kotlin provides a convenient way of creating entity class by declaring them as data class. When using Spring Data JDBC we have to set the primary key for the entity by annotating the field with @Id. It assumes the key is automatically incremented by the database. If you are not using auto-increment columns, you have to use a BeforeSaveEvent listener, which sets the ID of the entity. However, I tried to set such a listener for my entity, but it just didn’t work with the reactive version of Spring Data JDBC.
Here’s an implementation of Employee entity class. What is worth mentioning Spring Data JDBC will automatically map class field organizationId into database column organization_id.

data class Employee(val name: String, val salary: Int, val organizationId: Int) {
    @Id 
    var id: Int? = null
}

Here’s an implementation of Organization entity class.

data class Organization(var name: String) {
    @Id 
    var id: Int? = null
}

R2DBC does not support any lists or sets. Because I’d like to return a list with employees inside Organization object in one of API endpoints I have created a DTO containing such a list as shown below.

data class OrganizationDTO(var id: Int?, var name: String) {
    var employees : MutableList = ArrayList()
    constructor(employees: MutableList) : this(null, "") {
        this.employees = employees
    }
}

The SQL scripts corresponding to the created entities are visible below. Field type serial will automatically create a sequence and attach it to the field id.

CREATE TABLE employee (
    name character varying NOT NULL,
    salary integer NOT NULL,
    id serial PRIMARY KEY,
    organization_id integer
);
CREATE TABLE organization (
    name character varying NOT NULL,
    id serial PRIMARY KEY
);

Building sample web applications

For the demo purposes we will build two independent applications employee-service and organization-service. Application organization-service is communicating with employee-service using WebFlux WebClient. It gets the list of employees assigned to the organization, and includes them to response together with Organization object. Sample applications source code is available on GitHub under repository sample-spring-data-webflux: https://github.com/piomin/sample-spring-data-webflux.
Ok, let’s begin from declaring Spring Boot main class. We need to enable Spring Data JDBC repositories by annotating the main class with @EnableJdbcRepositories.

@SpringBootApplication
@EnableJdbcRepositories
class EmployeeApplication

fun main(args: Array<String>) {
    runApplication<EmployeeApplication>(*args)
}

Working with R2DBC and Postgres requires some configuration. Probably due to an early stage of progress in development of Spring Data JDBC and R2DBC there is no Spring Boot auto-configuration for Postgres. We need to declare connection factory, client, and repository inside @Configuration bean.

@Configuration
class EmployeeConfiguration {

    @Bean
    fun repository(factory: R2dbcRepositoryFactory): EmployeeRepository {
        return factory.getRepository(EmployeeRepository::class.java)
    }

    @Bean
    fun factory(client: DatabaseClient): R2dbcRepositoryFactory {
        val context = RelationalMappingContext()
        context.afterPropertiesSet()
        return R2dbcRepositoryFactory(client, context)
    }

    @Bean
    fun databaseClient(factory: ConnectionFactory): DatabaseClient {
        return DatabaseClient.builder().connectionFactory(factory).build()
    }

    @Bean
    fun connectionFactory(): PostgresqlConnectionFactory {
        val config = PostgresqlConnectionConfiguration.builder() //
                .host("192.168.99.100") //
                .port(5432) //
                .database("reactive") //
                .username("reactive") //
                .password("reactive123") //
                .build()

        return PostgresqlConnectionFactory(config)
    }

}

Finally, we can create REST controllers that contain the definition of our reactive API methods. With Kotlin it does not take much space. The following controller definition contains three GET methods that allows to find all employees, all employees assigned to a given organization or a single employee by id.

@RestController
@RequestMapping("/employees")
class EmployeeController {

    @Autowired
    lateinit var repository : EmployeeRepository

    @GetMapping
    fun findAll() : Flux<Employee> = repository.findAll()

    @GetMapping("/{id}")
    fun findById(@PathVariable id : Int) : Mono<Employee> = repository.findById(id)

    @GetMapping("/organization/{organizationId}")
    fun findByorganizationId(@PathVariable organizationId : Int) : Flux<Employee> = repository.findByOrganizationId(organizationId)

    @PostMapping
    fun add(@RequestBody employee: Employee) : Mono<Employee> = repository.save(employee)

}

Inter-service Communication

For the OrganizationController the implementation is a little bit more complicated. Because organization-service is communicating with employee-service, we first need to declare reactive WebFlux WebClient builder.

@Bean
fun clientBuilder() : WebClient.Builder {
   return WebClient.builder()
}

Then, similar to the repository bean the builder is being injected into the controller. It is used inside findByIdWithEmployees method for calling method GET /employees/organization/{organizationId} exposed by employee-service. As you can see on the code fragment below it provides a reactive API and returns Flux object containing a list of found employees. This list is injected into OrganizationDTO object using zipWith Reactor method.

@RestController
@RequestMapping("/organizations")
class OrganizationController {

    @Autowired
    lateinit var repository : OrganizationRepository
    @Autowired
    lateinit var clientBuilder : WebClient.Builder

    @GetMapping
    fun findAll() : Flux<Organization> = repository.findAll()

    @GetMapping("/{id}")
    fun findById(@PathVariable id : Int) : Mono<Organization> = repository.findById(id)

    @GetMapping("/{id}/withEmployees")
    fun findByIdWithEmployees(@PathVariable id : Int) : Mono<OrganizationDTO> {
        val employees : Flux<Employee> = clientBuilder.build().get().uri("http://localhost:8090/employees/organization/$id")
                .retrieve().bodyToFlux(Employee::class.java)
        val org : Mono = repository.findById(id)
        return org.zipWith(employees.collectList())
                .map { tuple -> OrganizationDTO(tuple.t1.id as Int, tuple.t1.name, tuple.t2) }
    }

    @PostMapping
    fun add(@RequestBody employee: Organization) : Mono<Organization> = repository.save(employee)

}

How it works?

Before running the tests we need to start a Postgres database. Here’s the Docker command used for running a Postgres container. It is creating a user with a password, and setting up a default database.

$ docker run -d --name postgres -p 5432:5432 -e POSTGRES_USER=reactive -e POSTGRES_PASSWORD=reactive123 -e POSTGRES_DB=reactive postgres

Then we need to create some test tables, so you have to run a SQL script placed in the section Implementing Entities and DTOs. After that you can start our test applications. If you do not override default settings provided inside application.yml files employee-service is listening on port 8090, and organization-service on port 8095. The following picture illustrates the architecture of our sample system.

Now, let’s add some test data using the reactive API exposed by the applications.

$ curl -d '{"name":"Test1"}' -H "Content-Type: application/json" -X POST http://localhost:8095/organizations
$ curl -d '{"name":"Name1", "balance":5000, "organizationId":1}' -H "Content-Type: application/json" -X POST http://localhost:8090/employees
$ curl -d '{"name":"Name2", "balance":10000, "organizationId":1}' -H "Content-Type: application/json" -X POST http://localhost:8090/employees

Finally you can call GET organizations/{id}/withEmployees method, for example using your web browser. The result should be similar to the result visible on the following picture.

The post Introduction to Reactive APIs with Postgres, R2DBC, Spring Data JDBC and Spring WebFlux appeared first on Piotr's TechBlog.

What is Gatling?

Gatling is a powerful tool for load testing, written in Scala. It has full support of HTTP protocols and can also be used for testing JDBC connections and JMS. When using Gatling you have to define the test scenario as a Scala dsl code. It is worth mentioning that it provides comprehensive informative HTML load reports and has plugins for integration with Gradle, Maven and Jenkins.

Building sample application

Before we run any tests we need to have something for tests. Our sample application is really simple. Its source code is available as usual on GitHub. It exposes a RESTful HTTP API with CRUD operations for adding and searching entities in the database. I use Postgres as a backend store for the application repository. The application is built on the top of the Spring Boot framework. It also uses the Spring Data project as a persistence layer implementation.

plugins {
   id 'org.springframework.boot' version '1.5.9.RELEASE'
}
dependencies {
   compile group: 'org.springframework.boot', name: 'spring-boot-starter-web'
   compile group: 'org.springframework.boot', name: 'spring-boot-starter-data-jpa'
   compile group: 'org.postgresql', name: 'postgresql', version: '42.1.4'
   testCompile group: 'org.springframework.boot', name: 'spring-boot-starter-test'
}

There is one entity Person which is mapped to the table person.

@Entity
@SequenceGenerator(name = "seq_person", initialValue = 1, allocationSize = 1)
public class Person {
   @Id
   @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "seq_person")
   private Long id;
   @Column(name = "first_name")
   private String firstName;
   @Column(name = "last_name")
   private String lastName;
   @Column(name = "birth_date")
   private Date birthDate;
   @Embedded
   private Address address;
   // ...
}

Database connection settings and hibernate properties are configured in application.yml file.

spring:
  application:
    name: gatling-service
  datasource:
    url: jdbc:postgresql://192.168.99.100:5432/gatling
    username: gatling
    password: gatling123
  jpa:
    properties:
      hibernate:
        hbm2ddl:
          auto: update

server:
  port: 8090

Like I have already mentioned the application exposes API methods for adding and searching persons in the database. Here’s our Spring REST controller implementation.

@RestController
@RequestMapping("/persons")
public class PersonsController {

   private static final Logger LOGGER = LoggerFactory.getLogger(PersonsController.class);

   @Autowired
   PersonsRepository repository;

   @GetMapping
   public List<Person> findAll() {
      return (List<Person>) repository.findAll();
   }

   @PostMapping
   public Person add(@RequestBody Person person) {
      Person p = repository.save(person);
      LOGGER.info("add: {}", p.toString());
      return p;
   }

   @GetMapping("/{id}")
   public Person findById(@PathVariable("id") Long id) {
      LOGGER.info("findById: id={}", id);
      return repository.findOne(id);
   }

}

Running database

The next after the sample application development is to run the database. The most suitable way of running it for the purposes is by Docker image. Here’s a Docker command that starts Postgres containerand initializes Gatling user and database.

$ docker run -d --name postgres -e POSTGRES_DB=gatling -e POSTGRES_USER=gatling -e POSTGRES_PASSWORD=gatling123 -p 5432:5432 postgres

Providing test scenario

Every Gatling test suite should extend the Simulation class. Inside it you may declare a list of scenarios using Gatling Scala DSL. Our goal is to run 30 clients which simultaneously sends requests 1000 times. First, the client adds a new person into the database by calling POST /persons method. Then they try to search the person using its id by calling GET /persons/{id} method. So, totally 60k would be sent to the application: 30k to POST endpoint and 30k to GET method. Like you see on the code below the test scenario is quite simple. ApiGatlingSimulationTest is available under directory src/test/scala.

class ApiGatlingSimulationTest extends Simulation {

val scn = scenario("AddAndFindPersons").repeat(1000, "n") {
   exec(
      http("AddPerson-API")
        .post("http://localhost:8090/persons")
        .header("Content-Type", "application/json")
        .body(StringBody("""{"firstName":"John${n}","lastName":"Smith${n}","birthDate":"1980-01-01", "address": {"country":"pl","city":"Warsaw","street":"Test${n}","postalCode":"02-200","houseNo":${n}}}"""))
         .check(status.is(200))
      ).pause(Duration.apply(5, TimeUnit.MILLISECONDS))
   }.repeat(1000, "n") {
     exec(
        http("GetPerson-API")
          .get("http://localhost:8090/persons/${n}")
          .check(status.is(200))
     )
   }

setUp(scn.inject(atOnceUsers(30))).maxDuration(FiniteDuration.apply(10, "minutes"))

}

To enable Gatling framework for the project we should also define the following dependency in the Gradle build file.

testCompile group: 'io.gatling.highcharts', name: 'gatling-charts-highcharts', version: '2.3.0'

Running tests

There are some Gradle plugins available, which provides support for running tests during project build. However, we may also define a simple Gradle task that just runs tests using io.gatling.app.Gatling class.

task loadTest(type: JavaExec) {
   dependsOn testClasses
   description = "Load Test With Gatling"
   group = "Load Test"
   classpath = sourceSets.test.runtimeClasspath
   jvmArgs = [
      "-Dgatling.core.directory.binaries=${sourceSets.test.output.classesDir.toString()}"
   ]
   main = "io.gatling.app.Gatling"
   args = [
      "--simulation", "pl.piomin.services.gatling.ApiGatlingSimulationTest",
      "--results-folder", "${buildDir}/gatling-results",
      "--binaries-folder", sourceSets.test.output.classesDir.toString(),
      "--bodies-folder", sourceSets.test.resources.srcDirs.toList().first().toString() + "/gatling/bodies",
   ]
}

The Gradle task defined above may be run with command gradle loadTest. Of course, before running tests you should launch the application. You may perform it from your IDE by starting the main class pl.piomin.services.gatling.ApiApplication or by running command java -jar build/libs/sample-load-test-gatling.jar.

Test reports

After test execution the report is printed in a text format.

================================================================================
---- Global Information --------------------------------------------------------
> request count                                      60000 (OK=60000  KO=0     )
> min response time                                      2 (OK=2      KO=-     )
> max response time                                   1338 (OK=1338   KO=-     )
> mean response time                                    80 (OK=80     KO=-     )
> std deviation                                        106 (OK=106    KO=-     )
> response time 50th percentile                         50 (OK=50     KO=-     )
> response time 75th percentile                         93 (OK=93     KO=-     )
> response time 95th percentile                        253 (OK=253    KO=-     )
> response time 99th percentile                        564 (OK=564    KO=-     )
> mean requests/sec                                319.149 (OK=319.149 KO=-     )
---- Response Time Distribution ------------------------------------------------
> t < 800 ms                                         59818 (100%) 
> 800 ms < t < 1200 ms                                 166 (  0%) > t > 1200 ms                                           16 (  0%)
> failed                                                 0 (  0%)
================================================================================

What is really cool in Gatling is an ability to generate reports in a graphical form. HTML reports are available under directory build/gatling-results. The first report shows global information with total number of requests and maximum response time by percentiles. For example, you may see that maximum response time in 95% of responses for GetPerson-API is 206 ms.

We may check out such reports for all requests or filter them to see only those generated by selected API. In the picture below there is visualization only for GetPerson-API.

Here’s the graph with the percentage of requests grouped by average response time.

Here’s the graph which illustrates the timeline with average response times. Additionally, that timeline also shows the statistics by percentiles.

Here’s the graph with the number of requests processed succesfully by the application in a second.

The post Performance Testing with Gatling appeared first on Piotr's TechBlog.