Source Code

If you would like to try it by yourself, you may always take a look at my source code. To do that you need to clone my GitHub repository. For the source code repository with my custom Spring Boot Logging library go here. Then you should just follow my instructions.

Using Loki4j Logback Appender

In order to use Loki4j Logback appended we need to include a single dependency in Maven pom.xml. The current version of that library is 1.4.1:

<dependency>
    <groupId>com.github.loki4j</groupId>
    <artifactId>loki-logback-appender</artifactId>
    <version>1.4.1</version>
</dependency>

Then we need to create the logback-spring.xml file in the src/main/resources directory. Our instance of Loki is available under the http://localhost:3100 address (1). Loki does not index the contents of the logs – but only metadata labels. There are some static labels like the app name, log level, or hostname. We can set them in the format.label field (2). We will also set some dynamic labels and therefore we enable the Logback markers feature (3). Finally, we are setting the log format pattern (4). In order to simplify, potential transformations with LogQL (Loki query language) we will use JSON notation.

<?xml version="1.0" encoding="UTF-8"?>
<configuration>

  <springProperty name="name" source="spring.application.name" />

  <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
    <encoder>
      <pattern>
        %d{HH:mm:ss.SSS} %-5level %logger{36} %X{X-Request-ID} - %msg%n
      </pattern>
    </encoder>
  </appender>

  <appender name="LOKI" class="com.github.loki4j.logback.Loki4jAppender">
    <!-- (1) -->
    <http>
      <url>http://localhost:3100/loki/api/v1/push</url>
    </http>
    <format>
      <!-- (2) -->
      <label>
        <pattern>app=${name},host=${HOSTNAME},level=%level</pattern>
        <!-- (3) -->
        <readMarkers>true</readMarkers>
      </label>
      <message>
        <!-- (4) -->
        <pattern>
{
   "level":"%level",
   "class":"%logger{36}",
   "thread":"%thread",
   "message": "%message",
   "requestId": "%X{X-Request-ID}"
}
        </pattern>
      </message>
    </format>
  </appender>

  <root level="INFO">
    <appender-ref ref="CONSOLE" />
    <appender-ref ref="LOKI" />
  </root>

</configuration>

Besides the static labels, we may send dynamic data, e.g. something specific just for the current request. Assuming we have a service that manages persons, we want to log the id of the target person from the request. As I mentioned before, with Loki4j we can use Logback markers for that. In classic Logback, markers are mostly used to filter log records. With Loki, we just need to define the LabelMarker object containing the key/value Map of dynamic fields (1). Then we pass the object to the current log line (2).

@RestController
@RequestMapping("/persons")
public class PersonController {

    private final Logger LOG = LoggerFactory
       .getLogger(PersonController.class);
    private final List<Person> persons = new ArrayList<>();

    @GetMapping
    public List<Person> findAll() {
        return persons;
    }

    @GetMapping("/{id}")
    public Person findById(@PathVariable("id") Long id) {
        Person p = persons.stream().filter(it -> it.getId().equals(id))
                .findFirst()
                .orElseThrow();
        LabelMarker marker = LabelMarker.of("personId", () -> 
           String.valueOf(p.getId())); // (1)
        LOG.info(marker, "Person successfully found"); // (2)
        return p;
    }

    @GetMapping("/name/{firstName}/{lastName}")
    public List<Person> findByName(
       @PathVariable("firstName") String firstName,
       @PathVariable("lastName") String lastName) {
       
       return persons.stream()
          .filter(it -> it.getFirstName().equals(firstName)
                        && it.getLastName().equals(lastName))
          .toList();
    }

    @PostMapping
    public Person add(@RequestBody Person p) {
        p.setId((long) (persons.size() + 1));
        LabelMarker marker = LabelMarker.of("personId", () -> 
           String.valueOf(p.getId()));
        LOG.info(marker, "New person successfully added");
        persons.add(p);
        return p;
    }

    @DeleteMapping("/{id}")
    public void delete(@PathVariable("id") Long id) {
        Person p = persons.stream()
           .filter(it -> it.getId().equals(id))
           .findFirst()
           .orElseThrow();
        persons.remove(p);
        LabelMarker marker = LabelMarker.of("personId", () -> 
           String.valueOf(id));
        LOG.info(marker, "Person successfully removed");
    }

    @PutMapping
    public void update(@RequestBody Person p) {
        Person person = persons.stream()
                .filter(it -> it.getId().equals(p.getId()))
                .findFirst()
                .orElseThrow();
        persons.set(persons.indexOf(person), p);
        LabelMarker marker = LabelMarker.of("personId", () -> 
            String.valueOf(p.getId()));
        LOG.info(marker, "Person successfully updated");
    }

}

Assuming we have multiple dynamic fields in the single log line, we have to create the LabelMarker object in this way:

LabelMarker marker = LabelMarker.of(() -> Map.of("audit", "true",
                    "X-Request-ID", MDC.get("X-Request-ID"),
                    "X-Correlation-ID", MDC.get("X-Correlation-ID")));

Running Loki with Spring Boot App

The simplest way to run Loki on the local machine is with a Docker container. Besides just the Loki instance, we will also run Grafana to display and search logs. Here’s the docker-compose.yml with all the required services. You can run them with the docker compose up command. However, there is another way – directly with the Spring Boot app.

version: "3"

networks:
  loki:

services:
  loki:
    image: grafana/loki:2.8.2
    ports:
      - "3100:3100"
    command: -config.file=/etc/loki/local-config.yaml
    networks:
      - loki

  grafana:
    environment:
      - GF_PATHS_PROVISIONING=/etc/grafana/provisioning
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
    entrypoint:
      - sh
      - -euc
      - |
        mkdir -p /etc/grafana/provisioning/datasources
        cat <<EOF > /etc/grafana/provisioning/datasources/ds.yaml
        apiVersion: 1
        datasources:
        - name: Loki
          type: loki
          access: proxy
          orgId: 1
          url: http://loki:3100
          basicAuth: false
          isDefault: true
          version: 1
          editable: false
        EOF
        /run.sh
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    networks:
      - loki

In order to take advantage of the Spring Boot Docker Compose support we need to place the docker-compose.yml in the app root directory. Then, we have to include the spring-boot-docker-compose dependency in the Maven pom.xml:

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-docker-compose</artifactId>
  <optional>true</optional>
</dependency>

Once we do all the required things, we can run the app. For example, with the following Maven command:

$ mvn spring-boot:run

Now, before the app, Spring Boot starts containers defined in the docker-compose.yml.

Let’s just display a list of running containers. As you see, everything will work fine since Loki listens on the local port 3100:

$ docker ps            
CONTAINER ID   IMAGE                    COMMAND                  CREATED         STATUS         PORTS                    NAMES
d23390fbee06   grafana/loki:2.8.2       "/usr/bin/loki -conf…"   4 minutes ago   Up 2 minutes   0.0.0.0:3100->3100/tcp   sample-spring-boot-web-loki-1
84a47637a50b   grafana/grafana:latest   "sh -euc 'mkdir -p /…"   2 days ago      Up 2 minutes   0.0.0.0:3000->3000/tcp   sample-spring-boot-web-grafana-1

Testing Logging on the Spring Boot REST App

After running the app, we can make some test calls of our REST API. In the beginning, let’s add some persons:

$ curl 'http://localhost:8080/persons' \
  -H 'Content-Type: application/json' \
  -d '{"firstName": "AAA","lastName": "BBB","age": 20,"gender": "MALE"}'

$ curl 'http://localhost:8080/persons' \
  -H 'Content-Type: application/json' \
  -d '{"firstName": "CCC","lastName": "DDD","age": 30,"gender": "FEMALE"}'

$ curl 'http://localhost:8080/persons' \
  -H 'Content-Type: application/json' \
  -d '{"firstName": "EEE","lastName": "FFF","age": 40,"gender": "MALE"}'

Then, we may call the “find” endpoints several times with different criteria:

$ curl http://localhost:8080/persons/1
$ curl http://localhost:8080/persons/2
$ curl http://localhost:8080/persons/3

Here are the application logs from the console. There are just simple log lines – not formatted in JSON.

Now, let’s switch to Grafana. We already have integration with Loki configured. In the new dashboard, we need to choose Loki.

Here’s the history of app logs stored in Loki.

As you see we are logging in the JSON format. Some log lines contain dynamic labels included with the Loki4j Logback appended.

We added the personId label to some logs, so we can easily filter records only with requests for a particular person. Here’s the LogQL query that filters records for the personId=1:

{app="first-service"} |= `` | personId = `1`

Here’s the result visible in the Grafana dashboard:

We can also format logs using LogQL. Thanks to JSON format it is possible to prepare a query that parses the whole log message.

{app="first-service"} |= `` | json

As you see, now Loki treats all the JSON fields as metadata labels:

Using Spring Boot Loki Starter Library

If you don’t want to configure those things by yourself, you can use my Spring Boot library which provides auto-configuration for that. Additionally, it automatically logs all the incoming HTTP requests and outgoing HTTP responses. If default settings are enough you just need to include the single Spring Boot starter as a dependency:

<dependency>
  <groupId>com.github.piomin</groupId>
  <artifactId>logstash-logging-spring-boot-starter</artifactId>
  <version>2.0.2</version>
</dependency>

The library logs each request and response with several default labels including e.g. requestId or correlationId.

If you need more information about my Spring Boot Logging library you can refer to my previous articles about it. Here’s the article with a detailed explanation with some implementation details. Another one is more focused on the usage guide.

The post Logging in Spring Boot with Loki appeared first on Piotr's TechBlog.

Of course, we usually consider such performance issues within the context of microservices or serverless apps. They should not consume many OS resources and should be easily auto-scalable. We can easily control resource usage on Kubernetes. If you are interested in Java virtual threads you can read my previous article about using them to create an HTTP server available here. For more details about Knative as serverless on Kubernetes, you can refer to the following article.

Introduction

Let’s start with the plan for our exercise today. In the first step, we will create a simple Java web app that uses virtual threads for processing incoming HTTP requests. Before we run the sample app we will install Knative on Kubernetes to quickly test autoscaling based on HTTP traffic. We will also install Prometheus on Kubernetes. This monitoring stack allows us to compare the performance of the app without/with GraalVM and virtual threads on Kubernetes. Then, we can proceed with the deployment. In order to easily build and run our native app on Kubernetes we will use Cloud Native Buildpacks. Finally, we will perform some load tests and compare metrics.

Source Code

If you would like to try it by yourself, you may always take a look at my source code. In order to do that you need to clone my GitHub repository. After that, you should follow my instructions.

Create Java App with Virtual Threads

In the first step, we will create a simple Java app that acts as an HTTP server and handles incoming requests. In order to do that, we can use the HttpServer object from the core Java API. Once we create the server we can override a default thread executor with the setExecutor method. In the end, we will try to compare the app using standard threads with the same app using virtual threads. Therefore, we allow overriding the type of executor using an environment variable. The name of that is THREAD_TYPE. If you want to enable virtual threads you need to set the value virtual for that env. Here’s the main method of our app.

public class MainApp {

   public static void main(String[] args) throws IOException {
      HttpServer httpServer = HttpServer
         .create(new InetSocketAddress(8080), 0);

      httpServer.createContext("/example", 
         new SimpleCPUConsumeHandler());

      if (System.getenv("THREAD_TYPE").equals("virtual")) {
         httpServer.setExecutor(
            Executors.newVirtualThreadPerTaskExecutor());
      } else {
         httpServer.setExecutor(Executors.newFixedThreadPool(200));
      }
      httpServer.start();
   }

}

In order to process incoming requests, the HTTP server uses the handler that implements the HttpHandler interface. In our case, the handler is implemented inside the SimpleCPUConsumeHandler class as shown below. It consumes a lot of CPU since it creates an instance of BigInteger with the constructor that performs a lot of computations under the hood. It will also consume some time, so we have the simulation of processing time in the same step. As a response, we just return the next number in the sequence with the Hello_ prefix.

public class SimpleCPUConsumeHandler implements HttpHandler {

   Logger LOG = Logger.getLogger("handler");
   AtomicLong i = new AtomicLong();
   final Integer cpus = Runtime.getRuntime().availableProcessors();

   @Override
   public void handle(HttpExchange exchange) throws IOException {
      new BigInteger(1000, 3, new Random());
      String response = "Hello_" + i.incrementAndGet();
      LOG.log(Level.INFO, "(CPU->{0}) {1}", 
         new Object[] {cpus, response});
      exchange.sendResponseHeaders(200, response.length());
      OutputStream os = exchange.getResponseBody();
      os.write(response.getBytes());
      os.close();
   }
}

In order to use virtual threads in Java 19 we need to enable preview mode during compilation. With Maven we need to enable preview features using maven-compiler-plugin as shown below.

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-compiler-plugin</artifactId>
  <version>3.10.1</version>
  <configuration>
    <release>19</release>
    <compilerArgs>
      --enable-preview
    </compilerArgs>
  </configuration>
</plugin>

Install Knative on Kubernetes

This and the next step are not required to run the native application on Kubernetes. We will use Knative to easily autoscale the app in reaction to the volume of incoming traffic. In the next section, I’ll describe how to run a monitoring stack on Kubernetes.

The simplest way to install Knative on Kubernetes is with the kubectl command. We just need the Knative Serving component without any additional features. The Knative CLI (kn) is not required. We will deploy the application from the YAML manifest using Skaffold.

First, let’s install the required custom resources with the following command:

$ kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.8.3/serving-crds.yaml

Then, we can Install the core components of Knative Serving by running the command:

$ kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.8.3/serving-core.yaml

In order to access Knative services outside of the Kubernetes cluster we also need to install a networking layer. By default, Knative uses Kourier as an ingress. We can install the Kourier controller by running the following command.

$ kubectl apply -f https://github.com/knative/net-kourier/releases/download/knative-v1.8.1/kourier.yaml

Finally, let’s configure Knative Serving to use Kourier with the following command:

kubectl patch configmap/config-network \
  --namespace knative-serving \
  --type merge \
  --patch '{"data":{"ingress-class":"kourier.ingress.networking.knative.dev"}}'

If you don’t have an external domain configured or you are running Knative on the local cluster you need to configure DNS. Otherwise, you would have to run curl commands with a host header. Knative provides a Kubernetes Job that sets sslip.io as the default DNS suffix.

$ kubectl apply -f https://github.com/knative/serving/releases/download/knative-v1.8.3/serving-default-domain.yaml

The generated URL contains the name of the service, the namespace, and the address of your Kubernetes cluster. Since I’m running my service on the local Kubernetes cluster in the demo-sless namespace my service is available under the following address:

But before we deploy the sample app on Knative, let’s do some other things.

Install Prometheus Stack on Kubernetes

As I mentioned before, we can also install a monitoring stack on Kubernetes.

The simplest way to install it is with the kube-prometheus-stack Helm chart. The package contains Prometheus and Grafana. It also includes all required rules and dashboards to visualize the basic metrics of your Kubernetes cluster. Firstly, let’s add the Helm repository containing our chart:

$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

Then we can install the kube-prometheus-stack Helm chart in the prometheus namespace with the following command:

$ helm install prometheus-stack prometheus-community/kube-prometheus-stack  \
    -n prometheus \
    --create-namespace

If everything goes fine, you should see a similar list of Kubernetes services:

$ kubectl get svc -n prometheus
NAME                                        TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
alertmanager-operated                       ClusterIP   None             <none>        9093/TCP,9094/TCP,9094/UDP   11s
prometheus-operated                         ClusterIP   None             <none>        9090/TCP                     10s
prometheus-stack-grafana                    ClusterIP   10.96.218.142    <none>        80/TCP                       23s
prometheus-stack-kube-prom-alertmanager     ClusterIP   10.105.10.183    <none>        9093/TCP                     23s
prometheus-stack-kube-prom-operator         ClusterIP   10.98.190.230    <none>        443/TCP                      23s
prometheus-stack-kube-prom-prometheus       ClusterIP   10.111.158.146   <none>        9090/TCP                     23s
prometheus-stack-kube-state-metrics         ClusterIP   10.100.111.196   <none>        8080/TCP                     23s
prometheus-stack-prometheus-node-exporter   ClusterIP   10.102.39.238    <none>        9100/TCP                     23s

We will analyze Grafana dashboards with memory and CPU statistics. We can enable port-forward to access it locally on the defined port, for example 9080:

$ kubectl port-forward svc/prometheus-stack-grafana 9080:80 -n prometheus

The default username for Grafana is admin and password prom-operator.

We will create two panels in the custom Grafana dashboard. First of them will show the memory usage per single pod in the demo-sless namespace.

sum(container_memory_working_set_bytes{namespace="demo-sless"} / (1024 * 1024)) by (pod)

The second of them will show the average CPU usage per single pod in the demo-sless namespace. You can import both of these directly to Grafana from the k8s/grafana-dasboards.json file from the GitHub repo.

rate(container_cpu_usage_seconds_total{namespace="demo-sless"}[3m])

Build and Deploy a native Java Application

We have already created the sample app and then configured the Kubernetes environment. Now, we may proceed to the deployment phase. Our goal here is to simplify the process of building a native image and running it on Kubernetes as much as possible. Therefore, we will use Cloud Native Buildpacks and Skaffold. With Buildpacks we don’t need to have anything installed on our laptop besides Docker. Skaffold can be easily integrated with Buildpacks to automate the whole process of building and running the app on Kubernetes. You just need to install the skaffold CLI on your machine.

For building a native image of a Java application we may use Paketo Buildpacks. It provides a dedicated buildpack for GraalVM called Paketo GraalVM Buildpack. We should include it in the configuration using the paketo-buildpacks/graalvm name. Since Skaffold supports Buildpacks, we should set all the properties inside the skaffold.yaml file. We need to override some default settings with environment variables. First of all, we have to set the version of Java to 19 and enable preview features (virtual threads). The Kubernetes deployment manifest is available under the k8s/deployment.yaml path.

apiVersion: skaffold/v2beta29
kind: Config
metadata:
  name: sample-java-concurrency
build:
  artifacts:
  - image: piomin/sample-java-concurrency
    buildpacks:
      builder: paketobuildpacks/builder:base
      buildpacks:
        - paketo-buildpacks/graalvm
        - paketo-buildpacks/java-native-image
      env:
        - BP_NATIVE_IMAGE=true
        - BP_JVM_VERSION=19
        - BP_NATIVE_IMAGE_BUILD_ARGUMENTS=--enable-preview
  local:
    push: true
deploy:
  kubectl:
    manifests:
    - k8s/deployment.yaml

Knative simplifies not only autoscaling, but also Kubernetes manifests. Here’s the manifest for our sample app available in the k8s/deployment.yaml file. We need to define a single object Service containing details of the application container. We will change the autoscaling target from the default 200 concurrent requests to 80. It means that if a single instance of the app will process more than 80 requests simultaneously Knative will create a new instance of the app (or a pod – to be more precise). In order to enable virtual threads for our app we also need to set the environment variable THREAD_TYPE to virtual.

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: sample-java-concurrency
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/target: "80"
    spec:
      containers:
        - name: sample-java-concurrency
          image: piomin/sample-java-concurrency
          ports:
            - containerPort: 8080
          env:
            - name: THREAD_TYPE
              value: virtual
            - name: JAVA_TOOL_OPTIONS
              value: --enable-preview

Assuming you already installed Skaffold, the only thing you need to do is to run the following command:

$ skaffold run -n demo-sless

Or you can just deploy a ready image from my registry on Docker Hub. However, in that case, you need to change the image tag in the deployment.yaml manifest to virtual-native.

Once you deploy the app you can verify a list of Knative Service. The name of our target service is sample-java-concurrency. The address of the service is returned in the URL field.

$ kn service list -n demo-sless

Load Testing

We will run three testing scenarios today. In the first of them, we will test a standard compilation and a standard thread pool of 100 size. In the second of them, we will test a standard compilation with virtual threads. The final test will check native compilation in conjunction with virtual threads. In all these scenarios, we will set the same autoscaling target – 80 concurrent requests. I’m using the k6 tool for load tests. Each test scenario consists of 4 same steps. Each step takes 2 minutes. In the first step, we are simulating 50 users.

$ k6 run -u 50 -d 120s k6-test.js

Then, we are simulating 100 users.

$ k6 run -u 100 -d 120s k6-test.js

Finally, we run the test for 200 users twice. So, in total, there are four tests with 50, 100, 200, and 200 users, which takes 8 minutes.

$ k6 run -u 200 -d 120s k6-test.js

Let’s verify the results. By the way, here is our test for the k6 tool in javascript.

import http from 'k6/http';
import { check } from 'k6';

export default function () {
  const res = http.get(`http://sample-java-concurrency.demo-sless.127.0.0.1.sslip.io/example`);
  check(res, {
    'is status 200': (res) => res.status === 200,
    'body size is > 0': (r) => r.body.length > 0,
  });
}

Test for Standard Compilation and Threads

The diagram visible below shows memory usage at each phase of the test scenario. After simulating 200 users Knative scales up the number of instances. Theoretically, it should do that during 100 users test. But Knative measures incoming traffic at the level of the sidecar container inside the pod. The memory usage for the first instance is around ~900MB (it includes also sidecar container usage).

Here’s a similar view as before but for the CPU usage. The highest consumption was before autoscaling occurs at the level of ~1.2 core. Then, depending on the number of instances ranges from ~0.4 core to ~0.7 core. As I mentioned before, we are using a time-consuming BigInteger constructor to simulate CPU usage under a heavy load.

Here are the test results for 50 users. The application was able to process ~105k requests in 2 minutes. The highest processing time value was ~3 seconds.

Here are the test results for 100 users. The application was able to process ~130k requests in 2 minutes with an average response time of ~90ms.

Finally, we have results for 200 users test. The application was able to process ~135k requests in 2 minutes with an average response time of ~175ms. The failure threshold was at the level of 0.02%.

Test for Standard Compilation and Virtual Threads

The same as in the previous section, here’s the diagram that shows memory usage at each phase of the test scenario. After simulating 100 users Knative scales up the number of instances. Theoretically, it should run the third instance of the app for 200 users. The memory usage for the first instance is around ~850MB (it includes also sidecar container usage).

Here’s a similar view as before but for the CPU usage. The highest consumption was before autoscaling occurs at ~1.1 core. Then, depending on the number of instances ranges from ~0.3 core to ~0.7 core.

Here are the test results for 50 users. The application was able to process ~105k requests in 2 minutes. The highest processing time value was ~2.2 seconds.

Here are the test results for 100 users. The application was able to process ~115k requests in 2 minutes with an average response time of ~100ms.

Finally, we have results for 200 users test. The application was able to process ~135k requests in 2 minutes with an average response time of ~180ms. The failure threshold was at the level of 0.02%.

Test for Native Compilation and Virtual Threads

The same as in the previous section, here’s the diagram that shows memory usage at each phase of the test scenario. After simulating 100 users Knative scales up the number of instances. Theoretically, it should run the third instance of the app for 200 users (the third pod visible on the diagram was in fact in the Terminating phase for some time). The memory usage for the first instance is around ~50MB.

Here’s a similar view as before but for the CPU usage. The highest consumption was before autoscaling occurs at ~1.3 core. Then, depending on the number of instances ranges from ~0.3 core to ~0.9 core.

Here are the test results for 50 users. The application was able to process ~75k requests in 2 minutes. The highest processing time value was ~2 seconds.

Here are the test results for 100 users. The application was able to process ~85k requests in 2 minutes with an average response time of ~140ms

Finally, we have results for 200 users test. The application was able to process ~100k requests in 2 minutes with an average response time of ~240ms. Plus – there were no failures at the second 200 users attempt.

Summary

In this article, I tried to compare the behavior of the Java app for GraalVM native compilation with virtual threads on Kubernetes with a standard approach. There are several conclusions after running all described tests:

The post Native Java with GraalVM and Virtual Threads on Kubernetes appeared first on Piotr's TechBlog.

If you like articles about Java you can also read my post about unknown and useful Java features. It is not my first article about virtual threads. I have already written about Java 19 virtual threads and support for them in the Quarkus framework in this article.

Source Code

If you would like to try it by yourself, you may always take a look at my source code. In order to do that you need to clone my GitHub repository. After that, you should follow my instructions.

Prerequisites

In order to do the exercise on your laptop you need to have JDK 19+ and Maven installed.

Enable Virtual Threads

Even if you have Java 19 that’s not all. Since virtual threads are still a preview feature in Java 19 we need to enable it during compilation. With Maven we need to enable preview features using maven-compiler-plugin as shown below.

<build>
  <plugins>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-compiler-plugin</artifactId>
      <version>3.10.1</version>
      <configuration>
        <release>19</release>
        <compilerArgs>
          --enable-preview
        </compilerArgs>
      </configuration>
    </plugin>
  </plugins>
</build>

Create HTTP Server with Virtual Threads

We don’t need much to create an HTTP or even HTTPS server with Java. In Java API, an object called HttpServer allows us to achieve it very easily. Once we will create the server we can override a default thread executor with the setExecutor method. No matter which type of executor we choose, there is one requirement that must be fulfilled by our server. It needs to be able to handle 200 requests simultaneously. Therefore for standard Java threads, we will create a pool with a maximum size of 200. For virtual threads, there is no sense to create any pools. They do not consume many resources since they are related directly to the OS.

Let’s take a look at the fragment of code visible below. That’s our method for creating an HTTP server. It will listen on 8080 port (1) under the /example context path (2). The SimpleDelayedHandler object handles all incoming requests. Depending on the value of the withLock variable, it will simulate delay without locking (false) or with ReentrantLock (true). In order to simplify the exercise, we can switch between standard (4) and virtual threads executor (3) using the single boolean parameter. After setting all required parameters, we can start the server (5).

private static void runServer(boolean virtual, boolean withLock) 
      throws IOException {
   
   HttpServer httpServer = HttpServer
         .create(new InetSocketAddress(8080), 0); // (1)

   httpServer.createContext("/example", 
      new SimpleDelayedHandler(withLock)); // (2)
   
   if (virtual) {
      httpServer.setExecutor(
            Executors.newVirtualThreadPerTaskExecutor()
      ); // (3)
   } else {
      httpServer.setExecutor(
            Executors.newFixedThreadPool(200)
      ); // (4)
   }

   httpServer.start(); // (5)
}

Then, we need to call the runServer method from the main method. We will test 4 scenarios depending on the value of two input arguments. We will discuss it in the next section.

public static void main(String[] args) throws IOException {
   runServer(true, false);
}

After running the server you can make a test call using the following command:

$ curl http://localhost:8080/example

Build Test Scenarios

As mentioned before, we will run four test scenarios. In the first two of them, we just compare the performance of the HTTP server with the standard thread pool and with virtual threads. We will simulate the processing time with the Thread.sleep method. In the next two scenarios, we will simulate the usage of the workers’ pool (1). For example, it can be something similar to using a JDBC connection pool in the REST app. There are 50 workers handling 200 requests (2). Those workers will also delay the thread execution with the Thread.sleep method, but this time they will lock the thread at the beginning of execution and unlock it at the end.

Depending on the value of the withLock input argument we will use the workers’ pool (3) or we will just sleep the thread (4). In both cases, we will finally return the response Ping_ and incremented number (5) represented by the AtomicLong object. Here’s the implementation of our handler.

public class SimpleDelayedHandler implements HttpHandler {

   private final List<SimpleWork> workers = 
      new ArrayList<>(); // (1)
   private final int workersCount = 50;
   private final boolean withLock;
   AtomicLong id = new AtomicLong();

   public SimpleDelayedHandler(boolean withLock) {
      this.withLock = withLock;
      if (withLock) {
         for (int i = 0; i < workersCount; i++) { // (2)
            workers.add(new SimpleWork());
         }
      }
   }

   @Override
   public void handle(HttpExchange t) throws IOException {
      String response = null;
      if (withLock) {
         response = workers
            .get((int) (id.incrementAndGet() % workersCount))
            .doJob();
      } else {
         try {
            Thread.sleep(200);
         } catch (InterruptedException e) {
            throw new RuntimeException(e);
         }
         response = "Ping_" + id.incrementAndGet();
      }

      t.sendResponseHeaders(200, response.length());
      OutputStream os = t.getResponseBody();
      os.write(response.getBytes());
      os.close();
   }
}

Here’s the implementation of our worker. As you it also sleeps the thread (this time for 100 milliseconds). However, during that time it locks the object. Since we have 50 worker objects in the pool only 50 threads may use it at the same time. Others will wait until the lock will be released.

public class SimpleWork {

   AtomicLong id = new AtomicLong();
   ReentrantLock lock = new ReentrantLock();

   public String doJob() {
      String response = null;
      lock.lock();
      try {
         Thread.sleep(100);
         response = "Ping_" + id.incrementAndGet();
      } catch (InterruptedException e) {
         throw new RuntimeException();
      } finally {
         lock.unlock();
      }
      return response;
   }

}

Load Test for Java Virtual vs Standard Threads

Let’s begin with the first scenario. We will test standard threads without any locking workers simulation.

public static void main(String[] args) throws IOException {
   runServer(false, false);
}

We can make some warmup tests as shown below. I’m using the siege tool for load testing. We can define the number of concurrent threads and the number of repetitions.

In the right test, we will simulate 200 concurrent requests.

$  siege http://localhost:8080/example -c 200 -r 500

Let’s switch to the profiler view. Here you can see heap memory usage during the test. The usage is around 300 MB, while the reservation is more than 500 MB.

Let’s take a look at the telemetry view. As you see there are ~200 running threads.

Now, we will run the same test for the HTTP server using virtual threads. Let’s restart the application with the following arguments:

public static void main(String[] args) throws IOException {
   runServer(true, false);
}

Let’s switch to the profiler view once again. Here you can see heap memory usage during the test. You can compare it to the previous results. Now the usage is around 180 MB, while the reservation is around 300 MB.

Here’s the telemetry view. There are just some (~10) platform threads that “carry” virtual threads.

Here’s the visualization of the thread pool from the beginning of the test. As you see there are just some platform threads (CarrierThreads) and a lot of short-lived virtual threads.

Locks with Virtual Threads

In the end, let’s make the same checks, but this time with our worker objects pool that uses ReentrantLock to synchronize threads. Firstly, we will start the app with the following arguments to test standard threads.

public static void main(String[] args) throws IOException {
   runServer(false, true);
}

In fact, for standard threads, the main difference is in thread pool visualization. As you see, now many threads waiting for the lock to release. Our workers’ pool became a bottleneck for the app.

It doesn’t have any impact on RAM usage in comparison to the previous test for standard Java threads.

And finally the last scenario. Now, we will do the same check for virtual threads.

public static void main(String[] args) throws IOException {
   runServer(true, true);
}

Here are the results for memory usage.

In thread pool visualization we have just some “carrier” threads. As you see they are not “locked”.

In the “Thread Monitor” view there are a lot of virtual threads that wait a moment until the lock is released.

Of course, you can clone my GitHub repo and make your own tests. I was using JProfiler for memory and threads visualization.

Final Thoughts

Java virtual threads are really long-awaited feature. Since they are still in the preview status in Java 19 we need to wait for their wide adoption in the most popular Java libraries. Unfortunately, even Java 19 is not an LTS and if you are working for one of those companies that only use LTS versions you will have to wait for Java 21 which should be released in September 2023. Nevertheless, virtual threads can reduce the effort of writing, maintaining, and observing especially for high-throughput concurrent applications. We can use them as simply as the standard Java threads. The aim of this article was to show you how you can start with virtual threads to build your own solution, for example, an HTTP server. Then you can easily compare the difference in performance between standard and virtual threads.

The post Java HTTP Server and Virtual Threads appeared first on Piotr's TechBlog.