• Use Quarkus Dekorate extension to automatically generate Kubernetes manifests basing on the source code and configuration
• Build and push images to Docker registry with Jib extension
• Deploy your application on Kubernetes without any manually created YAML in one click
• Use Quarkus Kubernetes Config to inject configuration properties from ConfigMap

This guide is the second in series about Quarkus framework. If you are interested in the introduction to building Quarkus REST applications with Kotlin you may refer to my article Guide to Quarkus with Kotlin.

Source code

The source code with the sample Quarkus applications is available on GitHub. First, you need to clone the following repository: https://github.com/piomin/sample-quarkus-applications.git. Then, you need to go to the employee-service directory. We use the same repository as in my previous article about Quarkus.

1. Dependencies

Quarkus does not implement mechanisms for generating Kubernetes manifests, deploying them on the platform, or building images. It adds some logic to the existing tools. To enable extensions to Dekorate and Jib we should include the following dependencies.

<dependency>
   <groupId>io.quarkus</groupId>
   <artifactId>quarkus-kubernetes</artifactId>
</dependency>
<dependency>
   <groupId>io.quarkus</groupId>
   <artifactId>quarkus-container-image-jib</artifactId>
</dependency>

Jib builds optimized images for Java applications without a Docker daemon, and without deep mastery of Docker best-practices. It is available as plugins for Maven and Gradle and as a Java library. Dekorate is a Java library that makes generating and decorating Kubernetes manifests as simple as adding a dependency to your project. It may generate manifests basing on the source code, annotations, and configuration properties.

2. Preparation

In the first part of my guide to Kotlin, we were running our application in development mode with an embedded H2 database. In this part of the tutorial, we will integrate our application with Postgres deployed on Kubernetes. To do that we first need to change configuration settings for the data source. H2 database will be active only in dev and test mode. The configuration of Postgresql data source would be based on environment variables.

# kubernetes
quarkus.datasource.db-kind=postgresql
quarkus.datasource.username=${POSTGRES_USER}
quarkus.datasource.password=${POSTGRES_PASSWORD}
quarkus.datasource.jdbc.url=jdbc:postgresql://${POSTGRES_HOST}:5432/${POSTGRES_DB}
# dev
%dev.quarkus.datasource.db-kind=h2
%dev.quarkus.datasource.username=sa
%dev.quarkus.datasource.password=password
%dev.quarkus.datasource.jdbc.url=jdbc:h2:mem:testdb
# test
%test.quarkus.datasource.db-kind=h2
%test.quarkus.datasource.username=sa
%test.quarkus.datasource.password=password
%test.quarkus.datasource.jdbc.url=jdbc:h2:mem:testdb

3. Configure Kubernetes extension

With Quarkus Kubernetes extension we may customize the behavior of the manifest generator. To do that we need to provide configuration settings with the prefix quarkus.kubernetes.*. There are pretty many options like defining labels, annotations, environment variables, Secret and ConfigMap references, or mounting volumes. First, let’s take a look at the Secret and ConfigMap prepared for Postgres.

apiVersion: v1
kind: ConfigMap
metadata:
  name: postgres-config
  labels:
    app: postgres
data:
  POSTGRES_DB: quarkus
  POSTGRES_USER: quarkus
  POSTGRES_HOST: postgres
---
apiVersion: v1
kind: Secret
metadata:
  name: postgres-secret
  labels:
    app: postgres
data:
  POSTGRES_PASSWORD: YWRtaW4xMjM=

In this fragment of configuration, besides simple label and annotation, we are adding reference to all the keys inside postgres-config and postgres-secret.

quarkus.kubernetes.labels.app-type=demo
quarkus.kubernetes.annotations.app-type=demo
quarkus.kubernetes.env.secrets=postgres-secret
quarkus.kubernetes.env.configmaps=postgres-config

4. Build image and deploy

Before executing build and deploy we need to apply manifest with Postgres. Here’s Deployment definition of Postgres.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:latest
          imagePullPolicy: "IfNotPresent"
          ports:
            - containerPort: 5432
          env:
            - name: POSTGRES_DB
              valueFrom:
                configMapKeyRef:
                  key: POSTGRES_DB
                  name: postgres-config
            - name: POSTGRES_USER
              valueFrom:
                configMapKeyRef:
                  key: POSTGRES_USER
                  name: postgres-config
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  key: POSTGRES_PASSWORD
                  name: postgres-secret
          volumeMounts:
            - mountPath: /var/lib/postgresql/data
              name: postgredb
      volumes:
        - name: postgredb
          persistentVolumeClaim:
            claimName: postgres-claim

Let’s apply it on Kubernetes together with required ConfigMap, Secret, PersistenceVolume and PersistenceVolumeClaim. All the objects are available inside example repository in the file employee-service/k8s/postgres-deployment.yaml.

$ kubectl apply -f employee-service\k8s\postgresql-deployment.yaml

After deploying Postgres we may proceed to the main task. In order to build a Docker image with the application, we need to enable option quarkus.container-image.build during Maven build. If you also want to deploy and run a container with the application on your local Kubernetes instance you need to enable option quarkus.kubernetes.deploy.

$ clean package -Dquarkus.container-image.build=true -Dquarkus.kubernetes.deploy=true

If your Kubernetes cluster is located on the hosted cloud you should push the image to a remote Docker registry before deployment. To do that we should also activate option quarkus.container-image.push during Maven build. If you do not push to the default Docker registry you have to set parameter quarkus.container-image.registry=gcr.io inside the application.properties file. The only thing I need to set for building images is the following property, which is the same as my login to docker.io site.

quarkus.container-image.group=piomin

After running the required Maven command our application is deployed on Kubernetes. Let’s take a look at what happened during the Maven build. Here’s the fragment of logs during that build. You see that Quarkus extension generated two files kubernetes.yaml and kubernetes.json inside target/kubernetes directory. Then it proceeded to build a Docker image with our application. Because we didn’t specify any base image it takes a default one for Java 11 – fabric8/java-alpine-openjdk11-jre.

Let’s take a look on the Deployment definition automatically generated by Quarkus.

1. It adds some annotations like port or path to metrics endpoint used by Prometheus to monitor application and enabled scraping. It also adds Git commit id, repository URL, and our custom annotation defined in application.properties.
2. It adds labels with the application name, version (taken from Maven pom.xml), and our custom label app-type.
3. It injects Kubernetes namespace name into the container.
4. It injects the reference to the postgres-secret defined in application.properties.
5. It injects the reference to the postgres-config defined in application.properties.
6. The name of the image is automatically created. It is based on Maven artifactId and version.
7. The definition of liveness and readiness is generated if Maven module quarkus-smallrye-health is present

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations: # (1)
    prometheus.io/path: /metrics
    prometheus.io/port: 8080
    app.quarkus.io/commit-id: f6ae37288ed445177f23291c921c6099cfc58c6e
    app.quarkus.io/vcs-url: https://github.com/piomin/sample-quarkus-applications.git
    app.quarkus.io/build-timestamp: 2020-08-10 - 13:22:32 +0000
    app-type: demo
    prometheus.io/scrape: "true"
  labels: # (2)
    app.kubernetes.io/name: employee-service
    app.kubernetes.io/version: 1.1
    app-type: demo
  name: employee-service
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: employee-service
      app.kubernetes.io/version: 1.1
  template:
    metadata:
      annotations:
        prometheus.io/path: /metrics
        prometheus.io/port: 8080
        app.quarkus.io/commit-id: f6ae37288ed445177f23291c921c6099cfc58c6e
        app.quarkus.io/vcs-url: https://github.com/piomin/sample-quarkus-applications.git
        app.quarkus.io/build-timestamp: 2020-08-10 - 13:22:32 +0000
        app-type: demo
        prometheus.io/scrape: "true"
      labels:
        app.kubernetes.io/name: employee-service
        app.kubernetes.io/version: 1.1
        app-type: demo
    spec:
      containers:
      - env:
        - name: KUBERNETES_NAMESPACE # (3)
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        envFrom:
        - secretRef: # (4)
            name: postgres-secret
        - configMapRef: # (5)
            name: postgres-config
        image: piomin/employee-service:1.1 # (6)
        imagePullPolicy: IfNotPresent
        livenessProbe: # (7)
          failureThreshold: 3
          httpGet:
            path: /health/live
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 0
          periodSeconds: 30
          successThreshold: 1
          timeoutSeconds: 10
        name: employee-service
        ports:
        - containerPort: 8080
          name: http
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /health/ready
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 0
          periodSeconds: 30
          successThreshold: 1
          timeoutSeconds: 10
      serviceAccount: employee-service

Once the image has been built it is available in local registry. Quarkus automatically deploy it to the current cluster using already generated Kubernetes manifests.

Here’s the list of pods in default namespace.

5. Using Kubernetes Config extension

With Kubernetes Config extension you can use ConfigMap as a configuration source, without having to mount them into the pod with the application. To use that extension we need to include the following Maven dependency.

<dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-kubernetes-config</artifactId>
</dependency>

This extension works directly with Kubernetes API using fabric8 KubernetesClient. That’s why we should set the proper permissions for ServiceAccount. Fortunately, all the required configuration is automatically generated by Quarkus Kubernetes extension. The RoleBinding object is appied automatically if quarkus-kubernetes-config module is present.

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  annotations:
    prometheus.io/path: /metrics
    prometheus.io/port: 8080
    app.quarkus.io/commit-id: a5da459af01637657ebb0ec3a606eb53d13b8524
    app.quarkus.io/vcs-url: https://github.com/piomin/sample-quarkus-applications.git
    app.quarkus.io/build-timestamp: 2020-08-10 - 14:25:20 +0000
    app-type: demo
    prometheus.io/scrape: "true"
  labels:
    app.kubernetes.io/name: employee-service
    app.kubernetes.io/version: 1.1
    app-type: demo
  name: employee-service:view
roleRef:
  kind: ClusterRole
  apiGroup: rbac.authorization.k8s.io
  name: view
subjects:
- kind: ServiceAccount
  name: employee-service

Here’s our example ConfigMap that contains a single property property1.

apiVersion: v1
kind: ConfigMap
metadata:
  name: employee-config
data:
  application.properties: |-
    property1=one

The same property is defined inside application.properties available on the classpath, but there it has a different value.

property1=test

Before deploying a new version of application we need to add the following properties. First of them enables Kubernetes ConfigMap injection, while the second specifies the name of injected ConfigMap.

quarkus.kubernetes-config.enabled=true
quarkus.kubernetes-config.config-maps=employee-config

Finally we just need to implement a simple endpoint that injects and returns configuration property.

@ConfigProperty(name = "property1")
lateinit var property1: String

@GET
@Path("/property1")
fun property1(): String = property1

The properties obtained from the ConfigMap have a higher priority than any properties of the same name that are found in application.properties available on the classpath. Let’s test it.

The post Guide to Quarkus on Kubernetes appeared first on Piotr's TechBlog.

Source Code

By default, Spring Cloud Kubernetes also does not support ConfigMap or Secret versioning. That feature is available in my forked version of this library. For accessing it go to my GitHub repository: https://github.com/piomin/spring-cloud-kubernetes.

Concept

The concept around Kubernetes ConfigMap versioning is pretty easy. Especially with Spring Cloud Kubernetes. We are creating ConfigMap that needs to be labelled with app and version. The label app contains the name of Spring Boot application, that is configured by property spring.application.name. The label version indicates a number of application version, which is fetched by the library from property info.app.version. Both these properties need to be available inside application.yml orbootstrap.yml. Here’s sample version 1.0 of ConfigMap for application api-test.

apiVersion: v1
kind: ConfigMap
metadata:
  name: api-test-1
  labels:
    version: "1.0"
    app: api-test
data:
  application.yaml: |-
    property1: v1.0

We may maintain many versions of ConfigMap used by a given application just by changing name, labels and content. Here’s version 1.1 of ConfigMap for the same api-test application.

apiVersion: v1
kind: ConfigMap
metadata:
  name: api-test-2
  labels:
    version: "1.1"
    app: api-test
data:
  application.yaml: |-
    property1: v1.1

By default, Spring Cloud Kubernetes Config is able to inject ConfigMap to the application based on its metadata.name. I redefined this mechanism to base on the label app. Let’s take a look at some details.

Implementation

Now the question is – how to use that feature in our application. Of course, you need to include the required library to your Maven dependencies. Remember to use my forked version Spring Cloud Kubernetes Config instead of the official one.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-kubernetes-config</artifactId>
</dependency>

Then the bootstrap.yaml file in your application needs to contains at least the following properties to enable ConfigMap versioning mechanism.

spring:
  application:
    name: api-test
  cloud:
    kubernetes:
      config:
        enableVersioning: true
info:
  app:
    version: 1.0

Let’s take a look on the key fragment of versioning mechanism implementation inside Spring Cloud Kubernetes Config. If it is not enabled with property spring.cloud.kubernetes.config.enableVersioning it works based on metadata.name – the same as before changes. We are also checking if property info.app.version is set inside the application settings. If not, the versioning mechanism is being disabled. Otherwise we are checking all ConfigMaps in the selected namespace, and filtering them by app and version labels.

String version = environment.getProperty("info.app.version");
LOG.info("Get Config: versioning->" + enableVersioning + ", name->" + name + ", version->" + version);
Map result = new HashMap<>();
ConfigMap map = null;
if (!enableVersioning || version == null) {
   map = StringUtils.isEmpty(namespace) ? client.configMaps().withName(name).get()
         : client.configMaps().inNamespace(namespace).withName(name).get();
} else {
   Optional optMap = StringUtils.isEmpty(namespace)
      ? client.configMaps().list().getItems().stream()
         .filter(configMap -> configMap.getMetadata().getLabels()
            .containsKey("app")
               && configMap.getMetadata().getLabels().get("app").equals(name)
               && configMap.getMetadata().getLabels().containsKey("version")
               && configMap.getMetadata().getLabels().get("version").equals(version))
            .findFirst()
      : client.configMaps().inNamespace(namespace).list().getItems().stream()
         .filter(configMap -> configMap.getMetadata().getLabels()
            .containsKey("app")
               && configMap.getMetadata().getLabels().get("app").equals(name)
               && configMap.getMetadata().getLabels().containsKey("version")
               && configMap.getMetadata().getLabels().get("version").equals(version))
            .findFirst();
   if (optMap.isPresent()) {
      map = optMap.get();
   }
}

Demo of Kubernetes ConfigMap versioning

Our sample Spring Boot application is very simple. It just exposes a single HTTP endpoint that displays the value of property injected from ConfigMap as shown below.

@RestController
public class ApiController {

   @Value("${property1}")
   private String property1;

   @GetMapping("/property")
   public String getProperty1() {
      return property1;
   }
   
}

Of course it is using the currently described versioning mechanism. First let’s run an instance of Kubernetes locally using Kind (Kubernetes IN Docker). Here’s the command that creates the local cluster.

$ kind create cluster --config=cluster.yaml

To make it available under virtual address 192.168.99.100 we need to provide the following configuration file cluster.yaml.

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
networking:
  apiServerAddress: 192.168.99.100
  apiServerPort: 6443

After running the following command, kind is ready to use.

Now, let’s create two test ConfigMaps for our application with kubectl apply command. Let’s take a look at the result.

Then we should deploy our sample app. The only difference between two subsequent deployments is in info.app.version property, that is 1.0 for first deployment, and 1.1 for the second deployment. Here’s the Deployment manifest. As you see below it does not inject any ConfigMap or Secret using Kubernetes mechanisms.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-test
spec:
  selector:
    matchLabels:
      app: api-test
  template:
    metadata:
      labels:
        app: api-test
    spec:
      containers:
      - name: api-test
        image: piomin/api-test
        ports:
        - containerPort: 8080

In the newest version of our sample application I just changed the value of property info.app.version from 1.0 to 1.1.

spring:
  application:
    name: api-test
  cloud:
    kubernetes:
      config:
        enableVersioning: true
info:
  app:
    version: 1.1

Let’s call our test endpoint. It returns the value of property1 taken from ConfigMap labelled with 1.1.

$ curl http://localhost:8080/property
v1.1

Here’s the history of deployments for api-test application. We may rollback the version of deployment to the previous one (1) by using command kubectl rollout undo.

Now we may call our test endpoint one more time. As you can see below it returns v1.0, which is set as a property1 value in ConfigMap labelled with version 1.0.

$ curl http://localhost:8080/property
v1.0

Conclusion

To use described versioning mechanism on Kubernetes you just need to create ConfigMap labelled with properly, include forked Spring Cloud Kubernetes Config from my repository and add property info.app.version to the Spring Boot application properties.

The post Kubernetes ConfigMap Versioning for Spring Boot Apps appeared first on Piotr's TechBlog.