We will do the same thing as described in that article, but fully with the GitOps approach applied by Argo CD. The main goal here is to integrate Postgres with the Vault database secrets engine to generate database credentials dynamically and initialize the DB schema for the sample Spring Boot app. In order to achieve these goals, we are going to install two Kubernetes operators: Atlas and Vault Config. Atlas is a tool for managing the database schema as code. Its Kubernetes Operator allows us to define the schema and apply it to our database using the CRD objects. The Vault Config Operator provided by the Red Hat Community of Practice does a very similar thing but for Hashicorp Vault.

Source Code

If you would like to try it by yourself, you may always take a look at my source code. In order to do that you need to clone my GitHub repository. I will explain the structure of our sample in detail later. So after cloning the Git repository you should just follow my instructions

How It Works

Before we start, let’s describe our sample scenario. Thanks to the database secrets engine Vault integrates with Postgres and generates its credentials dynamically based on configured roles. On the other hand, our sample Spring Boot app integrates with Vault and uses its database engine to authenticate against Postgres. All the aspects of that scenario are managed in the GitOps style. Argo CD installs Vault, Postgres, and additional operators on Kubernetes via their Helm charts. Then, it applies all the required CRD objects to configure both Vault and Postgres. We keep the whole configuration in a single Git repository in the form of YAML manifests.

Argo CD prepares the configuration on Vault and creates a table on Postgres for the sample Spring Boot app. Our app integrates with Vault through the Spring Cloud Vault project. It also uses Spring Data JPA to interact with the database. Here’s the illustration of our scenario.

Install Argo CD on Kubernetes

Traditionally, we need to start our GitOps exercise by installing Argo CD on the Kubernetes cluster. Of course, we can do it using the Helm chart. In the first step, we need to add the following repository:

$ helm repo add argo https://argoproj.github.io/argo-helm

We will add one parameter to the argo-cm ConfigMap to ignore the MutatingWebhookConfiguration kind. This step is not necessary. It allows us to ignore the specific resource generated by one of the Helm charts used in the further steps. Thanks to that we will have everything in Argo CD in the “green” color Here’s the Helm values.yaml file with the required configuration:

configs:
  cm:
    resource.exclusions: |
      - apiGroups:
        - admissionregistration.k8s.io
        kinds:
        - MutatingWebhookConfiguration
        clusters:
        - "*"

Now, we can install the Argo CD in the argocd namespace using the configuration previously defined in the values.yml file:

$ helm install argo-cd argo/argo-cd \
    --version 6.7.8 \
    -n argo \
    --create-namespace

That’s not all. Since the Atlas operator is available in the OCI-type Helm repository, we need to apply the following Secret in the argocd namespace. By default, Argo CD doesn’t allow the OCI-type repo, so we need to include the enableOCI parameter in the definition.

apiVersion: v1
kind: Secret
metadata:
  name: ghcr-io-helm-oci
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repository
stringData:
  name: ariga
  url: ghcr.io/ariga
  enableOCI: "true"
  type: helm

Let’s take a look at the list of repositories in the Argo CD UI dashboard. You should see the “Successful” connection status.

Prepare Configuration Manifests for Argo CD

Config Repository Structure

Let me first explain the structure of our Git config repository. The additional configuration is stored in the apps directory. It includes the CRD objects required to initialize the database schema or Vault engines. In the bootstrap directory, we keep the values.yaml file for each Helm chart managed by Argo CD. It’s all that we need. The bootstrap-via-appset/bootstrap.yaml contains the definition of Argo CD ApplicationSet we need to apply to the Kubernetes cluster. This ApplicationSet will generate all required Argo CD applications responsible for installing the charts and creating CRD objects.

.
├── apps
│   ├── postgresql
│   │   ├── database.yaml
│   │   ├── policies.yaml
│   │   ├── roles.yaml
│   │   └── schema.yaml
│   └── vault
│       └── job.yaml
├── bootstrap
│   ├── values
│   │   ├── atlas
│   │   │   └── values.yaml
│   │   ├── cert-manager
│   │   │   └── values.yaml
│   │   ├── postgresql
│   │   │   └── values.yaml
│   │   ├── vault
│   │   │   └── values.yaml
│   │   └── vault-config-operator
│   │       └── values.yaml
└── bootstrap-via-appset
    └── bootstrap.yaml

Bootstrap with the Argo CD ApplicationSet

Let’s take a look at the ApplicationSet. It’s pretty interesting (I hope :)). I’m using here some relatively new Argo CD features like multiple sources (Argo CD 2.6) or application sets template patch (Argo CD 2.10). We need to generate an Argo CD Application per each tool we want to install on Kubernetes (1). In the generators section, we define parameters for Vault, PostgreSQL, Atlas Operator, Vault Config Operator, and Cert Manager (which is required by the Vault Config Operator). In the templatePatch section, we prepare a list of source repositories used by each Argo CD Application (2). There is always a Helm chart repo, which refers to our Git repository containing dedicated values.yaml files. For the Vault and PostgreSQL charts, we include another source containing CRDs or additional Kubernetes objects. We will discuss it later.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: bootstrap-config
  namespace: argocd
spec:
  goTemplate: true
  generators:
  - list:
      elements:
        - chart: vault
          name: vault
          repo: https://helm.releases.hashicorp.com
          revision: 0.27.0
          namespace: vault
          postInstall: true
        - chart: postgresql
          name: postgresql
          repo: https://charts.bitnami.com/bitnami
          revision: 12.12.10
          namespace: default
          postInstall: true
        - chart: cert-manager
          name: cert-manager
          repo: https://charts.jetstack.io
          revision: v1.14.4
          namespace: cert-manager
          postInstall: false
        - chart: vault-config-operator
          name: vault-config-operator
          repo: https://redhat-cop.github.io/vault-config-operator
          revision: v0.8.25
          namespace: vault-config-operator
          postInstall: false
        - chart: charts/atlas-operator
          name: atlas
          repo: ghcr.io/ariga
          revision: 0.4.2
          namespace: atlas
          postInstall: false
  template:
    metadata:
      name: '{{.name}}'
      annotations:
        argocd.argoproj.io/sync-wave: "1"
    spec:
      syncPolicy:
        automated: {}
        syncOptions:
          - CreateNamespace=true
      destination:
        namespace: '{{.namespace}}'
        server: https://kubernetes.default.svc
      project: default
  templatePatch: |
    spec:
      sources:
        - repoURL: '{{ .repo }}'
          chart: '{{ .chart }}'
          targetRevision: '{{ .revision }}'
          helm:
            valueFiles:
              - $values/bootstrap/values/{{ .name }}/values.yaml
        - repoURL: https://github.com/piomin/kubernetes-config-argocd.git
          targetRevision: HEAD
          ref: values
        {{- if .postInstall }}
        - repoURL: https://github.com/piomin/kubernetes-config-argocd.git
          targetRevision: HEAD
          path: apps/{{ .name }}
        {{- end }}

Once we apply the bootstrap-config ApplicationSet to the argocd namespace, all the magic just happens. You should see five applications in the Argo CD UI dashboard. All of them are automatically synchronized (Argo CD autoSync enabled) to the cluster. It does the whole job. Now, let’s analyze step-by-step what we have to put in that configuration.

The Argo CD ApplicationSet generates five applications for installing all required tools. Here’s the Application generated for installing Vault with Helm charts and applying an additional configuration stored in the apps/vault directory.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: vault
  namespace: argocd
spec:
  destination:
    namespace: vault
    server: https://kubernetes.default.svc
  project: default
  sources:
    - chart: vault
      helm:
        valueFiles:
          - $values/bootstrap/values/vault/values.yaml
      repoURL: https://helm.releases.hashicorp.com
      targetRevision: 0.27.0
    - ref: values
      repoURL: https://github.com/piomin/kubernetes-config-argocd.git
      targetRevision: HEAD
    - path: apps/vault
      repoURL: https://github.com/piomin/kubernetes-config-argocd.git
      targetRevision: HEAD
  syncPolicy:
    automated: {}
    syncOptions:
      - CreateNamespace=true

Configure Vault on Kubernetes

Customize Helm Charts

Let’s take a look at the Vault values.yaml file. We run it in the development mode (single, in-memory node, no unseal needed). We will also enable the UI dashboard.

server:
  dev:
    enabled: true
ui:
  enabled: true

With the parameters visible above Argo CD installs Vault in the vault namespace. Here’s a list of running pods:

$ kubectl get po -n vault
NAME                                    READY   STATUS      RESTARTS      AGE
vault-0                                 1/1     Running     0            1h
vault-agent-injector-7f7f68d457-fvsd2   1/1     Running     0            1h

It also exposes Vault API under the 8200 port in the vault Kubernetes Service.

$ kubectl get svc -n vault
NAME                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)             AGE
vault                      ClusterIP   10.110.69.159    <none>        8200/TCP,8201/TCP   21h
vault-agent-injector-svc   ClusterIP   10.111.24.183    <none>        443/TCP             21h
vault-internal             ClusterIP   None             <none>        8200/TCP,8201/TCP   21h
vault-ui                   ClusterIP   10.110.160.239   <none>        8200/TCP            21h

For the Vault Config Operator, we need to override the default address of Vault API to vault.vault.svc:8200 (an a). In order to do that, we need to set the VAULT_ADDR env variable in the values.yaml file. We also disable Prometheus monitoring and enable integration with Cert Manager. Thanks to “cert-manager” we don’t need to generate any certificates or keys manually.

enableMonitoring: false
enableCertManager: true
env:
  - name: VAULT_ADDR
    value: http://vault.vault:8200

Enable Vault Config Operator

The Vault Config Operator needs to authenticate against Vault API using Kubernetes Authentication. So we need to configure a root Kubernetes Authentication mount point and role. Then we can create more roles or other Vault objects via the operator. Here’s the Kubernetes Job responsible for configuring Kubernetes mount point and role. It uses the Vault image and the vault CLI available inside that image. As you see, it creates the vault-admin role allowed in the default namespace.

apiVersion: batch/v1
kind: Job
metadata:
  name: vault-admin-initializer
  annotations:
    argocd.argoproj.io/sync-wave: "3"
spec:
  template:
    spec:
      containers:
        - name: vault-admin-initializer
          image: hashicorp/vault:1.15.2
          env:
            - name: VAULT_ADDR
              value: http://vault.vault.svc:8200
          command:
            - /bin/sh
            - -c
            - |
              export VAULT_TOKEN=root
              sleep 10
              vault auth enable kubernetes
              vault secrets enable database
              vault write auth/kubernetes/config kubernetes_host=https://kubernetes.default.svc:443 kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
              vault write auth/kubernetes/role/vault-admin bound_service_account_names=default bound_service_account_namespaces=default policies=vault-admin ttl=1h
              vault policy write vault-admin - <<EOF
                path "/*" {
                  capabilities = ["create", "read", "update", "delete", "list","sudo"]
                }          
              EOF
      restartPolicy: Never

Argo CD applies such a Job after installing the Vault chart.

$ kubectl get job -n vault
NAME                      COMPLETIONS   DURATION   AGE
vault-admin-initializer   1/1           15s        1h

Configure Vault via CRDs

Once a root Kubernetes authentication is ready, we can proceed to the CRD object creation. In the first step, we create objects responsible for configuring a connection to the Postgres database. In the DatabaseSecretEngineConfig we set the connection URL, credentials, and the name of a Vault plugin used to interact with the database (postgresql-database-plugin). We also define a list of allowed roles (postgresql-default-role). In the next step, we define the postgresql-default-role DatabaseSecretEngineRole object. Of course, the name of the role should be the same as the name passed in the allowedRoles list in the previous step. The role defines a target database connection name in Vault and the SQL statement for creating new users with privileges.

kind: DatabaseSecretEngineConfig
apiVersion: redhatcop.redhat.io/v1alpha1
metadata:
  name: postgresql-database-config
  annotations:
    argocd.argoproj.io/sync-wave: "3"
    argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
spec:
  allowedRoles:
    - postgresql-default-role
  authentication:
    path: kubernetes
    role: vault-admin
  connectionURL: 'postgresql://{{username}}:{{password}}@postgresql.default:5432?sslmode=disable'
  path: database
  pluginName: postgresql-database-plugin
  rootCredentials:
    passwordKey: postgres-password
    secret:
      name: postgresql
  username: postgres
---
apiVersion: redhatcop.redhat.io/v1alpha1
kind: DatabaseSecretEngineRole
metadata:
  name: postgresql-default-role
  annotations:
    argocd.argoproj.io/sync-wave: "3"
    argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
spec:
  creationStatements:
    - CREATE ROLE "{{name}}" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; GRANT SELECT ON ALL TABLES IN SCHEMA public TO "{{name}}"; GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO "{{name}}";
  maxTTL: 10m0s
  defaultTTL: 1m0s
  authentication:
    path: kubernetes
    role: vault-admin
  dBName: postgresql-database-config
  path: database

Once Argo CD applies both DatabaseSecretEngineConfig and DatabaseSecretEngineRole objects, we can verify it works fine by generating database credentials using the vault read command. We need to pass the name of the previously created role (postgresql-default-role). Our sample app will do the same thing but through the Spring Cloud Vault module.

Finally, we can create a policy and role for our sample Spring Boot. The policy requires only the privilege to generate new credentials:

kind: Policy
apiVersion: redhatcop.redhat.io/v1alpha1
metadata:
  name: database-creds-view
  annotations:
    argocd.argoproj.io/sync-wave: "3"
    argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
spec:
  authentication:
    path: kubernetes
    role: vault-admin
  policy: |
    path "database/creds/default" {
      capabilities = ["read"]
    }

Now, we have everything to proceed to the last step in this section. We need to create a Vault role with the Kubernetes authentication method dedicated to our sample app. In this role, we set the name and location of the Kubernetes ServiceAccount and the name of the Vault policy created in the previous step.

kind: KubernetesAuthEngineRole
apiVersion: redhatcop.redhat.io/v1alpha1
metadata:
  name: database-engine-creds-role
  annotations:
    argocd.argoproj.io/sync-wave: "3"
    argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
spec:
  authentication:
    path: kubernetes
    role: vault-admin
  path: kubernetes
  policies:
    - database-creds-view
  targetServiceAccounts:
    - default
  targetNamespaces:
    targetNamespaces:
      - default

Managing Postgres Schema with Atlas Operator

Finally, we can proceed to the last step in the configuration part. We will use the AtlasSchema CRD object to configure the database schema for our sample app. The object contains two sections: credentials and schema. In the credentials section, we refer to the PostgreSQL Secret to obtain a password. In the schema section, we create the person table with the id primary key.

apiVersion: db.atlasgo.io/v1alpha1
kind: AtlasSchema
metadata:
  name: sample-spring-cloud-vault
  annotations:
    argocd.argoproj.io/sync-wave: "4"
    argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true
spec:
  credentials:
    scheme: postgres
    host: postgresql.default
    user: postgres
    passwordFrom:
      secretKeyRef:
        key: postgres-password
        name: postgresql
    database: postgres
    port: 5432
    parameters:
      sslmode: disable
  schema:
    sql: |
      create table person (
        id serial primary key,
        name varchar(255),
        gender varchar(255),
        age int,
        external_id int
      );

Here’s the corresponding app @Entity model class in the sample Spring Boot app.

@Entity
public class Person {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Integer id;
    private String name;
    private int age;
    @Enumerated(EnumType.STRING)
    private Gender gender;
    private Integer externalId;   
    
   // GETTERS AND SETTERS ...
   
}

Once Argo CD applies the AtlasSchema object, we can verify its status. As you see, it has been successfully executed on the target database.

We can log in to the database using psql CLI and verify that the person table exists in the postgres database:

Run Sample Spring Boot App

Dependencies

For this demo, I created a simple Spring Boot application. It exposes REST API and connects to the PostgreSQL database. It uses Spring Data JPA to interact with the database. Here are the most important dependencies of our app in the Maven pom.xml:

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-bootstrap</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-vault-config-databases</artifactId>
</dependency>
<dependency>
  <groupId>org.postgresql</groupId>
  <artifactId>postgresql</artifactId>
  <scope>runtime</scope>
</dependency>

The first of them enables bootstrap.yml processing on the application startup. The third one includes Spring Cloud Vault Database engine support.

Integrate with Vault using Spring Cloud Vault

The only thing we need to do is to provide the right configuration settings. Here’s the minimal set of the required dependencies to make it work without any errors. The following configuration is provided in the bootstrap.yml file:

spring:
  application:
    name: sample-db-vault
  datasource:
    url: jdbc:postgresql://postgresql:5432/postgres #(1)
  jpa:
    hibernate:
      ddl-auto: update
  cloud:
    vault:
      config.lifecycle: #(2)
        enabled: true
        min-renewal: 10s
        expiry-threshold: 30s
      kv.enabled: false #(3)
      uri: http://vault.vault:8200 #(4)
      authentication: KUBERNETES #(5)
      postgresql: #(6)
        enabled: true
        role: postgresql-default-role
        backend: database
      kubernetes: #(7)
        role: database-engine-creds-role

Let’s analyze the configuration visible above in the details:

(1) Firstly, we need to set the database connection URL without any credentials. Our application uses standard properties for authentication against the database (spring.datasource.username and spring.datasource.password). Thanks to that, we don’t need to do anything else

(2) As you probably remember, the maximum TTL for the database lease is 10 minutes. We enable lease renewal every 30 seconds. Just for the demo purpose. You will see that Spring Cloud Vault will create new credentials in PostgreSQL every 30 seconds, and the application still works without any errors

(3) Vault KV is not needed here, since I’m using only the database engine

(4) The application is going to be deployed in the default namespace, while Vault is running in the vault namespace. So, the address of Vault should include the namespace name

(5) (7) Our application uses the Kubernetes authentication method to access Vault. We just need to set the role name, which is database-engine-creds-role. All other settings should be left with the default values

(6) We also need to enable postgres database backend support. The name of the backend in Vault is database and the name of the Vault role used for that engine is postgresql-default-role.

Run the App on Kubernetes

Finally, we can run our sample app on Kubernetes by applying the following YAML manifest:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-app-deployment
spec:
  selector:
    matchLabels:
      app: sample-app
  template:
    metadata:
      labels:
        app: sample-app
    spec:
      containers:
        - name: sample-app
          image: piomin/sample-app:1.0-gitops
          ports:
            - containerPort: 8080
      serviceAccountName: default
---
apiVersion: v1
kind: Service
metadata:
  name: sample-app
spec:
  type: ClusterIP
  selector:
    app: sample-app
  ports:
  - port: 8080

Our app exposes REST API under the /persons path. We can easily test it with curl after enabling port forwarding as shown below:

$ kubectl port-forward svc/sample-app 8080:8080
$ curl http://localhost:8080/persons

Final Thoughts

This article proves that we can effectively configure and manage tools like Postgres database or Hashicorp Vault on Kubernetes with Argo CD. The database schema or Vault configuration can be stored in the Git repository in the form of YAML manifests thanks to Atlas and Vault Config Kubernetes operators. Argo CD applies all required CRDs automatically, which results in the integration between Vault, Postgres, and our sample Spring Boot app.

The post GitOps on Kubernetes for Postgres and Vault with Argo CD appeared first on Piotr's TechBlog.

Source Code

If you would like to try it by yourself, you can always take a look at my source code. In order to do that, you need to clone my GitHub repository. It contains several different Quarkus apps. For the current article, please refer to the person-grpc-service app. You should go to that directory and then just follow my instructions

Generate Model Classes and Services for gRPC

In the first step, we will generate model classes and gRPC services using the .proto manifests. The same as for the Spring Boot app we will create the Protobuf manifest and place it inside the src/main/proto directory. We need to include some additional Protobuf schemas to use the google.protobuf.* package (1). Our gRPC service will provide methods for searching persons using various criteria and a single method for adding a new person (2). Those methods will use primitives from the google.protobuf.* package and model classes defined inside the .proto file as messages. The Person message represents a single model class. It contains three fields: id, name, age and gender (3). The Persons message contains a list of Person objects (4). The gender field inside the Person message is an enum (5).

syntax = "proto3";

package model;

option java_package = "pl.piomin.quarkus.grpc.model";
option java_outer_classname = "PersonProto";

// (1)
import "google/protobuf/empty.proto";
import "google/protobuf/wrappers.proto";

// (2)
service PersonsService {
  rpc FindByName(google.protobuf.StringValue) returns (Persons) {}
  rpc FindByAge(google.protobuf.Int32Value) returns (Persons) {}
  rpc FindById(google.protobuf.Int64Value) returns (Person) {}
  rpc FindAll(google.protobuf.Empty) returns (Persons) {}
  rpc AddPerson(Person) returns (Person) {}
}

// (3)
message Person {
  int64 id = 1;
  string name = 2;
  int32 age = 3;
  Gender gender = 4;
}

// (4)
message Persons {
  repeated Person person = 1;
}

// (5)
enum Gender {
  MALE = 0;
  FEMALE = 1;
}

Once again I will refer here to my previous article about Spring Boot for gRPC. With Quarkus we don’t need to include any Maven plugin responsible for generating Java classes. This feature is automatically included by the Quarkus gRPC module. This saves a lot of time – especially at the beginning with Java gRPC (I know this from my own experience). Of course, if you want to override a default behavior, you can include your own plugin. Here’s the example from the official Quarkus docs.

Now, we just need to build the project with the mvn clean package command. It will automatically generate the following list of classes (I highlighted the two most important for us):

By default, Quarkus generates our gRPC classes in the target/generated-sources/grpc directory. Let’s include it as the source directory using the build-helper-maven-plugin Maven plugin.

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>build-helper-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>add-source</id>
      <phase>generate-sources</phase>
      <goals>
        <goal>add-source</goal>
      </goals>
      <configuration>
        <sources>
          <source>target/generated-sources/grpc</source>
        </sources>
      </configuration>
    </execution>
  </executions>
</plugin>

Dependencies

By default, the quarkus-grpc extension relies on the reactive programming model. Therefore we will include a reactive database driver and a reactive version of the Panache Hibernate module. It is also worth adding the RESTEasy Reactive module. Thanks to that, we will be able to run e.g. Quarkus Dev UI, which also provides useful features for the gRPC services. Of course, we are going to write some JUnit tests to verify core functionalities, so that’s why quarkus-junit5 is included in the Maven pom.xml.

<dependencies>
  <dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-resteasy-reactive-jackson</artifactId>
  </dependency>
  <dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-grpc</artifactId>
  </dependency>
  <dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-hibernate-reactive-panache</artifactId>
  </dependency>
  <dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-reactive-pg-client</artifactId>
  </dependency>
  <dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-junit5</artifactId>
    <scope>test</scope>
  </dependency>
</dependencies>

Using the Quarkus gRPC Extension

Once we included all the required libraries and generated classes for Protobuf integration we can with the implementation. Let’s begin with the persistence layer. We already have message classes generated, but we still need to create an entity class. Here’s the PersonEntity class. We will take advantage of PanacheEntity, and thanks to that we don’t need to e.g. define getters/setters.

@Entity
public class PersonEntity extends PanacheEntity {

    public String name;
    public int age;
    public Gender gender;

}

After that, we can create the repository class. It implements the reactive version of the PanacheRepository interface. The important thing is that the PanacheRepository interface returns Mutiny Uni objects. Therefore, if need to add some additional methods inside the repository we should also return results as the Uni object.

@ApplicationScoped
public class PersonRepository implements PanacheRepository<PersonEntity> {

    public Uni<List<PersonEntity>> findByName(String name){
        return find("name", name).list();
    }

    public Uni<List<PersonEntity>> findByAge(int age){
        return find("age", age).list();
    }
}

Finally, we can proceed to the most important element in our tutorial – the implementation of the gRPC service. The implementation class should be annotated with @GrpcService (1). In order to interact with the database reactively I also had to annotate it with @WithSession (2). It creates a Mutiny session for each method gRPC method inside. We can also register optional interceptors, for example, to log incoming requests and outgoing responses (3). Our service class needs to implement the PersonsService interface generated by the Quarkus gRPC extension (4).

Then we can go inside the class. In the first step, we will inject the repository bean (5). After that, we will override all the gRPC methods generated from the .proto manifest (6). All those methods use the PersonRepository bean to interact with the database. Once they obtain a result it is required to convert it to the Protobuf object (7). When we add a new person to the database, we need to do it in the transaction scope (8).

@GrpcService // (1)
@WithSession // (2)
@RegisterInterceptor(LogInterceptor.class) // (3)
public class PersonsServiceImpl implements PersonsService { // (4)

    private PersonRepository repository; // (5)

    public PersonsServiceImpl(PersonRepository repository) {
        this.repository = repository;
    }

    @Override // (6)
    public Uni<PersonProto.Persons> findByName(StringValue request) {
        return repository.findByName(request.getValue())
                .map(this::mapToPersons); // (7)
    }

    @Override
    public Uni<PersonProto.Persons> findByAge(Int32Value request) {
        return repository.findByAge(request.getValue())
                .map(this::mapToPersons);
    }

    @Override
    public Uni<PersonProto.Person> findById(Int64Value request) {
        return repository.findById(request.getValue())
                .map(this::mapToPerson);
    }

    @Override
    public Uni<PersonProto.Persons> findAll(Empty request) {
        return repository.findAll().list()
                .map(this::mapToPersons);
    }

    @Override
    @WithTransaction // (8)
    public Uni<PersonProto.Person> addPerson(PersonProto.Person request) {
        PersonEntity entity = new PersonEntity();
        entity.age = request.getAge();
        entity.name = request.getName();
        entity.gender = Gender.valueOf(request.getGender().name());
        return repository.persist(entity)
           .map(personEntity -> mapToPerson(entity));
    }

    private PersonProto.Persons mapToPersons(List<PersonEntity> list) {
        PersonProto.Persons.Builder builder = 
           PersonProto.Persons.newBuilder();
        list.forEach(p -> builder.addPerson(mapToPerson(p)));
        return builder.build();
    }

    private PersonProto.Person mapToPerson(PersonEntity entity) {
        PersonProto.Person.Builder builder = 
           PersonProto.Person.newBuilder();
        if (entity != null) {
            return builder.setAge(entity.age)
                    .setName(entity.name)
                    .setId(entity.id)
                    .setGender(PersonProto.Gender
                       .valueOf(entity.gender.name()))
                    .build();
        } else {
            return null;
        }
    }
}

At the end let’s discuss the optional step. With Quarkus gRPC we can implement a server interceptor by creating the @ApplicationScoped bean implementing ServerInterceptor. In order to apply an interceptor to all exposed services, we should annotate it with @GlobalInterceptor. In our case, the interceptor is registered to a single service with @RegisterInterceptor annotation. Then we will the SimpleForwardingServerCall class to log outgoing messages, and SimpleForwardingServerCallListener to log outgoing messages.

@ApplicationScoped
public class LogInterceptor  implements ServerInterceptor {

    Logger log;

    public LogInterceptor(Logger log) {
        this.log = log;
    }

    @Override
    public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(ServerCall<ReqT, RespT> call, Metadata headers, ServerCallHandler<ReqT, RespT> next) {

        ServerCall<ReqT, RespT> listener = new ForwardingServerCall.SimpleForwardingServerCall<ReqT, RespT>(call) {
            @Override
            public void sendMessage(RespT message) {
                log.infof("[Sending message] %s",  message.toString().replaceAll("\n", " "));
                super.sendMessage(message);
            }
        };

        return new ForwardingServerCallListener.SimpleForwardingServerCallListener<ReqT>(next.startCall(listener, headers)) {
            @Override
            public void onMessage(ReqT message) {
                log.infof("[Received message] %s", message.toString().replaceAll("\n", " "));
                super.onMessage(message);
            }
        };
    }
}

Quarkus JUnit gRPC Tests

Of course, there must be tests in our app. Thanks to the Quarkus dev services and built-in integration with Testcontainers we don’t have to take care of starting the database. Just remember to run the Docker daemon on your laptop. After annotating the test class with @QuarkusTest we can inject the gRPC client generated from the .proto manifest with @GrpcClient (1). Then, we can use the PersonsService interface to call our gRPCS methods. By default, Quarkus starts gRPC endpoints on 9000 port. The Quarkus gRPC client works in the reactive mode, so we will leverage the CompletableFuture class to obtain and verify results in the tests (2).

@QuarkusTest
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class PersonsServiceTests {

    static Long newId;

    @GrpcClient // (1)
    PersonsService client;

    @Test
    @Order(1)
    void shouldAddNew() throws ExecutionException, InterruptedException, TimeoutException {
        CompletableFuture<Long> message = new CompletableFuture<>(); // (2)
        client.addPerson(PersonProto.Person.newBuilder()
                        .setName("Test")
                        .setAge(20)
                        .setGender(PersonProto.Gender.MALE)
                        .build())
                .subscribe().with(res -> message.complete(res.getId()));
        Long id = message.get(1, TimeUnit.SECONDS);
        assertNotNull(id);
        newId = id;
    }

    @Test
    @Order(2)
    void shouldFindAll() throws ExecutionException, InterruptedException, TimeoutException {
        CompletableFuture<List<PersonProto.Person>> message = new CompletableFuture<>();
        client.findAll(Empty.newBuilder().build())
                .subscribe().with(res -> message.complete(res.getPersonList()));
        List<PersonProto.Person> list = message.get(1, TimeUnit.SECONDS);
        assertNotNull(list);
        assertFalse(list.isEmpty());
    }

    @Test
    @Order(2)
    void shouldFindById() throws ExecutionException, InterruptedException, TimeoutException {
        CompletableFuture<PersonProto.Person> message = new CompletableFuture<>();
        client.findById(Int64Value.newBuilder().setValue(newId).build())
                .subscribe().with(message::complete);
        PersonProto.Person p = message.get(1, TimeUnit.SECONDS);
        assertNotNull(p);
        assertEquals("Test", p.getName());
        assertEquals(newId, p.getId());
    }

    @Test
    @Order(2)
    void shouldFindByAge() throws ExecutionException, InterruptedException, TimeoutException {
        CompletableFuture<PersonProto.Persons> message = new CompletableFuture<>();
        client.findByAge(Int32Value.newBuilder().setValue(20).build())
                .subscribe().with(message::complete);
        PersonProto.Persons p = message.get(1, TimeUnit.SECONDS);
        assertNotNull(p);
        assertEquals(1, p.getPersonCount());
    }

    @Test
    @Order(2)
    void shouldFindByName() throws ExecutionException, InterruptedException, TimeoutException {
        CompletableFuture<PersonProto.Persons> message = new CompletableFuture<>();
        client.findByName(StringValue.newBuilder().setValue("Test").build())
                .subscribe().with(message::complete);
        PersonProto.Persons p = message.get(1, TimeUnit.SECONDS);
        assertNotNull(p);
        assertEquals(1, p.getPersonCount());
    }
}

Running and Testing Quarkus Locally

Let’s run our Quarkus app locally in the dev mode:

$ mvn quarkus:dev

Quarkus will start the Postgres database and expose gRPC services on the port 9000:

We can go to the Quarkus Dev UI console. It is available under the address http://localhost:8080/q/dev-ui. Once you do it, you should see the gRPC tile as shown below:

Click the Services link inside that tile. You will be redirected to the site with a list of available gRPC services. After that, we may expand the row with the model.PersonsService name. It allows to perform a test call of the selected gRPC method. Let’s choose the AddPerson tab. Then we can insert the request in the JSON format and send it to the server by clicking the Send button.

If you don’t like UI interfaces you can use the grpcurl CLI instead. By default, the gRPC server is started on a port 9000 in the PLAINTEXT mode. In order to print a list of available services we need to execute the following command:

$ grpcurl --plaintext localhost:9000 list
grpc.health.v1.Health
model.PersonsService

Then, let’s print the list of methods exposed by the model.PersonsService:

$ grpcurl --plaintext localhost:9000 list model.PersonsService
model.PersonsService.AddPerson
model.PersonsService.FindAll
model.PersonsService.FindByAge
model.PersonsService.FindById
model.PersonsService.FindByName

We can also print the details about each method by using the describe keyword in the command:

$ grpcurl --plaintext localhost:9000 describe model.PersonsService.FindById
model.PersonsService.FindById is a method:
rpc FindById ( .google.protobuf.Int64Value ) returns ( .model.Person );

Finally, let’s call the endpoint described with the command visible above. We are going to find the previously added person (via UI) by the id field value.

$ grpcurl --plaintext -d '1' localhost:9000 model.PersonsService.FindById
{
  "id": "1",
  "name": "Test",
  "age": 20,
  "gender": "FEMALE"
}

Running Quarkus gRPC on OpenShift

In the last step, we will run our app on OpenShift and try to interact with the gRPC service through the OpenShift Route. Fortunately, we can leverage Quarkus extension for OpenShift. We can include the quarkus-openshift dependency in the optional Maven profile.

<profile>
  <id>openshift</id>
  <activation>
  <property>
    <name>openshift</name>
  </property>
  </activation>
  <dependencies>
    <dependency>
      <groupId>io.quarkus</groupId>
      <artifactId>quarkus-openshift</artifactId>
    </dependency>
  </dependencies>
  <properties>
    <quarkus.kubernetes.deploy>true</quarkus.kubernetes.deploy>
    <quarkus.profile>openshift</quarkus.profile>
  </properties>
</profile>

Once we run a Maven build with the option -Popenshift it will activate the profile. Thanks to that Quarkus will handle all things required for building image and running it on the target cluster.

$ mvn clean package -Popenshift -DskipTests

In order to test our gRPC service via the OpenShift Route we need to expose it over SSL/TLS. Here’s the secret that contains both the certificate and private key. It was issued by the cert-manager for the the Route hostname.

We need to configure several things in the Quarkus app. We didn’t have to take care of it when running in the dev mode. First of all, we need to provide Postgres database connection settings (1). Thanks to the Quarkus OpenShift module we can generate YAML manifest using configuration properties. The database is available on OpenShift under the person-db address. The rest of the credentials can be taken from the person-db secret (2).

Then we will mount a secret with a TLS certificate and private key to the DeploymentConfig (3). It will be available inside the pod under the /mnt/app-secret path. Then, we can enable SSL for the gRPC service by setting the certificate and private key for the server (4). We should also enable the reflection service, to allow tools like grpcurl to interact with our services. Once the SSL/TLS for the service is configured, we can create the passthrough Route that exposes it outside the OpenShift cluster (5).

# (1)
%prod.quarkus.datasource.username = ${POSTGRES_USER}
%prod.quarkus.datasource.password = ${POSTGRES_PASSWORD}
%prod.quarkus.datasource.reactive.url = vertx-reactive:postgresql://person-db:5432/${POSTGRES_DB}

# (2)
quarkus.openshift.env.mapping.postgres_user.from-secret = person-db
quarkus.openshift.env.mapping.postgres_user.with-key = database-user
quarkus.openshift.env.mapping.postgres_password.from-secret = person-db
quarkus.openshift.env.mapping.postgres_password.with-key = database-password
quarkus.openshift.env.mapping.postgres_db.from-secret = person-db
quarkus.openshift.env.mapping.postgres_db.with-key = database-name

# (3)
quarkus.openshift.app-secret = secure-callme-cert

# (4)
%prod.quarkus.grpc.server.ssl.certificate = /mnt/app-secret/tls.crt
%prod.quarkus.grpc.server.ssl.key = /mnt/app-secret/tls.key
%prod.quarkus.grpc.server.enable-reflection-service = true

# (5)
quarkus.openshift.route.expose = true
quarkus.openshift.route.target-port = grpc
quarkus.openshift.route.tls.termination = passthrough

quarkus.kubernetes-client.trust-certs = true
%openshift.quarkus.container-image.group = demo-grpc
%openshift.quarkus.container-image.registry = image-registry.openshift-image-registry.svc:5000

Here are the logs of the Quarkus app after running on OpenShift. As you see the gRPC server enables reflection service and is exposed over SSL/TLS.

Let’s also display information about our app Route. We will interact with the person-grpc-service-demo-grpc.apps-crc.testing address using the grpcurl tool.

$ oc get route
NAME                  HOST/PORT                                        PATH   SERVICES              PORT   TERMINATION   WILDCARD
person-grpc-service   person-grpc-service-demo-grpc.apps-crc.testing          person-grpc-service   grpc   passthrough   None

Finally, we have to set the key and certificates used for securing the gRPC server as the parameters of grpcurl. It should work properly as shown below. You can also try other gRPC commands used previously for the app running locally in dev mode.

$ grpcurl -key grpc.key -cert grpc.crt -cacert ca.crt \
  person-grpc-service-demo-grpc.apps-crc.testing:443 list
grpc.health.v1.Health
model.PersonsService

Final Thoughts

Quarkus simplifies the development of gRPC services. For example, it allows us to easily generate Java classes from .proto files or configure SSL/TLS for the server with application.properties. No matter if we run it locally or remotely e.g. on the OpenShift cluster, we can easily achieve it by using Quarkus features like Dev Services or Kubernetes Extension. This article shows how to take advantage of Quarkus gRPC features and use gRPC services on OpenShift.

The post Introduction to gRPC with Quarkus appeared first on Piotr's TechBlog.

If you are looking for more guides to Spring Boot 3, you can look at other posts on my blog. In that article, I’m describing how to build a microservices architecture with Spring Boot 3 and Spring Cloud. You can also read about the latest changes in observability with Spring Boot 3 and learn how to integrate your app with Grafana Stack in that article. Of course, these are just a few examples – you can find more content in this area on my blog.

Source Code

If you would like to try it by yourself, you may always take a look at my source code. In order to do that, you need to clone my GitHub repository. It contains two apps inside employee-service and organization-service directories. After that, you should just follow my instructions.

Dependencies

In the first step, we will add several dependencies related to Kotlin. Besides the standard libraries, we can include Kotlin support for Jackson (JSON serialization/deserialization):

<dependency>
  <groupId>org.jetbrains.kotlin</groupId>
  <artifactId>kotlin-stdlib</artifactId>
</dependency>
<dependency>
  <groupId>com.fasterxml.jackson.module</groupId>
  <artifactId>jackson-module-kotlin</artifactId>
</dependency>
<dependency>
  <groupId>org.jetbrains.kotlin</groupId>
  <artifactId>kotlin-reflect</artifactId>
</dependency>

We also need to include two Spring Boot Starters. In order to create a reactive Spring @Controller, we need to use the Spring WebFlux module. With Spring Boot Data R2DBC Starter we can use Spring Data Repositories in a reactive way. Finally, we have to include the Postgres driver provided by R2DBC.

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-data-r2dbc</artifactId>
</dependency>
<dependency>
  <groupId>org.postgresql</groupId>
  <artifactId>r2dbc-postgresql</artifactId>
  <scope>runtime</scope>
</dependency>

There are several testing dependencies in our project. We need to include a standard Spring Boot Test Starter, Testcontainers with JUnit 5, Postgres, and R2DBC support, and finally Mock Server Netty module for mocking reactive API. It is also worth adding the spring-boot-testcontainers module to take advantage of built-in integration between Spring Boot and Testcontainers.

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-test</artifactId>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.testcontainers</groupId>
  <artifactId>r2dbc</artifactId>
  <version>1.18.3</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.testcontainers</groupId>
  <artifactId>postgresql</artifactId>
  <version>1.18.3</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.testcontainers</groupId>
  <artifactId>junit-jupiter</artifactId>
  <version>1.18.3</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-testcontainers</artifactId>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.mock-server</groupId>
  <artifactId>mockserver-netty</artifactId>
  <version>5.15.0</version>
  <scope>test</scope>
</dependency>

The last dependency is optional. We can include Spring Boot Actuator in our apps. It adds R2DBC connection status to health checks and several metrics with the pool status.

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-actuator</artifactId>
</dependency>

Implement the Spring Reactive App

Here’s our model class for the first app – employee-service:

class Employee(val name: String, 
               val salary: Int, 
               val organizationId: Int) {
    @Id var id: Int? = null
}

Here’s the repository interface. It needs to extend the R2dbcRepository interface. The same as with the standard Spring Data Repositories we can define several find methods. However, instead of the entities they wrap the return objects with the Reactor Mono or Flux.

interface EmployeeRepository: R2dbcRepository<Employee, Int> {
    fun findByOrganizationId(organizationId: Int): Flux<Employee>
}

Here’s the implementation of our @RestController. We need to inject the EmployeeRepository bean. Then we use the repository bean to interact with the database in a reactive way. Our endpoints also return objects wrapped by the Reactor Mono and Flux. There are three find endpoints and a single POST endpoint:

• Searching all the employees (1)
• Searching by the employee id (2)
• Searching all employees by the organization id (3)
• Adding new employees (4)

@RestController
@RequestMapping("/employees")
class EmployeeController {

   @Autowired
   lateinit var repository : EmployeeRepository

   @GetMapping // (1)
   fun findAll() : Flux<Employee> = repository.findAll()

   @GetMapping("/{id}") // (2)
   fun findById(@PathVariable id: Int) : Mono<Employee> = 
      repository.findById(id)

   @GetMapping("/organization/{organizationId}") // (3)
   fun findByOrganizationId(@PathVariable organizationId: Int): 
      Flux<Employee> = repository.findByOrganizationId(organizationId)

   @PostMapping // (4)
   fun add(@RequestBody employee: Employee) : Mono<Employee> = 
      repository.save(employee)

}

We also need to configure database connection settings in the Spring Boot application.yml:

spring:
  application:
    name: employee-service
  r2dbc:
    url: r2dbc:postgresql://localhost:5432/spring
    username: spring
    password: spring123

Here’s our main class. We want our app to create a table in the database on startup. With R2DBC we need to prepare a fragment of code for populating the schema with the schema.sql file.

@SpringBootApplication
class EmployeeApplication {

   @Bean
   fun initializer(connectionFactory: ConnectionFactory): ConnectionFactoryInitializer? {
      val initializer = ConnectionFactoryInitializer()
      initializer.setConnectionFactory(connectionFactory)
      initializer.setDatabasePopulator(
         ResourceDatabasePopulator(ClassPathResource("schema.sql")))
      return initializer
   }

}

fun main(args: Array<String>) {
   runApplication<EmployeeApplication>(*args)
}

Then just place the schema.sql file in the src/main/resources directory.

CREATE TABLE employee (
  id SERIAL PRIMARY KEY, 
  name VARCHAR(255), 
  salary INT, 
  organization_id INT
);

Let’s switch to the organization-service. The implementation is pretty similar. Hore’s our domain model class:

class Organization(var name: String) {
    @Id var id: Int? = null
}

Our app is communicating with the employee-service. Therefore, we need to define the WebClient bean. It gets the address of the target service from application properties.

@SpringBootApplication
class OrganizationApplication {

   @Bean
   fun initializer(connectionFactory: ConnectionFactory): ConnectionFactoryInitializer? {
      val initializer = ConnectionFactoryInitializer()
      initializer.setConnectionFactory(connectionFactory)
      initializer.setDatabasePopulator(
         ResourceDatabasePopulator(ClassPathResource("schema.sql")))
      return initializer
   }

   @Value("\${employee.client.url}")
   private lateinit var employeeUrl: String

   @Bean
   fun webClient(builder: WebClient.Builder): WebClient {
      return builder.baseUrl(employeeUrl).build()
   }
}

fun main(args: Array<String>) {
    runApplication<OrganizationApplication>(*args)
}

There is also the repository interface OrganizationRepository. Our @RestController uses a repository bean to interact with the database and the WebClient bean to call the endpoint exposed by the employee-service. As the response from the GET /employees/{id}/with-employees it returns the OrganizationDTO.

@RestController
@RequestMapping("/organizations")
class OrganizationController {

   @Autowired
   lateinit var repository : OrganizationRepository
   @Autowired
   lateinit var client : WebClient

   @GetMapping
   fun findAll() : Flux<Organization> = repository.findAll()

   @GetMapping("/{id}")
   fun findById(@PathVariable id : Int): Mono<Organization> = 
      repository.findById(id)

   @GetMapping("/{id}/with-employees")
   fun findByIdWithEmployees(@PathVariable id : Int) : Mono<OrganizationDTO> {
      val employees : Flux<Employee> = client.get().uri("/employees/organization/$id")
             .retrieve().bodyToFlux(Employee::class.java)
      val org : Mono<Organization> = repository.findById(id)
      return org.zipWith(employees.collectList()).log()
             .map { tuple -> OrganizationDTO(tuple.t1.id as Int, tuple.t1.name, tuple.t2) }
    }

   @PostMapping
   fun add(@RequestBody employee: Organization) : Mono<Organization> = 
      repository.save(employee)

}

Here’s the implementation of our DTO:

data class OrganizationDTO(var id: Int?, var name: String) {
    var employees : MutableList<Employee> = ArrayList()
    constructor(employees: MutableList<Employee>) : this(null, "") {
        this.employees = employees
    }
    constructor(id: Int, name: String, employees: MutableList<Employee>) : this(id, name) {
        this.employees = employees
    }
}

Testing with Integrations

Once we finished the implementation we can prepare several integration tests. As I mentioned at the begging, we will use Testcontainers for running the Postgres container during the tests. Our test runs the app and leverages the auto-configured instance of WebTestClient to call the API endpoints (1). We need to start the Postgres container before the tests. So we need to define the container bean inside the companion object section (2). With the @ServiceConnection annotation we don’t have to manually set the properties – Spring Boot will do it for us (3).

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@Testcontainers
@TestMethodOrder(OrderAnnotation::class)
public class EmployeeControllerTests {

   @Autowired
   private lateinit var webTestClient: WebTestClient // (1)

   companion object { // (2)

      @Container
      @ServiceConnection // (3)
      val container = PostgreSQLContainer<Nothing>("postgres:14").apply {
         withDatabaseName("spring")
         withUsername("spring")
         withPassword("spring123")
      }

   }

   @Test
   @Order(1)
   fun shouldStart() {

   }

   @Test
   @Order(2)
   fun shouldAddEmployee() {
      webTestClient.post().uri("/employees")
          .contentType(MediaType.APPLICATION_JSON)
          .bodyValue(Employee("Test", 1000, 1))
          .exchange()
          .expectStatus().is2xxSuccessful
          .expectBody()
          .jsonPath("$.id").isNotEmpty
   }

   @Test
   @Order(3)
   fun shouldFindEmployee() {
      webTestClient.get().uri("/employees/1")
          .accept(MediaType.APPLICATION_JSON)
          .exchange()
          .expectStatus().is2xxSuccessful
          .expectBody()
          .jsonPath("$.id").isNotEmpty
   }

   @Test
   @Order(3)
   fun shouldFindEmployees() {
      webTestClient.get().uri("/employees")
          .accept(MediaType.APPLICATION_JSON)
          .exchange()
          .expectStatus().is2xxSuccessful
          .expectBody().jsonPath("$.length()").isEqualTo(1)
          .jsonPath("$[0].id").isNotEmpty
   }

}

The test class for the organization-service is a little bit more complicated. That’s because we need to mock the communication with the employee-service. In order to do that we use the ClientAndServer object (1). It is started once before all the tests (2) and stopped after the tests (3). We are mocking the GET /employees/organization/{id} endpoint, which is invoked by the organization-service (4). Then we are calling the organization-service GET /organizations/{id}/with-employees endpoint (5). Finally, we are verifying if it returns the list of employees inside the JSON response.

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@Testcontainers
@TestMethodOrder(OrderAnnotation::class)
public class OrganizationControllerTests {

   @Autowired
   private lateinit var webTestClient: WebTestClient

   companion object {

      @Container
      @ServiceConnection
      val container = PostgreSQLContainer<Nothing>("postgres:14").apply {
         withDatabaseName("spring")
         withUsername("spring")
         withPassword("spring123")
      }

      private var mockServer: ClientAndServer? = null // (1)

      @BeforeAll
      @JvmStatic
      internal fun beforeAll() { // (2)
         mockServer = ClientAndServer.startClientAndServer(8090);
      }

      @AfterAll
      @JvmStatic
      internal fun afterAll() { // (3)
         mockServer!!.stop()
      }
   }

   @Test
   @Order(1)
   fun shouldStart() {

   }

   @Test
   @Order(2)
   fun shouldAddOrganization() {
        webTestClient.post().uri("/organizations")
          .contentType(MediaType.APPLICATION_JSON)
          .bodyValue(Organization("Test"))
          .exchange()
          .expectStatus().is2xxSuccessful
          .expectBody()
          .jsonPath("$.id").isNotEmpty
   }

   @Test
   @Order(3)
   fun shouldFindOrganization() {
        webTestClient.get().uri("/organizations/1")
          .accept(MediaType.APPLICATION_JSON)
          .exchange()
          .expectStatus().is2xxSuccessful
          .expectBody()
          .jsonPath("$.id").isNotEmpty
   }

   @Test
   @Order(3)
   fun shouldFindOrganizations() {
        webTestClient.get().uri("/organizations")
          .accept(MediaType.APPLICATION_JSON)
          .exchange()
          .expectStatus().is2xxSuccessful
          .expectBody().jsonPath("$.length()").isEqualTo(1)
          .jsonPath("$[0].id").isNotEmpty
   }

   @Test
   @Order(3)
   fun shouldFindOrganizationWithEmployees() { // (4)
        mockServer!!.`when`(request()
               .withMethod("GET")
               .withPath("/employees/organization/1"))
            .respond(response()
                .withStatusCode(200)
                .withContentType(MediaType.APPLICATION_JSON)
                .withBody(createEmployees()))

      webTestClient.get().uri("/organizations/1/with-employees")
          .accept(MediaType.APPLICATION_JSON) // (5)
          .exchange()
          .expectStatus().is2xxSuccessful
          .expectBody()
          .jsonPath("$.id").isNotEmpty
          .jsonPath("$.employees.length()").isEqualTo(2)
          .jsonPath("$.employees[0].id").isEqualTo(1)
          .jsonPath("$.employees[1].id").isEqualTo(2)
   }

   private fun createEmployees(): String {
      val employees: List<Employee> = listOf<Employee>(
         Employee(1, "Test1", 10000, 1),
         Employee(2, "Test2", 20000, 1)
      )
      return jacksonObjectMapper().writeValueAsString(employees)
   }
}

You can easily verify that all the tests are finished successfully by running them on your laptop. After cloning the repository you need to run Docker and build the apps with the following Maven command:

$ mvn clean package

We can also prepare the build definition for our apps on CircleCI. Since we need to run Testcontainers, we need a machine with a Docker daemon. Here’s the configuration of a built pipeline for CircleCI inside the .circle/config.yml file:

version: 2.1

jobs:
  build:
    docker:
      - image: 'cimg/openjdk:20.0'
    steps:
      - checkout
      - run:
          name: Analyze on SonarCloud
          command: mvn verify sonar:sonar -DskipTests

executors:
  machine_executor_amd64:
    machine:
      image: ubuntu-2204:2022.04.2
    environment:
      architecture: "amd64"
      platform: "linux/amd64"

orbs:
  maven: circleci/maven@1.4.1

workflows:
  maven_test:
    jobs:
      - maven/test:
          executor: machine_executor_amd64
      - build:
          context: SonarCloud

Here’s the result of the build on CircleCI:

If you have Docker running you can also start our Spring Boot reactive apps with the Postgres container. It is possible thanks to the spring-boot-testcontainers module. There is a dedicated @TestConfiguration class that may be used to run Postgres in dev mode:

@TestConfiguration
class PostgresContainerDevMode {

    @Bean
    @ServiceConnection
    fun postgresql(): PostgreSQLContainer<*>? {
        return PostgreSQLContainer("postgres:14.0")
            .withUsername("spring")
            .withPassword("spring123")
    }
}

Now, we need to define the “test” main class that uses the configuration provided within the PostgresContainerDevMode class.

class EmployeeApplicationTest

fun main(args: Array<String>) {
    fromApplication<EmployeeApplication>()
       .with(PostgresContainerDevMode::class)
       .run(*args)
}

In order to run the app in dev Postgres on Docker just execute the following Maven command:

$ mvn spring-boot:test-run

The post Reactive Spring Boot with WebFlux, R2DBC and Postgres appeared first on Piotr's TechBlog.

Why Spring Cloud Vault on Kubernetes?

First of all, let me explain why I decided to use Spring Cloud instead of Hashicorp’s Vault Agent. It is important to know that Vault Agent is always injected as a sidecar container into the application pod. So even if we have a single secret in Vault and we inject it once on startup there is always one additional container running. I’m not saying it’s wrong, since it is a standard approach on Kubernetes. However, I’m not very happy with it. I also had some problems in troubleshooting with Vault Agent. To be honest, it wasn’t easy to find my mistake in configuration based just on its logs. Anyway, Spring Cloud is an interesting alternative to the solution provided by Hashicorp. It allows you to easily integrate Spring Boot configuration properties with the Vault Database engine. In fact, you just need to include a single dependency to use it.

Source Code

If you would like to try it by yourself, you may always take a look at my source code. In order to do that you need to clone my GitHub repository. To see the sample application go to the kubernetes/sample-db-vault directory. Then you should just follow my instructions

Prerequisites

Before we start, there are some required tools. Of course, we need to have a Kubernetes cluster locally or remotely. Personally, I use Docker Desktop, but you may use any other option you prefer. In order to run Vault on Kubernetes, we need to install Helm.

If you would like to build the application from the source code you need to have Skaffold, Java 17, and Maven. Alternatively, you may use a ready image from my Docker Hub account piomin/sample-app.

Install Vault on Kubernetes with Helm

The recommended way to run Vault on Kubernetes is via the Helm chart. Helm installs and configures all the necessary components to run Vault in several different modes. Firstly, let’s add the HashiCorp Helm repository.

$ helm repo add hashicorp https://helm.releases.hashicorp.com

Before proceeding it is worth updating all the repositories to ensure helm uses the latest versions of the components.

$ helm repo update

Since I will run Vault in the dedicated namespace, we first need to create it.

$ kubectl create ns vault

Finally, we can install the latest version of the Vault server and run it in development mode.

$ helm install vault hashicorp/vault \
    --set "server.dev.enabled=true" \
    -n vault

We can verify the installation by displaying a list of running pods in the vault namespace. As you see the Vault Agent is installed by the Helm Chart, so you can try using it as well. If you wish to just go to this tutorial prepared by HashiCorp.

$ kubectl get pod -n vault
NAME                                    READY   STATUS     RESTARTS   AGE
vault-0                                 1/1     Running    0          1h
vault-agent-injector-678dc584ff-wc2r7   1/1     Running    0          1h

Access Vault on Kubernetes

Before we run our application on Kubernetes, we need to configure several things on Vault. I’ll show you how to do it using the vault CLI. The simplest way to use CLI on Kubernetes is just by getting a shell of a running Vault container:

$ kubectl exec -it vault-0 -n vault -- /bin/sh

Alternatively, we can use Vault Web Console available at the 8200 port. To access it locally we should first enable port forwarding:

$ kubectl port-forward service/vault 8200:8200 -n vault

Now, you access it locally in your web browser at http://localhost:8200. In order to log in there use the Token method (a default token value is root). Then you may do the same as with the vault CLI but with the nice UI.

Configure Kubernetes authentication

Vault provides a Kubernetes authentication method that enables clients to authenticate with a Kubernetes service account token. This token is available to every single pod. Assuming you have already started an interactive shell session on the vault-0 pod just execute the following command:

$ vault auth enable kubernetes

In the next step, we are going to configure the Kubernetes authentication method. We need to set the location of the Kubernetes API, the service account token, its certificate, and the name of the Kubernetes service account issuer (required for Kubernetes 1.21+).

$ vault write auth/kubernetes/config \
    kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443" \
    token_reviewer_jwt="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
    kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt \
    issuer="https://kubernetes.default.svc.cluster.local"

Ok, now very important. You need to understand what happened here. We need to create a Vault policy that allows us to generate database credentials dynamically. We will enable the Vault database engine in the next section. For now, we are just creating a policy that will be assigned to the authentication role. The name of our Vault policy is internal-app:

$ vault policy write internal-app - <<EOF
path "database/creds/default" {
  capabilities = ["read"]
}
EOF

The next important thing is related to the Kubernetes RBAC. Although the Vault server is running in the vault namespace our sample application will be running in the default namespace. Therefore, the service account used by the application is also in the default namespace. Let’s create ServiceAccount for the application:

$ kubectl create sa internal-app

Now, we have everything to do the last step in this section. We need to create a Vault role for the Kubernetes authentication method. In this role, we set the name and location of the Kubernetes ServiceAccount and the Vault policy created in the previous step.

$ vault write auth/kubernetes/role/internal-app \
    bound_service_account_names=internal-app \
    bound_service_account_namespaces=default \
    policies=internal-app \
    ttl=24h

After that, we may proceed with the next steps. Let’s enable the Vault database engine.

Enable Vault Database Engine

Just to clarify, we are still inside the vault-0 pod. Let’s enable the Vault database engine.

$ vault secrets enable database

Of course, we need to run a database on Kubernetes. We will PostgreSQL since it is supported by Vault. The full deployment manifest is available on my GitHub repository in /kubernetes/k8s/postgresql-deployment.yaml. Here’s just the Deployment object:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:latest
          imagePullPolicy: "IfNotPresent"
          ports:
            - containerPort: 5432
          env:
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  key: POSTGRES_PASSWORD
                  name: postgres-secret
          volumeMounts:
            - mountPath: /var/lib/postgresql/data
              name: postgredb
      volumes:
        - name: postgredb
          persistentVolumeClaim:
            claimName: postgres-claim

Let’s apply the whole manifest to deploy Postgres in the default namespace:

$ kubectl apply -f postgresql-deployment.yaml

Following Vault documentation, we first need to configure a plugin for the PostgreSQL database and then provide connection settings and credentials:

$ vault write database/config/postgres \
    plugin_name=postgresql-database-plugin \
    allowed_roles="default" \
    connection_url="postgresql://{{username}}:{{password}}@postgres.default:5432?sslmode=disable" \
    username="postgres" \
    password="admin123"

I have disabled SSL for connection with Postgres by setting the property sslmode=disable. There is only one role allowed to use the Vault PostgresSQL plugin: default. The name of the role should be the same as the name passed in the field allowed_roles in the previous step. We also have to set a target database name and SQL statement that creates users with privileges. We set the max TTL of the lease to 10 minutes just to present revocation and renewal features of Spring Cloud Vault. It means that 10 minutes after your application has started it can no longer authenticate with the database.

$ vault write database/roles/default db_name=postgres \
    creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';GRANT SELECT, UPDATE, INSERT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";GRANT USAGE,  SELECT ON ALL SEQUENCES IN SCHEMA public TO \"{{name}}\";" \
    default_ttl="1m" \
    max_ttl="10m"

And that’s all on the Vault server side. Now, we can test our configuration using a vault CLI as shown below. You can log in to the database using returned credentials. By default, they are valid for one minute (the default_ttl parameter in the previous command).

$ vault read database/creds/default

We can also verify a connection to the instance of PostgreSQL in Vault UI:

Now, we can generate new credentials just by renewing the Vault lease (vault lease renew LEASE_ID). Hopefully, Spring Cloud Vault does it automatically for our app. Let’s see how it works.

Use Spring Cloud Vault on Kubernetes

For the purpose of this demo, I created a simple Spring Boot application. It exposes REST API and connects to the PostgreSQL database. It uses Spring Data JPA to interact with the database. However, the most important thing here are the following two dependencies:

<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-starter-bootstrap</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.cloud</groupId>
  <artifactId>spring-cloud-vault-config-databases</artifactId>
</dependency>

The first of them enables bootstrap.yml processing on the application startup. The second of them include Spring Cloud Vault Database engine support.

The only thing we need to do is to provide the right configuration settings Here’s the minimal set of the required dependencies to make it work without any errors. The following configuration is provided in the bootstrap.yml file:

spring:
  application:
    name: sample-db-vault
  datasource:
    url: jdbc:postgresql://postgres:5432/postgres #(1)
  jpa:
    hibernate:
      ddl-auto: update
  cloud:
    vault:
      config.lifecycle: #(2)
        enabled: true
        min-renewal: 10s
        expiry-threshold: 30s
      kv.enabled: false #(3)
      uri: http://vault.vault:8200 #(4)
      authentication: KUBERNETES #(5)
      postgresql: #(6)
        enabled: true
        role: default
        backend: database
      kubernetes: #(7)
        role: internal-app

Let’s analyze the configuration visible above in the details:

(1) We need to set the database connection URI, but WITHOUT any credentials. Assuming our application uses standard properties for authentication against the database (spring.datasource.username and spring.datasource.password) we don’t need to anything else

(2) As you probably remember, the max TTL for the database lease is 10 minutes. We enable lease renewal every 30 seconds. Just for the demo purpose. You will see that Spring Cloud Vault will create new credentials in PostgreSQL every 30 seconds, and the application still works without any errors

(3) Vault KV is not needed here, since I’m using only the database engine

(4) The application is going to be deployed in the default namespace, while Vault is running in the vault namespace. So, the address of Vault should include the namespace name

(5) (7) Our application uses the Kubernetes authentication method to access Vault. We just need to set the role name, which is internal-app. All other settings should be left with the default values

(6) We also need to enable postgres database backend support. The name of the backend in Vault is database and the name of Vault role used for that engine is default.

Run Spring Boot application on Kubernetes

The Deployment manifest is rather simple. But what is important here – we need to use the ServiceAccount internal-app used by the Vault Kubernetes authentication method.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-app-deployment
spec:
  selector:
    matchLabels:
      app: sample-app
  template:
    metadata:
      labels:
        app: sample-app
    spec:
      containers:
      - name: sample-app
        image: piomin/sample-app
        ports:
        - containerPort: 8080
      serviceAccountName: internal-app

Our application requires Java 17. Since I’m using Jib Maven Plugin for building images I also have to override the default base image. Let’s use openjdk:17.0.1-slim-buster.

<plugin>
  <groupId>com.google.cloud.tools</groupId>
  <artifactId>jib-maven-plugin</artifactId>
  <version>3.1.4</version>
  <configuration>
    <from>
      <image>openjdk:17.0.1-slim-buster</image>
    </from>
  </configuration>
</plugin>

The repository is configured to easily deploy the application with Skaffold. Just go to the /kubernetes/sample-db-vault directory and run the following command in order to build and deploy our sample application on Kubernetes:

$ skaffold dev --port-forward

After that, you can call one of the REST endpoints to test if the application works properly:

$ curl http://localhost:8080/persons

Everything works fine? In the background, Spring Cloud Vault creates new credentials every 30 seconds. You can easily verify it inside the PostgreSQL container. Just connect to the postgres pod and run the psql process:

$ kubectl exec svc/postgres -i -t -- psql -U postgres

Now you can list users with the \du command. Repeat the command several times to see if the credentials have been regenerated. Of course, the application is able to renew the lease until the max TTL (10 minutes) is not exceeded.

The post Vault on Kubernetes with Spring Cloud appeared first on Piotr's TechBlog.

Fortunately, there are two types of tools that perfectly fit the described process. Firstly, we need a tool that allows us to easily deploy applications in multiple environments. That’s what we may achieve on Kubernetes with ArgoCD. In the next step, we need a tool that automatically updates a database schema on demand. There are many tools for that. Since we need something lightweight and easy to containerize my choice fell on Liquibase. It is not my first article about Liquibase on Kubernetes. You can also read more about the blue-green deployment approach with Liquibase here.

However, in this article, we will focus on a little bit different problem. First, let’s describe it.

Introduction

I think that one of the biggest challenges around continuous delivery is integration to the databases. Consequently, we should treat this integration the same as a standard configuration. It’s time to treat database code like application code. Otherwise, our CI/CD process fails at the database.

Usually, when we consider the CI/CD process for the application we have multiple target environments. According to the cloud-native patterns, each application has its own separated database. Moreover, it is unique per each environment. So each time, we run our delivery pipeline, we have to update the database instance on the particular environment. We should do it just before running a new version application (e.g. with the blue-green approach).

Here’s the visualization of our scenario. We are releasing the Spring Boot application in three different environments. Those environments are just different namespaces on Kubernetes: dev, test and prod. The whole process is managed by ArgoCD and Liquibase. In the dev environment, we don’t use any migration tool. Let’s say we leave it to developers. Our mechanism is active for the test and prod namespaces.

Modern Java frameworks like Spring Boot offer built-in integration with Liquibase. In that concept, we just need to create a Liquibase changelog and set its location in Spring configuration. Our framework is running such a script on application startup. Since it is a very useful approach in development, I would not recommend it for production deployment. Especially if you deploy your application on Kubernetes. Why? You will find a detailed explanation in the article I have already mentioned in the first paragraph.

Source Code

If you would like to try it by yourself, you may always take a look at my source code. In order to do that you need to clone my GitHub repository. Then you should just follow my instructions

Also, one thing before we start. Here I described the whole process of building and deploying applications on Kubernetes with ArgoCD and Tekton. Typically there is a continuous integration phase, which is realized with Tekton. In this phase, we are just building and pushing the image. We are not releasing changes to the database. Especially, that we have multiple environments.

The picture visible below illustrates our approach in that context. ArgoCD synchronizes configuration and applies the changes into the Kubernetes cluster and a database using Liquibase. It doesn’t matter if a database is running on Kubernetes or not. However, in our case, we assume it is deployed on the namespace as the application.

Docker Image with Liquibase

There is an official Liquibase image on Docker Hub. We need to execute the update command using this image. To do that I prepared a custom image based on the official Liquibase image. You can see the Dockerfile below. But you can as well pull the image I published in my Docker registry docker.io/piomin/liquibase:latest.

FROM liquibase/liquibase
ENV URL=jdbc:postgresql://postgresql:5432/test
ENV USERNAME=postgres
ENV PASSWORD=postgres
ENV CHANGELOGFILE=changelog.xml
CMD ["sh", "-c", "docker-entrypoint.sh --url=${URL} --username=${USERNAME} --password=${PASSWORD} --classpath=/liquibase/changelog --changeLogFile=${CHANGELOGFILE} update"]

We will run that image as the init container inside the pod with our application. Thanks to that approach we can be sure that it updates database schema just before starting the container with the application.

Sample Spring Boot application

We will use one of my sample Spring Boot applications in this exercise. You may find it in my GitHub repository here. It connects to the PostgreSQL database:

spring:
  application:
    name: person-service
  datasource:
    url: jdbc:postgresql://person-db:5432/${DATABASE_NAME}
    username: ${DATABASE_USER}
    password: ${DATABASE_PASSWORD}

You can clone the application source code by yourself. But you can as well pull the ready image located here: quay.io/pminkows/person-app. The application is compiled with Java 17 and uses Spring Data JPA as an ORM layer to integrate with the database.

<properties>
  <java.version>17</java.version>
</properties>
<dependencies>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-jpa</artifactId>
  </dependency>
  <dependency>
    <groupId>org.postgresql</groupId>
    <artifactId>postgresql</artifactId>
    <scope>runtime</scope>
  </dependency>
</dependencies>

Use Liquibase with ArgoCD and Kustomize

In our scenario, the Liquibase init container should be included in the Deployment only for test and prod namespaces. On the dev environment, the pod should just contain a single container with the Spring Boot application. In order to implement this behavior with ArgoCD, we may use Kustomize. Kustomize has the concepts of bases and overlays. A base is a directory with a kustomization.yaml, which contains a set of resources and associated customization. Thanks to overlays, we may include additional elements into the base manifests. So, here’s the structure of our configuration repository in GitHub:

Here’s our base Deployment file. It’s pretty simple. The only thing we need to do is to inject database credentials from secrets:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-spring-deployment
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sample-spring
  template:
    metadata:
      labels:
        app: sample-spring
    spec:
      containers:
        - name: sample-spring
          image: quay.io/pminkows/person-app:1.0
          ports:
            - containerPort: 8080
          env:
            - name: DATABASE_USER
              valueFrom:
                secretKeyRef:
                  name: postgres
                  key: database-user
            - name: DATABASE_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgres
                  key: database-password
            - name: DATABASE_NAME
              valueFrom:
                secretKeyRef:
                  name: postgres
                  key: database-name

We also have the Liquibase changeLog.sql file in the base directory. We should place it inside Kubernetes ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  name: changelog-cm
data:
  changeLog.sql: |
    --liquibase formatted sql
    --changeset piomin:1
    create table person (
      id serial primary key,
      name varchar(255),
      gender varchar(255),
      age int,
      externalId int
    );
    insert into person(name, age, gender) values('John Smith', 25, 'MALE');
    insert into person(name, age, gender) values('Paul Walker', 65, 'MALE');
    insert into person(name, age, gender) values('Lewis Hamilton', 35, 'MALE');
    insert into person(name, age, gender) values('Veronica Jones', 20, 'FEMALE');
    insert into person(name, age, gender) values('Anne Brown', 60, 'FEMALE');
    insert into person(name, age, gender) values('Felicia Scott', 45, 'FEMALE');

Also, let’s take look at the base/kustomization.yaml file:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
  - deployment.yaml
  - changelog.yaml

In the overlays/liquibase/liquibase-container.yaml file we are defining the init container that should be included in our base Deployment. There are four parameters available to override. Therefore, we will set the address of a target database, username, password, and location of the Liquibase changelog file. The changeLog.sql file is available to the container as a mounted volume under location /liquibase/changelog. Of course, I’m using the image described in the Docker Image with Liquibase section.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-spring-deployment
spec:
  template:
    spec:
      initContainers:
        - name: liquibase
          image: docker.io/piomin/liquibase:latest
          env:
            - name: URL
              value: jdbc:postgresql://person-db:5432/sampledb
            - name: USERNAME
              valueFrom:
                secretKeyRef:
                  name: postgres
                  key: database-user
            - name: PASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgres
                  key: database-password
            - name: CHANGELOGFILE
              value: changeLog.sql
          volumeMounts:
            - mountPath: /liquibase/changelog
              name: changelog
      volumes:
        - name: changelog
          configMap:
            name: changelog-cm

And last manifest in the repository. Here’s the overlay kustomization.yaml file. It uses the whole structure from the base catalog and includes the init container to the application Deployment:

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
bases:
  - ../../base
patchesStrategicMerge:
  - liquibase-container.yaml

Create ArgoCD applications

Since our configuration is ready, we may proceed to the last step. Let’s create ArgoCD applications responsible for synchronization between Git repository and both Kubernetes cluster and a target database. In order to create the ArgoCD application, we need to apply the following manifest. It refers to the Kustomize overlay defined in the /overlays/liquibase directory. The declaration for test or prod environment looks as shown below:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: liquibase-prod
spec:
  destination:
    namespace: prod
    server: 'https://kubernetes.default.svc'
  project: default
  source:
    path: overlays/liquibase
    repoURL: 'https://github.com/piomin/sample-argocd-liquibase-kustomize.git'
    targetRevision: HEAD

On the other hand, the dev environment doesn’t require an init container with Liquibase. Therefore, it uses the base directory:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: liquibase-dev
spec:
  destination:
    namespace: dev
    server: 'https://kubernetes.default.svc'
  project: default
  source:
    path: base
    repoURL: 'https://github.com/piomin/sample-argocd-liquibase-kustomize.git'
    targetRevision: HEAD

Since ArgoCD supports Kustomize, we just need to create applications. Alternatively, we could have created them using ArgoCD UI. Finally, here’s a list of ArgoCD applications responsible for applying changes to the Postgres database. Here’s the UI view for all the environments after synchronization (Sync button).

We can also verify the logs printed by the Liquibase init container after ArgoCD synchronization:

We may also verify a list of running pods in one of the target namespace, e.g. prod.

$ kubectl get pod                                               
NAME                                        READY   STATUS    RESTARTS   AGE
person-db-1-vn8kw                           1/1     Running   0          82m
sample-spring-deployment-7695d64c54-9hf75   1/1     Running   0          7m16s

And also print out Liquibase logs using kubectl:

$ kubectl logs sample-spring-deployment-7695d64c54-9hf75 -c liquibase

The post Continuous Delivery on Kubernetes with Database using ArgoCD and Liquibase appeared first on Piotr's TechBlog.

If you run your applications on Kubernetes, Quarkus is obviously a good choice. It starts fast and does not consume much memory. You may easily compile it natively with GraalVM. It provides a lot of useful developers features like e.g. hot reload. I hope you will find there tips and techniques that help to boost your productivity in Quarkus development. Or maybe just convince you to take a look at it, if don’t have any experience yet.

I have already published all these Quarkus tips on Twitter in a graphical form visible below. You may access them using the #QuarkusTips hashtag. I’m a huge fan of Quarkus (and to be honest Spring Boot also :)). So, if you have suggestions or your own favorite features just ping me on Twitter (@piotr_minkowski). I will definitely retweet your tweet.

Tip 1. Use Quarkus command-line tool

How do you start a new application when using one of the popular Java frameworks? You can go to the online generator website, which is usually provided by those frameworks. Did you hear about Spring Initializr? Quarkus offers a similar site available at https://code.quarkus.io/. But what you may not know, there is also the Quarkus CLI tool. It allows you to create projects, manage extensions and execute build and dev commands. For example, you can create a source code for a new application using a single command as shown below.

$ quarkus create app --package-name=pl.piomin.samples.quarkus \
  -x resteasy-jackson,hibernate-orm-panache,jdbc-postgresql \
  -o person-service \
  pl.piomin.samples:person-service

After executing the command visible above you should a similar screen.

This command creates a simple REST application that uses the PostgreSQL database and Quarkus ORM layer. Also, it sets the name of the application, Maven groupId, and artifactId. After that, you can just run the application. To do that go to the generated directory and run the following command. Alternatively, you can execute the mvn quarkus:dev command.

$ quarkus dev

The application does not start successfully, since there is no database connection configured. Do we have to do that? No! Let’s proceed to the next section to see why.

Tip 2. Use Dev Services with databases

Did you hear about Testcontainers? It is a Java library that allows you to run containers automatically during JUnit tests. You can run common databases, Selenium web browsers, or anything else that can run in a Docker container. Quarkus provides built-in integration with Testcontainers when running applications in dev or test modes. This feature is called Dev Services. Moreover, you don’t have to do anything to enable it. Just DO NOT PROVIDE connection URL and credentials.

Let’s back to our scenario. We have already created the application using Quarkus CLI. It contains all the required libraries. So, the only thing we need to do now is to run a Docker daemon. Thanks to that Quarkus will try to run PostgreSQL with Testcontainers in development mode. What’s the final result? Our application is working and it is connected with PostgreSQL started with Docker as shown below.

Then, we may proceed to the development. With the quarkus dev command we have already enabled dev mode. Thanks to this, we may take advantage of the live reload feature.

Tip 3. Use simplified ORM with Panache

Let’s add some code to our sample application. We will implement a data layer using Quarkus Panache ORM. That’s a very interesting module that focuses on making your entities trivial and fun to write in Quarkus. Here’s our entity class. Thanks to the Quarkus field access rewrite, when you read person.name you will actually call your getName() accessor, and similarly for field writes and the setter. This allows for proper encapsulation at runtime as all fields calls will be replaced by the corresponding getter or setter calls. The PanacheEntity also takes care of the primary key implementation.

@Entity
public class Person extends PanacheEntity {
   public String name;
   public int age;
   @Enumerated(EnumType.STRING)
   public Gender gender;
}

In the next step, we are going to define the repository class. Since it implements the PanacheRepository interface, we only need to add our custom find methods.

@ApplicationScoped
public class PersonRepository implements PanacheRepository<Person> {

    public List<Person> findByName(String name) {
        return find("name", name).list();
    }

    public List<Person> findByAgeGreaterThan(int age) {
        return find("age > ?1", age).list();
    }
}

Finally, let’s add a resource class with REST endpoints.

@Path("/persons")
public class PersonResource {

    @Inject
    PersonRepository personRepository;

    @POST
    @Transactional
    public Person addPerson(Person person) {
        personRepository.persist(person);
        return person;
    }

    @GET
    public List<Person> getPersons() {
        return personRepository.listAll();
    }

    @GET
    @Path("/name/{name}")
    public List<Person> getPersonsByName(@PathParam("name") String name) {
        return personRepository.findByName(name);
    }

    @GET
    @Path("/age-greater-than/{age}")
    public List<Person> getPersonsByName(@PathParam("age") int age) {
        return personRepository.findByAgeGreaterThan(age);
    }

    @GET
    @Path("/{id}")
    public Person getPersonById(@PathParam("id") Long id) {
        return personRepository.findById(id);
    }

}

Also, let’s create the import.sql file in the src/main/resources directory. It loads SQL statements when Hibernate ORM starts.

insert into person(id, name, age, gender) values(1, 'John Smith', 25, 'MALE');
insert into person(id, name, age, gender) values(2, 'Paul Walker', 65, 'MALE');
insert into person(id, name, age, gender) values(3, 'Lewis Hamilton', 35, 'MALE');
insert into person(id, name, age, gender) values(4, 'Veronica Jones', 20, 'FEMALE');
insert into person(id, name, age, gender) values(5, 'Anne Brown', 60, 'FEMALE');
insert into person(id, name, age, gender) values(6, 'Felicia Scott', 45, 'FEMALE');

Finally, we can call our REST endpoint.

$ curl http://localhost:8080/persons

Tip 4. Unified configuration as option

Assuming we don’t want to run a database on Docker, we should configure the connection in application.properties. By default, Quarkus provides 3 profiles: prod, test, dev. We can define properties for multiple profiles inside a single application.properties using the syntax  %{profile-name}.config.name. In our case, there is an H2 instance used in dev and test modes, and an external PostgreSQL instance in the prod mode.

quarkus.datasource.db-kind=postgresql
quarkus.datasource.username=${POSTGRES_USER}
quarkus.datasource.password=${POSTGRES_PASSWORD}
quarkus.datasource.jdbc.url=jdbc:postgresql://person-db:5432/${POSTGRES_DB}

%test.quarkus.datasource.db-kind=h2
%test.quarkus.datasource.username=sa
%test.quarkus.datasource.password=password
%test.quarkus.datasource.jdbc.url=jdbc:h2:mem:testdb

%dev.quarkus.datasource.db-kind=h2
%dev.quarkus.datasource.username=sa
%dev.quarkus.datasource.password=password
%dev.quarkus.datasource.jdbc.url=jdbc:h2:mem:testdb

Before running a new version application we have to include H2 dependency in Maven pom.xml.

<dependency>
   <groupId>io.quarkus</groupId>
   <artifactId>quarkus-jdbc-h2</artifactId>
</dependency>

You can also define your custom profile and provide properties using it as a prefix. Of course, you may still define profile-specific files like application-{profile}.properties.

Tip 5. Deploy to Kubernetes with Maven

Quarkus in a Kubernetes native framework. You may easily deploy your Quarkus application to the Kubernetes cluster without creating any YAML files manually. For more advanced configurations like e.g. mapping secrets to environment variables, you can use application.properties. Other things like e.g. health checks are detected in the source code. To enable this we need to include the quarkus-kubernetes module. There is also an implementation for OpenShift.

<dependency>
   <groupId>io.quarkus</groupId>
   <artifactId>quarkus-openshift</artifactId>
</dependency>

After that, Quarkus will generate deployment manifests during the Maven build. We can enable automatic deployment to the current Kubernetes cluster by setting the property quarkus.kubernetes.deploy to true. For OpenShift deploy, we have to change the default deployment target from kubernetes to openshift.

quarkus.container-image.build = true
quarkus.kubernetes.deploy = true
quarkus.kubernetes.deployment-target = openshift

Let’s assume we have some custom configuration to set on the Deployment manifest. Our application will run in two pods and is automatically exposed outside the cluster. It also injects values from Secret in order to connect with the PostgreSQL database.

quarkus.openshift.expose = true
quarkus.openshift.replicas = 2
quarkus.openshift.labels.app = person-app
quarkus.openshift.annotations.app-type = demo
quarkus.openshift.env.mapping.postgres_user.from-secret = person-db
quarkus.openshift.env.mapping.postgres_user.with-key = database-user
quarkus.openshift.env.mapping.postgres_password.from-secret = person-db
quarkus.openshift.env.mapping.postgres_password.with-key = database-password
quarkus.openshift.env.mapping.postgres_db.from-secret = person-db
quarkus.openshift.env.mapping.postgres_db.with-key = database-name

Then we just need to build our application with Maven. Alternatively, we may remove the quarkus.kubernetes.deploy property from application.properties and enable it on the Maven command.

$ maven clean package -Dquarkus.kubernetes.deploy=true

Tip 6. Access Dev UI console

After running the Quarkus app in dev mode (mvn quarkus:dev) you can access the Dev UI console under the address http://localhost:8080/q/dev. The more modules you include the more options you can configure there. One of my favorite features here is the ability to deploy applications to OpenShift. Instead of running the Maven command for building an application, we can just run it in dev mode and deploy using graphical UI.

Tip 7. Test continuously

Quarkus supports continuous testing, where tests run immediately after code changes. This allows you to get instant feedback on your code changes. Quarkus detects which tests cover which code, and uses this information to only run the relevant tests when code is changed. After running the application in development you will be prompted to enable that feature as shown below. Just press r to enable it.

Ok, so let’s add some tests for our sample application. Firstly, we need to include the Quarkus Test module and REST Assured library.

<dependency>
   <groupId>io.quarkus</groupId>
   <artifactId>quarkus-junit5</artifactId>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>io.rest-assured</groupId>
   <artifactId>rest-assured</artifactId>
   <scope>test</scope>
</dependency>

Then we will add some simple API tests. The test class has to be annotated with @QuarkusTest. The rest of the implementation is typical for the REST Assured library.

@QuarkusTest
public class PersonResourceTests {

    @Test
    void getPersons() {
        List<Person> persons = given().when().get("/persons")
                .then()
                .statusCode(200)
                .extract()
                .body()
                .jsonPath().getList(".", Person.class);
        assertEquals(persons.size(), 6);
    }

    @Test
    void getPersonById() {
        Person person = given()
                .pathParam("id", 1)
                .when().get("/persons/{id}")
                .then()
                .statusCode(200)
                .extract()
                .body().as(Person.class);
        assertNotNull(person);
        assertEquals(1L, person.id);
    }

    @Test
    void newPersonAdd() {
        Person newPerson = new Person();
        newPerson.age = 22;
        newPerson.name = "TestNew";
        newPerson.gender = Gender.FEMALE;
        Person person = given()
                .body(newPerson)
                .contentType(ContentType.JSON)
                .when().post("/persons")
                .then()
                .statusCode(200)
                .extract()
                .body().as(Person.class);
        assertNotNull(person);
        assertNotNull(person.id);
    }
}

We can run those JUnit tests from the Dev UI console as well. Firstly, you should go to the Dev UI console. At the bottom of the page, you will find the panel responsible testing module. Just click the Test result icon and you will see a screen similar to the visible below.

Tip 8. Compile natively with GraalVM on OpenShift

You can easily build and run a native Quarkus GraalVM image on OpenShift using a single command and ubi-quarkus-native-s2i builder. OpenShift builds the application using the S2I (source-2-image) approach. Of course, you just need a running OpenShift cluster (e.g. local CRC or Developer Sandbox https://developers.redhat.com/products/codeready-containers/overview) and the oc client installed locally.

$ oc new-app --name person-native \
             --context-dir basic-with-db/person-app \
  quay.io/quarkus/ubi-quarkus-native-s2i:21.2-java11~https://github.com/piomin/openshift-quickstart.git

Tip 9. Rollback transaction after each test

If you need to roll back changes in the data after each test, avoid doing it manually. Instead, you just need to annotate your test class with @TestTransaction. Rollback is performed each time the test method is complete.

@QuarkusTest
@TestTransaction
public class PersonRepositoryTests {

    @Inject
    PersonRepository personRepository;

    @Test
    void addPerson() {
        Person newPerson = new Person();
        newPerson.age = 22;
        newPerson.name = "TestNew";
        newPerson.gender = Gender.FEMALE;
        personRepository.persist(newPerson);
        Assertions.assertNotNull(newPerson.id);
    }
}

Tip 10. Take advantage of GraphQL support

That’s the last of Quarkus tips in this article. However, it is one of my favorite Quarkus features. GraphQL support is not a strong side of Spring Boot. On the other hand, Quarkus provides very cool and simple extensions for GraphQL for the client and server-side.

Firstly, let’s add the Quarkus modules responsible for GraphQL support.

<dependency>
   <groupId>io.quarkus</groupId>
   <artifactId>quarkus-smallrye-graphql</artifactId>
</dependency>
<dependency>
   <groupId>io.quarkus</groupId>
   <artifactId>quarkus-smallrye-graphql-client</artifactId>
   <scope>test</scope>
</dependency>

Then we may create a code responsible for exposing GraphQL API. The class has to be annotated with @GraphQLAPI. Quarkus automatically generates GraphQL schema from the source code.

@GraphQLApi
public class EmployeeFetcher {

    private EmployeeRepository repository;

    public EmployeeFetcher(EmployeeRepository repository){
        this.repository = repository;
    }

    @Query("employees")
    public List<Employee> findAll() {
        return repository.listAll();
    }

    @Query("employee")
    public Employee findById(@Name("id") Long id) {
        return repository.findById(id);
    }

    @Query("employeesWithFilter")
    public List<Employee> findWithFilter(@Name("filter") EmployeeFilter filter) {
        return repository.findByCriteria(filter);
    }

}

Then, let’s create a client interface to call two endpoints. We need to annotate that interface with @GraphQLClientApi.

@GraphQLClientApi(configKey = "employee-client")
public interface EmployeeClient {

    List<Employee> employees();
    Employee employee(Long id);
}

Finally, we can add a simple JUnit test. We just need to inject EmployeeClient, and then call methods. If you are interested in more details about Quarkus GraphQL support read my article An Advanced GraphQL with Quarkus.

@QuarkusTest
public class EmployeeFetcherTests {

    @Inject
    EmployeeClient employeeClient;

    @Test
    void fetchAll() {
        List<Employee> employees = employeeClient.employees();
        Assertions.assertEquals(10, employees.size());
    }

    @Test
    void fetchById() {
        Employee employee = employeeClient.employee(10L);
        Assertions.assertNotNull(employee);
    }
}

Final Thougths

In my opinion, Quarkus is a very interesting and promising framework. With these tips, you can easily start the development of your first application with Quarkus. There are some new interesting features in each new release. So maybe, I will have to update this list of Quarkus tips soon

The post Quarkus Tips, Tricks and Techniques appeared first on Piotr's TechBlog.

Example

The source code with sample applications is as usual available on GitHub in the repository: https://github.com/piomin/sample-spring-microservices-transactions.git.

Architecture

First, we need to add a new component to our system. It is responsible just for managing distributed transactions across microservices. That element is described as transaction-server on the diagram below. We also use another popular component in microservices-based architecture discovery-server. There are three applications: order-service, account-service and product-service. The application order-service is communicating with account-service and product-service. All these applications are using Postgres database as a backend store. Just for simplification I have run a single database with multiple tables. In a normal situation we would have a single database per each microservice.

Now, we will consider the following situation (it is visualized on the diagram below). The application order-service is creating an order, storing it in the database, and then starting a new distributed transaction (1). After that, it is communicating with application product-service to update the current number of stored products and get their price (2). At the same time product-service is sending information to transaction-server that it is participating in the transaction (3). Then order-service is trying to withdraw the required funds from the customer account and transfer them into another account related to a seller (4). Finally, we are rolling back the transaction by throwing an exception inside the transaction method from order-service (6). This rollback should cause a rollback of the whole distributed transaction.

Building transaction server

We are starting implementation from transaction-server. A transaction server is responsible for managing distributed transactions across all microservices in our sample system. It exposes REST API available for all other microservices for adding new transactions and updating their status. It also sends asynchronous broadcast events after receiving transaction confirmation or rollback from a source microservice. It uses RabbitMQ message broker for sending events to other microservices via topic exchange. All other microservices are listening for incoming events, and after receiving them they are committing or rolling back transactions. We can avoid using a message broker for exchanging events and use communication over HTTP endpoints, but that makes sense only if we have a single instance of every microservice. Here’s the picture that illustrates the currently described architecture.

Let’s take a look on the list of required dependencies. It would be pretty the same for other applications. We need spring-boot-starter-amqp for integration with RabbitMQ, spring-boot-starter-web for exposing REST API over HTTP, spring-cloud-starter-netflix-eureka-client for integration with Eureka discovery server and some basic Kotlin libraries.

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
   <groupId>com.fasterxml.jackson.module</groupId>
   <artifactId>jackson-module-kotlin</artifactId>
</dependency>
<dependency>
   <groupId>org.jetbrains.kotlin</groupId>
   <artifactId>kotlin-reflect</artifactId>
</dependency>
<dependency>
   <groupId>org.jetbrains.kotlin</groupId>
   <artifactId>kotlin-stdlib</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-netflix-eureka-client</artifactId>
</dependency>

In the main class we are defining a topic exchange for events sent to microservices. The name of exchange is trx-events, and it is automatically created on RabbitMQ after application startup.

@SpringBootApplication
class TransactionServerApp {

    @Bean
    fun topic(): TopicExchange = TopicExchange("trx-events")
}

fun main(args: Array) {
    runApplication(*args)
}

Here are domain model classes used by a transaction server. The same classes are used by the microservices during communication with transaction-server.

data class DistributedTransaction(var id: String? = null,var status: DistributedTransactionStatus,
                                  val participants: MutableList<DistributedTransactionParticipant> = mutableListOf())
                          
class DistributedTransactionParticipant(val serviceId: String, var status: DistributedTransactionStatus)

enum class DistributedTransactionStatus {
    NEW, CONFIRMED, ROLLBACK, TO_ROLLBACK
}   

Here’s the controller class. It is using a simple in-memory implementation of repository and RabbitTemplate for sending events to RabbitMQ. The HTTP API provides methods for adding new transaction, finishing existing transaction with a given status (CONFIRM or ROLLBACK), searching transaction by id and adding participants (new services) into a transaction.

@RestController
@RequestMapping("/transactions")
class TransactionController(val repository: TransactionRepository,
                            val template: RabbitTemplate) {

    @PostMapping
    fun add(@RequestBody transaction: DistributedTransaction): DistributedTransaction =
            repository.save(transaction)

    @GetMapping("/{id}")
    fun findById(@PathVariable id: String): DistributedTransaction? = repository.findById(id)

    @PutMapping("/{id}/finish/{status}")
    fun finish(@PathVariable id: String, @PathVariable status: DistributedTransactionStatus) {
        val transaction: DistributedTransaction? = repository.findById(id)
        if (transaction != null) {
            transaction.status = status
            repository.update(transaction)
            template.convertAndSend("trx-events", DistributedTransaction(id, status))
        }
    }

    @PutMapping("/{id}/participants")
    fun addParticipant(@PathVariable id: String,
                       @RequestBody participant: DistributedTransactionParticipant) =
            repository.findById(id)?.participants?.add(participant)

    @PutMapping("/{id}/participants/{serviceId}/status/{status}")
    fun updateParticipant(@PathVariable id: String,
                          @PathVariable serviceId: String,
                          @PathVariable status: DistributedTransactionStatus) {
        val transaction: DistributedTransaction? = repository.findById(id)
        if (transaction != null) {
            val index = transaction.participants.indexOfFirst { it.serviceId == serviceId }
            if (index != -1) {
                transaction.participants[index].status = status
                template.convertAndSend("trx-events", DistributedTransaction(id, status))
            }
        }
    }

}   

Handling transactions in downstream services

Let’s analyze how our microservices are handling transactions on the example of account. Here’s the implementation of AccountService that is called by the controller for transfering funds from/to account. All methods here are @Transactional and here we need an attention – @Async. It means that each method is running in a new thread and is processing asynchronously. Why? That’s a key concept here. We will block the transaction in order to wait for confirmation from transaction-server, but the main thread used by the controller will not be blocked. It returns the response with the current state of Account immediately.

@Service
@Transactional
@Async
class AccountService(val repository: AccountRepository,
                     var applicationEventPublisher: ApplicationEventPublisher) {
    
    fun payment(id: Int, amount: Int, transactionId: String) =
            transfer(id, amount, transactionId)

    fun withdrawal(id: Int, amount: Int, transactionId: String) =
            transfer(id, (-1) * amount, transactionId)

    private fun transfer(id: Int, amount: Int, transactionId: String) {
        val accountOpt: Optional<Account> = repository.findById(id)
        if (accountOpt.isPresent) {
            val account: Account = accountOpt.get()
            account.balance += amount
            applicationEventPublisher.publishEvent(AccountTransactionEvent(transactionId, account))
            repository.save(account)
        }
    }

}

Here’s the implementation of @Controller class. As you see it is calling methods from AccountService, that are being processed asynchronously. The returned Account object is taken from EventBus bean. This bean is responsible for exchanging asynchronous events within the application scope. En event is sent by the AccountTransactionListener bean responsible for handling Spring transaction events.

@RestController
@RequestMapping("/accounts")
class AccountController(val repository: AccountRepository,
                        val service: AccountService,
                        val eventBus: EventBus) {

    @PostMapping
    fun add(@RequestBody account: Account): Account = repository.save(account)

    @GetMapping("/customer/{customerId}")
    fun findByCustomerId(@PathVariable customerId: Int): List<Account> =
            repository.findByCustomerId(customerId)

    @PutMapping("/{id}/payment/{amount}")
    fun payment(@PathVariable id: Int, @PathVariable amount: Int,
                @RequestHeader("X-Transaction-ID") transactionId: String): Account {
        service.payment(id, amount, transactionId)
        return eventBus.receiveEvent(transactionId)!!.account
    }

    @PutMapping("/{id}/withdrawal/{amount}")
    fun withdrawal(@PathVariable id: Int, @PathVariable amount: Int,
                   @RequestHeader("X-Transaction-ID") transactionId: String): Account {
        service.withdrawal(id, amount, transactionId)
        return eventBus.receiveEvent(transactionId)!!.account
    }

}

The event object exchanged between bean is very simple. It contains an id of transaction and the current Account object.

class AccountTransactionEvent(val transactionId: String, val account: Account)

Finally, let’s take a look at the implementation of AccountTransactionListener bean responsible for handling transactional events. We are using Spring @TransactionalEventListener for annotating methods that should handle incoming events. There are 4 possible event types to handle: BEFORE_COMMIT, AFTER_COMMIT, AFTER_ROLLBACK and AFTER_COMPLETION. There is one very important thing in @TransactionalEventListener, which may be not very intuitive. It is being processed in the same thread as the transaction. So if you would do something that should not block the thread with transaction you should annotate it with @Async. However, in our case this behaviour is required, since we need to block a transactional thread until we receive a confirmation or rollback from transaction-server for a given transaction. These events are sent by transaction-server through RabbitMQ, and they are also exchanged between beans using EventBus. If the status of the received event is different than CONFIRMED we are throwing the exception to rollback transaction.
The AccountTransactionListener is also listening on AFTER_ROLLBACK and AFTER_COMPLETION. After receiving such an event type it is changing the status of the transaction by calling endpoint exposed by transaction-server.

@Component
class AccountTransactionListener(val restTemplate: RestTemplate,
                                 val eventBus: EventBus) {

    @TransactionalEventListener(phase = TransactionPhase.BEFORE_COMMIT)
    @Throws(AccountProcessingException::class)
    fun handleEvent(event: AccountTransactionEvent) {
        eventBus.sendEvent(event)
        var transaction: DistributedTransaction? = null
        for (x in 0..100) {
            transaction = eventBus.receiveTransaction(event.transactionId)
            if (transaction == null)
                Thread.sleep(100)
            else break
        }
        if (transaction == null || transaction.status != DistributedTransactionStatus.CONFIRMED)
            throw AccountProcessingException()
    }

    @TransactionalEventListener(phase = TransactionPhase.AFTER_ROLLBACK)
    fun handleAfterRollback(event: AccountTransactionEvent) {
        restTemplate.put("http://transaction-server/transactions/transactionId/participants/{serviceId}/status/{status}",
                null, "account-service", "TO_ROLLBACK")
    }

    @TransactionalEventListener(phase = TransactionPhase.AFTER_COMPLETION)
    fun handleAfterCompletion(event: AccountTransactionEvent) {
        restTemplate.put("http://transaction-server/transactions/transactionId/participants/{serviceId}/status/{status}",
                null, "account-service", "CONFIRM")
    }
    
}

Here’s the implementation of the bean responsible for receiving asynchronous events from a message broker. As you see after receiving such an event it is using EventBus to forward that event to other beans.

@Component
class DistributedTransactionEventListener(val eventBus: EventBus) {

    @RabbitListener(bindings = [
        QueueBinding(exchange = Exchange(type = ExchangeTypes.TOPIC, name = "trx-events"),
                value = Queue("trx-events-account"))
    ])
    fun onMessage(transaction: DistributedTransaction) {
        eventBus.sendTransaction(transaction)
    }

}

Integration with database

Of course our application is using Postgres as a backend store, so we need to provide integration. In fact, that is the simplest step of our implementation. First we need to add the following 2 dependencies. We will use Spring Data JPA for integration with Postgres.

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
   <groupId>org.postgresql</groupId>
   <artifactId>postgresql</artifactId>
   <scope>runtime</scope>
</dependency>

Our entity is very simple. Besides the id field it contains two fields: customerId and balance.

@Entity
data class Account(@Id @GeneratedValue(strategy = GenerationType.AUTO) val id: Int,
                   val customerId: Int,
                   var balance: Int)

We are using the well-known Spring Data repository pattern.

interface AccountRepository: CrudRepository<Account, Int> {

    fun findByCustomerId(id: Int): List<Account>

}

Here’s the suggested list of configuration settings.

spring:
  application:
    name: account-service
  datasource:
    url: jdbc:postgresql://postgresql:5432/trx
    username: trx
    password: trx
    hikari:
      connection-timeout: 2000
      initialization-fail-timeout: 0
  jpa:
    database-platform: org.hibernate.dialect.PostgreSQLDialect
    hibernate:
      ddl-auto: create
    show-sql: true
    properties:
      hibernate:
        format_sql: true
  rabbitmq:
    host: rabbitmq
    port: 5672
    connection-timeout: 2000

Building order-service

Ok, we have already finished the implementation of transaction-server, and two microservices account-service and product-service. Since the implementation of product-service is very similar to account-service, I have explained everything on the example of account-service. Now, we may proceed to the last part – the implementation of order-service. It is responsible for starting a new transaction and marking it as finished. It also may finish it with rollback.Of course, rollback events may be sent by another two applications as well.
The implementation of @Controller class is visible below. I’ll describe it step by step. We are starting a new distributed transaction by calling POST /transactions endpoint exposed by transaction-server (1). Then we are storing a new order in database (2). When we are calling a transactional method from downstream service we need to set HTTP header X-Transaction-ID. The first transactional method that is called here is PUT /products/{id}/count/{count}(3). It updates the number of products in the store and calculates a final price (4). In the step it is calling another transaction method – this time from account-service (5). It is responsible for withdrawing money from customer accounts. We are enabling Spring transaction events processing (6). In the last step we are generating a random number, and then basing on its value application is throwing an exception to rollback transaction (7).

@RestController
@RequestMapping("/orders")
class OrderController(val repository: OrderRepository,
                      val restTemplate: RestTemplate,
          var applicationEventPublisher: ApplicationEventPublisher) {

    @PostMapping
    @Transactional
    @Throws(OrderProcessingException::class)
    fun addAndRollback(@RequestBody order: Order) {
        var transaction  = restTemplate.postForObject("http://transaction-server/transactions",
                DistributedTransaction(), DistributedTransaction::class.java) // (1)
        val orderSaved = repository.save(order) // (2)
        val product = updateProduct(transaction!!.id!!, order) // (3)
        val totalPrice = product.price * product.count // (4)
        val accounts = restTemplate.getForObject("http://account-service/accounts/customer/{customerId}",
                Array<Account>::class.java, order.customerId)
        val account  = accounts!!.first { it.balance >= totalPrice}
        updateAccount(transaction.id!!, account.id, totalPrice) // (5)
        applicationEventPublisher.publishEvent(OrderTransactionEvent(transaction.id!!)) // (6)
        val r = Random.nextInt(100) // (7)
        if (r % 2 == 0)
            throw OrderProcessingException()
    }

    fun updateProduct(transactionId: String, order: Order): Product {
        val headers = HttpHeaders()
        headers.set("X-Transaction-ID", transactionId)
        val entity: HttpEntity<*> = HttpEntity<Any?>(headers)
        val product = restTemplate.exchange("http://product-service/products/{id}/count/{count}",
                HttpMethod.PUT, null, Product::class.java, order.id, order.count)
        return product.body!!
    }

    fun updateAccount(transactionId: String, accountId: Int, totalPrice: Int): Account {
        val headers = HttpHeaders()
        headers.set("X-Transaction-ID", transactionId)
        val entity: HttpEntity<*> = HttpEntity<Any?>(headers)
        val account = restTemplate.exchange("http://account-service/accounts/{id}/withdrawal/{amount}",
                HttpMethod.PUT, null, Account::class.java, accountId, totalPrice)
        return account.body!!
    }
}

Conclusion

Even a trivial implementation of distributed transactions in microservices, like the one, demonstrated in this article, can be complicated. As you see we need to add a new element to our architecture, transaction-server, responsible only for distributed transaction management. We also have to add a message broker in order to exchange events between our applications and transaction-server. However, many of you were asking me about distributed transactions in the microservices world, so I decided to build that simple demo. I’m waiting for your feedback and opinions.

The post Distributed Transactions in Microservices with Spring Boot appeared first on Piotr's TechBlog.

Ok, we know how it works, now the question is how to automatically test it. With Testcontainers it is possible with just a few lines of code.

1. Building application

Let’s begin from a short intro to the application code. It is very simple. Here’s the list of dependencies required for building an application that exposes REST API, and integrates with Postgres and Vault.

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-vault-config</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-vault-config-databases</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
   <groupId>org.postgresql</groupId>
   <artifactId>postgresql</artifactId>
   <version>42.2.5</version>
</dependency>

Application connects to Postgres, enables integration with Vault via Spring Cloud Vault, and automatically creates/updates tables on startup.

spring:
  application:
    name: callme-service
  cloud:
    vault:
      uri: http://192.168.99.100:8200
      token: ${VAULT_TOKEN}
      postgresql:
        enabled: true
        role: default
        backend: database
  datasource:
    url: jdbc:postgresql://192.168.99.100:5432/postgres
  jpa.hibernate.ddl-auto: update

It exposes the single endpoint. The following method is responsible for handling incoming requests. It just inserts a record to the database and returns a response with app name, version and id of inserted record.

@RestController
@RequestMapping("/callme")
public class CallmeController {

   private static final Logger LOGGER = LoggerFactory.getLogger(CallmeController.class);
   
   @Autowired
   Optional<BuildProperties> buildProperties;
   @Autowired
   CallmeRepository repository;
   
   @GetMapping("/message/{message}")
   public String ping(@PathVariable("message") String message) {
      Callme c = repository.save(new Callme(message, new Date()));
      if (buildProperties.isPresent()) {
         BuildProperties infoProperties = buildProperties.get();
         LOGGER.info("Ping: name={}, version={}", infoProperties.getName(), infoProperties.getVersion());
         return infoProperties.getName() + ":" + infoProperties.getVersion() + ":" + c.getId();
      } else {
         return "callme-service:"  + c.getId();
      }
   }
   
}

2. Enabling Testcontainers

To enable Testcontainers for our project we need to include some dependencies to our Maven pom.xml. We have dedicated modules for Postgres and Vault. We also include Spring Boot Test dependency, because we would like to test the whole Spring Boot app.

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-test</artifactId>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.testcontainers</groupId>
   <artifactId>vault</artifactId>
   <version>1.10.5</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.testcontainers</groupId>
   <artifactId>testcontainers</artifactId>
   <version>1.10.5</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.testcontainers</groupId>
   <artifactId>postgresql</artifactId>
   <version>1.10.5</version>
   <scope>test</scope>
</dependency>

3. Running Vault test container

Testcontainers framework supports JUnit 4/JUnit 5 and Spock. The Vault container can be started before tests if it is annotated with @Rule or @ClassRule. By default it uses version 0.7, but we can override it with the newest version, which is 1.0.2. We also may set a root token, which is then required by Spring Cloud Vault for integration with Vault.

@ClassRule
public static VaultContainer vaultContainer = new VaultContainer<>("vault:1.0.2")
   .withVaultToken("123456")
   .withVaultPort(8200);

That root token can be overridden before starting a JUnit test on the test class.

@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT, properties = {
    "spring.cloud.vault.token=123456"
})
public class CallmeTest { ... }

4. Running Postgres with Testcontainers

As an alternative to @ClassRule, we can manually start the container in a @BeforeClass or @Before method in the test. With this approach you will also have to stop it manually in @AfterClass or @After method. We start the Postgres container manually, because by default it is exposed on a dynamically generated port, which needs to be set for the Spring Boot application before starting the test. The listen port is returned by method getFirstMappedPort invoked on PostgreSQLContainer.

private static PostgreSQLContainer postgresContainer = new PostgreSQLContainer()
   .withDatabaseName("postgres")
   .withUsername("postgres")
   .withPassword("postgres123");
   
@BeforeClass
public static void init() throws IOException, InterruptedException {
   postgresContainer.start();
   int port = postgresContainer.getFirstMappedPort();
   System.setProperty("spring.datasource.url", String.format("jdbc:postgresql://192.168.99.100:%d/postgres", postgresContainer.getFirstMappedPort()));
   // ...
}

@AfterClass
public static void shutdown() {
   postgresContainer.stop();
}

5. Integrating Vault and Postgres containers

Once we have succesfully started both Vault and Postgres containers, we need to integrate them via Vault secret engine. First, we need to enable database secret engine Vault. After that we must configure a connection to Postgres. The last step is to configure a role. A role is a logical name that maps to a policy used to generate those credentials. All these actions may be performed using Vault commands. You can launch commands on the Vault container using execInContainer method. Vault configuration commands should be executed just after Postgres container startup.

@BeforeClass
public static void init() throws IOException, InterruptedException {
   postgresContainer.start();
   int port = postgresContainer.getFirstMappedPort();
   System.setProperty("spring.datasource.url", String.format("jdbc:postgresql://192.168.99.100:%d/postgres", postgresContainer.getFirstMappedPort()));
   vaultContainer.execInContainer("vault", "secrets", "enable", "database");
   String url = String.format("connection_url=postgresql://{{username}}:{{password}}@192.168.99.100:%d?sslmode=disable", port);
   vaultContainer.execInContainer("vault", "write", "database/config/postgres", "plugin_name=postgresql-database-plugin", "allowed_roles=default", url, "username=postgres", "password=postgres123");
   vaultContainer.execInContainer("vault", "write", "database/roles/default", "db_name=postgres",
      "creation_statements=CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';GRANT SELECT, UPDATE, INSERT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";GRANT USAGE,  SELECT ON ALL SEQUENCES IN SCHEMA public TO \"{{name}}\";",
      "default_ttl=1h", "max_ttl=24h");
}

6. Running application tests

Finally, we may run application tests. We just call the single endpoint exposed by the app using TestRestTemplate, and verify the output.

@Autowired
TestRestTemplate template;

@Test
public void test() {
   String res = template.getForObject("/callme/message/{message}", String.class, "Test");
   Assert.assertNotNull(res);
   Assert.assertTrue(res.endsWith("1"));
}

If you are interested in what exactly happens during the test you can set a breakpoint inside the test method and execute docker ps command manually.

The post Testing Spring Boot Integration with Vault and Postgres using Testcontainers Framework appeared first on Piotr's TechBlog.

Today I’m going to show you how to integrate your Spring Boot application with HashiCorp’s Vault in order to store your sensitive data properly. The first good news is that you don’t have to create any keys or certificates for encryption and decryption, because Vault will do it in your place. In this article in a few areas I’ll refer to my previous article about HashiCorp’s solutions Deploying Spring Cloud Microservices on HashiCorp’s Nomad. Now, as then, I also deploy my sample applications on Nomad to take advantage of built-in integration between those two very interesting HashiCorp’s tools. We will also use another HashiCorp’s solution for service discovery in inter-service communication – Consul. It’s also worth mentioning that Spring Cloud provides a dedicated project for integration with Vault – Spring Cloud Vault.

Architecture

The sample presented in this article will consist of two applications deployed on HashiCorp’s Nomad callme-service and caller-service. Microservice caller-service is the endpoint exposed by callme-service. An inter-service communication is performed using the name of the target application registered in Consul server. Microservice callme-service will store the history of all interactions triggered by caller-service in the database. The credentials to the database are stored on Vault. Nomad is integrated with Vault and stores the root token, which is not visible by the applications. The architecture of the described solution is visible on the following picture.

The current sample is pretty similar to the sample presented in my article Deploying Spring Cloud Microservices on Hashicorp’s Nomad. It is also available in the same repository on GitHub sample-nomad-java-service, but in the different branch vault. The current sample add an integration with PostgreSQL and Vault server for managing credentials to database.

1. Running Vault

We will run Vault inside the Docker container in a development mode. Server in development mode does not require any further setup, it is ready to use just after startup. It provides in-memory encrypted storage and unsecure (HTTP) connection, which is not a problem for a demo purposes. We can override default server IP address and listening port by setting environment property VAULT_DEV_LISTEN_ADDRESS, but we won’t do that. After startup our instance of Vault is available on port 8200. We can use the admin web console, which is for me available under address http://192.168.99.100:8200. The current version of Vault is 1.0.0.

$ docker run --cap-add=IPC_LOCK -d --name vault -p 8200:8200 vault

It is possible to login using different methods, but the most suitable way for us is through a token. To do that we have to display container logs using command docker logs vault, and then copy Root Token as shown below.

Now you can login to the Vault web console.

2. Integration with Postgres database

In Vault we can create Secret Engine that connects to other services and generates dynamic credentials on demand. Secrets engines are available under path. There is a dedicated engine for the various databases, for example PostgreSQL. Before activating such an engine we should run an instance of Postgres database. This time we will also use a Docker container. It is possible to set login and password to the database using environment variables.

$ docker run -d --name postgres -p 5432:5432 -e POSTGRES_PASSWORD=postgres123456 -e POSTGRES_USER=postgres postgres

After starting the database, we may proceed to the engine configuration in the Vault web console. First, let’s create our first secret engine. We may choose between some different types of engine. The right choice for now is Databases.

You can apply a new configuration to Vault using vault command or by HTTP API. Vault web console provides a terminal for running CLI commands, but it could be problematic in some cases. For example, I have a problem with escaping strings in some SQL commands, and therefore I had to add it using HTTP API. No matter which method you use, the next steps are the same. Following Vault documentation we first need to configure a plugin for PostgreSQL database and then provide connection settings and credentials.

$ vault write database/config/postgres plugin_name=postgresql-database-plugin allowed_roles="default" connection_url="postgresql://{{username}}:{{password}}@192.168.99.100:5432?sslmode=disable" username="postgres" password="postgres123456"

Alternatively, you can perform the same action using the HTTP API method. To authenticate against Vault we need to add header X-Vault-Token with root token. I have disabled SSL for connection with Postgres by setting sslmode=disable. There is only one role allowed to use this plugin: default. Now, let’s configure that role.

$ curl --header "X-Vault-Token: s.44GiacPqbV78fNbmoWK4mdYq" --request POST --data '{"plugin_name": "postgresql-database-plugin","allowed_roles": "default","connection_url": "postgresql://{{username}}:{{password}}@localhost:5432?sslmode=disable","username": "postgres","password": "postgres123456"}' http://192.168.99.100:8200/v1/database/config/postgres

The role can be created either with CLI or with HTTP API. The name of the role should be the same as the name passed in field allowed_roles in the previous step. We also have to set a target database name and SQL statement that creates users with privileges.

$ vault write database/roles/default db_name=postgres creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';GRANT SELECT, UPDATE, INSERT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";GRANT USAGE,  SELECT ON ALL SEQUENCES IN SCHEMA public TO \"{{name}}\";" default_ttl="1h" max_ttl="24h"

Alternatively you can call the following HTTP API endpoint.

$ curl --header "X-Vault-Token: s.44GiacPqbV78fNbmoWK4mdYq" --request POST --data '{"db_name":"postgres", "creation_statements": ["CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}';GRANT SELECT, UPDATE, INSERT ON ALL TABLES IN SCHEMA public TO \"{{name}}\";GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO \"{{name}}\";"]}' http://192.168.99.100:8200/v1/database/roles/default

And that’s all. Now, we can test our configuration using command with role’s name vault read database/creds/default as shown below. You can login to the database using returned credentials. By default, they are valid for one hour.

3. Enabling Spring Cloud Vault

We have succesfully configured a secret engine that is responsible for creating users on Postgres. Now, we can proceed to the development and integrate our application with Vault. Fortunately, there is a project Spring Cloud Vault, which provides out-of-the-box integration with Vault database secret engines. The only thing we have to do is to include Spring Cloud Vault to our project and provide some configuration settings. Let’s start from setting Spring Cloud Release Train. We use the newest stable version Finchley.SR2.

<dependencyManagement>
   <dependencies>
      <dependency>
         <groupId>org.springframework.cloud</groupId>
         <artifactId>spring-cloud-dependencies</artifactId>
         <version>Finchley.SR2</version>
         <type>pom</type>
         <scope>import</scope>
      </dependency>
   </dependencies>
</dependencyManagement>

We have to include two dependencies to our pom.xml. Starter spring-cloud-starter-vault-config is responsible for loading configuration from Vault and spring-cloud-vault-config-databases responsible for integration with secret engines for databases.

<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-starter-vault-config</artifactId>
</dependency>
<dependency>
   <groupId>org.springframework.cloud</groupId>
   <artifactId>spring-cloud-vault-config-databases</artifactId>
</dependency>

The sample application also connects to Postgres database, so we will include the following dependencies.

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
   <groupId>org.postgresql</groupId>
   <artifactId>postgresql</artifactId>
   <version>42.2.5</version>
</dependency>

The only thing we have to do is to configure integration with Vault via Spring Cloud Vault. The following configuration settings should be placed in bootstrap.yml (no application.yml). Because we run our application on Nomad server, we use the port number dynamically set by Nomad available under environment property NOMAD_HOST_PORT_http and secret token from Vault available under environment property VAULT_TOKEN.

server:
  port: ${NOMAD_HOST_PORT_http:8091}

spring:
  application:
    name: callme-service
  cloud:
    vault:
      uri: http://192.168.99.100:8200
      token: ${VAULT_TOKEN}
      postgresql:
        enabled: true
        role: default
        backend: database
  datasource:
    url: jdbc:postgresql://192.168.99.100:5432/postgres

The important part of the configuration visible above is under the property spring.cloud.vault.postgresql. Following Spring Cloud documentation “Username and password are stored in spring.datasource.username and spring.datasource.password so using Spring Boot will pick up the generated credentials for your DataSource without further configuration”. Spring Cloud Vault is connecting with Vault, and then using role default (previously created on Vault) to generate new credentials to database. Those credentials are injected into spring.datasource properties. Then, the application is connected to the database using injected credentials. Everything works fine. Now, let’s try to run our applications on Nomad.

4. Deploying apps on Nomad

Before starting Nomad node we should also run Consul using its Docker container. Here’s a Docker command that starts a single node Consul instance.

$ docker run -d --name consul -p 8500:8500 consul

After that we can configure connection settings to Consul and Vault in Nomad configuration. I have created the file nomad.conf. Nomad is authenticating itself against Vault using root token. Connection with Consul is not secured. Sometimes it is also required to set network interface name and total CPU on the machine for the Nomad client. Most clients are able to determine it automatically, but it does not work for me.

client {
  network_interface = "Połączenie lokalne 4"
  cpu_total_compute = 10400
}

consul {
  address = "192.168.99.100:8500"
}

vault {
  enabled = true
  address = "http://192.168.99.100:8200"
  token = "s.6jhQ1WdcYrxpZmpa0RNd0LMw"
}

Let’s run Nomad in development mode passing configuration file location.

$ nomad agent -dev -config=nomad.conf

If everything works fine you should see the similar log on startup.

Once we have succesfully started Nomad agent integrated with Consul and Vault, we can proceed to the application deployment. First build the whole project with mvn clean install command. The next step is to prepare Nomad’s job descriptor file. For more details about the Nomad deployment process and its descriptor file you can refer to my previous article about it (mentioned in the preface of this article). Descriptor file is available inside application GitHub under path callme-service/job.nomad for callme-service, and caller-service/job.nomad for caller-service.

job "callme-service" {
   datacenters = ["dc1"]
   type = "service"
   group "callme" {
      count = 2
      task "api" {
         driver = "java"
         config {
            jar_path    = "C:\\Users\\minkowp\\git\\sample-nomad-java-services-idea\\callme-service\\target\\callme-service-1.0.0-SNAPSHOT.jar"
            jvm_options = ["-Xmx256m", "-Xms128m"]
         }
         resources {
            cpu    = 500 # MHz
            memory = 300 # MB
            network {
               port "http" {}
            }
         }
         service {
            name = "callme-service"
            port = "http"
         }
         vault {
            policies = ["nomad"]
         }
      }
      restart {
         attempts = 1
      }
   }
}

You will have to change the value of jar_path property with your path of application binaries. Before applying this deployment to Nomad we will have to add some additional configuration on Vault. When adding integration with Vault we have to pass the name of policies used for checking permissions. I set the policy with the name nomad, which now has to be created in Vault. Our application requires a permission for reading paths /secret/* and /database/* as shown below.

Finally, we can deploy our application callme-service on Nomad by executing the following command.

$ nomad job run job.nomad

The similar descriptor file is available for caller-service, so we can also deploy it. All the microservice has been registered in Consul as shown below.

Here is the list of registered instances of caller-service. As you can see on the picture below it is available under port 25816.

You can also take a look at the Nomad jobs view.

The post Secure Spring Cloud Microservices with Vault and Nomad appeared first on Piotr's TechBlog.