circleci Archives - Piotr's TechBlog

Backstage on Kubernetes

piotr.minkowski — Fri, 28 Jun 2024 15:06:35 +0000

In this article, you will learn how to integrate Backstage with Kubernetes. We will run Backstage in two different ways. Firstly, it will run outside the cluster and connect with Kubernetes via the API. In the second scenario, we will deploy it directly on the cluster using the official Helm chart. Our instance of Backstage will connect Argo CD and Prometheus deployed on Kubernetes, to visualize the status of Argo CD synchronization and basic metrics related to the app.

This exercise continues the work described in my previous article about Backstage. So, before you start, you should read that article to understand the whole concept. In many places, I will refer to something that was described and done in the previous article. I’m describing there how to configure and run Backstage, and also how to build a basic template for the sample Spring Boot app. You should be familiar with all those basic terms, to fully understand what happens in the current exercise.

Source Code

If you would like to try it by yourself, you may always take a look at my source code. Our sample GitHub repository contains software templates written in the Backstage technology called Skaffolder. In this article, we will analyze a template dedicated to Kubernetes available in the templates/spring-boot-basic-on-kubernetes directory. After cloning this repository, you should just follow my instructions.

Here’s the structure of our repository. Besides the template, it also contains the Argo CD template with YAML deployment manifests to apply on Kubernetes.

.
├── skeletons
│   └── argocd
│       └── manifests
│           ├── deployment.yaml
│           └── service.yaml
├── templates
│   └── spring-boot-basic-on-kubernetes
│       ├── skeleton
│       │   ├── README.md
│       │   ├── catalog-info.yaml
│       │   ├── k8s
│       │   │   ├── deployment.yaml
│       │   │   └── kind-cluster-test.yaml
│       │   ├── pom.xml
│       │   ├── renovate.json
│       │   ├── skaffold.yaml
│       │   └── src
│       │       ├── main
│       │       │   ├── java
│       │       │   │   └── ${{values.javaPackage}}
│       │       │   │       ├── Application.java
│       │       │   │       ├── controller
│       │       │   │       │   └── ${{values.domainName}}Controller.java
│       │       │   │       └── domain
│       │       │   │           └── ${{values.domainName}}.java
│       │       │   └── resources
│       │       │       └── application.yml
│       │       └── test
│       │           ├── java
│       │           │   └── ${{values.javaPackage}}
│       │           │       └── ${{values.domainName}}ControllerTests.java
│       │           └── resources
│       │               └── k6
│       │                   └── load-tests-add.js
│       └── template.yaml
└── templates.yaml

ShellSession

There is also another Git repository related to this article. It contains the modified source code of Backstage with several plugins installed and configured. The process of extending Backstage with plugins is described in detail in this article. So, you can start from scratch and apply my instructions step by step. But you can clone the final version of the code committed inside that repo and run it on your laptop as well.

Run and Prepare Kubernetes

Before we start with Backstage, we need to run and configure our instance of the Kubernetes cluster. It can be, for example, Minikube. Once you have the running cluster, you can obtain its control plane URL by executing the following command. As you see, my Minikube is available under the https://127.0.0.1:55782 address, so I will have to set it in the Backstage configuration later.

$ kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:55782
...
$ export K8S_URL=https://127.0.0.1:55782

ShellSession

We need to install Prometheus and Argo CD on our Kubernetes. In order to install Prometheus, we will use the kube-prometheus-stack Helm chart. Firstly, we should add the Prometheus chart repository with the following command:

$ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

ShellSession

Then, we can run the following command to install Prometheus in the monitoring namespace:

$ helm install kube-prometheus-stack prometheus-community/kube-prometheus-stack \
  --version 60.3.0 \
  -n monitoring --create-namespace

ShellSession

The same as with Prometheus, for Argo CD we need to add the chart repository first:

$ helm repo add argo https://argoproj.github.io/argo-helm

ShellSession

For Argo CD we need an additional configuration to be provided inside the values.yaml file. We have to create the user for the Backstage with privileges to call HTTP API with the apiKey authentication. It is required to automatically create an Argo CD Application from the Skaffolder template.

configs:
  cm:
    accounts.backstage: apiKey,login
  rbac:
    policy.csv: |
      p, backstage, applications, *, */*, allow

YAML

Let’s install Argo CD in the argocd namespace using the settings from values.yaml file:

$ helm install argo-cd argo/argo-cd \
  --version 7.2.0 \
  -f values.yaml \
  -n argocd --create-namespace

ShellSession

That’s not all. We still need to generate the apiKey for the backstage user. Firstly, let’s enable port forwarding for both Argo CD and Prometheus services to access their APIs over localhost.

$ kubectl port-forward svc/argo-cd-argocd-server 8443:443 -n argocd
$ kubectl port-forward svc/kube-prometheus-stack-prometheus 9090 -n monitoring

ShellSession

In order to generate the apiKey for the backstage user we need to sign in to Argo CD with the argocd CLI as the admin user. Then, we need to run the following command for the backstage account and export the generated token as the ARGOCD_TOKEN env variable:

$ argocd account generate-token --account backstage
$ export ARGOCD_TOKEN='argocd.token='

ShellSession

Finally, let’s obtain the long-lived API token for Kubernetes by creating a secret:

apiVersion: v1
kind: Secret
metadata:
  name: default-token
  namespace: default
  annotations:
    kubernetes.io/service-account.name: default
type: kubernetes.io/service-account-token

YAML

Then, we can copy and export it as the K8S_TOKEN environment variable with the following command:

$ export K8S_TOKEN=$(kubectl get secret default-token -o go-template='{{.data.token | base64decode}}')

ShellSession

Just for the testing purposes, we add the cluster-admin role to the default ServiceAccount.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: default-admin
subjects:
- kind: ServiceAccount
  name: default
  namespace: default
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

YAML

Modify App Source Code Skeleton for Kubernetes

First of all, we will modify several things in the application source code skeleton. In order to build the container image, we include the jib-maven-plugin in the Maven pom.xml. This plugin will be activated under the jib Maven profile.


  
    jib
    
      false
    
    
      
        
          com.google.cloud.tools
          jib-maven-plugin
          3.4.3
          
            
              eclipse-temurin:21-jdk-ubi9-minimal

XML

Our source code repository will also contain the Skaffold configuration file. With Skaffold we can easily build an image and deploy an app to Kubernetes in a single step. The address of the image depends on the orgName and appName parameters in the Skaffolder template. During the image build we skip the tests and activate the Maven jib profile.

apiVersion: skaffold/v4beta5
kind: Config
metadata:
  name: ${{ values.appName }}
build:
  artifacts:
    - image: ${{ values.orgName }}/${{ values.appName }}
      jib:
        args:
          - -Pjib
          - -DskipTests
manifests:
  rawYaml:
    - k8s/deployment.yaml
deploy:
  kubectl: {}

YAML

In order to deploy the app on Kubernetes, Skaffold is looking for the k8s/deployment.yaml manifest. We will use this deployment manifest only for development and automated test purposes. In the “production” we will keep the YAML manifests in a separate Git repository and apply them through Argo CD. Once we provide a change in the source CircleCI will try to deploy the app on the temporary Kind cluster. Therefore, our Service is exposed as a NodePort under the 30000 port.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ${{ values.appName }}
spec:
  selector:
    matchLabels:
      app: ${{ values.appName }}
  template:
    metadata:
      annotations:
        prometheus.io/path: /actuator/prometheus
        prometheus.io/scrape: "true"
        prometheus.io/port: "8080"
      labels:
        app: ${{ values.appName }}
    spec:
      containers:
        - name: ${{ values.appName }}
          image: ${{ values.orgName }}/${{ values.appName }}
          ports:
            - containerPort: 8080
          readinessProbe:
            httpGet:
              port: 8080
              path: /actuator/health/readiness
              scheme: HTTP
            timeoutSeconds: 1
            periodSeconds: 10
            successThreshold: 1
            failureThreshold: 3
          resources:
            limits:
              memory: 1024Mi
---
apiVersion: v1
kind: Service
metadata:
  name: ${{ values.appName }}
spec:
  type: NodePort
  selector:
    app: ${{ values.appName }}
  ports:
    - port: 8080
      nodePort: 30000

YAML

Let’s switch to the CircleCi configuration file. It also contains several changes related to Kubernetes. We need to include the image-build job responsible for building and pushing the app image to the target registry using Jib. We also include the deploy-k8s job to perform a test deployment to the Kind cluster. In this job, we have to install Skaffold and Kind tools on the CircleCI executor machine. Once the Kind cluster is up and ready, we deploy the app there by executing the skaffold run command.

version: 2.1

jobs:
  analyze:
    docker:
      - image: 'cimg/openjdk:21.0.2'
    steps:
      - checkout
      - run:
          name: Analyze on SonarCloud
          command: mvn verify sonar:sonar -DskipTests
  test:
    executor: machine_executor_amd64
    steps:
      - checkout
      - run:
          name: Install OpenJDK 21
          command: |
            java -version
            sudo apt-get update && sudo apt-get install openjdk-21-jdk
            sudo update-alternatives --set java /usr/lib/jvm/java-21-openjdk-amd64/bin/java
            sudo update-alternatives --set javac /usr/lib/jvm/java-21-openjdk-amd64/bin/javac
            java -version
            export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
      - run:
          name: Maven Tests
          command: mvn test
  deploy-k8s:
    executor: machine_executor_amd64
    steps:
      - checkout
      - run:
          name: Install Kubectl
          command: |
            curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
            chmod +x kubectl
            sudo mv ./kubectl /usr/local/bin/kubectl
      - run:
          name: Install Skaffold
          command: |
            curl -Lo skaffold https://storage.googleapis.com/skaffold/releases/latest/skaffold-linux-amd64
            chmod +x skaffold
            sudo mv skaffold /usr/local/bin
      - run:
          name: Install Kind
          command: |
            [ $(uname -m) = x86_64 ] && curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.20.0/kind-linux-amd64
            chmod +x ./kind
            sudo mv ./kind /usr/local/bin/kind
      - run:
          name: Install OpenJDK 21
          command: |
            java -version
            sudo apt-get update && sudo apt-get install openjdk-21-jdk
            sudo update-alternatives --set java /usr/lib/jvm/java-21-openjdk-amd64/bin/java
            sudo update-alternatives --set javac /usr/lib/jvm/java-21-openjdk-amd64/bin/javac
            java -version
            export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
      - run:
          name: Create Kind Cluster
          command: |
            kind create cluster --name c1 --config k8s/kind-cluster-test.yaml
      - run:
          name: Deploy to K8s
          command: |
            export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
            skaffold run
      - run:
          name: Delete Kind Cluster
          command: |
            kind delete cluster --name c1
  image-push:
    docker:
      - image: 'cimg/openjdk:21.0.2'
    steps:
      - checkout
      - run:
          name: Build and push image to DockerHub
          command: mvn compile jib:build -Pjib -Djib.to.image=${{ values.orgName }}/${{ values.appName }}:latest -Djib.to.auth.username=${DOCKER_LOGIN} -Djib.to.auth.password=${DOCKER_PASSWORD} -DskipTests

executors:
  machine_executor_amd64:
    machine:
      image: ubuntu-2204:2023.10.1
    environment:
      architecture: "amd64"
      platform: "linux/amd64"

workflows:
  maven_test:
    jobs:
      - test
      - analyze:
          context: SonarCloud
      - deploy-k8s:
          requires:
            - test
      - image-push:
          context: Docker
          requires:
            - deploy-k8s

YAML

Install Backstage Plugins for Kubernetes

In the previous article about Backstage, we learned how to install plugins for GitHub, CircleCI, and Sonarqube integration. We will still use those plugins but also extend our Backstage instance with some additional plugins dedicated mostly to the Kubernetes-native environment. We will install the following plugins: Kubernetes (backend + frontend), HTTP Request Action (backend), Argo CD (frontend), and Prometheus (frontend). Let’s begin with the Kubernetes plugin.

Install the Kubernetes Plugin

In the first step, we install the Kubernetes frontend plugin. It allows us to view the app pods running on Kubernetes in the Backstage UI. In order to install it, we need to execute the following yarn command:

$ yarn --cwd packages/app add @backstage/plugin-kubernetes

ShellSession

Then, we have to make some changes in the packages/app/src/components/catalog/EntityPage.tsx file. We should import the EntityKubernetesContent component, and then include it in the serviceEntityPage object as a new route on the frontend.

... );" style="color:#d8dee9ff;display:none" aria-label="Copy" class="code-block-pro-copy-button">

import { EntityKubernetesContent } from '@backstage/plugin-kubernetes';

const serviceEntityPage = (
  <EntityLayout>
    ...
    <EntityLayout.Route path="/kubernetes" title="Kubernetes">
      <EntityKubernetesContent refreshIntervalMs={30000} />
    EntityLayout.Route>
    ...
  EntityLayout>
);

TypeScript

We also need to install the Kubernetes backend plugin, to make it work on the frontend site. Here’s the required yarn command:

$ yarn --cwd packages/backend add @backstage/plugin-kubernetes-backend

ShellSession

Then, we should register the plugin-kubernetes-backend module in the packages/backend/src/index.ts file.

import { createBackend } from '@backstage/backend-defaults';

const backend = createBackend();

backend.add(import('@backstage/plugin-app-backend/alpha'));
backend.add(import('@backstage/plugin-proxy-backend/alpha'));
backend.add(import('@backstage/plugin-scaffolder-backend/alpha'));
backend.add(import('@backstage/plugin-techdocs-backend/alpha'));
backend.add(import('@backstage/plugin-auth-backend'));
backend.add(import('@backstage/plugin-auth-backend-module-guest-provider'));
backend.add(import('@backstage/plugin-catalog-backend/alpha'));
backend.add(
  import('@backstage/plugin-catalog-backend-module-scaffolder-entity-model'),
);
backend.add(import('@backstage/plugin-permission-backend/alpha'));
backend.add(import('@backstage/plugin-permission-backend-module-allow-all-policy'));
backend.add(import('@backstage/plugin-search-backend/alpha'));
backend.add(import('@backstage/plugin-search-backend-module-catalog/alpha'));
backend.add(import('@backstage/plugin-search-backend-module-techdocs/alpha'));

backend.add(import('@backstage/plugin-scaffolder-backend-module-github'));
backend.add(import('@backstage-community/plugin-sonarqube-backend'));
backend.add(import('@backstage/plugin-kubernetes-backend/alpha'));

backend.start();

TypeScript

Install the Argo CD Plugin

We also integrate our instance of Backstage with Argo CD running on Kubernetes. Firstly, we should execute the following yarn command:

$ yarn --cwd packages/app add @roadiehq/backstage-plugin-argo-cd

ShellSession

Then, we need to update the EntityPage.tsx file. We will add the EntityArgoCDOverviewCard component inside the overviewContent object.

... Boolean(isArgocdAvailable(e))}> ... );" style="color:#d8dee9ff;display:none" aria-label="Copy" class="code-block-pro-copy-button">

import {
  EntityArgoCDOverviewCard,
  isArgocdAvailable
} from '@roadiehq/backstage-plugin-argo-cd';

const overviewContent = (
  <Grid container spacing={3} alignItems="stretch">
  ...
    <EntitySwitch>
      <EntitySwitch.Case if={e => Boolean(isArgocdAvailable(e))}>
        <Grid item sm={4}>
          <EntityArgoCDOverviewCard />
        Grid>
      EntitySwitch.Case>
    EntitySwitch>
  ...
  Grid>
);

TSX

Install Prometheus Plugin

The steps for the Prometheus Plugin are pretty similar to those for the Argo CD Plugin. Firstly, we should execute the following yarn command:

$ yarn --cwd packages/app add @roadiehq/backstage-plugin-prometheus

ShellSession

Then, we need to update the EntityPage.tsx file. We will add the EntityPrometheusContent component inside the seerviceEntityPage object.

... );" style="color:#d8dee9ff;display:none" aria-label="Copy" class="code-block-pro-copy-button">

import {
  EntityPrometheusContent,
} from '@roadiehq/backstage-plugin-prometheus';

const serviceEntityPage = (
  <EntityLayout>
    ...
    <EntityLayout.Route path="/kubernetes" title="Kubernetes">
      <EntityKubernetesContent refreshIntervalMs={30000} />
    EntityLayout.Route>
    <EntityLayout.Route path="/prometheus" title="Prometheus">
      <EntityPrometheusContent />
    EntityLayout.Route>
    ...
  EntityLayout>
);

TSX

Install HTTP Request Action Plugin

This plugin is not related to Kubernetes. It allows us to integrate with third-party solutions through the HTTP API services. As you probably remember, we have already integrated with Sonarcloud and CircleCI in the Backstage UI. However, we didn’t create any projects there. We could just view the history of builds or scans for the previously created projects in Sonarcloud or CircleCI. It’s time to change it in our template! Thanks to the HTTP Request Action plugin we will create the Argo CD Application through the REST API. As always, we need to execute the yarn add command to install the backend plugin:

$ yarn --cwd packages/backend add @roadiehq/scaffolder-backend-module-http-request

ShellSession

Then, we will register it in the index.ts file:

import { createBackend } from '@backstage/backend-defaults';

const backend = createBackend();

backend.add(import('@backstage/plugin-app-backend/alpha'));
backend.add(import('@backstage/plugin-proxy-backend/alpha'));
backend.add(import('@backstage/plugin-scaffolder-backend/alpha'));
backend.add(import('@backstage/plugin-techdocs-backend/alpha'));
backend.add(import('@backstage/plugin-auth-backend'));
backend.add(import('@backstage/plugin-auth-backend-module-guest-provider'));
backend.add(import('@backstage/plugin-catalog-backend/alpha'));
backend.add(
  import('@backstage/plugin-catalog-backend-module-scaffolder-entity-model'),
);
backend.add(import('@backstage/plugin-permission-backend/alpha'));
backend.add(
  import('@backstage/plugin-permission-backend-module-allow-all-policy'),
);
backend.add(import('@backstage/plugin-search-backend/alpha'));
backend.add(import('@backstage/plugin-search-backend-module-catalog/alpha'));
backend.add(import('@backstage/plugin-search-backend-module-techdocs/alpha'));

backend.add(import('@backstage/plugin-scaffolder-backend-module-github'));
backend.add(import('@backstage-community/plugin-sonarqube-backend'));
backend.add(import('@backstage/plugin-kubernetes-backend/alpha'));
backend.add(import('@roadiehq/scaffolder-backend-module-http-request/new-backend'));

backend.start();

TypeScript

After that, we can modify a Skaffolder template used in the previous article with some additional steps.

Prepare Backstage Template for Kubernetes

Once we have all the things in place, we can modify a previous template for the standard Spring Boot app to adapt it to the Kubernetes requirements.

Create Skaffolder Template

First of all, we add a single input parameter that indicates the target namespace in Kubernetes for running our app (1). Then, we include some additional action steps. In the first of them, we generate the repository with the YAML configuration manifests for Argo CD (2). Then, we will publish that repository on GitHub under the ${{parameters.appName}}-gitops name (3).

After that, we will use the HTTP Request Action plugin to automatically follow a new repository in CircleCI (5). Once we create such a repository in the previous step, CircleCI automatically starts a build after detecting it. We also use the HTTP Request Action plugin to create a new repository on Sonarcloud under the same name as the ${{parameters.appName}} (4). Finally, we integrate with Argo CD through the API to create a new Application responsible for applying app Deployment to Kubernetes (6). This Argo CD Application will access the previously published config repository with the -config suffix in the name and apply manifests inside the manifests directory

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: spring-boot-basic-on-kubernetes-template
  title: Create a Spring Boot app for Kubernetes
  description: Create a Spring Boot app for Kubernetes
  tags:
    - spring-boot
    - java
    - maven
    - circleci
    - renovate
    - sonarqube
    - kubernetes
    - argocd
spec:
  owner: piomin
  system: microservices
  type: service

  parameters:
    - title: Provide information about the new component
      required:
        - orgName
        - appName
        - domainName
        - repoBranchName
        - groupId
        - javaPackage
        - apiPath
        - namespace
        - description
      properties:
        orgName:
          title: Organization name
          type: string
          default: piomin
        appName:
          title: App name
          type: string
          default: sample-spring-boot-app-k8s
        domainName:
          title: Name of the domain object
          type: string
          default: Person
        repoBranchName:
          title: Name of the branch in the Git repository
          type: string
          default: master
        groupId:
          title: Maven Group ID
          type: string
          default: pl.piomin.services
        javaPackage:
          title: Java package directory
          type: string
          default: pl/piomin/services
        apiPath:
          title: REST API path
          type: string
          default: /api/v1
        # (1)
        namespace:
          title: The target namespace on Kubernetes
          type: string
          default: demo
        description:
          title: Description
          type: string
          default: Spring Boot App Generated by Backstage
  steps:
    - id: sourceCodeTemplate
      name: Generating the Source Code Component
      action: fetch:template
      input:
        url: ./skeleton
        values:
          orgName: ${{ parameters.orgName }}
          appName: ${{ parameters.appName }}
          domainName: ${{ parameters.domainName }}
          groupId: ${{ parameters.groupId }}
          javaPackage: ${{ parameters.javaPackage }}
          apiPath: ${{ parameters.apiPath }}

    - id: publish
      name: Publishing to the Source Code Repository
      action: publish:github
      input:
        allowedHosts: ['github.com']
        description: ${{ parameters.description }}
        repoUrl: github.com?owner=${{ parameters.orgName }}&repo=${{ parameters.appName }}
        defaultBranch: ${{ parameters.repoBranchName }}
        repoVisibility: public

    - id: register
      name: Registering the Catalog Info Component
      action: catalog:register
      input:
        repoContentsUrl: ${{ steps.publish.output.repoContentsUrl }}
        catalogInfoPath: /catalog-info.yaml

    # (2)
    - id: configCodeTemplate
      name: Generating the Config Code Component
      action: fetch:template
      input:
        url: ../../skeletons/argocd
        values:
          orgName: ${{ parameters.orgName }}
          appName: ${{ parameters.appName }}
        targetPath: ./gitops

    # (3)
    - id: publish
      name: Publishing to the Config Code Repository
      action: publish:github
      input:
        allowedHosts: ['github.com']
        description: ${{ parameters.description }}
        repoUrl: github.com?owner=${{ parameters.orgName }}&repo=${{ parameters.appName }}-config
        defaultBranch: ${{ parameters.repoBranchName }}
        sourcePath: ./gitops
        repoVisibility: public

    # (4)
    - id: sonarqube
      name: Follow new project on Sonarcloud
      action: http:backstage:request
      input:
        method: 'POST'
        path: '/proxy/sonarqube/projects/create?name=${{ parameters.appName }}&organization=${{ parameters.orgName }}&project=${{ parameters.orgName }}_${{ parameters.appName }}'
        headers:
          content-type: 'application/json'

    # (5)
    - id: circleci
      name: Follow new project on CircleCI
      action: http:backstage:request
      input:
        method: 'POST'
        path: '/proxy/circleci/api/project/gh/${{ parameters.orgName }}/${{ parameters.appName }}/follow'
        headers:
          content-type: 'application/json'

    # (6)
    - id: argocd
      name: Create New Application in Argo CD
      action: http:backstage:request
      input:
        method: 'POST'
        path: '/proxy/argocd/api/applications'
        headers:
          content-type: 'application/json'
        body:
          metadata:
            name: ${{ parameters.appName }}
            namespace: argocd
          spec:
            project: default
            source:
              # (7)
              repoURL: https://github.com/${{ parameters.orgName }}/${{ parameters.appName }}-config.git
              targetRevision: master
              path: manifests
            destination:
              server: https://kubernetes.default.svc
              namespace: ${{ parameters.namespace }}
            syncPolicy:
              automated:
                prune: true
                selfHeal: true
              syncOptions:
                - CreateNamespace=true

  output:
    links:
      - title: Open the Source Code Repository
        url: ${{ steps.publish.output.remoteUrl }}
      - title: Open the Catalog Info Component
        icon: catalog
        entityRef: ${{ steps.register.output.entityRef }}

YAML

Create Catalog Component

Our catalog-info.yaml file should contain several additional annotations related to the plugins installed in the previous section. The argocd/app-name annotation indicates the name of the target Argo CD Application responsible for deployment on Kubernetes. The backstage.io/kubernetes-id annotation contains the value of the label used to search the pods on Kubernetes displayed in the Backstage UI. Finally, the prometheus.io/rule annotation contains a comma-separated list of the Prometheus queries. We will create graphs displaying app pod CPU and memory usage.

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: ${{ values.appName }}
  title: ${{ values.appName }}
  annotations:
    circleci.com/project-slug: github/${{ values.orgName }}/${{ values.appName }}
    github.com/project-slug: ${{ values.orgName }}/${{ values.appName }}
    sonarqube.org/project-key: ${{ values.orgName }}_${{ values.appName }}
    backstage.io/kubernetes-id: ${{ values.appName }}
    argocd/app-name: ${{ values.appName }}
    prometheus.io/rule: container_memory_usage_bytes{pod=~"${{ values.appName }}-.*"}|pod,rate(container_cpu_usage_seconds_total{pod=~"${{ values.appName }}-.*"}[5m])|pod
  tags:
    - spring-boot
    - java
    - maven
    - circleci
    - renovate
    - sonarqube
spec:
  type: service
  owner: piotr.minkowski@gmail.com
  lifecycle: experimental

YAML

Provide Configuration Settings

We need to include several configuration settings inside the app-config.yaml file. It includes the proxy section, which should contain all APIs required by the HTTP Request Action plugin and frontend plugins. We should include proxy addresses for CircleCI (1), Sonarcloud (2), Argo CD (3), and Prometheus (4). After that, we include the address of our Skaffolder template (5). We also have to include the kubernetes section with the address of the Minikube cluster and previously generated service account token (6).

app:
  title: Scaffolded Backstage App
  baseUrl: http://localhost:3000

organization:
  name: piomin

backend:
  baseUrl: http://localhost:7007
  listen:
    port: 7007
  csp:
    connect-src: ["'self'", 'http:', 'https:']
  cors:
    origin: http://localhost:3000
    methods: [GET, HEAD, PATCH, POST, PUT, DELETE]
    credentials: true
  database:
    client: better-sqlite3
    connection: ':memory:'

integrations:
  github:
    - host: github.com
      token: ${GITHUB_TOKEN}

proxy:
  # (1)
  '/circleci/api':
    target: https://circleci.com/api/v1.1
    headers:
      Circle-Token: ${CIRCLECI_TOKEN}
  # (2)
  '/sonarqube':
    target: https://sonarcloud.io/api
    allowedMethods: [ 'GET', 'POST' ]
    auth: "${SONARCLOUD_TOKEN}:"
  # (3)
  '/argocd/api':
    target: https://localhost:8443/api/v1/
    changeOrigin: true
    secure: false
    headers:
      Cookie:
        $env: ARGOCD_TOKEN
  # (4)
  '/prometheus/api':
    target: http://localhost:9090/api/v1/

auth:
  providers:
    guest: {}

catalog:
  import:
    entityFilename: catalog-info.yaml
    pullRequestBranchName: backstage-integration
  rules:
    - allow: [Component, System, API, Resource, Location]
  locations:
    - type: file
      target: ../../examples/entities.yaml

    - type: file
      target: ../../examples/template/template.yaml
      rules:
        - allow: [Template]
    
    # (5)
    - type: url
      target: https://github.com/piomin/backstage-templates/blob/master/templates/spring-boot-basic-on-kubernetes/template.yaml
      rules:
        - allow: [ Template ]

    - type: file
      target: ../../examples/org.yaml
      rules:
        - allow: [User, Group]


sonarqube:
  baseUrl: https://sonarcloud.io
  apiKey: ${SONARCLOUD_TOKEN}

# (6)
kubernetes:
  serviceLocatorMethod:
    type: 'multiTenant'
  clusterLocatorMethods:
    - type: 'config'
      clusters:
        - url: ${K8S_URL}
          name: minikube
          authProvider: 'serviceAccount'
          skipTLSVerify: false
          skipMetricsLookup: true
          serviceAccountToken: ${K8S_TOKEN}
          dashboardApp: standard
          caFile: '/Users/pminkows/.minikube/ca.crt'

YAML

Build Backstage Image

Our source code repository with Backstage contains all the required plugins and the configuration. Now, we will build it using the yarn tool. Here’s a list of required commands to perform a build.

$ yarn clean
$ yarn install
$ yarn tsc
$ yarn build:backend

ShellSession

The repository with Backstage already contains the Dockerfile. You can find it in the packages/backend directory.

FROM node:18-bookworm-slim

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
    --mount=type=cache,target=/var/lib/apt,sharing=locked \
    apt-get update && \
    apt-get install -y --no-install-recommends python3 g++ build-essential && \
    yarn config set python /usr/bin/python3

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
    --mount=type=cache,target=/var/lib/apt,sharing=locked \
    apt-get update && \
    apt-get install -y --no-install-recommends libsqlite3-dev

USER node

WORKDIR /app

ENV NODE_ENV production

COPY --chown=node:node yarn.lock package.json packages/backend/dist/skeleton.tar.gz ./
RUN tar xzf skeleton.tar.gz && rm skeleton.tar.gz

RUN --mount=type=cache,target=/home/node/.cache/yarn,sharing=locked,uid=1000,gid=1000 \
    yarn install --frozen-lockfile --production --network-timeout 300000

COPY --chown=node:node packages/backend/dist/bundle.tar.gz app-config*.yaml ./
RUN tar xzf bundle.tar.gz && rm bundle.tar.gz

CMD ["node", "packages/backend", "--config", "app-config.yaml"]

Dockerfile

In order to build the image using the Dockerfile from the packages/backend directory, we need to run the following command from the project root directory.

$ yarn build-image

ShellSession

If you see a similar result, it means that the build was successfully finished.

The image is available locally as backstage:latest. We can run it on Docker with the following command:

$ docker run -it -p 7007:7007 \
  -e GITHUB_TOKEN=${GITHUB_TOKEN} \
  -e SONARCLOUD_TOKEN=${SONARCLOUD_TOKEN} \
  -e CIRCLECI_TOKEN=${CIRCLECI_TOKEN} \
  -e ARGOCD_TOKEN=${ARGOCD_TOKEN} \
  -e K8S_TOKEN=${K8S_TOKEN} \
  -e K8S_URL=${K8S_URL} \
  -e NODE_ENV=development \
  backstage:latest

ShellSession

However, our main goal today is to run it directly on Kubernetes. You can find our custom Backstage image in my Docker registry: piomin/backstage:latest.

Deploy Backstage on Kubernetes

We will use the official Helm chart for installing Backstage on Kubernetes. In the first step, let’s add the following chart repository:

$ helm repo add backstage https://backstage.github.io/charts

ShellSession

Here’s our values.yaml file for Helm installation. We need to set all the required tokens as extra environment variables inside the Backstage pod. We also changed the default image used in the installation into the previously built custom image. To simplify the exercise, we can disable the external database and use the internal SQLite instance. It is possible to pass extra configuration files by defining them as ConfigMap, without rebuilding the Docker image (my-app-config).

backstage:
  extraEnvVars:
    - name: NODE_ENV
      value: development
    - name: GITHUB_TOKEN
      value: ${GITHUB_TOKEN}
    - name: SONARCLOUD_TOKEN
      value: ${SONARCLOUD_TOKEN}
    - name: CIRCLECI_TOKEN
      value: ${CIRCLECI_TOKEN}
    - name: ARGOCD_TOKEN
      value: ${ARGOCD_TOKEN}
  image:
    registry: docker.io
    repository: piomin/backstage
  extraAppConfig:
    - filename: app-config.extra.yaml
      configMapRef: my-app-config
postgresql:
  enabled: false

YAML

We will change the addresses of the Kubernetes cluster, Argo CD, and Prometheus into the internal cluster locations by modifying the app-config.yaml file.

proxy:
  .
  '/argocd/api':
    target: https://argo-cd-argocd-server.argocd.svc/api/v1/
    changeOrigin: true
    secure: false
    headers:
      Cookie:
        $env: ARGOCD_TOKEN
  '/prometheus/api':
    target: http://kube-prometheus-stack-prometheus.monitoring.svc:9090/api/v1/

catalog:
  locations:
    ...
    - type: url
      target: https://github.com/piomin/backstage-templates/blob/master/templates/spring-boot-basic-on-kubernetes/template.yaml
      rules:
        - allow: [ Template ]
            
kubernetes:
  serviceLocatorMethod:
    type: 'multiTenant'
  clusterLocatorMethods:
    - type: 'config'
      clusters:
        - url: https://kubernetes.default.svc
          name: minikube
          authProvider: 'serviceAccount'
          skipTLSVerify: false
          skipMetricsLookup: true

app-config-kubernetes.yaml

Then, we will create the backstage namespace and extra ConfigMap that contains a new configuration for the Backstage running inside the Kubernetes cluster.

$ kubectl create ns backstage
$ kubectl create configmap my-app-config \
  --from-file=app-config.extra.yaml=app-config-kubernetes.yaml -n backstage

ShellSession

Finally, let’s install our custom instance of Backstage in the backstage namespace by executing the following command:

$ envsubst < values.yaml | helm install backstage backstage/backstage \
  --values - -n backstage

ShellSession

As I result, there is a running Backstage pod on Kubernetes:

$ kubectl get po -n backstage
NAME                         READY   STATUS    RESTARTS   AGE
backstage-7bfbc55647-8cj5d   1/1     Running   0          16m

ShellSession

Let’s enable port forwarding to access the Backstage UI on the http://localhost:7007:

$ kubectl port-forward svc/backstage 7007 -n backstage

ShellSession

This time we increase the privileges for default ServiceAccount in the backstage namespace used by our instance of Backstage:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: default-admin
subjects:
- kind: ServiceAccount
  name: default
  namespace: backstage
roleRef:
  kind: ClusterRole
  name: cluster-admin
  apiGroup: rbac.authorization.k8s.io

YAML

Final Test

After accessing Backstage UI we can create a new Spring Boot app from the template. Choose the “Create a Spring Boot app for Kubernetes” template as shown below:

If you would like to try it by yourself, you need to change the organization name to your GitHub account name. Then click “Review” and “Create” on the next page.

There are two GitHub repositories created. The first one contains the sample app source code.

The second one contains YAML manifests with Deployment for Argo CD.

The Argo CD Application is automatically created. We can verify the synchronization status in the Backstage UI.

Our application is running in the demo namespace. We can display a list of pods in the “KUBERNETES” tab.

We can also verify the detailed status of each pod.

Or take a look at the logs.

Final Thoughts

In this article, we learned how to install and integrate Backstage with Kubernetes-native services like Argo CD or Prometheus. We built the customized image with Backstage and then deployed it on Kubernetes using the Helm chart.

The post Backstage on Kubernetes appeared first on Piotr's TechBlog.

Getting Started with Backstage

piotr.minkowski — Thu, 13 Jun 2024 13:47:07 +0000

This article will teach you how to use Backstage in your app development and create software templates to generate a typical Spring Boot app. Backstage is an open-source framework for building developer portals. It allows us to automate the creation of the infrastructure, CI/CD, and operational knowledge needed to run an application or product. It offers a centralized software catalog and unifies all infrastructure tooling, services, and documentation within a single and intuitive UI. With Backstage, we can create any new software component, such as a new microservice, just with a few clicks. Developers can choose between several standard templates. Platform engineers will create such templates to meet the organization’s best practices.

From the technical point of view, Backstage is a web frontend designed to run on Node.js. It is mostly written in TypeScript using the React framework. It has an extensible nature. Each time we need to integrate Backstage with some third-party software we need to install a dedicated plugin. Plugins are essentially individually packaged React components. Today you will learn how to install and configure plugins to integrate with GitHub, CircleCI, and Sonarqube.

It is the first article about Backstage on my blog. You can expect more in the future. However, before proceeding with this article, it is worth reading the following post. It explains how I create my repositories on GitHub and what tools I’m using to check the quality of the code and be up-to-date.

Source Code

If you would like to try it by yourself, you may always take a look at my source code. This time, there are two sample Git repositories. The first of them contains software templates written in the Backstage technology called Skaffolder. On the other hand, the second repository contains the source code of the simple Spring Boot generated from the Backstage template. Once you clone both of those repos, you should just follow my further instructions.

Writing Software Templates in Backstage

We can create our own software templates using YAML notation or find existing examples on the web. Such software templates are very similar to the definition of Kubernetes objects. They have the apiVersion, kind (Template) fields, metadata, and spec fields. Each template must define a list of input variables and then a list of actions that the scaffolding service executes.

The Structure of the Repository with Software Templates

Let’s take a look at the structure of the repository containing our Spring Boot template. As you see, there is the template.yaml file with the software template YAML manifest and the skeleton directory with our app source code. Besides Java files, there is the Maven pom.xml, Renovate and CircleCI configuration manifests. The Scaffolder template input parameters determine the name of Java classes or packages. In the Skaffolder template, we set the default base package name and a domain object name. The catalog-info.yaml file contains a definition of the object required to register the app in the software catalog. As you can see, we are also parametrizing the names of the files with the domain object names.

.
├── skeleton
│   ├── .circleci
│   │   └── config.yml
│   ├── README.md
│   ├── catalog-info.yaml
│   ├── pom.xml
│   ├── renovate.json
│   └── src
│       ├── main
│       │   ├── java
│       │   │   └── ${{values.javaPackage}}
│       │   │       ├── Application.java
│       │   │       ├── controller
│       │   │       │   └── ${{values.domainName}}Controller.java
│       │   │       └── domain
│       │   │           └── ${{values.domainName}}.java
│       │   └── resources
│       │       └── application.yml
│       └── test
│           └── java
│               └── ${{values.javaPackage}}
│                   └── ${{values.domainName}}ControllerTests.java
└── template.yaml

13 directories, 11 files

ShellSession

Building Templates with Scaffolder

Now, let’s take a look at the most important element in our repository – the template manifest. It defines several input parameters with default values. We can set the name of our app (appName), choose a default branch name inside the Git repository (repoBranchName), Maven group ID (groupId), the name of the default Java package (javaPackage), or the base REST API controller path (apiPath). All these parameters are then used during code generation. If you need more customization in the templates, you should add other parameters in the manifest.

The steps section in the manifest defines the actions required to create a new app in Backstage. We are doing three things here. In the first step, we need to generate the Spring Boot app source by filling the templates inside the skeleton directory with parameters defined in the Scaffolder manifest. Then, we are publishing the generated code in the newly created GitHub repository. The name of the repository is the same as the app name (the appName parameter). The owner of the GitHub repository is determined by the values of the orgName parameter. Finally, we are registering the new component in the Backstage catalog by calling the catalog:register action.

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  name: spring-boot-basic-template
  title: Create a Spring Boot app
  description: Create a Spring Boot app
  tags:
    - spring-boot
    - java
    - maven
    - circleci
    - renovate
    - sonarqube
spec:
  owner: piomin
  system: piomin
  type: service

  parameters:
    - title: Provide information about the new component
      required:
        - orgName
        - appName
        - domainName
        - repoBranchName
        - groupId
        - javaPackage
        - apiPath
        - description
      properties:
        orgName:
          title: Organization name
          type: string
          default: piomin
        appName:
          title: App name
          type: string
          default: sample-spring-boot-app
        domainName:
          title: Name of the domain object
          type: string
          default: Person
        repoBranchName:
          title: Name of the branch in the Git repository
          type: string
          default: master
        groupId:
          title: Maven Group ID
          type: string
          default: pl.piomin.services
        javaPackage:
          title: Java package directory
          type: string
          default: pl/piomin/services
        apiPath:
          title: REST API path
          type: string
          default: /api/v1
        description:
          title: Description
          type: string
          default: Sample Spring Boot App
          
  steps:
    - id: sourceCodeTemplate
      name: Generating the Source Code Component
      action: fetch:template
      input:
        url: ./skeleton
        values:
          orgName: ${{ parameters.orgName }}
          appName: ${{ parameters.appName }}
          domainName: ${{ parameters.domainName }}
          groupId: ${{ parameters.groupId }}
          javaPackage: ${{ parameters.javaPackage }}
          apiPath: ${{ parameters.apiPath }}

    - id: publish
      name: Publishing to the Source Code Repository
      action: publish:github
      input:
        allowedHosts: ['github.com']
        description: ${{ parameters.description }}
        repoUrl: github.com?owner=${{ parameters.orgName }}&repo=${{ parameters.appName }}
        defaultBranch: ${{ parameters.repoBranchName }}
        repoVisibility: public

    - id: register
      name: Registering the Catalog Info Component
      action: catalog:register
      input:
        repoContentsUrl: ${{ steps.publish.output.repoContentsUrl }}
        catalogInfoPath: /catalog-info.yaml

  output:
    links:
      - title: Open the Source Code Repository
        url: ${{ steps.publish.output.remoteUrl }}
      - title: Open the Catalog Info Component
        icon: catalog
        entityRef: ${{ steps.register.output.entityRef }}

YAML

Generating Spring Boot Source Code

Here’s the template of the Maven pom.xml. Our app uses the current latest version of the Spring Boot framework and Java 21 for compilation. The Maven groupId and artifactId are taken from the Scaffolder template parameters. The pom.xml file also contains data required to integrate with a specific project on Sonarcloud (sonar.projectKey and sonar.organization).

4.0.0 ${{ values.groupId }} ${{ values.appName }} 1.0-SNAPSHOT org.springframework.boot spring-boot-starter-parent 3.3.0 ${{ values.orgName }}_${{ values.appName }} ${{ values.orgName }} https://sonarcloud.io 21 org.springframework.boot spring-boot-starter-web org.springframework.boot spring-boot-starter-test test org.springframework.boot spring-boot-starter-actuator org.springdoc springdoc-openapi-starter-webmvc-ui 2.5.0 org.springframework.boot spring-boot-devtools true org.instancio instancio-junit 4.7.0 test org.springframework.boot spring-boot-maven-plugin build-info ${java.version} ${maven.build.timestamp} pl.project13.maven git-commit-id-plugin false org.jacoco jacoco-maven-plugin 0.8.12 prepare-agent report test report " style="color:#d8dee9ff;display:none" aria-label="Copy" class="code-block-pro-copy-button">

 xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    4.0.0

    ${{ values.groupId }}
    ${{ values.appName }}
    1.0-SNAPSHOT

    
        org.springframework.boot
        spring-boot-starter-parent
        3.3.0
    

    
        ${{ values.orgName }}_${{ values.appName }}
        ${{ values.orgName }}
        https://sonarcloud.io
        21
    

    
        
            org.springframework.boot
            spring-boot-starter-web
        
        
            org.springframework.boot
            spring-boot-starter-test
            test
        
        
            org.springframework.boot
            spring-boot-starter-actuator
        
        
            org.springdoc
            springdoc-openapi-starter-webmvc-ui
            2.5.0
        
        
            org.springframework.boot
            spring-boot-devtools
            true
        
        
            org.instancio
            instancio-junit
            4.7.0
            test
        
    

    
        
            
                org.springframework.boot
                spring-boot-maven-plugin
                
                    
                        
                            build-info
                        
                        
                            
                                ${java.version}
                                ${maven.build.timestamp}
                            
                        
                    
                
            
            
                pl.project13.maven
                git-commit-id-plugin
                
                    false
                
            
            
                org.jacoco
                jacoco-maven-plugin
                0.8.12
                
                    
                        
                            prepare-agent
                        
                    
                    
                        report
                        test
                        
                            report

XML

There are several Java classes generated during bootstrap. Here’s the @RestController class template. It uses three parameters defined in the Scaffolder template: groupId, domainName and apiPath. It imports the domain object class and exposes REST methods for CRUD operations. As you see, the implementation is very simple. It just uses the in-memory Java List to store the domain objects. However, it perfectly shows the idea behind Scaffolder templates.

objs = new ArrayList<>(); @GetMapping public List<${{ values.domainName }}> findAll() { return objs; } @GetMapping("/{id}") public ${{ values.domainName }} findById(@PathVariable("id") Long id) { ${{ values.domainName }} obj = objs.stream().filter(it -> it.getId().equals(id)) .findFirst() .orElseThrow(); LOG.info("Found: {}", obj.getId()); return obj; } @PostMapping public ${{ values.domainName }} add(@RequestBody ${{ values.domainName }} obj) { obj.setId((long) (objs.size() + 1)); LOG.info("Added: {}", obj); objs.add(obj); return obj; } @DeleteMapping("/{id}") public void delete(@PathVariable("id") Long id) { ${{ values.domainName }} obj = objs.stream().filter(it -> it.getId().equals(id)).findFirst().orElseThrow(); objs.remove(obj); LOG.info("Removed: {}", id); } @PutMapping public void update(@RequestBody ${{ values.domainName }} obj) { ${{ values.domainName }} objTmp = objs.stream() .filter(it -> it.getId().equals(obj.getId())) .findFirst() .orElseThrow(); objs.set(objs.indexOf(objTmp), obj); LOG.info("Updated: {}", obj.getId()); } }" style="color:#d8dee9ff;display:none" aria-label="Copy" class="code-block-pro-copy-button">

package ${{ values.groupId }}.controller;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.*;
import ${{ values.groupId }}.domain.${{ values.domainName }};

import java.util.ArrayList;
import java.util.List;

@RestController
@RequestMapping("${{ values.apiPath }}")
public class ${{ values.domainName }}Controller {

    private final Logger LOG = LoggerFactory.getLogger(${{ values.domainName }}Controller.class);
    private final List<${{ values.domainName }}> objs = new ArrayList<>();

    @GetMapping
    public List<${{ values.domainName }}> findAll() {
        return objs;
    }

    @GetMapping("/{id}")
    public ${{ values.domainName }} findById(@PathVariable("id") Long id) {
        ${{ values.domainName }} obj = objs.stream().filter(it -> it.getId().equals(id))
                .findFirst()
                .orElseThrow();
        LOG.info("Found: {}", obj.getId());
        return obj;
    }

    @PostMapping
    public ${{ values.domainName }} add(@RequestBody ${{ values.domainName }} obj) {
        obj.setId((long) (objs.size() + 1));
        LOG.info("Added: {}", obj);
        objs.add(obj);
        return obj;
    }

    @DeleteMapping("/{id}")
    public void delete(@PathVariable("id") Long id) {
        ${{ values.domainName }} obj = objs.stream().filter(it -> it.getId().equals(id)).findFirst().orElseThrow();
        objs.remove(obj);
        LOG.info("Removed: {}", id);
    }

    @PutMapping
    public void update(@RequestBody ${{ values.domainName }} obj) {
        ${{ values.domainName }} objTmp = objs.stream()
                .filter(it -> it.getId().equals(obj.getId()))
                .findFirst()
                .orElseThrow();
        objs.set(objs.indexOf(objTmp), obj);
        LOG.info("Updated: {}", obj.getId());
    }

}

Java

Then, we can generate a test class to verify @RestController endpoints. The app is starting on the random port during the JUnit tests. In the first test, we are adding a new object into the store. Then we are verifying the GET /{id} endpoint works fine. Finally, we are removing the object from the store by calling the DELETE /{id} endpoint.

0); } @Test @Order(2) void findById() { ${{ values.domainName }} obj = restTemplate.getForObject(API_PATH + "/{id}", ${{ values.domainName }}.class, 1L); assertNotNull(obj); assertEquals(1, obj.getId()); } @Test @Order(3) void delete() { restTemplate.delete(API_PATH + "/{id}", 1L); ${{ values.domainName }} obj = restTemplate.getForObject(API_PATH + "/{id}", ${{ values.domainName }}.class, 1L); assertNull(obj.getId()); } }" style="color:#d8dee9ff;display:none" aria-label="Copy" class="code-block-pro-copy-button">

package ${{ values.groupId }};

import org.instancio.Instancio;
import org.junit.jupiter.api.MethodOrderer;
import org.junit.jupiter.api.Order;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestMethodOrder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.boot.test.web.client.TestRestTemplate;
import ${{ values.groupId }}.domain.${{ values.domainName }};

import static org.junit.jupiter.api.Assertions.*;

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class ${{ values.domainName }}ControllerTests {

    private static final String API_PATH = "${{values.apiPath}}";

    @Autowired
    private TestRestTemplate restTemplate;

    @Test
    @Order(1)
    void add() {
        ${{ values.domainName }} obj = restTemplate.postForObject(API_PATH, Instancio.create(${{ values.domainName }}.class), ${{ values.domainName }}.class);
        assertNotNull(obj);
        assertEquals(1, obj.getId());
    }

    @Test
    @Order(2)
    void findAll() {
        ${{ values.domainName }}[] objs = restTemplate.getForObject(API_PATH, ${{ values.domainName }}[].class);
        assertTrue(objs.length > 0);
    }

    @Test
    @Order(2)
    void findById() {
        ${{ values.domainName }} obj = restTemplate.getForObject(API_PATH + "/{id}", ${{ values.domainName }}.class, 1L);
        assertNotNull(obj);
        assertEquals(1, obj.getId());
    }

    @Test
    @Order(3)
    void delete() {
        restTemplate.delete(API_PATH + "/{id}", 1L);
        ${{ values.domainName }} obj = restTemplate.getForObject(API_PATH + "/{id}", ${{ values.domainName }}.class, 1L);
        assertNull(obj.getId());
    }

}

Java

Integrate with CircleCI and Renovate

Once I create a new repository on GitHub I want to automatically integrate it with CircleCI builds. I also want to update Maven dependencies versions automatically with Renovate to keep the project up to date. Since my GitHub account is connected to the CircleCI account and the Renovate app is installed there I just need to provide two configuration manifests inside the generated repository. Here’s the CircleCI config.yaml file. It runs the Maven build with JUnit tests and performs the Sonarqube scan in the sonarcloud.io portal.

version: 2.1

jobs:
  analyze:
    docker:
      - image: 'cimg/openjdk:21.0.2'
    steps:
      - checkout
      - run:
          name: Analyze on SonarCloud
          command: mvn verify sonar:sonar

executors:
  jdk:
    docker:
      - image: 'cimg/openjdk:21.0.2'

orbs:
  maven: circleci/maven@1.4.1

workflows:
  maven_test:
    jobs:
      - maven/test:
          executor: jdk
      - analyze:
          context: SonarCloud

YAML

Here’s the renovate.json manifest. Renovate will create a PR in GitHub each time it detects a new version of Maven dependency.

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": [
    "config:base",":dependencyDashboard"
  ],
  "packageRules": [
    {
      "matchUpdateTypes": ["minor", "patch", "pin", "digest"],
      "automerge": true
    }
  ],
  "prCreation": "not-pending"
}

YAML

Register a new Component in the Software Catalog

Once we generate the whole code and publish it as the GitHub repository, we need to register a new component in the Backstage catalog. In order to achieve this, our repository needs to contain the Component manifest as shown below. Once again, we need to fill it with the parameter values during the bootstrap phase. It contains a reference to the CircleCI and Sonarqube projects and a generated GitHub repository in the annotations section. Here’s the catalog-info.yaml template:

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: ${{ values.appName }}
  title: ${{ values.appName }}
  annotations:
    circleci.com/project-slug: github/${{ values.orgName }}/${{ values.appName }}
    github.com/project-slug: ${{ values.orgName }}/${{ values.appName }}
    sonarqube.org/project-key: ${{ values.orgName }}_${{ values.appName }}
  tags:
    - spring-boot
    - java
    - maven
    - circleci
    - renovate
    - sonarqube
spec:
  type: service
  owner: piotr.minkowski@gmail.com
  lifecycle: experimental

YAML

Running Backstage

Once we have the whole template ready, we can proceed to run the Backstage on our local machine. As I mentioned before, Backstage is a Node.js app, so we need to have several tools installed to be able to run it. By the way, the list of prerequisites is pretty large. You can find it under the following link. First of all, I had to downgrade the version of Node.js from the latest 22 to 18. We also need to install yarn and npx. If you have the following versions of those tools, you shouldn’t have any problems with running Backstage according to further instructions.

$ node --version
v18.20.3

$ npx --version
10.7.0

$ yarn --version
1.22.22

ShellSession

Running a Standalone Server Locally

We are going to run Backstage locally in the development mode as a standalone server. In order to achieve this, we first need to run the following command. It will create a new directory with a Backstage app inside.

$ npx @backstage/create-app@latest

ShellSession

This may take some time. But if you see a similar result, it means that your instance is ready. However, before we start it we need to install some plugins and include some configuration settings. In that case, the name of our instance is backstage1.

Firstly, we should go to the backstage1 directory and take a look at the project structure. The most important elements for us are: the app-config.yaml file with configuration and the packages directory with the source code of the backend and frontend (app) modules.

├── README.md
├── app-config.local.yaml
├── app-config.production.yaml
├── app-config.yaml
├── backstage.json
├── catalog-info.yaml
├── dist-types
├── examples
├── lerna.json
├── node_modules
├── package.json
├── packages
│   ├── app
│   └── backend
├── playwright.config.ts
├── plugins
├── tsconfig.json
└── yarn.lock

ShellSession

The app is not configured according to our needs yet. However, we can run it with the following command just to try it out:

$ yarn dev

ShellSession

We can visit the UI available under the http://localhost:3000:

Provide Configuration and Install Plugins

In the first, we will analyze the app-config.yaml and add some configuration settings there. I will focus only on the aspects important to our exercise. The default configuration comes with enabled built-in integration with GitHub. We just need to generate a personal access token in GitHub and provide it as the GITHUB_TOKEN environment variable (1). Then, we need to integrate with Sonarqube also through the access token (2). Our portal will also display a list of CircleCI builds. Therefore, we need to include the CircleCI token as well (3). Finally, we should include the URL address of our custom Scaffolder template in the catalog section (4). It is located in our sample GitHub repository: https://github.com/piomin/backstage-templates/blob/master/templates/spring-boot-basic/skeleton/catalog-info.yaml.

app:
  title: Scaffolded Backstage App
  baseUrl: http://localhost:3000

organization:
  name: piomin

backend:
  baseUrl: http://localhost:7007
  listen:
    port: 7007
  csp:
    connect-src: ["'self'", 'http:', 'https:']
  cors:
    origin: http://localhost:3000
    methods: [GET, HEAD, PATCH, POST, PUT, DELETE]
    credentials: true
  database:
    client: better-sqlite3
    connection: ':memory:'

# (1)
integrations:
  github:
    - host: github.com
      token: ${GITHUB_TOKEN}

# (2)
sonarqube:
  baseUrl: https://sonarcloud.io
  apiKey: ${SONARCLOUD_TOKEN}

# (3)
proxy:
  '/circleci/api':
    target: https://circleci.com/api/v1.1
    headers:
      Circle-Token: ${CIRCLECI_TOKEN}
      
auth:
  providers:
    guest: {}

catalog:
  import:
    entityFilename: catalog-info.yaml
    pullRequestBranchName: backstage-integration
  rules:
    - allow: [Component, System, API, Resource, Location]
  locations:`
    - type: file
      target: ../../examples/entities.yaml
    - type: file
      target: ../../examples/template/template.yaml
      rules:
        - allow: [Template]
    # (4)
    - type: url
      target: https://github.com/piomin/backstage-templates/blob/master/templates/spring-boot-basic/template.yaml
      rules:
        - allow: [ Template ]
    - type: file
      target: ../../examples/org.yaml
      rules:
        - allow: [User, Group]

YAML

So, before starting Backstage we need to export both GitHub and Sonarqube tokens.

$ export GITHUB_TOKEN=<YOUR_GITHUB_TOKEN>
$ export SONARCLOUD_TOKEN=<YOUR_SONARCLOUD_TOKEN>
$ export CIRCLECI_TOKEN=<YOUR_CIRCLECI_TOKEN>

ShellSession

For those of you who didn’t generate Sonarcloud tokens before:

And similar operation for CicleCI:

Unfortunately, that is not all. Now, we need to install several required plugins. To be honest with you, plugin installation in Backstage is quite troublesome. Usually, we not only need to install such a plugin with yarn but also provide some changes in the packages directory. Let’s begin!

Enable GitHub Integration

Although integration with GitHub is enabled by default in the configuration settings we still need to install the plugin to be able to make some actions related to repositories. Firstly, from your Backstage instance root directory, you need to execute the following command:

$ yarn --cwd packages/backend add @backstage/plugin-scaffolder-backend-module-github

ShellSession

Then, go to the packages/backend/app/index.ts file and add a single highlighted line there. It imports the @backstage/plugin-scaffolder-backend-module-github module to the backend.

import { createBackend } from '@backstage/backend-defaults';

const backend = createBackend();

backend.add(import('@backstage/plugin-app-backend/alpha'));
backend.add(import('@backstage/plugin-proxy-backend/alpha'));
backend.add(import('@backstage/plugin-scaffolder-backend/alpha'));
backend.add(import('@backstage/plugin-techdocs-backend/alpha'));
backend.add(import('@backstage/plugin-auth-backend'));
backend.add(import('@backstage/plugin-auth-backend-module-guest-provider'));
backend.add(import('@backstage/plugin-catalog-backend/alpha'));
backend.add(
  import('@backstage/plugin-catalog-backend-module-scaffolder-entity-model'),
);
backend.add(import('@backstage/plugin-permission-backend/alpha'));
backend.add(
  import('@backstage/plugin-permission-backend-module-allow-all-policy'),
);
backend.add(import('@backstage/plugin-search-backend/alpha'));
backend.add(import('@backstage/plugin-search-backend-module-catalog/alpha'));
backend.add(import('@backstage/plugin-search-backend-module-techdocs/alpha'));

backend.add(import('@backstage/plugin-scaffolder-backend-module-github'));
backend.add(import('@backstage-community/plugin-sonarqube-backend'));

backend.start();

TypeScript

After that, it will be possible to call the publish:github action defined in our template, which is responsible for creating a new GitHub repository with the Spring Boot app source code.

Enable Sonarqube Integration

In the next step, we need to install and configure the Sonarqube plugin. This time, we need to install both backend and frontend modules. Let’s begin with a frontend part. Firstly, we have to execute the following yarn command from the project root directory:

$ yarn --cwd packages/app add @backstage-community/plugin-sonarqube

ShellSession

Then, we need to edit the packages/app/src/components/catalog/EntityPage.tsx file to import the EntitySonarQubeCard object from the @backstage-community/plugin-sonarqube plugin. After that, we can include EntitySonarQubeCard component to the frontend page. For example, it can be placed as a part of the overview content.

{entityWarningContent} );" style="color:#d8dee9ff;display:none" aria-label="Copy" class="code-block-pro-copy-button">

import { EntitySonarQubeCard } from '@backstage-community/plugin-sonarqube';

// ... other imports
// ... other contents

const overviewContent = (
  <Grid container spacing={3} alignItems="stretch">
    {entityWarningContent}
    <Grid item md={6}>
      <EntityAboutCard variant="gridItem" />
    Grid>
    <Grid item md={6} xs={12}>
      <EntityCatalogGraphCard variant="gridItem" height={400} />
    Grid>
    <Grid item md={6}>
      <EntitySonarQubeCard variant="gridItem" />
    Grid>
    <Grid item md={4} xs={12}>
      <EntityLinksCard />
    Grid>
    <Grid item md={8} xs={12}>
      <EntityHasSubcomponentsCard variant="gridItem" />
    Grid>
  Grid>
);

TypeScript

Then, we can proceed with the backend plugin. Once again, we are installing with the yarn command:

$ yarn --cwd packages/backend add @backstage-community/plugin-sonarqube-backend

ShellSession

Finally, the same as for the GitHub plugin, go to the packages/backend/app/index.ts file and add a single highlighted line there to import the @backstage-community/plugin-sonarqube-backend to the backend module.

import { createBackend } from '@backstage/backend-defaults';

const backend = createBackend();

backend.add(import('@backstage/plugin-app-backend/alpha'));
backend.add(import('@backstage/plugin-proxy-backend/alpha'));
backend.add(import('@backstage/plugin-scaffolder-backend/alpha'));
backend.add(import('@backstage/plugin-techdocs-backend/alpha'));
backend.add(import('@backstage/plugin-auth-backend'));
backend.add(import('@backstage/plugin-auth-backend-module-guest-provider'));
backend.add(import('@backstage/plugin-catalog-backend/alpha'));
backend.add(
  import('@backstage/plugin-catalog-backend-module-scaffolder-entity-model'),
);
backend.add(import('@backstage/plugin-permission-backend/alpha'));
backend.add(
  import('@backstage/plugin-permission-backend-module-allow-all-policy'),
);
backend.add(import('@backstage/plugin-search-backend/alpha'));
backend.add(import('@backstage/plugin-search-backend-module-catalog/alpha'));
backend.add(import('@backstage/plugin-search-backend-module-techdocs/alpha'));

backend.add(import('@backstage/plugin-scaffolder-backend-module-github'));
backend.add(import('@backstage-community/plugin-sonarqube-backend'));

backend.start();

TypeScript

Note that previously, we added the sonarqube section with the access token to the app-config.yaml file and we included annotation with the SonarCloud project key into the Backstage Component manifest. Thanks to that, we don’t need to do anything more in this part.

Enable CircleCI Integration

In order to install the CircleCI plugin, we need to execute the following yarn command:

$ yarn add --cwd packages/app @circleci/backstage-plugin

ShellSession

Then, we have to edit the packages/app/src/components/catalog/EntityPage.tsx file. The same as before we need to include the import section and choose a place on the frontend page to display the content.

Read more } /> );" style="color:#d8dee9ff;display:none" aria-label="Copy" class="code-block-pro-copy-button">

// ... other imports

import {
  EntityCircleCIContent,
  isCircleCIAvailable,
} from '@circleci/backstage-plugin';

// ... other contents

const cicdContent = (
  <EntitySwitch>
    <EntitySwitch.Case if={isCircleCIAvailable}>
      <EntityCircleCIContent />
    EntitySwitch.Case>
    <EntitySwitch.Case>
      <EmptyState
        title="No CI/CD available for this entity"
        missing="info"
        description="You need to add an annotation to your component if you want to enable CI/CD for it. You can read more about annotations in Backstage by clicking the button below."
        action={
          <Button
            variant="contained"
            color="primary"
            href="https://backstage.io/docs/features/software-catalog/well-known-annotations"
          >
            Read more
          
        }
      />
    EntitySwitch.Case>
  EntitySwitch>
);

TypeScript

Final Run

That’s all we need to configure before running the Backstage instance. Once again, we need to start the instance with the yarn dev command. After running the app, we should go to the “Create…” section in the left menu pane. You should see our custom template under the name “Create a Spring Boot app”. Click the “CHOOSE” button to create a new component from that template.

Then, we will see the form with several input parameters. I will just change the app name to the sample-spring-boot-app-backstage and leave the default values everywhere else.

Then, let’s just click the “CREATE” button on the next page.

After that, Backstage will generate all the required things from our sample template.

We can go to the app page in the Backstage catalog. As you see it contains the “Code Quality” section with the latest Sonrqube report for our newly generated app.

We can also switch to the “CI/CD” tab to see the history of the app builds in the CircleCI.

If you want to visit the example repository generated from the sample template discussed today, you can find it here.

Final Thoughts

Backstage is an example of a no-code IDP (Internal Developer Portal). IDP is an important part of the relatively new trend in software development called “Platform Engineering”. In this article, I showed you how to create “Golden Path Templates” using the technology called Scaffolder. Then, you could see how to run Backstage on the local machine and how to create an app source from the template. We installed some useful plugins to integrate our portal with GitHub, CircleCI, and Sonarqube. Plugins installation may cause some problems, especially for people without experience in Node.js and React. Hope it helps!

The post Getting Started with Backstage appeared first on Piotr's TechBlog.

Kubernetes Testing with CircleCI, Kind, and Skaffold

piotr.minkowski — Tue, 28 Nov 2023 13:04:18 +0000

In this article, you will learn how to use tools like Kind or Skaffold to build integration tests on CircleCI for apps running on Kubernetes. Our main goal in this exercise is to build the app image and verify the Deployment on Kubernetes in the CircleCI pipeline. Skaffold and Jib Maven plugin build the image from the source and deploy it on Kind using YAML manifests. Finally, we will run some load tests on the deployed app using the Grafana k6 tool and its integration with CircleCI.

If you want to build and run tests against Kubernetes, you can read my article about integration tests with JUnit. On the other hand, if you are looking for other testing tools for testing in a Kubernetes-native environment you can refer to that article about Testkube.

Introduction

Before we start, let’s do a brief introduction. There are three simple Spring Boot apps that communicate with each other. The first-service app calls the endpoint exposed by the caller-service app, and then the caller-service app calls the endpoint exposed by the callme-service app. The diagram visible below illustrates that architecture.

So in short, our goal is to deploy all the sample apps on Kind during the CircleCI build and then test the communication by calling the endpoint exposed by the first-service through the Kubernetes Service.

Source Code

If you would like to try it by yourself, you may always take a look at my source code. In order to do that you need to clone my GitHub repository. It contains three apps: first-service, caller-service, and callme-service. The main Skaffold config manifest is available in the project root directory. Required Kubernetes YAML manifests are always placed inside the k8s directory. Once you take a look at the source code, you should just follow my instructions. Let’s begin.

Our sample Spring Boot apps are very simple. They are exposing a single “ping” endpoint over HTTP and call “ping” endpoints exposed by other apps. Here’s the @RestController in the first-service app:

@RestController
@RequestMapping("/first")
public class FirstController {

   private static final Logger LOGGER = LoggerFactory
      .getLogger(FirstController.class);

   @Autowired
   Optional buildProperties;
   @Autowired
   RestTemplate restTemplate;
   @Value("${VERSION}")
   private String version;

   @GetMapping("/ping")
   public String ping() {
      LOGGER.info("Ping: name={}, version={}", buildProperties.isPresent() 
         ? buildProperties.get().getName() : "first-service", version);
      String response = restTemplate.getForObject(
         "http://caller-service:8080/caller/ping", String.class);
      LOGGER.info("Calling: response={}", response);
      return "I'm first-service " + version + ". Calling... " + response;
   }

}

Here’s the @RestController inside the caller-service app. The endpoint is called by the first-service app through the RestTemplate bean.

@RestController
@RequestMapping("/caller")
public class CallerController {

   private static final Logger LOGGER = LoggerFactory
      .getLogger(CallerController.class);

   @Autowired
   Optional buildProperties;
   @Autowired
   RestTemplate restTemplate;
   @Value("${VERSION}")
   private String version;

   @GetMapping("/ping")
   public String ping() {
      LOGGER.info("Ping: name={}, version={}", 
         buildProperties.or(Optional::empty), version);
      String response = restTemplate.getForObject(
         "http://callme-service:8080/callme/ping", String.class);
      LOGGER.info("Calling: response={}", response);
      return "I'm caller-service " + version + ". Calling... " + response;
   }

}

Finally, here’s the @RestController inside the callme-service app. It also exposes a single GET /callme/ping endpoint called by the caller-service app:

@RestController
@RequestMapping("/callme")
public class CallmeController {

   private static final Logger LOGGER = LoggerFactory
      .getLogger(CallmeController.class);
   private static final String INSTANCE_ID = UUID.randomUUID().toString();
   private Random random = new Random();

   @Autowired
   Optional buildProperties;
   @Value("${VERSION}")
   private String version;

   @GetMapping("/ping")
   public String ping() {
      LOGGER.info("Ping: name={}, version={}", buildProperties.isPresent() 
         ? buildProperties.get().getName() : "callme-service", version);
      return "I'm callme-service " + version;
   }

}

Build and Deploy Images with Skaffold and Jib

Firstly, let’s take a look at the main Maven pom.xml in the project root directory. We use the latest version of Spring Boot and the latest LTS version of Java for compilation. All three app modules inherit settings from the parent pom.xml. In order to build the image with Maven we are including jib-maven-plugin. Since it is still using Java 17 in the default base image, we need to override this behavior with the . tag. We will declare eclipse-temurin:21-jdk-ubi9-minimal as the base image. Note that jib-maven-plugin is activated only if we enable the jib Maven profile during the build.

4.0.0


  org.springframework.boot
  spring-boot-starter-parent
  3.2.0
  


pl.piomin.services
sample-istio-services
1.1.0
pom


  21



  caller-service
  callme-service
  first-service



  
    jib
    
      
        
          com.google.cloud.tools
          jib-maven-plugin
          3.4.0
          
            
              eclipse-temurin:21-jdk-ubi9-minimal

Now, let’s take a look at the main skaffold.yaml file. Skaffold builds the image using Jib support and deploys all three apps on Kubernetes using manifests available in the k8s/deployment.yaml file inside each app module. Skaffold disables JUnit tests for Maven and activates the jib profile. It is also able to deploy Istio objects after activating the istio Skaffold profile. However, we won’t use it today.

apiVersion: skaffold/v4beta5
kind: Config
metadata:
  name: simple-istio-services
build:
  artifacts:
    - image: piomin/first-service
      jib:
        project: first-service
        args:
          - -Pjib
          - -DskipTests
    - image: piomin/caller-service
      jib:
        project: caller-service
        args:
          - -Pjib
          - -DskipTests
    - image: piomin/callme-service
      jib:
        project: callme-service
        args:
          - -Pjib
          - -DskipTests
  tagPolicy:
    gitCommit: {}
manifests:
  rawYaml:
    - '*/k8s/deployment.yaml'
deploy:
  kubectl: {}
profiles:
  - name: istio
    manifests:
      rawYaml:
        - k8s/istio-*.yaml
        - '*/k8s/deployment-versions.yaml'
        - '*/k8s/istio-*.yaml'

Here’s the typical Deployment for our apps. The app is running on the 8080 port.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: first-service
spec:
  replicas: 1
  selector:
    matchLabels:
      app: first-service
  template:
    metadata:
      labels:
        app: first-service
    spec:
      containers:
        - name: first-service
          image: piomin/first-service
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
          env:
            - name: VERSION
              value: "v1"

For testing purposes, we need to expose the first-service outside of the Kind cluster. In order to do that, we will use the Kubernetes NodePort Service. Our app will be available under the 30000 port.

apiVersion: v1
kind: Service
metadata:
  name: first-service
  labels:
    app: first-service
spec:
  type: NodePort
  ports:
  - port: 8080
    name: http
    nodePort: 30000
  selector:
    app: first-service

Note that all other Kubernetes services (“caller-service” and “callme-service”) are exposed only internally using a default ClusterIP type.

How It Works

In this section, we will discuss how we would run the whole process locally. Of course, our goal is to configure it as the CircleCI pipeline. In order to expose the Kubernetes Service outside Kind we need to define the externalPortMappings section in the configuration manifest. As you probably remember, we are exposing our app under the 30000 port. The following file is available in the repository under the k8s/kind-cluster-test.yaml path:

apiVersion: kind.x-k8s.io/v1alpha4
kind: Cluster
nodes:
  - role: control-plane
    extraPortMappings:
      - containerPort: 30000
        hostPort: 30000
        listenAddress: "0.0.0.0"
        protocol: tcp

Assuming we already installed kind CLI on our machine, we need to execute the following command to create a new cluster:

$ kind create cluster --name c1 --config k8s/kind-cluster-test.yaml

You should have the same result as visible on my screen:

We have a single-node Kind cluster ready. There is a single c1-control-plane container running on Docker. As you see, it exposes 30000 port outside of the cluster:

The Kubernetes context is automatically switched to kind-c1. So now, we just need to run the following command from the repository root directory to build and deploy the apps:

$ skaffold run

If you see a similar output in the skaffold run logs, it means that everything works fine.

We can verify a list of Kubernetes services. The first-service is exposed under the 30000 port as expected.

$ kubectl get svc
NAME             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
caller-service   ClusterIP   10.96.47.193           8080/TCP         2m24s
callme-service   ClusterIP   10.96.98.53            8080/TCP         2m24s
first-service    NodePort    10.96.241.11           8080:30000/TCP   2m24s

Assuming you have already installed the Grafana k6 tool locally, you may run load tests using the following command:

$ k6 run first-service/src/test/resources/k6/load-test.js

That’s all. Now, let’s define the same actions with the CircleCI workflow.

Test Kubernetes Deployment with the CircleCI Workflow

The CircleCI config.yml file should be placed in the .circle directory. We are doing two things in our pipeline. In the first step, we are executing Maven unit tests without the Kubernetes cluster. That’s why we need a standard executor with OpenJDK 21 and the maven ORB. In order to run Kind during the CircleCI build, we need to have access to the Docker daemon. Therefore, we use the latest version of the ubuntu-2204 machine.

version: 2.1

orbs:
  maven: circleci/maven@1.4.1

executors:
  jdk:
    docker:
      - image: 'cimg/openjdk:21.0'
  machine_executor_amd64:
    machine:
      image: ubuntu-2204:2023.10.1
    environment:
      architecture: "amd64"
      platform: "linux/amd64"

After that, we can proceed to the job declaration. The name of our job is deploy-k8s. It uses the already-defined machine executor. Let’s discuss the required steps after running a standard checkout command:

We need to install the kubectl CLI and copy it to the /usr/local/bin directory. Skaffold uses kubectl to interact with the Kubernetes cluster.
After that, we have to install the skaffold CLI
Our job also requires the kind CLI to be able to create or delete Kind clusters on Docker…
… and the Grafana k6 CLI to run load tests against the app deployed on the cluster
There is a good chance that this step won’t required once CircleCI releases a new version of ubuntu-2204 machine (probably 2024.1.1 according to the release strategy). For now, ubuntu-2204 provides OpenJDK 17, so we need to install OpenJDK 17 to successfully build the app from the source code
After installing all the required tools we can create a new Kubernetes with the kind create cluster command.
Once a cluster is ready, we can deploy our apps using the skaffold run command.
Once the apps are running on the cluster, we can proceed to the tests phase. We are running the test defined inside the first-service/src/test/resources/k6/load-test.js file.
After doing all the required steps, it is important to remove the Kind cluster

 jobs:
  deploy-k8s:
    executor: machine_executor_amd64
    steps:
      - checkout
      - run: # (1)
          name: Install Kubectl
          command: |
            curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
            chmod +x kubectl
            sudo mv ./kubectl /usr/local/bin/kubectl
      - run: # (2)
          name: Install Skaffold
          command: |
            curl -Lo skaffold https://storage.googleapis.com/skaffold/releases/latest/skaffold-linux-amd64
            chmod +x skaffold
            sudo mv skaffold /usr/local/bin
      - run: # (3)
          name: Install Kind
          command: |
            [ $(uname -m) = x86_64 ] && curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.20.0/kind-linux-amd64
            chmod +x ./kind
            sudo mv ./kind /usr/local/bin/kind
      - run: # (4)
          name: Install Grafana K6
          command: |
            sudo gpg -k
            sudo gpg --no-default-keyring --keyring /usr/share/keyrings/k6-archive-keyring.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69
            echo "deb [signed-by=/usr/share/keyrings/k6-archive-keyring.gpg] https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list
            sudo apt-get update
            sudo apt-get install k6
      - run: # (5)
          name: Install OpenJDK 21
          command: |
            java -version
            sudo apt-get update && sudo apt-get install openjdk-21-jdk
            sudo update-alternatives --set java /usr/lib/jvm/java-21-openjdk-amd64/bin/java
            sudo update-alternatives --set javac /usr/lib/jvm/java-21-openjdk-amd64/bin/javac
            java -version
            export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
      - run: # (6)
          name: Create Kind Cluster
          command: |
            kind create cluster --name c1 --config k8s/kind-cluster-test.yaml
      - run: # (7)
          name: Deploy to K8s
          command: |
            export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
            skaffold run
      - run: # (8)
          name: Run K6 Test
          command: |
            kubectl get svc
            k6 run first-service/src/test/resources/k6/load-test.js
      - run: # (9)
          name: Delete Kind Cluster
          command: |
            kind delete cluster --name c1

Here’s the definition of our load test. It has to be written in JavaScript. It defines some thresholds like a % of maximum failed requests or maximum response time for 95% of requests. As you see, we are testing the http://localhost:30000/first/ping endpoint:

import { sleep } from 'k6';
import http from 'k6/http';

export const options = {
  duration: '60s',
  vus: 10,
  thresholds: {
    http_req_failed: ['rate<0.25'],
    http_req_duration: ['p(95)<1000'],
  },
};

export default function () {
  http.get('http://localhost:30000/first/ping');
  sleep(2);
}

Finally, the last part of the CircleCI config file. It defines pipeline workflow. In the first step, we are running tests with Maven. After that, we proceeded to the deploy-k8s job.

workflows:
  build-and-deploy:
    jobs:
      - maven/test:
          name: test
          executor: jdk
      - deploy-k8s:
          requires:
            - test

Once we push a change to the sample Git repository we trigger a new CircleCI build. You can verify it by yourself here in my CircleCI project page.

As you see all the pipeline steps have been finished successfully.

We can display logs for every single step. Here are the logs from the k6 load test step.

There were some errors during the warm-up. However, the test shows that our scenario works on the Kubernetes cluster.

Final Thoughts

CircleCI is one of the most popular CI/CD platforms. Personally, I’m using it for running builds and tests for all my demo repositories on GitHub. For the sample projects dedicated to the Kubernetes cluster, I want to verify such steps as building images with Jib, Kubernetes deployment scripts, or Skaffold configuration. This article shows how to easily perform such tests with CircleCI and Kubernetes cluster running on Kind. Hope it helps

The post Kubernetes Testing with CircleCI, Kind, and Skaffold appeared first on Piotr's TechBlog.

Create and Release Your Own Helm Chart

piotr.minkowski — Tue, 28 Feb 2023 14:32:18 +0000

In this article, you will learn how to create your Helm chart and release it in the public repository. We will prepare a Helm chart for the typical Spring Boot REST-based app as an exercise. Our goal is to have a fully automated process to build, test, and release it. In order to do that, we will define a pipeline in CircleCI. This CI/CD pipeline will publish the Helm chart in the public Artifact Hub.

If you are interested in the Helm chart and CI/CD process you may refer to the following article. It shows how to design your continuous development process on Kubernetes and use Helm charts in the GitOps approach.

Source Code

If you would like to try it by yourself, you may always take a look at my source code. In order to do that you need to clone my GitHub repository. Then you should just follow my instructions.

Create Helm Chart

In this part of the exercise, we will use the helm CLI. Helm installed locally is not required in our whole process, but helps you understand what will happen in the next steps. Therefore, it is worth installing it. Please refer to the official Helm docs to find an installation approach suitable for your needs.

In the first step, we are going to create a sample chart. It is a typical chart for web apps. For example, it exposes the 8080 port outside of the containers or allows us to define liveness and readiness probes checking HTTP endpoints. This Helm chart should not be too complicated, but also not too simple, since we want to create automated tests for it.

Here’s our Deployment template. It adds some standard labels to the Deployment manifest (1). It also sets resource requests and limits (2). As I mentioned before, our chart is adding liveness probe (3), readiness probe (4), and exposes port 8080 outside of the container (5). We may also set environment variables (6), or inject them from ConfigMap (7) and Secret (8).

apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ .Values.app.name }}
  labels: # (1)
    app: {{ .Values.app.name }}
    env: {{ .Values.app.environment }}
    owner: {{ .Values.app.owner }}
spec:
  replicas: {{ .Values.replicaCount }}
  selector:
    matchLabels:
      app: {{ .Values.app.name }}
  template:
    metadata:
      labels:
        app: {{ .Values.app.name }}
        env: {{ .Values.app.environment }}
    spec:
      containers:
        - name: {{ .Values.app.name }}
          image: {{ .Values.image.repository }}:{{ .Values.image.tag }}
          resources: # (2)
            {{- toYaml .Values.resources | nindent 12 }}
          livenessProbe: # (3)
            initialDelaySeconds: {{ .Values.liveness.initialDelaySeconds }}
            httpGet:
              port: {{ .Values.liveness.port }}
              path: {{ .Values.liveness.path }}
            failureThreshold: {{ .Values.liveness.failureThreshold }}
            successThreshold: {{ .Values.liveness.successThreshold }}
            timeoutSeconds: {{ .Values.liveness.timeoutSeconds }}
            periodSeconds: {{ .Values.liveness.periodSeconds }}
          readinessProbe: # (4)
            initialDelaySeconds: {{ .Values.readiness.initialDelaySeconds }}
            httpGet:
              port: {{ .Values.readiness.port }}
              path: {{ .Values.readiness.path }}
            failureThreshold: {{ .Values.readiness.failureThreshold }}
            successThreshold: {{ .Values.readiness.successThreshold }}
            timeoutSeconds: {{ .Values.readiness.timeoutSeconds }}
            periodSeconds: {{ .Values.readiness.periodSeconds }}
          ports: # (5)
          {{- range .Values.ports }}
          - containerPort: {{ .value }}
            name: {{ .name }}
          {{- end }}
          {{- if .Values.envs }}
          env: # (6)
          {{- range .Values.envs }}
          - name: {{ .name }}
            value: {{ .value }}
          {{- end }}
          {{- end }}
          {{- if or .Values.extraEnvVarsConfigMap .Values.extraEnvVarsSecret }}
          envFrom:
          {{- if or .Values.extraEnvVarsConfigMap }}
          - configMapRef:
              name: {{ .Values.extraEnvVarsConfigMap }}
          {{- end }}
          {{- if or .Values.extraEnvVarsSecret }}
          - secretRef:
              name: {{ .Values.extraEnvVarsSecret }}
          {{- end }}
          {{- end }}
          securityContext:
            runAsNonRoot: true

We also have a template for the Service object.

apiVersion: v1
kind: Service
metadata:
  name: {{ .Values.app.name }}
  labels:
    app: {{ .Values.app.name }}
    env: {{ .Values.app.environment }}
    owner: {{ .Values.app.owner }}
spec:
  type: {{ .Values.service.type }}
  selector:
    app: {{ .Values.app.name }}
  ports:
  {{- range .Values.ports }}
  - port: {{ .value }}
    name: {{ .name }}
  {{- end }}

Now, we can fill our templates with the default values. The following values.yaml file is available in the sample repository.

replicaCount: 1

app:
  name: sample-spring-boot-api
  environment: dev
  owner: default

image:
  repository: piomin/sample-spring-kotlin-microservice
  tag: "1.1"

nameOverride: ""
fullnameOverride: ""

service:
  type: ClusterIP

ports:
  - name: http
    value: 8080

resources:
  limits:
    cpu: 1000m
    memory: 512Mi
  requests:
    cpu: 100m
    memory: 256Mi

liveness:
  initialDelaySeconds: 10
  port: http
  path: /actuator/health/liveness
  failureThreshold: 3
  successThreshold: 1
  timeoutSeconds: 3
  periodSeconds: 5

readiness:
  initialDelaySeconds: 10
  port: http
  path: /actuator/health/readiness
  failureThreshold: 3
  successThreshold: 1
  timeoutSeconds: 3
  periodSeconds: 5

envs:
  - name: INFO
    value: Spring Boot REST API

You can easily test the newly created templates with the helm CLI. In order to do that, just execute the following command in the repository root directory. As a result, you will see the YAML manifests created from our sample templates.

$ helm template charts/spring-boot-api-app

Such a testing method is fine… but just to run it locally during chart development. Assuming we need to create a delivery pipeline, we need a more advanced tool.

Unit Testing of Helm Charts

From my perspective, the most important thing in the CI/CD pipeline is automated testing. Without it, we are releasing unverified software, which may potentially result in many complications. The single Helm chart can be by several apps, so we should make every effort to carefully test it. Fortunately, there are some tools dedicated to Helm chart testing. My choice fell on the helm-unittest. It allows us to write unit tests file in pure YAML. We can install it as a Helm plugin or run it inside the Docker container. Let’s just it locally to verify our test work before pushing it to the Git repository:

$ helm plugin install https://github.com/helm-unittest/helm-unittest

We should place the unit test inside the test directory in our chart. Here’s the structure of our chart repository:

In the first step, we are creating the unit test file. As mentioned before, we can create a test using the YAML notation. It is pretty intuitive. We need to pass a location of the values file (1) and a location of the tested Helm template (2). In the test section, we have to define a list of asserts (3). I will not get into the details of the helm-unittest tool – for more information please refer to its docs. The important thing is that I can easily test each path of the YAML manifest. It can be an exact comparison or regex. It also supports JsonPath for mappings and arrays. Here’s our test in the deployment_test.yaml:

suite: test deployment
values:
  - ./values/test.yaml # (1)
templates:
  - templates/deployment.yaml # (2)
chart:
  version: 0.3.4+test
  appVersion: 1.0.0
tests:
  - it: should pass all kinds of assertion
    template: templates/deployment.yaml
    documentIndex: 0
    asserts: # (3)
      - equal:
          path: spec.template.spec.containers[?(@.name == "sample-spring-boot-api")].image
          value: piomin/sample-spring-kotlin-microservice:1.1
      - equal:
          path: metadata.labels.app
          value: sample-spring-boot-api
      - equal:
          path: metadata.labels.env
          value: dev
      - equal:
          path: metadata.labels.owner
          value: default
      - matchRegex:
          path: metadata.name
          pattern: ^.*-api$
      - contains:
          path: spec.template.spec.containers[?(@.name == "sample-spring-boot-api")].ports
          content:
            containerPort: 8080
            name: http
      - notContains:
          path: spec.template.spec.containers[?(@.name == "sample-spring-boot-api")].ports
          content:
            containerPort: 80
      - isNotEmpty:
          path: spec.template.spec.containers[?(@.name == "sample-spring-boot-api")].livenessProbe
      - isNotEmpty:
          path: spec.template.spec.containers[?(@.name == "sample-spring-boot-api")].readinessProbe
      - isKind:
          of: Deployment
      - isAPIVersion:
          of: apps/v1

Now, we can verify the test locally by executing the following command from the project root directory:

$ helm unittest charts/*

Currently, we have only a single chart in the charts directory. Assuming we would have more, it runs the tests for all charts. Here’s our result:

We can change something in the test to break it. Now, the result of the same will look like that:

Helm Chart Release Pipeline in CircleCI

Once we have a chart and tests created we may proceed to the delivery pipeline. In our CircleCI pipeline, we have to do not only the same steps as before, but also need to include a release part. First of all, we will use GitHub Releases and GitHub Pages to release and host our charts. In order to simplify the process, we may use a dedicated tool for releasing Helm charts – Chart Releaser.

We also need to create a personal token to pass to the Helm Chart Release workflow. Visit Settings > Developer Settings > Personal Access Token. Generate Personal the token with the repo scope. Then, we should put this token into the CircleCI context. You can choose any name for the context, but the name of the environment variable has to be CR_TOKEN. That name is required by the Chart Releaser. The name of my context is GitHub.

Here’s the list of steps we need to do in our pipeline:

Install the helm CLI on the machine (we will use the cimg/base image as the tests executor)
Install the Helm unit-test plugin
Run unit tests
Only if we make a change in the master branch, then we proceed to the release part. In the first step, we need to package the chart with helm package command
Install Chart Releaser
Release the chart in GitHub with the Chart Releaser upload command
Generate chart index.yaml and publish it on GitHub Pages

Let’s define our CircleCI pipeline. First, we need to create the .circleci directory in the repository root and place the config.yml file there. We can use the helm orb to simplify the process of the helm CLI installation (1) (2). Once we install the helm CLI, we can install the unit-test plugin and run the unit tests (3). Then we define a rule for filtering the master branch (4). If the change is pushed to the master branch we package the chart as the TAR archive and place it in the .deploy directory (5). Then we install Chart Releaser and create a GitHub release (6). In the last step, we generate the index.yaml file using Chart Releaser and commit it to the gh-pages branch (7).

version: 2.1

orbs:
  helm: circleci/helm@2.0.1 # (1)

jobs:
  build:
    docker:
      - image: cimg/base:2023.02
    steps:
      - checkout
      - helm/install-helm-client # (2)
      - run:
          name: Install Helm unit-test
          command: helm plugin install https://github.com/helm-unittest/helm-unittest
      - run: # (3)
          name: Run unit tests
          command: helm unittest charts/*
      - when:
          condition: # (4)
            equal: [ master, << pipeline.git.branch >> ]
          steps:
            - run:
                name: Package chart # (5)
                command: helm package charts/* -d .deploy
            - run:
                name: Install chart releaser
                command: |
                  curl -L -o /tmp/cr.tgz https://github.com/helm/chart-releaser/releases/download/v1.5.0/chart-releaser_1.5.0_linux_amd64.tar.gz
                  tar -xv -C /tmp -f /tmp/cr.tgz
                  mv /tmp/cr ~/bin/cr
            - run:
                name: Release chart # (6)
                command: cr upload -o piomin -r helm-charts -p .deploy
            - run:
                name: Create index on GitHub pages # (7)
                command: |
                  git config user.email "job@circleci.com"
                  git config user.name "CircleCI"
                  git checkout --orphan gh-pages
                  cr index -i ./index.yaml -p .deploy -o piomin -r helm-charts
                  git add index.yaml
                  git commit -m "New release"
                  git push --set-upstream --force origin gh-pages

workflows:
  helm_test:
    jobs:
      - build:
          context: GitHub

Execute Helm Chart Release Pipeline

Once we push a change to the helm-charts repository our pipeline is starting. Here is our result. As you see the pipeline finishes with success. We were releasing the 0.3.5 version of our chart.

Let’s see a list of GitHub releases. As you see, the 0.3.5 version has already been released.

How to access our Helm repository. In order to check it go to the repository Settings > Pages. The address of the GitHub Pages for that repository is the address of our Helm repository. We publish there the index.yaml file that contains a definition of charts inside the repository. As you see, the address of the Helm repository is piomin.github.io/helm-charts.

We can see the structure of the index.yaml file just by calling the following URL: https://piomin.github.io/helm-charts/index.yaml. Here’s the fragment of index.yaml for the currently published version.

apiVersion: v1
entries:
  spring-boot-api-app:
  - apiVersion: v2
    appVersion: 1.0.0
    created: "2023-02-28T13:06:28.835693321Z"
    description: A Helm chart for Kubernetes
    digest: b308cbdf9f93f79baf5b39de8a7c509834d7b858f33d79d0c76b528e0cd7ca11
    name: spring-boot-api-app
    type: application
    urls:
    - https://github.com/piomin/helm-charts/releases/download/spring-boot-api-app-0.3.5/spring-boot-api-app-0.3.5.tgz
    version: 0.3.5

Assuming we want to use our Helm chart we can easily access it. First, let’s add the Helm repository using CLI:

$ helm repo add piomin https://piomin.github.io/helm-charts/

Then, we can verify a list of Helm charts existing inside the repository:

$ helm search repo piomin
NAME                            CHART VERSION   APP VERSION     DESCRIPTION                
piomin/spring-boot-api-app      0.3.5           1.0.0           A Helm chart for Kubernetes

Publish Helm Chart to Artifact Hub

In order to publish your Helm repository and charts on Artifact Hub you need to go to that site and create an account. Once you do it, you can add a new repository just by clicking the button. Then you just need to choose the name of your repo and put the right address.

Now, we can find our spring-boot-api-app chart on the list of packages.

We can see its details. It’s worth publishing documentation in the README.md file. Once you do it, you can view it in the chart details on Artifact Hub.

Finally, we can easily use the chart and deploy the Spring Boot app, e.g. with Argo CD.

The post Create and Release Your Own Helm Chart appeared first on Piotr's TechBlog.

Advanced Testing with Quarkus

piotr.minkowski — Wed, 08 Feb 2023 09:52:45 +0000

This article will teach you how to build advanced testing scenarios with Quarkus. We will focus mainly on the integration tests. Quarkus simplifies them by leveraging the Testcontainers project. In many cases, it is a smooth integration process. You won’t even notice you are using Testcontainers under the hood.

Before starting with the test it is worth reading about the Quarkus framework. If you are familiar with Spring Boot, I especially recommend the following article about Quarkus. It shows some useful and interesting features of the Quarkus framework that Spring Boot doesn’t provide.

Introduction: The Basics

Let’s begin with the basics. Quarkus has three different launch modes: dev, test, and prod. It defines a built-in profile for each of those modes. As you probably guessed, today we will focus on the test mode. It is automatically activated when running tests during the build. The class containing tests should be annotated with @QuarkusTest. We may provide a configuration dedicated to the particular mode using the following semantics in the application.properties file:

%prod.quarkus.datasource.db-kind = postgresql
%prod.quarkus.datasource.username = ${PG_USER}
%prod.quarkus.datasource.password = ${PG_PASS}
%prod.quarkus.datasource.jdbc.url = jdbc:postgresql://pg:5432/${PG_DB}

Let’s assume we have a very simple endpoint that returns an object by its id:

@Path("/persons")
public class PersonResource {

   @Inject
   InMemoryPersonRepository inMemoryRepository;

   @GET
   @Path("/{id}")
   public Person getPersonById(@PathParam("id") Long id) {
      return inMemoryRepository.findById(id);
   }

}

Now, we have to create a test. We don’t need to take care of a port. By default, Quarkus runs on the 8081 port in test mode. It also automatically configures the Rest Assured library to interact with the server.

@QuarkusTest
public class PersonResourceTests {

   @Test
   void getById() {
      given().get("/persons/{id}", 1L)
         .then()
         .statusCode(200)
         .body("id", notNullValue());
   }

}

Source Code

If you would like to try it by yourself, you may always take a look at my source code. This time, we have multiple repositories with examples. All those repositories contain Quarkus testing scenarios for the different use cases. You can clone the repository with a single app that connects to the Postgres database. There are two other repositories with microservices. Here’s the repository with simple microservices. There is another one with Consul configuration and discovery. Then you should just follow my instructions.

Testing with External Services

Let’s include a database in our scenario. We will use Postgres. In order to interact with the database, we will leverage the Quarkus Panache ORM module. Firstly, we need to add the following two dependencies:


   io.quarkus
   quarkus-hibernate-orm-panache


   io.quarkus
   quarkus-jdbc-postgresql

Here’s our Person entity:

@Entity
public class Person extends PanacheEntityBase {

   @Id
   @GeneratedValue(strategy = GenerationType.IDENTITY)
   public Long id;
   public String name;
   public int age;
   @Enumerated(EnumType.STRING)
   public Gender gender;

}

We also need to create the repository:

@ApplicationScoped
public class PersonRepository implements PanacheRepository {

   public List findByName(String name) {
      return find("name", name).list();
   }

   public List findByAgeGreaterThan(int age) {
      return find("age > ?1", age).list();
   }

}

Finally, we have a resource endpoints implementation:

@Path("/persons")
public class PersonResource {

   private PersonRepository repository;

   public PersonResource(PersonRepository repository) {
      this.repository = repository;
   }

   @GET
   public List findAll() {
      return repository.findAll().list();
   }

   @GET
   @Path("/name/{name}")
   public List findByName(@PathParam("name") String name) {
      return repository.findByName(name);
   }

   @GET
   @Path("/{id}")
   public Person findById(@PathParam("id") Long id) {
      return repository.findById(id);
   }
}

The most important thing here is not to place any addresses for the test mode. We may set them, for example, only for the prod mode. Here’s our test. It doesn’t differ much from the previous one. We don’t need to add any special annotations, dependencies, or objects. Everything happens automatically. The only thing we need to guarantee is access to the Docker host. Quarkus will automatically start the Postgres container there and configure connection settings for the app.

@QuarkusTest
public class PersonResourceTest {

    @Test
    public void findAll() {
        given()
          .when().get("/persons")
          .then()
             .statusCode(200)
             .assertThat().body("size()", is(20));
    }

    @Test
    public void findById() {
        Person person = given()
                .when().get("/persons/1")
                .then()
                .statusCode(200)
                .extract()
                .body().as(Person.class);
        assertNotNull(person);
        assertEquals(1L, person.id);
    }

}

Let’s run our tests. Here’s the fragment of the logs. Before running the tests Quarkus starts the Postgres container using Testcontainers:

Then it runs our tests and exposes the app at the 8081 port.

Ok, our tests work fine on the local machine. However, the goal is to run them as a part of the CI process. This time we will use CircleCI. The process needs to have access to the Docker host. We may use a dedicated Linux machine as an executor or take advantage of the Testcontainers cloud. Here’s a build configuration for the second option.

version: 2.1

orbs:
  maven: circleci/maven@1.4.0
  tcc: atomicjar/testcontainers-cloud-orb@0.1.0

executors:
  j17:
    docker:
      - image: 'cimg/openjdk:17.0'

workflows:
  maven_test:
    jobs:
      - maven/test:
          executor: j17
          context: Testcontainers
          pre-steps:
            - tcc/setup

Now, we just need to create a job in CircleCI and run the pipeline.

Integration Testing with Quarkus

Instead of @QuarkusTest, we can annotate our test with @QuarkusIntegrationTest. It is a really powerful solution in conjunction with the Quarkus containers build feature. It allows us to run the tests against an already-built image containing the app. First, let’s include the Quarkus Jib module:


   io.quarkus
   quarkus-container-image-jib

We need to enable container build in the application.properties:

quarkus.container-image.build = true

This time instead of the RestAssured object we will use the real HTTP client. Quarkus provides a convenient method for creating declarative REST clients. We need to define an interface with endpoint methods:

public interface EmployeeService {

   @POST
   Employee add(@Valid Employee employee);

   @GET
   Set findAll();

   @Path("/{id}")
   @GET
   Employee findById(@PathParam("id") Long id);

}

In the test class, we use @TestHTTPResource and @TestHTTPEndpoint annotations to inject the test URL. Then we are creating a client with the RestClientBuilder and call the service started on the container. The name of the test class is not accidental. In order to be automatically detected as the integration test, it has the IT suffix.

@QuarkusIntegrationTest
public class EmployeeControllerIT {

    @TestHTTPEndpoint(EmployeeController.class)
    @TestHTTPResource
    URL url;

    @Test
    void add() {
        EmployeeService service = RestClientBuilder.newBuilder()
                .baseUrl(url)
                .build(EmployeeService.class);
        Employee employee = new Employee(1L, 1L, "Josh Stevens", 
                                         23, "Developer");
        employee = service.add(employee);
        assertNotNull(employee.getId());
    }

    @Test
    public void findAll() {
        EmployeeService service = RestClientBuilder.newBuilder()
                .baseUrl(url)
                .build(EmployeeService.class);
        Set employees = service.findAll();
        assertTrue(employees.size() >= 3);
    }

    @Test
    public void findById() {
        EmployeeService service = RestClientBuilder.newBuilder()
                .baseUrl(url)
                .build(EmployeeService.class);
        Employee employee = service.findById(1L);
        assertNotNull(employee.getId());
    }
}

Now, we just need to include the maven-failsafe-plugin. It will run our test during the verify or integration-test Maven phase.


  maven-failsafe-plugin
  ${surefire-plugin.version}
  
    
      
        integration-test
        verify

Let’s see how it works. In order to run the tests we need to execute the following Maven command:

$ mvn clean verify

Before running the tests, Quarkus build the app image using Jib:

Then, Maven runs the integration tests. Quarkus app starts as the container on Docker and exposes its endpoint over the default test URL http://localhost:8081.

Testcontainers with Quarkus

By default, Quarkus automatically runs several third-party services as containers. It includes databases, brokers like Kafka or RabbitMQ, and some others tools. Here’s a full list of supported software. What if we have a tool that’s not on that list? Let’s consider HashiCorp Consul. There’s a dedicated module for integrating it via Testcontainers:


  org.testcontainers
  consul
  1.17.6
  test


  org.testcontainers
  junit-jupiter
  1.17.6
  test

In order to start the container during the test we need to create a class that implements the QuarkusTestResourceLifecycleManager interface. It defines two methods: start() and stop(). Inside the start() method, we are creating and running the Consul container. Once it starts successfully we put a new key department.name under the config/department path. Then we need to override the address of the Consul used by the test to the dynamic address of the container run by Testcontainers.

public class ConsulResource implements QuarkusTestResourceLifecycleManager {

   private ConsulContainer consul;

   @Override
   public Map start() {
      consul = new ConsulContainer("consul:1.14")
         .withConsulCommand("kv put config/department department.name=abc");

      consul.start();

      String url = consul.getHost() + ":" + consul.getFirstMappedPort();

      return ImmutableMap.of("quarkus.consul-config.agent.host-port", url);
   }

   @Override
   public void stop() {
      consulContainer.stop();
   }
}

Here are the application settings related to the Consul instance. We use them only for storing configuration keys and values.

quarkus.consul-config.enabled=true
quarkus.consul-config.properties-value-keys=config/${quarkus.application.name}

Finally, we can go to the test implementation. In order to start the Consul container defined inside the ConsulResource class during the test we need to annotate the whole test with @QuarkusTestResource. By default, all test resources are global, even if they are defined on a test class or custom profile, which means they will all be activated for all tests. If you want to only enable a test resource on a single test class or test profile, you need to set the restrictToAnnotatedClass field to true. In the following test, I’m injecting the property department.name defined in our Consul instance under the /config/department key.

@QuarkusTest
@QuarkusTestResource(ConsulResource.class, 
                     restrictToAnnotatedClass = true)
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class DepartmentResourceConsulTests {

   @ConfigProperty(name = "department.name", defaultValue = "")
   private String name;

   @Test
   @Order(1)
   void add() {
      Department d = new Department();
      d.setOrganizationId(1L);
      d.setName(name);

      given().body(d).contentType(ContentType.JSON)
              .when().post("/departments").then()
              .statusCode(200)
              .body("id", notNullValue());
   }

   @Test
   @Order(2)
   void findAll() {
       when().get("/departments").then()
              .statusCode(200)
              .body("size()", is(1));
   }
}

Once again let’s run the test using the mvn clean verify command. Of course, don’t forget to run Docker on your laptop. Then, you verify the result.

Enable Profiles

Instead of creating an integration test that runs the Consul container, we can create a simple unit test with disabled interaction with Consul. In order to do that, we need to override a configuration setting. Of course, we can do it globally for the test profile using the following notation:

%test.quarkus.consul-config.enabled = false

However, assuming there are several different tests in the project, we may a different set of configuration properties per a single test class. In that case, we still have the integration test that interacts with the Consul container started on Docker. For such a scenario, Quarkus provides the QuarkusTestProfile interface. We need to create a class that implements it and overrides the value of the quarkus.consul-config.enabled property inside the getConfigOverrides() method.

public class DisableExternalProfile implements QuarkusTestProfile {

    @Override
    public Map getConfigOverrides() {
        return Map.of("quarkus.consul-config.enabled", "false");
    }
}

Then, we just need to annotate the test class with the @TestProfile holding of our implementation of the QuarkusTestProfile interface.

@QuarkusTest
@TestProfile(DisableExternalProfile.class)
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class DepartmentResourceTests {

    @Test
    @Order(1)
    void add() {
        Department d = new Department();
        d.setName("test");
        d.setOrganizationId(1L);

        given().body(d).contentType(ContentType.JSON)
                .when().post("/departments").then()
                .statusCode(200)
                .body("id", notNullValue());
    }

    @Test
    @Order(2)
    void findAll() {
        when().get("/departments").then()
                .statusCode(200)
                .body("size()", is(1));
    }

    @Test
    @Order(2)
    void findById() {
        when().get("/departments/{id}", 1).then()
                .statusCode(200)
                .body("id", is(1));
    }

}

Testing with Quarkus on Kubernetes

This topic couldn’t be missed in my article. The last part of the article will show how to deploy the app on Kubernetes and run tests against an application pod. There is no built-in support in Quarkus for the whole scenario, but we can simplify it with some separate features. First of all, Quarkus provides built-in support for building the image (we have already done it in one of the previous sections) and generating YAML manifests for Kubernetes. In order to use we need to include Quarkus Kubernetes Extension in the Maven dependencies. We will also include the fabric8 Kubernetes client for the test purposes:


  io.quarkus
  quarkus-kubernetes


  io.quarkus
  quarkus-kubernetes-client
  test

Now, if we set the property quarkus.kubernetes.deploy to true during the build, Quarkus will try to deploy the image to the current Kubernetes cluster. We can customize this process using Quarkus configuration properties. Of course, we need to build the image and push it to the container registry. In order to easily test the app in the @QuarkusIntegrationTest we will enable the NodePort Kubernetes service. Here’s the full set of configuration properties to perform the test.

quarkus.container-image.build = true
quarkus.container-image.group = piomin
quarkus.container-image.push = true

quarkus.kubernetes.deploy = true
quarkus.kubernetes.namespace = default
quarkus.kubernetes.service-type = node-port

It is important to run the test after the Maven build phase to deploy an already created and pushed image. The same in one of the previous scenarios we can use the @QuarkusIntegrationTest for that. We can use the KubernetesClient to detect the target port the employee-service running on Kubernetes. Then, we will use the Quarkus Rest client to call the target service as shown below.

@QuarkusIntegrationTest
public class EmployeeAppKubernetesIT {

   KubernetesClient client = new KubernetesClientBuilder().build();

   @Test
   void api() throws MalformedURLException {
      Service service = client.services()
         .inNamespace("default")
         .withName("employee-service")
         .get();
      ServicePort port = service.getSpec().getPorts().get(0);
      EmployeeService client = RestClientBuilder.newBuilder()
         .baseUrl(new URL("http://localhost:" + port.getNodePort() + "/employees"))
         .build(EmployeeService.class);
      Employee employee = new Employee(1L, 1L, "Josh Stevens", 23, "Developer");
      employee = client.add(employee);
      assertNotNull(employee.getId());
   }
}

That’s it. Let’s run the test. In the first step, the Quarkus Maven plugin builds the app image using Jib and pushes it to the registry:

Only after that, it tries to deploy the image on the current Kubernetes cluster. It creates Deployment and Service with the NodePort type.

Finally, it will run the test against the current Kubernetes cluster. As I mentioned before, there is no full built-in support for that scenario. So, for example, Quarkus still tries to run the Docker container with the app. In this scenario, our test ignores it and connects to the app deployed on Kubernetes.

Final Thoughts

Quarkus simplifies several things in automation testing. It effectively uses containers in the integration tests. Most of the things work out of the box without any additional configuration or annotations. Finally, we can easily include Kubernetes in our testing scenarios thanks to Quarkus Kubernetes Extension. I just included the most interesting Quarkus testing features. For detailed pieces of information, you may refer to the docs.

The post Advanced Testing with Quarkus appeared first on Piotr's TechBlog.

Manage Multiple GitHub Repositories with Renovate and CircleCI

piotr.minkowski — Thu, 12 Jan 2023 11:37:55 +0000

In this article, you will learn how to automatically update your GitHub repositories with Renovate and CircleCI. The problem we will try to solve today is strictly related to my blogging. As I always attach code examples to my posts, I have a lot of repositories to manage. I know that sometimes it is more convenient to have a single repo per all the demos, but I do not prefer it. My repo is always related to the particular technology or even to the case it is showing.

Let’s consider what’s the problem with that approach. I’m usually sharing the same repository across multiple articles if they are closely related to each other. But despite that, I have more than 100 repositories with code examples. Once I create a repository I usually don’t have time to keep it up to date. I need a tool that will do that automatically for me. This, however, forces me to improve automatic tests. If I configure a tool that automatically updates a code in GitHub repositories, I need to verify that the change is valid and will not break the demo app.

There is another problem related to that. Classics to the genre – lack of automation tests… I was always focusing on creating the example app to show the use case described in the post, but not on building the valuable tests. It’s time to fix that! This is my first New Year’s resolution As you probably guessed, my work is still in progress. But even now, I can show you which tools I’m using for that and how to configure them. I will also share some first thoughts. Let’s begin!

First Problem: Not Maintained Repositories

Did you ever try to run the app from the source code created some years ago? In theory, everything should go fine. But in practice, several things may have changed. I can use a different version of e.g. Java or Maven than before. Even if have automated tests, they may not work fine especially since I didn’t use any tool to run the build and tests remotely. Of course, I don’t have such many old, unmaintained repositories. Sometimes, I was updating them manually, in particular, those more popular and shared across several articles.

Let’s just take a look at this example. It is from the following repository. I’m trying to generate a class definition from the Protocol Buffers schema file. As you see, the plugin used for that is not able to find the protoc executable. Honestly, I don’t remember how it worked before. Maybe I installed something on my laptop… Anyway, the solution was to use another plugin that doesn’t require any additional executables. Of course, I need to do it manually.

Let’s analyze another example. This time it fails during integration tests from another repository. The test is trying to connect to the Docker container. The problem here is that I was using Windows some years ago and Docker Toolbox was, by default, available under the 192.168.99.100 address. I should not leave such an address in the test. However, once again, I was just running all the tests locally, and at that time they finished successfully.

By the way, moving such a test to the CircleCI pipeline is not a simple thing to do. In order to run some containers (pact-broker with posgtresql) before the pipeline I decided to use Docker Compose. To run containers with Docker Compose I had to enable remote docker for CicrleCI as described here.

Second Problem: Updating Dependencies

If you manage application repositories that use several libraries, you probably know that an update is sometimes not just a formality. Even if that’s a patch or a minor update. Although my applications are usually not very complicated, the update of the Spring Boot version may be challenging. In the following example of Netflix DGS usage (GraphQL framework), I tried to update from the 2.4.2 to the 2.7.7 version. Here’s the result.

In that particular case, my app was initiating the H2 database with some data from the data.sql file. But since one of the 2.4.X Spring Boot version, the records from the data.sql are loaded before database schema initialization. The solution is to replace that file with the import.sql script or add the property spring.jpa.defer-datasource-initialization=true to the application properties. After choosing the second option we solved the problem… and then another one occurs. This time it is related to Netflix DGS and GraphQL Java libraries as described here.

Currently, according to the comments, there is no perfect solution to that problem with Maven. Probably I will have to wait for the next release of Netflix DGS or wait until they will propose the right solution.

Let’s analyze another example – once again with the Spring Boot update. This time it is related to the Spring Data and Embedded Mongo. The case is very interesting since it fails just on the remote builder. When I’m running the test on my local machine everything works perfectly fine.

A similar issue has been described here. However, the described solution doesn’t help me anymore. Probably I will decide to migrate my tests to the Testcontainers. By the way, it is also a very interesting example, since it has an impact only on the tests. So, even with a high level of automation, you will still need to do manual work.

Third Problem: Lack of Automated Tests

It is some kind of paradox – although I’m writing a lot about continuous delivery or tests I have a lot of repositories without any tests. Of course, when I was creating real applications for several companies I was adding many tests to ensure they will fine on production. But even for simple demo apps it is worth adding several tests that verify if everything works fine. In that case, I don’t have many small unit tests but rather a test that runs a whole app and verifies e.g. all the endpoints. Fortunately, the frameworks like Spring Boot or Quarkus provide intuitive tools for that. There are helpers for almost all popular solutions. Here’s my @SprignBootTest for GraphQL queries.

@SpringBootTest(webEnvironment = 
      SpringBootTest.WebEnvironment.RANDOM_PORT)
public class EmployeeQueryResolverTests {

    @Autowired
    GraphQLTestTemplate template;

    @Test
    void employees() throws IOException {
        Employee[] employees = template
           .postForResource("employees.graphql")
           .get("$.data.employees", Employee[].class);
        Assertions.assertTrue(employees.length > 0);
    }

    @Test
    void employeeById() throws IOException {
        Employee employee = template
           .postForResource("employeeById.graphql")
           .get("$.data.employee", Employee.class);
        Assertions.assertNotNull(employee);
        Assertions.assertNotNull(employee.getId());
    }

    @Test
    void employeesWithFilter() throws IOException {
        Employee[] employees = template
           .postForResource("employeesWithFilter.graphql")
           .get("$.data.employeesWithFilter", Employee[].class);
        Assertions.assertTrue(employees.length > 0);
    }
}

In the previous test, I’m using an in-memory H2 database in the background. If I want to test smth with the “real” database I can use Testcontainers. This tool runs the required container on Docker during the test. In the following example, we run PostgreSQL. After that, the Spring Boot application automatically connects to the database thanks to the @DynamicPropertySource annotation that sets the generated URL as the Spring property.

@SpringBootTest(webEnvironment = 
      SpringBootTest.WebEnvironment.RANDOM_PORT)
@Testcontainers
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
public class PersonControllerTests {

   @Autowired
   TestRestTemplate restTemplate;

   @Container
   static PostgreSQLContainer postgres = 
      new PostgreSQLContainer<>("postgres:15.1")
           .withExposedPorts(5432);

   @DynamicPropertySource
   static void registerMySQLProperties(DynamicPropertyRegistry registry) {
       registry.add("spring.datasource.url", 
          postgres::getJdbcUrl);
       registry.add("spring.datasource.username", 
          postgres::getUsername);
       registry.add("spring.datasource.password", 
          postgres::getPassword);
   }

   @Test
   @Order(1)
   void add() {
       Person person = Instancio.of(Person.class)
               .ignore(Select.field("id"))
               .create();
       person = restTemplate
          .postForObject("/persons", person, Person.class);
       Assertions.assertNotNull(person);
       Assertions.assertNotNull(person.getId());
   }

   @Test
   @Order(2)
   void updateAndGet() {
       final Integer id = 1;
       Person person = Instancio.of(Person.class)
               .set(Select.field("id"), id)
               .create();
       restTemplate.put("/persons", person);
       Person updated = restTemplate
          .getForObject("/persons/{id}", Person.class, id);
       Assertions.assertNotNull(updated);
       Assertions.assertNotNull(updated.getId());
       Assertions.assertEquals(id, updated.getId());
   }

   @Test
   @Order(3)
   void getAll() {
       Person[] persons = restTemplate
          .getForObject("/persons", Person[].class);
       Assertions.assertEquals(1, persons.length);
   }

   @Test
   @Order(4)
   void deleteAndGet() {
       restTemplate.delete("/persons/{id}", 1);
       Person person = restTemplate
          .getForObject("/persons/{id}", Person.class, 1);
       Assertions.assertNull(person);
   }

}

In some cases, we may have multiple applications (or microservices) communicating with each other. We can mock that communication with the libraries like Mockito. On the other, we can simulate real HTTP traffic with the libraries like Hoverfly or Wiremock. Here’s the example with Hoverfly and the Spring Boot Test module.

@SpringBootTest(properties = { "POD_NAME=abc", "POD_NAMESPACE=default"}, 
   webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ExtendWith(HoverflyExtension.class)
public class CallerControllerTests {

   @LocalServerPort
   int port;
   @Autowired
   TestRestTemplate restTemplate;

   @Test
   void ping(Hoverfly hoverfly) {
      String msg = "callme-service v1.0-SNAPSHOT (id=1): abc in default";
      hoverfly.simulate(dsl(
            service("http://callme-service.serverless.svc.cluster.local")
               .get("/callme/ping")
               .willReturn(success(msg, "text/plain"))));

      String response = restTemplate
         .getForObject("/caller/ping", String.class);
      assertNotNull(response);

      String c = "caller-service(id=1): abc in default is calling " + msg;
      assertEquals(c, response);
   }
}

Of course, these are just examples of tests. There are a lot of different tests and technologies used in all my repositories. Some others would be added in the near future Now, let’s go to the point.

Choosing the Right Tools

As mentioned in the introduction, I will use CircleCI and Renovate for managing my GitHub repositories. CircleCI is probably the most popular choice for running builds of open-source projects stored in GitHub repositories. GitHub also provides a tool for updating dependencies called Dependabot. However, Renovate has some significant advantages over Dependabot. It provides a lot of configuration options, may be run anywhere (including Kubernetes – more details here), and can integrate also with GitLab or Bitbucket. We will also use SonarCloud for a static code quality analysis.

Renovate is able to analyze not only the descriptors of traditional package managers like npm, Maven, or Gradle but also e.g. CircleCI configuration files or Docker image tags. Here’s a list of my requirements that the following tool needs to meet:

It should be able to perform different actions depending on the dependency update type (major, patch, or minor)
It needs to create PR on change and auto-merge it only if the build performed by CircleCI finishes successfully. Therefore it needs to wait for the status of that build
Auto-merge should not be enabled for major updates. They require approval from the repository admin

Renovate meets all these requirements. We can also easily install Renovate on GitHub and use it to update CircleCI configuration files inside repositories. In order to install Renovate on GitHub you need to go to the marketplace. After you install it, go the Settings, and then the Applications menu item. In order to set the list of repositories enabled for Renovate click the Configure button.

Then in the Repository Access section, you can enable all your repositories or choose several from the whole list.

Configure Renovate and CircleCI inside the GitHub Repository

Each GitHub repository has to contain CircleCI and Renovate configuration files. Renovate tries to detect the renovate.json file in the repository root directory. We don’t provide many configuration settings to achieve the expected results. By default, Renovate creates a pull request once it detects a new version of dependency but does not auto-merge it. We want to auto-merge all non-major changes. Therefore, we need to set a list of all update types merged automatically (minor, patch, pin, and digest).

By default, Renovate creates PR just after it creates a branch with a new version of the dependency. Because we are auto-merging all non-major PRs we need to force Renovate to create them only after the build on CircleCI finishes successfully. Once, all the tests on the newly created branch will be passed, Renovate creates PR and auto-merge if it does not contain major changes. Otherwise, it leaves the PR for approval. To achieve it, we to set the property prCreation to not-pending. Here’s the renovate.json file I’m using for all my GitHub repositories.

{
  "$schema": "https://docs.renovatebot.com/renovate-schema.json",
  "extends": [
    "config:base",":dependencyDashboard"
  ],
  "packageRules": [
    {
      "matchUpdateTypes": ["minor", "patch", "pin", "digest"],
      "automerge": true
    }
  ],
  "prCreation": "not-pending"
}

The CircleCI configuration is stored in the .circleci/config.yml file. I mostly use Maven as a build tool. Here’s a typical CircleCI configuration file for my repositories. It defines two jobs: a standard maven/test job for building the project and running unit tests and a job for running SonarQube analysis.

version: 2.1

jobs:
  analyze:
    docker:
      - image: 'cimg/openjdk:17.0'
    steps:
      - checkout
      - run:
          name: Analyze on SonarCloud
          command: mvn verify sonar:sonar

executors:
  j17:
    docker:
      - image: 'cimg/openjdk:17.0'

orbs:
  maven: circleci/maven@1.4.0

workflows:
  maven_test:
    jobs:
      - maven/test:
          executor: j17
      - analyze:
          context: SonarCloud

By default, CicrcleCI runs builds on Docker containers. However, this approach is not suitable everywhere. For Testcontainers we need a machine executor that has full access to the Docker process. Thanks to that, it is able to run additional containers during tests with e.g. databases.

version: 2.1

jobs:
  analyze:
    docker:
      - image: 'cimg/openjdk:11.0'
    steps:
      - checkout
      - run:
          name: Analyze on SonarCloud
          command: mvn verify sonar:sonar -DskipTests

orbs:
  maven: circleci/maven@1.3.0

executors:
  machine_executor_amd64:
    machine:
      image: ubuntu-2204:2022.04.2
    environment:
      architecture: "amd64"
      platform: "linux/amd64"

workflows:
  maven_test:
    jobs:
      - maven/test:
          executor: machine_executor_amd64
      - analyze:
          context: SonarCloud

Finally, the last part of configuration – an integration between CircleCI and SonarCloud. We need to add some properties to Maven pom.xml to enable SonarCloud context.


  piomin_sample-spring-redis
  piomin
  https://sonarcloud.io

How It Works

Let’s verify how it works. Once you provide the required configuration for Renovate, CircleCI, and SonarCloud in your GitHub repository the process starts. Renovate initially detects a list of required dependency updates. Since I enabled the dependency dashboard, Renovate immediately creates an issue with a list of changes as shown below. It just provides a summary view showing a list of changes in the dependencies.

Here’s a list of detected package managers in this repository. Besides Maven and CircleCI, there is also Dockerfile and Gitlab CI configuration file there.

Some pull requests has already been automerged by Renovate, if the build on CircleCI has finished successfully.

Some other pull requests are still waiting in the Open state – waiting for approval (a major update from Java 11 to Java 17) or for a fix because the build on CicrcleCI failed.

We can go into the details of the selected PR. Let’s do that for the first PR (#11) on the list visible above. Renovate is trying to update Spring Boot from 2.6.1 to the latest 2.7.7. It created the branch renovate/spring-boot that contains the required changes.

The PR could be merged automatically. However the build failed, so it didn’t happen.

We can go to the details of the build. As you see in the CircleCI dashboard all the tests failed. In this particular case, I have already tried to fix the by updating the version of embedded Mongo. However, it didn’t solve the problem.

Here’s a list of commits in the master branch. As you see Renovate is automatically updating the repository after the build of the particular branch finishes successfully.

As you see, each time a new branch is created CircleCI runs a build to verify if it does not break the tests.

Conclusion

I have some conclusions after making the described changes in my repository:

1) Include automated tests in your projects even if you are creating an app for demo showcase, not for production usage. It will help you back to a project after some time. It will also ensure that everything works fine in your demo and helps other people when using it.
2) All these tools like Renovate, CircleCI, or SonarCloud can be easily used with your GitHub project for free. You don’t need to spend a lot of time configuring them, but the effect can be significant.
3) Keeping the repositories up to date is important. Sometimes people wrote to me that something doesn’t work properly in my examples. Even now, I found some small bugs in the code logic. Thanks to the described approach, I hope to give you a better quality of my examples – as you are my blog followers.

If you have something like that in my repository main site, it means that I already have reviewed the project and added all the described mechanisms in that article.

The post Manage Multiple GitHub Repositories with Renovate and CircleCI appeared first on Piotr's TechBlog.

Guide to building Spring Boot library

piotr.minkowski — Tue, 04 Aug 2020 08:32:58 +0000

In this article, I’m going to show you how to create and share your own custom Spring Boot library. If you decide to build such a product you should follow some best practices recommended by Spring Team. It’s a little bit more complicated than creating a plain Java library. Finally, you should publish your artifacts somewhere to share it with the community. Probably you need to obtain positive feedback from the community, so you should think about adding some extras. I’m also going to describe them. Let’s begin!

Examples

If you are looking for the examples of simple Spring Boot libraries you can take a look on my repositories: https://github.com/piomin/spring-boot-logging and https://github.com/piomin/spring-boot-istio.

1. Pick the right name

We should pick the right name for our library. Spring recommends creating special modules called “starters” that contain code with auto-configuration and customize the infrastructure of a given technology. The name of the third-party starter should end with spring-boot-starter and start with the name of the project or something related to the technology we are using in the library. It is contrary to the names of all official starters, which are created following the pattern spring-boot-starter-*. For example, the names of my libraries are logstash-logging-spring-boot-starter or istio-spring-boot-starter.

2. Create auto-configuration

Typically the “starter” module is separated from the “autoconfigure” module. However, it is not required. The autoconfigure module contains everything necessary for a start. Moreover, if I’m creating a simple library that does not consist of many classes, I’m inserting everything into a single starter module. Of course, that is my approach. You can still create a separate starter module that includes the required dependencies for the project. It is the most important that all the beans are registered inside the auto-configuration class. Do not annotate each of your beans inside a library with @Component or @Service, but define them in an auto-configured module. Here’s a simple auto-configuration class inside my logstash-logging-spring-boot-starter library.

@Configuration
@ConfigurationProperties(prefix = "logging.logstash")
public class SpringLoggingAutoConfiguration {

   private static final String LOGSTASH_APPENDER_NAME = "LOGSTASH";
   private String url = "localhost:8500";
   private String ignorePatterns;
   private boolean logHeaders;
   private String trustStoreLocation;
   private String trustStorePassword;

   @Value("${spring.application.name:-}")
   String name;

   @Autowired(required = false)
   Optional template;

   @Bean
   public UniqueIDGenerator generator() {
      return new UniqueIDGenerator();
   }

   @Bean
   public SpringLoggingFilter loggingFilter() {
      return new SpringLoggingFilter(generator(), ignorePatterns, logHeaders);
   }

   @Bean
   @ConditionalOnMissingBean(RestTemplate.class)
   public RestTemplate restTemplate() {
      RestTemplate restTemplate = new RestTemplate();
      List interceptorList = new ArrayList();
      interceptorList.add(new RestTemplateSetHeaderInterceptor());
      restTemplate.setInterceptors(interceptorList);
      return restTemplate;
   }

   @Bean
   @ConditionalOnProperty("logging.logstash.enabled")
   public LogstashTcpSocketAppender logstashAppender() {
      LoggerContext loggerContext = (LoggerContext) LoggerFactory.getILoggerFactory();
      LogstashTcpSocketAppender logstashTcpSocketAppender = new LogstashTcpSocketAppender();
      logstashTcpSocketAppender.setName(LOGSTASH_APPENDER_NAME);
      logstashTcpSocketAppender.setContext(loggerContext);
      logstashTcpSocketAppender.addDestination(url);
      if (trustStoreLocation != null) {
         SSLConfiguration sslConfiguration = new SSLConfiguration();
         KeyStoreFactoryBean factory = new KeyStoreFactoryBean();
         factory.setLocation(trustStoreLocation);
         if (trustStorePassword != null)
            factory.setPassword(trustStorePassword);
         sslConfiguration.setTrustStore(factory);
         logstashTcpSocketAppender.setSsl(sslConfiguration);
      }
      LogstashEncoder encoder = new LogstashEncoder();
      encoder.setContext(loggerContext);
      encoder.setIncludeContext(true);
      encoder.setCustomFields("{\"appname\":\"" + name + "\"}");
      encoder.start();
      logstashTcpSocketAppender.setEncoder(encoder);
      logstashTcpSocketAppender.start();
      loggerContext.getLogger(Logger.ROOT_LOGGER_NAME).addAppender(logstashTcpSocketAppender);
      return logstashTcpSocketAppender;
   }
}

To enable auto-configuration for the custom library we need to create file spring.factories in /src/main/resources/META-INF directory that contains a list of auto-configuration classes.

org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
pl.piomin.logging.config.SpringLoggingAutoConfiguration

3. Process annotations

Spring is an annotation-based framework. If you are creating your custom library you will usually define some annotations used to enable or disable features. With Spring Boot you can easily process such annotations. Here’s my custom annotation used to enable the Istio client on application startup. I’m following the Spring pattern widely used in Spring Cloud.

@Target({ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Documented
public @interface EnableIstio {
    int timeout() default 0;
    String version() default "";
    int weight() default 0;
    int numberOfRetries() default 0;
    int circuitBreakerErrors() default 0;
}

I need to process already defined annotation only once on startup. That’s why I’m creating a bean that implements the ApplicationListener interface to catch ContextRefreshedEvent emitted by Spring Boot.

public class ApplicationStartupListener implements
      ApplicationListener {
   private ApplicationContext context;
   private EnableIstioAnnotationProcessor processor;
   public ApplicationStartupListener(ApplicationContext context,
         EnableIstioAnnotationProcessor processor) {
      this.context = context;
      this.processor = processor;
   }
   @Override
   public void onApplicationEvent(ContextRefreshedEvent contextRefreshedEvent) {
      Optional annotation =
            context.getBeansWithAnnotation(EnableIstio.class).keySet().stream()
            .map(key -> context.findAnnotationOnBean(key, EnableIstio.class))
            .findFirst();
      annotation.ifPresent(enableIstio -> processor.process(enableIstio));
   }
}

4. Spring Boot library dependencies

Our library should reference only those artifacts or other starters, that are necessary for implementation. Here’s a minimal set of artifacts required for my istio-spring-boot-starter. Besides Spring and Spring Boot libraries I only use Kubernetes and Istio Java clients. We might as well declare a reference to spring-boot-starter-parent.


   
      me.snowdrop
      istio-client
      ${istio-client.version}
   
   
      io.fabric8
      kubernetes-client
      ${kubernetes-client.version}
   
   
      org.springframework
      spring-context-support
      ${spring.version}
      provided
   
   
      org.springframework.boot
      spring-boot-autoconfigure
      ${spring.boot.version}
      provided

5. Publish

Typically the implementation process of the Spring Boot library is divided into two phases. In the first phase, we need to implement the specific mechanisms related to our library. In the second phase, we should take care of following the Spring Boot best practices. Assuming we have already finished it, we may publish our custom starter to share it with the community. In my opinion, the best way to do it is by publishing it on the Maven Central repository.
You must go through several steps to publish JAR files to Maven Central. The list of necessary steps is listed below. For the more detailed description, you may refer to the article How to Publish Your Artifacts to Maven Central on DZone.
Here’s the list of prerequisites:

Create an account at Sonatype (https://oss.sonatype.org/)
Claim your product’s namespace by creating an issue in Sonatype’s Jira
Generate PGP private/public key pair to sign your JAR files
Publish your key to a public key server to one of GPG servers

After completing all the required steps you may proceed to a configuration in Maven POM file. You need to include there two sets of configurations. The first of them contains a piece of the necessary information about our project, author, and source code repository.

logstash-logging-spring-boot-starter
Library for HTTP logging with Spring Boot
https://github.com/piomin/spring-boot-logging

   
      Piotr Mińkowski
      piotr.minkowski@gmail.com
      https://github.com/piomin
   


   
      MIT License
      http://www.opensource.org/licenses/mit-license.php
      repo
   


   scm:git:git://github.com/piomin/spring-boot-logging.git
   scm:git:git@github.com:piomin/spring-boot-logging.git
   https://github.com/piomin/spring-boot-logging


   
      ossrh
      https://oss.sonatype.org/content/repositories/snapshots
   
   
      ossrh
      https://oss.sonatype.org/service/local/staging/deploy/maven2/

In the second step, we need to add some Maven plugins for signing JAR file, including source code files and Javadocs there. Here’s a required list of plugins activated only with release Maven profile enabled.

profiles>
   
      release
      
         
            
               org.apache.maven.plugins
               maven-gpg-plugin
               1.6
               
                  
                     sign-artifacts
                     verify
                     
                        sign
                     
                  
               
            
            
               org.apache.maven.plugins
               maven-source-plugin
               3.2.1
               
                  
                     attach-sources
                     
                        jar-no-fork
                     
                  
               
            
            
               org.apache.maven.plugins
               maven-javadoc-plugin
               3.2.0
               
                  
                     attach-javadocs
                     
                        jar

Finally you need to execute command mvn clean deploy -P release, and visit Sonatype site to confirm publication of your library.

6. Promote your Spring Boot library

Congratulations! You have already published your first Spring Boot library. But, the question is what’s next? You probably would like to encourage people to try it. Of course, you can advertise it on social media, or create articles on dev portals. But my first advice is to take care of the presentation site. If you are storing your source code on GitHub prepare a Readme file with a detailed description of your library. It is also worth adding some tags that describe your project.

It is relatively easy to integrate your GitHub repository with some third-party tools used for continuous integration or static source code analysis. Thanks to that you can continuously improve your library. Moreover, you can add some badges to your repository that indicate you are using such tools. In my repositories spring-boot-logging and spring-boot-istio, I have already added badges with Maven release, CircleCI builds status and SonarCloud analysis reports. Looks fine?

Conclusion

In this article, I describe the process of creating a Spring Boot library from the beginning to the end. You can take a look on my libraries https://github.com/piomin/spring-boot-logging and https://github.com/piomin/spring-boot-istio if you are looking for simple examples. Of course, there are many other third-party Spring Boot starters published on GitHub you can also take a look. If you interested in building your own Spring Boot library you should learn more about auto-configuration: A Magic Around Spring Boot Auto Configuration.

The post Guide to building Spring Boot library appeared first on Piotr's TechBlog.