You can read more about Istio, Spring Boot, and other Java and Kubernetes-related technologies in my latest book, Hands-On Java with Kubernetes.

You can also find many other articles on my blog about Istio. For example, this article is about Quarkus and tracing with Istio.

Source Code

Feel free to use my source code if you’d like to try it out yourself. To do that, you must clone my sample GitHub repository. Two sample applications for this exercise are available in the spring-boot-istio directory.

Prerequisites

We will start with the most straightforward Istio installation on a local Kubernetes cluster. This could be Minikube, which you can run with the following command. You can set slightly lower resource limits than I did.

minikube start --memory='8gb' --cpus='6'

minikube start --memory='8gb' --cpus='6'

To complete the exercise below, you need to install istioctl in addition to kubectl. Here you will find the available distributions for the latest versions of kubectl and istioctl. I install them on my laptop using Homebrew.

$ brew install kubectl
$ brew install istioctl

$ brew install kubectl
$ brew install istioctl

To install Istio with default parameters, run the following command:

istioctl install

istioctl install

After a moment, Istio should be running in the istio-system namespace.

It is also worth installing Kiali to verify the Istio resources we have created. Kiali is an observability and management tool for Istio that provides a web-based dashboard for service mesh monitoring. It visualizes service-to-service traffic, validates Istio configuration, and integrates with tools like Prometheus, Grafana, and Jaeger.

kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.28/samples/addons/kiali.yaml

kubectl apply -f https://raw.githubusercontent.com/istio/istio/release-1.28/samples/addons/kiali.yaml

Once Kiali is successfully installed on Kubernetes, we can expose its web dashboard locally with the following istioctl command:

istioctl dashboard kiali

istioctl dashboard kiali

To test access to the Istio mesh from outside the cluster, you need to expose the ingress gateway. To do this, run the minikube tunnel command.

Use Spring Boot Istio Library

To test our library’s functionality, we will create a simple Spring Boot application that exposes a single REST endpoint.

@RestController
@RequestMapping("/callme")
public class CallmeController {

    private static final Logger LOGGER = LoggerFactory
        .getLogger(CallmeController.class);

    @Autowired
    Optional<BuildProperties> buildProperties;
    @Value("${VERSION}")
    private String version;

    @GetMapping("/ping")
    public String ping() {
        LOGGER.info("Ping: name={}, version={}", buildProperties.isPresent() ?
            buildProperties.get().getName() : "callme-service", version);
        return "I'm callme-service " + version;
    }
    
}

@RestController
@RequestMapping("/callme")
public class CallmeController {

    private static final Logger LOGGER = LoggerFactory
        .getLogger(CallmeController.class);

    @Autowired
    Optional<BuildProperties> buildProperties;
    @Value("${VERSION}")
    private String version;

    @GetMapping("/ping")
    public String ping() {
        LOGGER.info("Ping: name={}, version={}", buildProperties.isPresent() ?
            buildProperties.get().getName() : "callme-service", version);
        return "I'm callme-service " + version;
    }
    
}

Then, in addition to the standard Spring Web starter, add the istio-spring-boot-starter dependency.

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
  <groupId>com.github.piomin</groupId>
  <artifactId>istio-spring-boot-starter</artifactId>
  <version>1.2.1</version>
</dependency>

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
  <groupId>com.github.piomin</groupId>
  <artifactId>istio-spring-boot-starter</artifactId>
  <version>1.2.1</version>
</dependency>

Finally, we must add the @EnableIstio annotation to our application’s main class. We can also enable Istio Gateway to expose the REST endpoint outside the cluster. An Istio Gateway is a component that controls how external traffic enters or leaves a service mesh.

@SpringBootApplication
@EnableIstio(enableGateway = true)
public class CallmeApplication {

	public static void main(String[] args) {
		SpringApplication.run(CallmeApplication.class, args);
	}
	
}

@SpringBootApplication
@EnableIstio(enableGateway = true)
public class CallmeApplication {

	public static void main(String[] args) {
		SpringApplication.run(CallmeApplication.class, args);
	}
	
}

Let’s deploy our application on the Kubernetes cluster. To do this, we must first create a role with the necessary permissions to manage Istio resources in the cluster. The role must be assigned to the ServiceAccount used by the application.

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: callme-service-with-starter
rules:
  - apiGroups: ["networking.istio.io"]
    resources: ["virtualservices", "destinationrules", "gateways"]
    verbs: ["create", "get", "list", "watch", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: callme-service-with-starter
subjects:
  - kind: ServiceAccount
    name: callme-service-with-starter
    namespace: spring
roleRef:
  kind: Role
  name: callme-service-with-starter
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: callme-service-with-starter

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: callme-service-with-starter
rules:
  - apiGroups: ["networking.istio.io"]
    resources: ["virtualservices", "destinationrules", "gateways"]
    verbs: ["create", "get", "list", "watch", "update", "patch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: callme-service-with-starter
subjects:
  - kind: ServiceAccount
    name: callme-service-with-starter
    namespace: spring
roleRef:
  kind: Role
  name: callme-service-with-starter
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: callme-service-with-starter

Here are the Deployment and Service resources.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: callme-service-with-starter
spec:
  replicas: 1
  selector:
    matchLabels:
      app: callme-service-with-starter
  template:
    metadata:
      labels:
        app: callme-service-with-starter
    spec:
      containers:
        - name: callme-service-with-starter
          image: piomin/callme-service-with-starter
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
          env:
            - name: VERSION
              value: "v1"
      serviceAccountName: callme-service-with-starter
---
apiVersion: v1
kind: Service
metadata:
  name: callme-service-with-starter
  labels:
    app: callme-service-with-starter
spec:
  type: ClusterIP
  ports:
  - port: 8080
    name: http
  selector:
    app: callme-service-with-starter

apiVersion: apps/v1
kind: Deployment
metadata:
  name: callme-service-with-starter
spec:
  replicas: 1
  selector:
    matchLabels:
      app: callme-service-with-starter
  template:
    metadata:
      labels:
        app: callme-service-with-starter
    spec:
      containers:
        - name: callme-service-with-starter
          image: piomin/callme-service-with-starter
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
          env:
            - name: VERSION
              value: "v1"
      serviceAccountName: callme-service-with-starter
---
apiVersion: v1
kind: Service
metadata:
  name: callme-service-with-starter
  labels:
    app: callme-service-with-starter
spec:
  type: ClusterIP
  ports:
  - port: 8080
    name: http
  selector:
    app: callme-service-with-starter

The application repository is configured to run it with Skaffold. Of course, you can apply YAML manifests to the cluster with kubectl apply. To do this, simply navigate to the callme-service-with-starter/k8s directory and apply the deployment.yaml file. As part of the exercise, we will run our applications in the spring namespace.

skaffold dev -n spring

skaffold dev -n spring

The sample Spring Boot application creates two Istio objects at startup: a VirtualService and a Gateway. We can verify them in the Kiali dashboard.

The default host name generated for the gateway includes the deployment name and the .ext suffix. We can change the suffix name using the domain field in the @EnableIstio annotation. Assuming you run the minikube tunnel command, you can call the service using the Host header with the hostname in the following way:

$ curl http://localhost/callme/ping -H "Host:callme-service-with-starter.ext"
I'm callme-service v1

$ curl http://localhost/callme/ping -H "Host:callme-service-with-starter.ext"
I'm callme-service v1

Additional Capabilities with Spring Boot Istio

The library’s behavior can be customized by modifying the @EnableIstio annotation. For example, you can enable the fault injection mechanism using the fault field. Both abort and delay are possible. You can make this change without redeploying the app. The library updates the existing VirtualService.

@SpringBootApplication
@EnableIstio(enableGateway = true, fault = @Fault(percentage = 50))
public class CallmeApplication {

	public static void main(String[] args) {
		SpringApplication.run(CallmeApplication.class, args);
	}
	
}

@SpringBootApplication
@EnableIstio(enableGateway = true, fault = @Fault(percentage = 50))
public class CallmeApplication {

	public static void main(String[] args) {
		SpringApplication.run(CallmeApplication.class, args);
	}
	
}

Now, you can call the same endpoint through the gateway. There is a 50% chance that you will receive this response.

Now let’s analyze a slightly more complex scenario. Let’s assume that we are running two different versions of the same application on Kubernetes. We use Istio to manage its versioning. Traffic is forwarded to the particular version based on the X-Version header in the incoming request. If the header value is v1 the request is sent to the application Pod with the label version=v1, and similarly, for the version v2. Here’s the annotation for the v1 application main class.

@SpringBootApplication
@EnableIstio(enableGateway = true, version = "v1",
	matches = { 
	  @Match(type = MatchType.HEADERS, key = "X-Version", value = "v1") })
public class CallmeApplication {

	public static void main(String[] args) {
		SpringApplication.run(CallmeApplication.class, args);
	}
	
}

@SpringBootApplication
@EnableIstio(enableGateway = true, version = "v1",
	matches = { 
	  @Match(type = MatchType.HEADERS, key = "X-Version", value = "v1") })
public class CallmeApplication {

	public static void main(String[] args) {
		SpringApplication.run(CallmeApplication.class, args);
	}
	
}

The Deployment manifest for the callme-service-with-starter-v1 should define two labels: app and version.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: callme-service-with-starter-v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: callme-service-with-starter
      version: v1
  template:
    metadata:
      labels:
        app: callme-service-with-starter
        version: v1
    spec:
      containers:
        - name: callme-service-with-starter
          image: piomin/callme-service-with-starter
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
          env:
            - name: VERSION
              value: "v1"
      serviceAccountName: callme-service-with-starter

apiVersion: apps/v1
kind: Deployment
metadata:
  name: callme-service-with-starter-v1
spec:
  replicas: 1
  selector:
    matchLabels:
      app: callme-service-with-starter
      version: v1
  template:
    metadata:
      labels:
        app: callme-service-with-starter
        version: v1
    spec:
      containers:
        - name: callme-service-with-starter
          image: piomin/callme-service-with-starter
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
          env:
            - name: VERSION
              value: "v1"
      serviceAccountName: callme-service-with-starter

Unlike the skaffold dev command, the skaffold run command simply launches the application on the cluster and terminates. Let’s first release version v1, and then move on to version v2.

skaffold run -n spring

skaffold run -n spring

Then, we can deploy the v2 version of our sample Spring Boot application. In this exercise, it is just an “artificial” version, since we deploy the same source code, but with different labels and environment variables injected in the Deployment manifest.

@SpringBootApplication
@EnableIstio(enableGateway = true, version = "v2",
	matches = { 
	  @Match(type = MatchType.HEADERS, key = "X-Version", value = "v2") })
public class CallmeApplication {

	public static void main(String[] args) {
		SpringApplication.run(CallmeApplication.class, args);
	}
	
}

@SpringBootApplication
@EnableIstio(enableGateway = true, version = "v2",
	matches = { 
	  @Match(type = MatchType.HEADERS, key = "X-Version", value = "v2") })
public class CallmeApplication {

	public static void main(String[] args) {
		SpringApplication.run(CallmeApplication.class, args);
	}
	
}

Here’s the callme-service-with-starter-v2 Deployment manifest.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: callme-service-with-starter-v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: callme-service-with-starter
      version: v2
  template:
    metadata:
      labels:
        app: callme-service-with-starter
        version: v2
    spec:
      containers:
        - name: callme-service-with-starter
          image: piomin/callme-service-with-starter
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
          env:
            - name: VERSION
              value: "v2"
      serviceAccountName: callme-service-with-starter

apiVersion: apps/v1
kind: Deployment
metadata:
  name: callme-service-with-starter-v2
spec:
  replicas: 1
  selector:
    matchLabels:
      app: callme-service-with-starter
      version: v2
  template:
    metadata:
      labels:
        app: callme-service-with-starter
        version: v2
    spec:
      containers:
        - name: callme-service-with-starter
          image: piomin/callme-service-with-starter
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8080
          env:
            - name: VERSION
              value: "v2"
      serviceAccountName: callme-service-with-starter

Service, on the other hand, remains unchanged. It still refers to Pods labeled with app=callme-service-with-starter. However, this time it includes both application instances marked as v1 or v2.

apiVersion: v1
kind: Service
metadata:
  name: callme-service-with-starter
  labels:
    app: callme-service-with-starter
spec:
  type: ClusterIP
  ports:
  - port: 8080
    name: http
  selector:
    app: callme-service-with-starter

apiVersion: v1
kind: Service
metadata:
  name: callme-service-with-starter
  labels:
    app: callme-service-with-starter
spec:
  type: ClusterIP
  ports:
  - port: 8080
    name: http
  selector:
    app: callme-service-with-starter

Version v2 should be run in the same way as before, using the skaffold run command. There are three Istio objects generated during apps startup: Gateway, VirtualService and DestinationRule.

A generated DestinationRule contains two subsets for both v1 and v2 versions.

The automatically generated VirtualService looks as follows.

kind: VirtualService
apiVersion: networking.istio.io/v1
metadata:
  name: callme-service-with-starter-route
spec:
  hosts:
  - callme-service-with-starter
  - callme-service-with-starter.ext
  gateways:
  - callme-service-with-starter
  http:
  - match:
    - headers:
        X-Version:
          prefix: v1
    route:
    - destination:
        host: callme-service-with-starter
        subset: v1
    timeout: 6s
    retries:
      attempts: 3
      perTryTimeout: 2s
      retryOn: 5xx
  - match:
    - headers:
        X-Version:
          prefix: v2
    route:
    - destination:
        host: callme-service-with-starter
        subset: v2
      weight: 100
    timeout: 6s
    retries:
      attempts: 3
      perTryTimeout: 2s
      retryOn: 5xx

kind: VirtualService
apiVersion: networking.istio.io/v1
metadata:
  name: callme-service-with-starter-route
spec:
  hosts:
  - callme-service-with-starter
  - callme-service-with-starter.ext
  gateways:
  - callme-service-with-starter
  http:
  - match:
    - headers:
        X-Version:
          prefix: v1
    route:
    - destination:
        host: callme-service-with-starter
        subset: v1
    timeout: 6s
    retries:
      attempts: 3
      perTryTimeout: 2s
      retryOn: 5xx
  - match:
    - headers:
        X-Version:
          prefix: v2
    route:
    - destination:
        host: callme-service-with-starter
        subset: v2
      weight: 100
    timeout: 6s
    retries:
      attempts: 3
      perTryTimeout: 2s
      retryOn: 5xx

To test the versioning mechanism generated using the Spring Boot Istio library, set the X-Version header for each call.

$ curl http://localhost/callme/ping -H "Host:callme-service-with-starter.ext" -H "X-Version:v1"
I'm callme-service v1

$ curl http://localhost/callme/ping -H "Host:callme-service-with-starter.ext" -H "X-Version:v2"
I'm callme-service v2

$ curl http://localhost/callme/ping -H "Host:callme-service-with-starter.ext" -H "X-Version:v1"
I'm callme-service v1

$ curl http://localhost/callme/ping -H "Host:callme-service-with-starter.ext" -H "X-Version:v2"
I'm callme-service v2

Conclusion

I am still working on this library, and new features will be added in the near future. I hope it will be helpful for those of you who want to get started with Istio without getting into the details of its configuration.

The post Istio Spring Boot Library Released appeared first on Piotr's TechBlog.

The other potential solution in that context is the Vertical Pod Autoscaler. In the latest version, it supports in-place pod resize. Vertical Pod Autoscaler (VPA) in Kubernetes automatically adjusts CPU and memory requests/limits for pods based on their actual usage, ensuring containers receive the appropriate resources. Unlike the Horizontal Pod Autoscaler (HPA), which scales resources, not replicas, it may restart pods to apply changes. For now, VPA does not support this use case, but once this feature is implemented, the situation will change.

On the other hand, Kube Startup CPU Boost is a dedicated feature for scenarios with high CPU requirements during app startup. It is a controller that increases CPU resource requests and limits during Kubernetes workload startup. Once the workload is up and running, the resources are set back to their original values. Let’s see how this solution works in practice!

If you’re looking to go deeper into the practical side of running Java applications on Kubernetes, I explore these topics in much more detail in my book, Hands-On Java with Kubernetes.

Source Code

Feel free to use my source code if you’d like to try it out yourself. To do that, you must clone my sample GitHub repository. The sample application is based on Spring Boot and exposes several REST endpoints. However, in this exercise, we will use a ready-made image published on my Quay: quay.io/pminkows/sample-kotlin-spring:1.5.1.1.

Install Kube Startup CPU Boost

The Kubernetes cluster you are using must enable the In-Place Pod Resize feature. The activation method may vary by Kubernetes distribution. For the Minikube I am using in today’s example, it looks like this:

minikube start --memory='8gb' --cpus='6' --feature-gates=InPlacePodVerticalScaling=true

minikube start --memory='8gb' --cpus='6' --feature-gates=InPlacePodVerticalScaling=true

After that, we can proceed with installing the Kube Startup CPU Boost controller. There are several ways to achieve it. The easiest way to do this is with a Helm chart. Let’s add the following Helm repository:

helm repo add kube-startup-cpu-boost https://google.github.io/kube-startup-cpu-boost

helm repo add kube-startup-cpu-boost https://google.github.io/kube-startup-cpu-boost

Then, we can install the kube-startup-cpu-boost chart in the dedicated kube-startup-cpu-boost-system namespace using the following command:

helm install -n kube-startup-cpu-boost-system kube-startup-cpu-boost \
  kube-startup-cpu-boost/kube-startup-cpu-boost --create-namespace

helm install -n kube-startup-cpu-boost-system kube-startup-cpu-boost \
  kube-startup-cpu-boost/kube-startup-cpu-boost --create-namespace

If the installation was successful, you should see the following pod running in the kube-startup-cpu-boost-system namespace as below.

$ kubectl get pod -n kube-startup-cpu-boost-system
NAME                                                         READY   STATUS    RESTARTS   AGE
kube-startup-cpu-boost-controller-manager-75f95d5fb6-692s6   1/1     Running   0          36s

$ kubectl get pod -n kube-startup-cpu-boost-system
NAME                                                         READY   STATUS    RESTARTS   AGE
kube-startup-cpu-boost-controller-manager-75f95d5fb6-692s6   1/1     Running   0          36s

Install Monitoring Stack (optional)

Then, we can install Prometheus monitoring. It is an optional step to verify pod resource usage in the graphical form. Firstly, let’s install the following Helm repository:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts

After that, we can install the latest version of the kube-prometheus-stack chart in the monitoring namespace.

helm install my-kube-prometheus-stack prometheus-community/kube-prometheus-stack \
  -n monitoring --create-namespace

helm install my-kube-prometheus-stack prometheus-community/kube-prometheus-stack \
  -n monitoring --create-namespace

Let’s verify the installation succeeded by listing the pods running in the monitoring namespace.

$ kubectl get pods -n monitoring
NAME                                                         READY   STATUS    RESTARTS   AGE
alertmanager-my-kube-prometheus-stack-alertmanager-0         2/2     Running   0          38s
my-kube-prometheus-stack-grafana-f8bb6b8b8-mzt4l             3/3     Running   0          48s
my-kube-prometheus-stack-kube-state-metrics-99f4574c-bf5ln   1/1     Running   0          48s
my-kube-prometheus-stack-operator-6d58dd9d6c-6srtg           1/1     Running   0          48s
my-kube-prometheus-stack-prometheus-node-exporter-tdwmr      1/1     Running   0          48s
prometheus-my-kube-prometheus-stack-prometheus-0             2/2     Running   0          38s

$ kubectl get pods -n monitoring
NAME                                                         READY   STATUS    RESTARTS   AGE
alertmanager-my-kube-prometheus-stack-alertmanager-0         2/2     Running   0          38s
my-kube-prometheus-stack-grafana-f8bb6b8b8-mzt4l             3/3     Running   0          48s
my-kube-prometheus-stack-kube-state-metrics-99f4574c-bf5ln   1/1     Running   0          48s
my-kube-prometheus-stack-operator-6d58dd9d6c-6srtg           1/1     Running   0          48s
my-kube-prometheus-stack-prometheus-node-exporter-tdwmr      1/1     Running   0          48s
prometheus-my-kube-prometheus-stack-prometheus-0             2/2     Running   0          38s

Finally, we can expose the Prometheus console over localhost using the port forwarding feature:

kubectl port-forward svc/my-kube-prometheus-stack-prometheus 9090:9090 -n monitoring

kubectl port-forward svc/my-kube-prometheus-stack-prometheus 9090:9090 -n monitoring

Configure Kube Startup CPU Boost

The Kube Startup CPU Boost configuration is pretty intuitive. We need to create a StartupCPUBoost resource. It can manage multiple applications based on a given selector. In our case, it is a single sample-kotlin-spring Deployment determined by the app.kubernetes.io/name label (1). The next step is to define the resource management policy (2). The Kube Startup CPU Boost increases both request and limit by 50%. Resources should only be increased for the duration of the startup (3). Therefore, once the readiness probe succeeds, the resource level will return to its initial state. Of course, everything happens in-place without restarting the container.

apiVersion: autoscaling.x-k8s.io/v1alpha1
kind: StartupCPUBoost
metadata:
  name: sample-kotlin-spring
  namespace: demo
selector:
  matchExpressions: # (1)
  - key: app.kubernetes.io/name
    operator: In
    values: ["sample-kotlin-spring"]
spec:
  resourcePolicy: # (2)
    containerPolicies:
    - containerName: sample-kotlin-spring
      percentageIncrease:
        value: 50
  durationPolicy: # (3)
    podCondition:
      type: Ready
      status: "True"

apiVersion: autoscaling.x-k8s.io/v1alpha1
kind: StartupCPUBoost
metadata:
  name: sample-kotlin-spring
  namespace: demo
selector:
  matchExpressions: # (1)
  - key: app.kubernetes.io/name
    operator: In
    values: ["sample-kotlin-spring"]
spec:
  resourcePolicy: # (2)
    containerPolicies:
    - containerName: sample-kotlin-spring
      percentageIncrease:
        value: 50
  durationPolicy: # (3)
    podCondition:
      type: Ready
      status: "True"

Next, we will deploy our sample application. Here’s the Deployment manifest of our Spring Boot app. The name of the app container is sample-kotlin-spring, which matches the target Deployment name defined inside the StartupCPUBoost object (1). Then, we set the CPU limit to 500 millicores (2). There’s also a new field resizePolicy. It tells Kubernetes whether a change to CPU or memory can be applied in-place or requires a Pod restart. (3). The NotRequired value means that changing the resource limit or request will not trigger a pod restart. The Deployment object also contains a readiness probe that calls the GET/actuator/health/readiness exposed with the Spring Boot Actuator (4).

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-kotlin-spring
  namespace: demo
  labels:
    app: sample-kotlin-spring
    app.kubernetes.io/name: sample-kotlin-spring
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sample-kotlin-spring
  template:
    metadata:
      labels:
        app: sample-kotlin-spring
        app.kubernetes.io/name: sample-kotlin-spring
    spec:
      containers:
      - name: sample-kotlin-spring # (1)
        image: quay.io/pminkows/sample-kotlin-spring:1.5.1.1
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 500m # (2)
            memory: "1Gi"
          requests:
            cpu: 200m
            memory: "256Mi"
        resizePolicy: # (3)
        - resourceName: "cpu"
          restartPolicy: "NotRequired"
        readinessProbe: # (4)
          httpGet:
            path: /actuator/health/readiness
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 15
          periodSeconds: 5
          successThreshold: 1
          failureThreshold: 3

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-kotlin-spring
  namespace: demo
  labels:
    app: sample-kotlin-spring
    app.kubernetes.io/name: sample-kotlin-spring
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sample-kotlin-spring
  template:
    metadata:
      labels:
        app: sample-kotlin-spring
        app.kubernetes.io/name: sample-kotlin-spring
    spec:
      containers:
      - name: sample-kotlin-spring # (1)
        image: quay.io/pminkows/sample-kotlin-spring:1.5.1.1
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 500m # (2)
            memory: "1Gi"
          requests:
            cpu: 200m
            memory: "256Mi"
        resizePolicy: # (3)
        - resourceName: "cpu"
          restartPolicy: "NotRequired"
        readinessProbe: # (4)
          httpGet:
            path: /actuator/health/readiness
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 15
          periodSeconds: 5
          successThreshold: 1
          failureThreshold: 3

Here are the pod requests and limits configured by Kube Startup CPU Boost. As you can see, the request is set to 300m, while the limit is completely removed.

Once the application startup process completes, Kube Startup CPU Boost restores the initial request and limit.

Now we can switch to the Prometheus console to see the history of CPU request values for our pod. As you can see, the request was temporarily increased during the pod startup.

The chart below illustrates CPU usage when the application is launched and then during normal operation.

We can also define the fixed resources for a target container. The CPU requests and limits of the selected container will be set to the given values (1). If you do not want the operator to remove the CPU limit during boost time, set the REMOVE_LIMITS environment variable to false in the kube-startup-cpu-boost-controller-manager Deployment.

apiVersion: autoscaling.x-k8s.io/v1alpha1
kind: StartupCPUBoost
metadata:
  name: sample-kotlin-spring
  namespace: demo
selector:
  matchExpressions:
  - key: app.kubernetes.io/name
    operator: In
    values: ["sample-kotlin-spring"]
spec:
  resourcePolicy:
    containerPolicies: # (1)
    - containerName: sample-kotlin-spring
      fixedResources:
        requests: "500m"
        limits: "2"
  durationPolicy:
    podCondition:
      type: Ready
      status: "True"

apiVersion: autoscaling.x-k8s.io/v1alpha1
kind: StartupCPUBoost
metadata:
  name: sample-kotlin-spring
  namespace: demo
selector:
  matchExpressions:
  - key: app.kubernetes.io/name
    operator: In
    values: ["sample-kotlin-spring"]
spec:
  resourcePolicy:
    containerPolicies: # (1)
    - containerName: sample-kotlin-spring
      fixedResources:
        requests: "500m"
        limits: "2"
  durationPolicy:
    podCondition:
      type: Ready
      status: "True"

Conclusion

There are many ways to address application CPU demand during startup. First, you don’t need to set a CPU limit for Deployment. What’s more, many people believe that setting a CPU limit doesn’t make sense, but for different reasons. In this situation, the request issue remains, but given the short timeframe and the significantly higher usage than in the declaration, it isn’t material.

Other solutions are related to strictly Java features. If we compile the application natively with GraalVM or use the CRaC feature, we will significantly speed up startup and reduce CPU requirements.

Finally, several solutions rely on in-place resizing. If you use Kyverno, consider its mutate policy, which can modify resources in response to an application startup event. The Kube Startup CPU Boost tool described in this article operates similarly but is designed exclusively for this use case. In the near future, Vertical Pod Autoscaler will also offer a CPU boost via in-place resize.

The post Startup CPU Boost in Kubernetes with In-Place Pod Resize appeared first on Piotr's TechBlog.

Here is a brief overview of all my published books.

Motivation

I won’t hide that this post is mainly directed at my blog subscribers and people who enjoy reading it and value my writing style. As you know, all posts and content on my blog, along with sample application repositories on GitHub, are always accessible to you for free. Over the past eight years, I have worked to publish high-quality content on my blog, and I plan to keep doing so. It is a part of my life, a significant time commitment, but also a lot of fun and a hobby.

I want to explain why I decided to write this book, why now, and why in this way. But first, a bit of background. I wrote my last book first, then my first book, over seven years ago. It focused on topics I was mainly involved with at the time, specifically Spring Boot and Spring Cloud. Since then, a lot of time has passed, and much has changed – not only in the technology itself but also a little in my personal life. Today, I am more involved in Kubernetes and container topics than, for example, Spring Cloud. For years, I have been helping various organizations transition from traditional application architectures to cloud-native models based on Kubernetes. Of course, Java remains my main area of expertise. Besides Spring Boot, I also really like the Quarkus framework. You can read a lot about both in my book on Kubernetes.

Based on my experience over the past few years, involving development teams is a key factor in the success of the Kubernetes platform within an organization. Ultimately, it is the applications developed by these teams that are deployed there. For developers to be willing to use Kubernetes, it must be easy for them to do so. That is why I persuade organizations to remove barriers to using Kubernetes and to design it in a way that makes it easier for development teams. On my blog and in this book, I aim to demonstrate how to quickly and simply launch applications on Kubernetes using frameworks such as Spring Boot and Quarkus.

It’s an unusual time to publish a book. AI agents are producing more and more technical content online. More often than not, instead of grabbing a book, people turn to an AI chatbot for a quick answer, though not always the best one. Still, a book that thoroughly introduces a topic and offers a step-by-step guide remains highly valuable.

Content of the Book

This book demonstrates that Java is an excellent choice for building applications that run on Kubernetes. In the first chapter, I’ll show you how to quickly build your application, create its image, and run it on Kubernetes without writing a single line of YAML or Dockerfile. This chapter also covers the minimum Kubernetes architecture you must understand to manage applications effectively in this environment. The second chapter, on the other hand, demonstrates how to effectively organize your local development environment to work with a Kubernetes cluster. You’ll see several options for running a distribution of your cluster locally and learn about the essential set of tools you should have. The third chapter outlines best practices for building applications on the Kubernetes platform. Most of the presented requirements are supported by simple examples and explanations of the benefits of meeting them. The fourth chapter presents the most valuable tools for the inner development loop with Kubernetes. After reading the first four chapters, you will understand the main Kubernetes components related to application management, enabling you to navigate the platform efficiently. You’ll also learn to leverage Spring Boot and Quarkus features to adapt your application to Kubernetes requirements.

In the following chapters, I will focus on the benefits of migrating applications to Kubernetes. The first area to cover is security. Chapter five discusses mechanisms and tools for securing applications running in a cluster. Chapter six describes Spring and Quarkus projects that enable native integration with the Kubernetes API from within applications. In chapter seven, you’ll learn about the service mesh tool and the benefits of using it to manage HTTP traffic between microservices. Chapter eight addresses the performance and scalability of Java applications in a Kubernetes environment. Chapter Eight demonstrates how to design a CI/CD process that runs entirely within the cluster, leveraging Kubernetes-native tools for pipeline building and the GitOps approach. This book also covers AI. In the final, ninth chapter, you’ll learn how to run a simple Java application that integrates with an AI model deployed on Kubernetes.

Publication

I decided to publish my book on Leanpub. Leanpub is a platform for writing, publishing, and selling books, especially popular among technical content authors. I previously published a book with Packt, but honestly, I was alone during the writing process. Leanpub is similar but offers several key advantages over publishers like Packt. First, it allows you to update content collaboratively with readers and keep it current. Even though my book is finished, I don’t rule out adding more chapters, such as on AI on Kubernetes. I also look forward to your feedback and plan to improve the content and examples in the repository continuously. Overall, this has been another exciting experience related to publishing technical content.

And when you buy such a book, you can be sure that most of the royalties go to me as the author, unlike with other publishers, where most of the royalties go to them as promoters. So, I’m looking forward to improving my book with you!

Conclusion

My book aims to bring together all the most interesting elements surrounding Java application development on Kubernetes. It is intended not only for developers but also for architects and DevOps teams who want to move to the Kubernetes platform.

The post A Book: Hands-On Java with Kubernetes appeared first on Piotr's TechBlog.

Source Code

Feel free to use my source code if you’d like to try it out yourself. To do that, you must clone my sample GitHub repository. Then you should only follow my instructions.

Quarkus Buildpacks Extension

Recently, support for Cloud Native Buildpacks in Quarkus has been significantly enhanced. Here you can access the repository containing the source code for the Paketo Quarkus buildpack. To implement this solution, add one dependency to your application.

<dependency>
  <groupId>io.quarkus</groupId>
  <artifactId>quarkus-container-image-buildpack</artifactId>
</dependency>

<dependency>
  <groupId>io.quarkus</groupId>
  <artifactId>quarkus-container-image-buildpack</artifactId>
</dependency>

Next, run the build command with Maven and activate the quarkus.container-image.build parameter. Also, set the appropriate Java version needed for your application. For the sample Quarkus application in this article, the Java version is 21.

mvn clean package \
  -Dquarkus.container-image.build=true \
  -Dquarkus.buildpack.builder-env.BP_JVM_VERSION=21

mvn clean package \
  -Dquarkus.container-image.build=true \
  -Dquarkus.buildpack.builder-env.BP_JVM_VERSION=21

To build, you need Docker or Podman running. Here’s the output from the command run earlier.

As you can see, Quarkus uses, among other buildpacks, the buildpack as mentioned earlier.

The new image is now available for use.

$ docker images sample-quarkus/person-service:1.0.0-SNAPSHOT
REPOSITORY                      TAG              IMAGE ID       CREATED        SIZE
sample-quarkus/person-service   1.0.0-SNAPSHOT   e0b58781e040   45 years ago   160MB

$ docker images sample-quarkus/person-service:1.0.0-SNAPSHOT
REPOSITORY                      TAG              IMAGE ID       CREATED        SIZE
sample-quarkus/person-service   1.0.0-SNAPSHOT   e0b58781e040   45 years ago   160MB

Quarkus with OpenShift Builds Shipwright

Install the Openshift Build Operator

Now, we will move the image building process to the OpenShift cluster. OpenShift offers built-in support for creating container images directly within the cluster through OpenShift Builds, using the BuildConfig solution. For more details, please refer to my previous article. However, in this article, we explore a new technology for building container images called OpenShift Builds with Shipwright. To enable this solution on OpenShift, you need to install the following operator.

After installing this operator, you will see a new item in the “Build” menu called “Shiwright”. Switch to it, then select the “ClusterBuildStrategies” tab. There are two strategies on the list designed for Cloud Native Buildpacks. We are interested in the buildpacks strategy.

Create and Run Build with Shipwright

Finally, we can create the Shiwright Build object. It contains three sections. In the first step, we define the address of the container image repository where we will push our output image. For simplicity, we will use the internal registry provided by the OpenShift cluster itself. In the source section, we specify the repository address where the application source code is located. In the last section, we need to set the build strategy. We chose the previously mentioned buildpacks strategy for Cloud Native Buildpacks. Some parameters need to be set for the buildpacks strategy: run-image and cnb-builder-image. The cnb-builder-image indicates the name of the builder image containing the buildpacks. The run-image refers to a base image used to run the application. We will also activate the buildpacks Maven profile during the build to set the Quarkus property that switches from fast-jar to uber-jar packaging.

apiVersion: shipwright.io/v1beta1
kind: Build
metadata:
  name: buildpack-quarkus-build
spec:
  env:
    - name: BP_JVM_VERSION
      value: '21'
  output:
    image: 'image-registry.openshift-image-registry.svc:5000/builds/sample-quarkus-microservice:1.0'
  paramValues:
    - name: run-image
      value: 'paketobuildpacks/run-java-21-ubi9-base:latest'
    - name: cnb-builder-image
      value: 'paketobuildpacks/builder-jammy-java-tiny:latest'
    - name: env-vars
      values:
        - value: BP_MAVEN_ADDITIONAL_BUILD_ARGUMENTS=-Pbuildpacks
  retention:
    atBuildDeletion: true
  source:
    git:
      url: 'https://github.com/piomin/sample-quarkus-microservice.git'
    type: Git
  strategy:
    kind: ClusterBuildStrategy
    name: buildpacks

apiVersion: shipwright.io/v1beta1
kind: Build
metadata:
  name: buildpack-quarkus-build
spec:
  env:
    - name: BP_JVM_VERSION
      value: '21'
  output:
    image: 'image-registry.openshift-image-registry.svc:5000/builds/sample-quarkus-microservice:1.0'
  paramValues:
    - name: run-image
      value: 'paketobuildpacks/run-java-21-ubi9-base:latest'
    - name: cnb-builder-image
      value: 'paketobuildpacks/builder-jammy-java-tiny:latest'
    - name: env-vars
      values:
        - value: BP_MAVEN_ADDITIONAL_BUILD_ARGUMENTS=-Pbuildpacks
  retention:
    atBuildDeletion: true
  source:
    git:
      url: 'https://github.com/piomin/sample-quarkus-microservice.git'
    type: Git
  strategy:
    kind: ClusterBuildStrategy
    name: buildpacks

Here’s the Maven buildpacks profile that sets a single Quarkus property quarkus.package.jar.type. We must change it to uber-jar, because the paketobuildpacks/builder-jammy-java-tiny builder expects a single jar instead of the multi-folder layout used by the default fast-jar format. Of course, I would prefer to use the paketocommunity/builder-ubi-base builder, which can recognize the fast-jar format. However, at this time, it does not function correctly with OpenShift Builds.

<profiles>
  <profile>
    <id>buildpacks</id>
    <activation>
      <property>
        <name>buildpacks</name>
      </property>
    </activation>
    <properties>
      <quarkus.package.jar.type>uber-jar</quarkus.package.jar.type>
    </properties>
  </profile>
</profiles>

<profiles>
  <profile>
    <id>buildpacks</id>
    <activation>
      <property>
        <name>buildpacks</name>
      </property>
    </activation>
    <properties>
      <quarkus.package.jar.type>uber-jar</quarkus.package.jar.type>
    </properties>
  </profile>
</profiles>

To start the build, you can use the OpenShift console or execute the following command:

shp build run buildpack-quarkus-build --follow

shp build run buildpack-quarkus-build --follow

We can switch to the OpenShift Console. As you can see, our build is running.

The history of such builds is available on OpenShift. You can also review the build logs.

Finally, you should see your image in the list of OpenShift internal image streams.

$ oc get imagestream
NAME                          IMAGE REPOSITORY                                                                                                    TAGS                UPDATED
sample-quarkus-microservice   default-route-openshift-image-registry.apps.pminkows.95az.p1.openshiftapps.com/builds/sample-quarkus-microservice   1.2,0.0.1,1.1,1.0   13 hours ago

$ oc get imagestream
NAME                          IMAGE REPOSITORY                                                                                                    TAGS                UPDATED
sample-quarkus-microservice   default-route-openshift-image-registry.apps.pminkows.95az.p1.openshiftapps.com/builds/sample-quarkus-microservice   1.2,0.0.1,1.1,1.0   13 hours ago

Conclusion

OpenShift Build Shipwright lets you perform the entire application image build process on the OpenShift cluster in a standardized manner. Cloud Native Buildpacks is a popular mechanism for building images without writing a Dockerfile yourself. In this case, support for Buildpacks on the OpenShift side is an interesting alternative to the Source-to-Image approach.

The post Quarkus with Buildpacks and OpenShift Builds appeared first on Piotr's TechBlog.

If you are interested in developing applications for OpenShift, you may also want to read my article on deploying Java applications using the odo tool.

Why Source-to-Image?

That’s probably the first question that comes to mind. Let’s start with a brief definition. Source-to-Image (S2I) is a framework and tool that enables you to write images using the application’s source code as input, producing a new image. In other words, it provides a clean, repeatable, and developer-friendly way to build container images directly from source code – especially in OpenShift, where it’s a core built-in mechanism. With S2I, there is no need to create Dockerfiles, and you can trust that the images will be built to run seamlessly on OpenShift without any issues.

Source Code

Feel free to use my source code if you’d like to try it out yourself. To do that, you must clone my sample GitHub repository. Then you should only follow my instructions.

Prerequisite – OpenShift cluster

There are several ways in which you can run an OpenShift cluster. I’m using a cluster that runs in AWS. But you can run it locally using OpenShift Local. This article describes how to install it on your laptop. You can also take advantage of the 30-day free Developer Sandbox service. However, it is worth mentioning that its use requires creating an account with Red Hat. To provision an OpenShift cluster in the developer sandbox, go here. You can also download and install Podman Desktop, which will help you set up both OpenShift Local and connect to the Developer Sandbox. Generally speaking, there are many possibilities. I assume you simply have an OpenShift cluster at your disposal.

Create a .NET application

I have created a slightly more complex application in terms of its modules. It consists of two main projects and two projects with unit tests. The WebApi.Library project is simply a module to be included in the main application, which is WebApi.App. Below is the directory structure of our sample repository.

.
├── README.md
├── WebApi.sln
├── src
│   ├── WebApi.App
│   │   ├── Controllers
│   │   │   └── VersionController.cs
│   │   ├── Program.cs
│   │   ├── Startup.cs
│   │   ├── WebApi.App.csproj
│   │   └── appsettings.json
│   └── WebApi.Library
│       ├── VersionService.cs
│       └── WebApi.Library.csproj
└── tests
    ├── WebApi.App.Tests
    │   ├── VersionControllerTests.cs
    │   └── WebApi.App.Tests.csproj
    └── WebApi.Library.Tests
        ├── VersionServiceTests.cs
        └── WebApi.Library.Tests.csproj

.
├── README.md
├── WebApi.sln
├── src
│   ├── WebApi.App
│   │   ├── Controllers
│   │   │   └── VersionController.cs
│   │   ├── Program.cs
│   │   ├── Startup.cs
│   │   ├── WebApi.App.csproj
│   │   └── appsettings.json
│   └── WebApi.Library
│       ├── VersionService.cs
│       └── WebApi.Library.csproj
└── tests
    ├── WebApi.App.Tests
    │   ├── VersionControllerTests.cs
    │   └── WebApi.App.Tests.csproj
    └── WebApi.Library.Tests
        ├── VersionServiceTests.cs
        └── WebApi.Library.Tests.csproj

Both the library and the application are elementary in nature. The library provides a single method in the VersionService class to return its version read from the .csproj file.

using System.Reflection;

namespace WebApi.Library;

public class VersionService
{
    private readonly Assembly _assembly;

    public VersionService(Assembly? assembly = null)
    {
        _assembly = assembly ?? Assembly.GetExecutingAssembly();
    }

    public string? GetVersion()
    {
        var informationalVersion = _assembly
            .GetCustomAttribute<AssemblyInformationalVersionAttribute>()?
            .InformationalVersion;

        return informationalVersion ?? _assembly.GetName().Version?.ToString();
    }
}

using System.Reflection;

namespace WebApi.Library;

public class VersionService
{
    private readonly Assembly _assembly;

    public VersionService(Assembly? assembly = null)
    {
        _assembly = assembly ?? Assembly.GetExecutingAssembly();
    }

    public string? GetVersion()
    {
        var informationalVersion = _assembly
            .GetCustomAttribute<AssemblyInformationalVersionAttribute>()?
            .InformationalVersion;

        return informationalVersion ?? _assembly.GetName().Version?.ToString();
    }
}

The application includes the library and uses its VersionService class to read and return the library version in the GET /api/version endpoint. There is no story behind it.

using Microsoft.AspNetCore.Mvc;
using WebApi.Library;

namespace WebApi.App.Controllers
{
    [ApiController]
    [Route("api/version")]
    public class VersionController : ControllerBase
    {
        private readonly VersionService _versionService;
        private readonly ILogger<VersionController> _logger;

        public VersionController(ILogger<VersionController> logger)
        {
            _versionService = new VersionService();
            _logger = logger;
        }

        [HttpGet]
        public IActionResult GetVersion()
        {
            _logger.LogInformation("GetVersion");
            var version = _versionService.GetVersion();
            return Ok(new { version });
        }
    }
}

using Microsoft.AspNetCore.Mvc;
using WebApi.Library;

namespace WebApi.App.Controllers
{
    [ApiController]
    [Route("api/version")]
    public class VersionController : ControllerBase
    {
        private readonly VersionService _versionService;
        private readonly ILogger<VersionController> _logger;

        public VersionController(ILogger<VersionController> logger)
        {
            _versionService = new VersionService();
            _logger = logger;
        }

        [HttpGet]
        public IActionResult GetVersion()
        {
            _logger.LogInformation("GetVersion");
            var version = _versionService.GetVersion();
            return Ok(new { version });
        }
    }
}

The application itself utilizes several other libraries, including those for generating Swagger API documentation, Prometheus metrics, and Kubernetes health checks.

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.OpenApi.Models;
using HealthChecks.UI.Client;
using Microsoft.AspNetCore.Diagnostics.HealthChecks;
using Prometheus;
using System;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Diagnostics.HealthChecks;

namespace WebApi.App
{
    public class Startup
    {
        private readonly IConfiguration _configuration;

        public Startup(IConfiguration configuration)
        {
            _configuration = configuration;
        }

        public void ConfigureServices(IServiceCollection services)
        {
            // Enhanced Health Checks
            services.AddHealthChecks()
                .AddCheck("memory", () =>
                    HealthCheckResult.Healthy("Memory usage is normal"),
                    tags: new[] { "live" });

            services.AddControllers();

            services.AddSwaggerGen(c =>
            {
                c.SwaggerDoc("v1", new OpenApiInfo {Title = "WebApi.App", Version = "v1"});
            });
        }

        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            // Enable prometheus metrics
            app.UseMetricServer();
            app.UseHttpMetrics();

            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }

            app.UseSwagger();
            app.UseSwaggerUI(c => c.SwaggerEndpoint("/swagger/v1/swagger.json", "person-service v1"));

            // Kubernetes probes
            app.UseHealthChecks("/health/live", new HealthCheckOptions
            {
                Predicate = reg => reg.Tags.Contains("live"),
                ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
            });

            app.UseHealthChecks("/health/ready", new HealthCheckOptions
            {
                Predicate = reg => reg.Tags.Contains("ready"),
                ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
            });

            app.UseRouting();

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllers();
                endpoints.MapMetrics();
            });

            using var scope = app.ApplicationServices.CreateScope();
        }
    }
}

using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.OpenApi.Models;
using HealthChecks.UI.Client;
using Microsoft.AspNetCore.Diagnostics.HealthChecks;
using Prometheus;
using System;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Diagnostics.HealthChecks;

namespace WebApi.App
{
    public class Startup
    {
        private readonly IConfiguration _configuration;

        public Startup(IConfiguration configuration)
        {
            _configuration = configuration;
        }

        public void ConfigureServices(IServiceCollection services)
        {
            // Enhanced Health Checks
            services.AddHealthChecks()
                .AddCheck("memory", () =>
                    HealthCheckResult.Healthy("Memory usage is normal"),
                    tags: new[] { "live" });

            services.AddControllers();

            services.AddSwaggerGen(c =>
            {
                c.SwaggerDoc("v1", new OpenApiInfo {Title = "WebApi.App", Version = "v1"});
            });
        }

        public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            // Enable prometheus metrics
            app.UseMetricServer();
            app.UseHttpMetrics();

            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
            }

            app.UseSwagger();
            app.UseSwaggerUI(c => c.SwaggerEndpoint("/swagger/v1/swagger.json", "person-service v1"));

            // Kubernetes probes
            app.UseHealthChecks("/health/live", new HealthCheckOptions
            {
                Predicate = reg => reg.Tags.Contains("live"),
                ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
            });

            app.UseHealthChecks("/health/ready", new HealthCheckOptions
            {
                Predicate = reg => reg.Tags.Contains("ready"),
                ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
            });

            app.UseRouting();

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllers();
                endpoints.MapMetrics();
            });

            using var scope = app.ApplicationServices.CreateScope();
        }
    }
}

As you can see, the WebApi.Library project is included as an internal module, while other dependencies are simply added from the external NuGet repository.

<Project Sdk="Microsoft.NET.Sdk.Web">

  <ItemGroup>
    <ProjectReference Include="..\WebApi.Library\WebApi.Library.csproj" />
    <PackageReference Include="Swashbuckle.AspNetCore" Version="9.0.4" />
    <PackageReference Include="prometheus-net.AspNetCore" Version="8.2.1" />
    <PackageReference Include="AspNetCore.HealthChecks.NpgSql" Version="9.0.0" />
    <PackageReference Include="AspNetCore.HealthChecks.UI.Client" Version="9.0.0" />
    <PackageReference Include="Microsoft.Extensions.Diagnostics.HealthChecks.EntityFrameworkCore" Version="9.0.8" />
  </ItemGroup>

  <PropertyGroup>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <Version>1.0.3</Version>
    <IsPackable>true</IsPackable>
    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
    <PackageId>WebApi.App</PackageId>
    <Authors>piomin</Authors>
    <Description>WebApi</Description>
  </PropertyGroup>

</Project>

<Project Sdk="Microsoft.NET.Sdk.Web">

  <ItemGroup>
    <ProjectReference Include="..\WebApi.Library\WebApi.Library.csproj" />
    <PackageReference Include="Swashbuckle.AspNetCore" Version="9.0.4" />
    <PackageReference Include="prometheus-net.AspNetCore" Version="8.2.1" />
    <PackageReference Include="AspNetCore.HealthChecks.NpgSql" Version="9.0.0" />
    <PackageReference Include="AspNetCore.HealthChecks.UI.Client" Version="9.0.0" />
    <PackageReference Include="Microsoft.Extensions.Diagnostics.HealthChecks.EntityFrameworkCore" Version="9.0.8" />
  </ItemGroup>

  <PropertyGroup>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <Version>1.0.3</Version>
    <IsPackable>true</IsPackable>
    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
    <PackageId>WebApi.App</PackageId>
    <Authors>piomin</Authors>
    <Description>WebApi</Description>
  </PropertyGroup>

</Project>

Using OpenShift Source-to-Image for .NET

S2I Locally with CLI

Before testing a mechanism on OpenShift, you try Source-to-Image locally. On macOS, you can install s2i CLI using Homebrew:

brew install source-to-image

brew install source-to-image

After installation, check its version:

$ s2i version
s2i v1.5.1

$ s2i version
s2i v1.5.1

Then, go to the repository root directory. At this point, we need to parameterize our build because the repository contains several projects. Fortunately, S2I provides a parameter that allows us to set the main project in a multi-module structure easily. It must be set as an environment variable for the s2i command. The following command sets the DOTNET_STARTUP_PROJECT environment variable and uses the registry.access.redhat.com/ubi8/dotnet-90:latest as a builder image.

s2i build . registry.access.redhat.com/ubi8/dotnet-90:latest webapi-app \
  -e DOTNET_STARTUP_PROJECT=src/WebApi.App

s2i build . registry.access.redhat.com/ubi8/dotnet-90:latest webapi-app \
  -e DOTNET_STARTUP_PROJECT=src/WebApi.App

Of course, you must have Docker or Podman running on your laptop to use s2i. So, before using a builder image, pull it to your host.

podman pull registry.access.redhat.com/ubi8/dotnet-90:latest

podman pull registry.access.redhat.com/ubi8/dotnet-90:latest

Let’s take a look at the s2i build command output. As you can see, s2i restored and built two projects, but then created a runnable output for the WebApi.App project.

What about our unit tests? To execute tests during the build, we must also set the DOTNET_TEST_PROJECTS environment variable.

s2i build . registry.access.redhat.com/ubi8/dotnet-90:latest webapi-app \
  -e DOTNET_STARTUP_PROJECT=src/WebApi.App \
  -e DOTNET_TEST_PROJECTS=tests/WebApi.App.Tests

s2i build . registry.access.redhat.com/ubi8/dotnet-90:latest webapi-app \
  -e DOTNET_STARTUP_PROJECT=src/WebApi.App \
  -e DOTNET_TEST_PROJECTS=tests/WebApi.App.Tests

Here’s the command output:

The webapi-app image is ready.

$ podman images webapi-app
REPOSITORY                    TAG         IMAGE ID      CREATED        SIZE
docker.io/library/webapi-app  latest      e9d94f983ac1  5 seconds ago  732 MB

$ podman images webapi-app
REPOSITORY                    TAG         IMAGE ID      CREATED        SIZE
docker.io/library/webapi-app  latest      e9d94f983ac1  5 seconds ago  732 MB

We can run it locally with Podman (or Docker):

S2I for .NET on OpenShift

Then, let’s switch to the OpenShift cluster. You need to log in to your cluster using the oc login command. After that, create a new project for testing purposes:

oc new-project dotnet

oc new-project dotnet

In OpenShift, a single command can handle everything necessary to build and deploy an application. We need to provide the address of the Git repository containing the source code, specify the branch name, and indicate the name of the builder image located within the cluster’s namespace. Additionally, we should include the same environment variables as we did previously. Since the version of source code we tested before is located in the dev branch, we must pass it together with the repository URL after #.

oc new-app openshift/dotnet:latest~https://github.com/piomin/web-api-2.git#dev --name webapi-app \
  --build-env DOTNET_STARTUP_PROJECT=src/WebApi.App \
  --build-env DOTNET_TEST_PROJECTS=tests/WebApi.App.Tests

oc new-app openshift/dotnet:latest~https://github.com/piomin/web-api-2.git#dev --name webapi-app \
  --build-env DOTNET_STARTUP_PROJECT=src/WebApi.App \
  --build-env DOTNET_TEST_PROJECTS=tests/WebApi.App.Tests

Here’s the oc new-app command output:

Then, let’s expose the application outside the cluster using OpenShift Route.

oc expose service/webapi-app

oc expose service/webapi-app

Finally, we can verify the build and deployment status:

There is also a really nice command you can use here. Try yourself

oc get all

oc get all

OpenShift Builds with Source-to-Image

Verify Build Status

Let’s verify what has happened after taking the steps from the previous section. Here’s the panel that summarizes the status of our application on the cluster. OpenShift automatically built the image from the .NET source code repository and then deployed it in the target namespace.

Here are the logs from the Pod with our application:

The build was entirely performed on the cluster. You can verify the logs from the build by accessing the Build object. After building, the image was pushed to the internal image registry in OpenShift.

Under the hood, the BuildConfig was created. This will be the starting point for the next example we will consider.

apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
  annotations:
    openshift.io/generated-by: OpenShiftNewApp
  labels:
    app: webapi-app
    app.kubernetes.io/component: webapi-app
    app.kubernetes.io/instance: webapi-app
  name: webapi-app
  namespace: dotnet
spec:
  output:
    to:
      kind: ImageStreamTag
      name: webapi-app:latest
  source:
    git:
      ref: dev
      uri: https://github.com/piomin/web-api-2.git
    type: Git
  strategy:
    sourceStrategy:
      env:
      - name: DOTNET_STARTUP_PROJECT
        value: src/WebApi.App
      - name: DOTNET_TEST_PROJECTS
        value: tests/WebApi.App.Tests
      from:
        kind: ImageStreamTag
        name: dotnet:latest
        namespace: openshift
    type: Source

apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
  annotations:
    openshift.io/generated-by: OpenShiftNewApp
  labels:
    app: webapi-app
    app.kubernetes.io/component: webapi-app
    app.kubernetes.io/instance: webapi-app
  name: webapi-app
  namespace: dotnet
spec:
  output:
    to:
      kind: ImageStreamTag
      name: webapi-app:latest
  source:
    git:
      ref: dev
      uri: https://github.com/piomin/web-api-2.git
    type: Git
  strategy:
    sourceStrategy:
      env:
      - name: DOTNET_STARTUP_PROJECT
        value: src/WebApi.App
      - name: DOTNET_TEST_PROJECTS
        value: tests/WebApi.App.Tests
      from:
        kind: ImageStreamTag
        name: dotnet:latest
        namespace: openshift
    type: Source

OpenShift with .NET and Azure Artifacts Proxy

Now let’s switch to the master branch in our Git repository. In this branch, the WebApi.Library library is no longer included as a path in the project, but as a separate dependency from an external repository. However, this library has not been published in the public NuGet repository, but in the internal Azure Artifacts repository. Therefore, the build process must take place via a proxy pointing to the address of our repository, or rather, a feed in Azure Artifacts.

<Project Sdk="Microsoft.NET.Sdk.Web">

  <ItemGroup>
    <PackageReference Include="WebApi.Library" Version="1.0.3" />
    <PackageReference Include="Swashbuckle.AspNetCore" Version="9.0.4" />
    <PackageReference Include="prometheus-net.AspNetCore" Version="8.2.1" />
    <PackageReference Include="AspNetCore.HealthChecks.NpgSql" Version="9.0.0" />
    <PackageReference Include="AspNetCore.HealthChecks.UI.Client" Version="9.0.0" />
    <PackageReference Include="Microsoft.Extensions.Diagnostics.HealthChecks.EntityFrameworkCore" Version="9.0.8" />
  </ItemGroup>

  <PropertyGroup>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <Version>1.0.3</Version>
    <IsPackable>true</IsPackable>
    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
    <PackageId>WebApi.App</PackageId>
    <Authors>piomin</Authors>
    <Description>WebApi</Description>
  </PropertyGroup>

</Project>

<Project Sdk="Microsoft.NET.Sdk.Web">

  <ItemGroup>
    <PackageReference Include="WebApi.Library" Version="1.0.3" />
    <PackageReference Include="Swashbuckle.AspNetCore" Version="9.0.4" />
    <PackageReference Include="prometheus-net.AspNetCore" Version="8.2.1" />
    <PackageReference Include="AspNetCore.HealthChecks.NpgSql" Version="9.0.0" />
    <PackageReference Include="AspNetCore.HealthChecks.UI.Client" Version="9.0.0" />
    <PackageReference Include="Microsoft.Extensions.Diagnostics.HealthChecks.EntityFrameworkCore" Version="9.0.8" />
  </ItemGroup>

  <PropertyGroup>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <Version>1.0.3</Version>
    <IsPackable>true</IsPackable>
    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
    <PackageId>WebApi.App</PackageId>
    <Authors>piomin</Authors>
    <Description>WebApi</Description>
  </PropertyGroup>

</Project>

This is how it looks in Azure Artifacts. The name of my feed is pminkows. To access the feed, I must be authenticated against Azure DevOps using a personal token. The full address of the NuGet registry exposed via my instance of Azure Artifacts is https://pkgs.dev.azure.com/pminkows/_packaging/pminkows/nuget/v3/index.json.

If you would like to build such an application locally using Azure Artifacts, you should create a nuget.config file with the configuration below. Then place it in the $HOME/.nuget/NuGet directory.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="pminkows" value="https://pkgs.dev.azure.com/pminkows/_packaging/pminkows/nuget/v3/index.json" />
  </packageSources>
  <packageSourceCredentials>
    <pminkows>
      <add key="Username" value="pminkows" />
      <add key="ClearTextPassword" value="<MY_PERSONAL_TOKEN>" />
    </pminkows>
  </packageSourceCredentials>
</configuration>

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <clear />
    <add key="pminkows" value="https://pkgs.dev.azure.com/pminkows/_packaging/pminkows/nuget/v3/index.json" />
  </packageSources>
  <packageSourceCredentials>
    <pminkows>
      <add key="Username" value="pminkows" />
      <add key="ClearTextPassword" value="<MY_PERSONAL_TOKEN>" />
    </pminkows>
  </packageSourceCredentials>
</configuration>

Our goal is to run this type of build on OpenShift instead of locally. To achieve this, we need to create a Kubernetes Secret containing the nuget.config file.

oc create secret generic nuget-config --from-file=nuget.config

oc create secret generic nuget-config --from-file=nuget.config

Then, we must update the contents of the BuildConfig object. The changed lines in the object have been highlighted. The most important element is spec.source.secrets. The Kubernetes Secret containing the nuget.config file must be mounted in the HOME directory of the base image with .NET. We also change the branch in the repository to master and increase the logging level for the builder to detailed.

apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
  annotations:
    openshift.io/generated-by: OpenShiftNewApp
  labels:
    app: webapi-app
    app.kubernetes.io/component: webapi-app
    app.kubernetes.io/instance: webapi-app
  name: webapi-app
  namespace: dotnet
spec:
  output:
    to:
      kind: ImageStreamTag
      name: webapi-app:latest
  source:
    git:
      ref: master
      uri: https://github.com/piomin/web-api-2.git
    type: Git
    secrets:
      - secret:
          name: nuget-config
        destinationDir: /opt/app-root/src/
  strategy:
    sourceStrategy:
      env:
      - name: DOTNET_STARTUP_PROJECT
        value: src/WebApi.App
      - name: DOTNET_TEST_PROJECTS
        value: tests/WebApi.App.Tests
      - name: DOTNET_VERBOSITY
        value: d
      from:
        kind: ImageStreamTag
        name: dotnet:latest
        namespace: openshift
    type: Source

apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
  annotations:
    openshift.io/generated-by: OpenShiftNewApp
  labels:
    app: webapi-app
    app.kubernetes.io/component: webapi-app
    app.kubernetes.io/instance: webapi-app
  name: webapi-app
  namespace: dotnet
spec:
  output:
    to:
      kind: ImageStreamTag
      name: webapi-app:latest
  source:
    git:
      ref: master
      uri: https://github.com/piomin/web-api-2.git
    type: Git
    secrets:
      - secret:
          name: nuget-config
        destinationDir: /opt/app-root/src/
  strategy:
    sourceStrategy:
      env:
      - name: DOTNET_STARTUP_PROJECT
        value: src/WebApi.App
      - name: DOTNET_TEST_PROJECTS
        value: tests/WebApi.App.Tests
      - name: DOTNET_VERBOSITY
        value: d
      from:
        kind: ImageStreamTag
        name: dotnet:latest
        namespace: openshift
    type: Source

Next, we can run the build again, but this time with new parameters using the command below. With increased logging level, you can confirm that all dependencies are being retrieved via the Azure Artifacts instance.

oc start-build webapi-app --follow

oc start-build webapi-app --follow

Conclusion

This article covers different scenarios about building and deploying .NET applications in developer mode on OpenShift. It demonstrates how to use various parameters to customize image building according to the application’s needs. My goal was to demonstrate that deploying .NET applications on OpenShift is straightforward with the help of Source-to-Image.

The post Running .NET Apps on OpenShift appeared first on Piotr's TechBlog.

For comparison, please refer to the article where I demonstrate how to prepare a Backstage instance for running on Kubernetes step-by-step. Today, for the sake of clarity, we will be working in a Developer Hub instance running on OpenShift using an operator. However, we can also easily install Developer Hub on vanilla Kubernetes using a Helm chart.

Source Code

Feel free to use my source code if you’d like to try it out yourself. To do that, you must clone my sample GitHub repository. We will also use another repository with a sample Backstage plugin. This time, I won’t create a plugin myself, but I will use an existing one. The plugin provides a collection of scaffolder actions for interacting with Kubernetes on Backstage, including apply and delete. Once you clone both of those repositories, you should only follow my instructions.

Prerequisites

You must have an OpenShift cluster with the Red Hat Developer Hub installed and configured with a Kubernetes plugin. I will briefly explain it in the next section without getting more into the details. You can find more information in the already mentioned article about Red Hat Developer Hub.

You must also have Node.js, NPM, and Yarn installed and configured on your laptop. It is used for plugin compilation and building. The npm package @janus-idp/cli used for developing and exporting Backstage plugins as dynamic plugins, requires Podman when working in image mode.

Motivation

Red Hat Developer Hub comes with a curated set of plugins preinstalled on its container image. It is easy to enable such a plugin just by changing the configuration available in Kubernetes ConfigMap. The situation becomes complicated when we attempt to install and configure a third-party plugin. In this case, I would like to extend Backstage with a set of actions that enable the creation and management of Kubernetes resources directly, rather than applying them via Argo CD. To do that, we must install the Backstage Scaffolder Actions for Kubernetes plugin. To use this plugin in Red Hat Developer Hub without rebuilding its image, you must export plugins as derived dynamic plugin packages. This is our goal.

Convert to a dynamic Backstage plugin

The backstage-k8s-scaffolder-actions is a backend plugin. It meets all the requirements to be converted to a dynamic plugin form. It has a valid package.json file in its root directory, containing all required metadata and dependencies. That plugin is compatible with the new Backstage backend system, which means that it was created using createBackendPlugin() or createBackendModule(). Let’s clone its repository first:

$ git clone https://github.com/kirederik/backstage-k8s-scaffolder-actions.git
$ cd backstage-k8s-scaffolder-actions

Then we must install the @janus-idp/cli npm package with the following command:

yarn add @janus-idp/cli

After that, you should run both those commands inside the plugin directory:

$ yarn install
$ yarn build

If the commands were successful, you can proceed with the plugin conversion procedure. The plugin defines some shared dependencies that must be explicitly specified with the --shared-package flag.

Here’s the command used to convert our plugin to a dynamic form supported by Red Hat Developer Hub:

npx @janus-idp/cli@latest package export-dynamic-plugin \
  --shared-package '!@backstage/cli-common' \
  --shared-package '!@backstage/cli-node' \
  --shared-package '!@backstage/config-loader' \
  --shared-package '!@backstage/config' \
  --shared-package '!@backstage/errors' \
  --shared-package '!@backstage/types'

Package and publish Backstage dynamic plugins

After exporting a third-party plugin, you can package the derived package into one of the following supported formats:

• Open Container Initiative (OCI) image (recommended)
• TGZ file
• JavaScript package

Since the OCI image option is recommended, we will proceed accordingly. However, first you must ensure that podman is running on your laptop and is logged in to your container registry.

podman login quay.io

Then you can run the following @janus-idp/cli command with npx. It must specify the target image repository address, including image name and tag. The target image address is quay.io/pminkows/backstage-k8s-scaffolder-actions:v0.5. It is tagged with the latest version of the plugin.

npx @janus-idp/cli@latest package package-dynamic-plugins \
  --tag quay.io/pminkows/backstage-k8s-scaffolder-actions:v0.5

Here’s the command output. Ultimately, it provides instructions on how to install and enable the plugin within the Red Hat Developer configuration. Copy that statement for future use.

The previous command packages the plugin and builds its image.

Finally, let’s push the image with our plugin to the target registry:

podman push quay.io/pminkows/backstage-k8s-scaffolder-actions:v0.5

Install and enable the Backstage dynamic plugins in Developer Hub

My instance of Developer Hub is running in the backstage namespace. The operator manages it.

Here’s the Backstage CR object responsible for creating a Developer Hub instance. The dynamicPluginsConfigMapName property specifies the name of the ConfigMap that stores the plugins’ configuration.

apiVersion: rhdh.redhat.com/v1alpha3
kind: Backstage
metadata:
  name: developer-hub
  namespace: backstage
spec:
  application:
    appConfig:
      configMaps:
        - name: app-config-rhdh
      mountPath: /opt/app-root/src
    dynamicPluginsConfigMapName: dynamic-plugins-rhdh
    extraEnvs:
      secrets:
        - name: app-secrets-rhdh
    extraFiles:
      mountPath: /opt/app-root/src
    replicas: 1
    route:
      enabled: true
  database:
    enableLocalDb: true

Then, we must modify the dynamic-plugins-rhdh ConfigMap to register our plugin in Red Hat Developer Hub. You must paste the previously copied two lines of code generated by the npx @janus-idp/cli@latest package package-dynamic-plugins command.

kind: ConfigMap
apiVersion: v1
metadata:
  name: dynamic-plugins-rhdh
  namespace: backstage
data:
  dynamic-plugins.yaml: |-
    plugins:
      # ... other plugins

      - package: oci://quay.io/pminkows/backstage-k8s-scaffolder-actions:v0.5!devangelista-backstage-scaffolder-kubernetes
        disabled: false

That’s all! After the change is applied to ConfigMap, the operator should restart the pod with Developer Hub. It can take some time, as the pod will be restarted, since all plugins must be enabled during pod startup.

$ oc get pod
NAME                                      READY   STATUS    RESTARTS   AGE
backstage-developer-hub-896c5f9d9-vvddb   1/1     Running   0          4m21s
backstage-psql-developer-hub-0            1/1     Running   0          8d

You can verify the logs with the oc logs command. Developer Hub prints a list of available actions provided by the installed plugins. You should see three actions delivered by the Backstage Scaffolder Actions for Kubernetes plugin, starting with the kube: prefix.

Prepare the Backstage template

Finally, we will test a new plugin by calling the kube:apply action from our Backstage template. It uses the kube:apply to create a Secret in the specified namespace under a given name. This template is available in the backstage-templates repository.

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
  description: Create a Secret in Kubernetes
  name: create-secret
  title: Create a Secret
spec:
  lifecycle: experimental
  owner: user
  type: example
  parameters:
    - properties:
        name:
          description: The namespace name
          title: Name
          type: string
          ui:autofocus: true
      required:
        - name
      title: Namespace Name
    - properties:
        secretName:
          description: The secret name
          title: Secret Name
          type: string
          ui:autofocus: true
      required:
        - secretName
      title: Secret Name
    - title: Cluster Name
      properties:
        cluster:
          type: string
          enum:
            - ocp
          ui:autocomplete:
            options:
              - ocp
  steps:
    - action: kube:apply
      id: k-apply
      name: Create a Resouce
      input:
        namespaced: true
        clusterName: ${{ parameters.cluster }}
        manifest: |
          kind: Secret
          apiVersion: v1
          metadata:
            name: ${{ parameters.secretName }}
            namespace: ${{ parameters.name }}
          data:
            username: YWRtaW4=

You should import the repository with templates into your Developer Hub instance. The app-config-rhdh ConfigMap should contain the full address templates list file in the repository, and the Kubernetes cluster address and connection credentials.

catalog:
  rules:
    - allow: [Component, System, API, Resource, Location, Template]
  locations:
    - type: url
      target: https://github.com/piomin/backstage-templates/blob/master/templates.yaml
      rules:
        - allow: [Template, Location]

kubernetes:
  clusterLocatorMethods:
    - clusters:
      - authProvider: serviceAccount
        name: ocp
        serviceAccountToken: ${OPENSHIFT_TOKEN}
        skipTLSVerify: true
        url: https://api.${DOMAIN}:6443
      type: config
  customResources:
    - apiVersion: v1beta1
      group: tekton.dev
      plural: pipelineruns
    - apiVersion: v1beta1
      group: tekton.dev
      plural: taskruns
    - apiVersion: v1
      group: route.openshift.io
      plural: routes
  serviceLocatorMethod:
    type: multiTenant

You can access the Developer Hub instance through the OpenShift Route:

$ oc get route
NAME                      HOST/PORT                                                                 PATH   SERVICES                  PORT           TERMINATION     WILDCARD
backstage-developer-hub   backstage-developer-hub-backstage.apps.piomin.ewyw.p1.openshiftapps.com   /      backstage-developer-hub   http-backend   edge/Redirect   None

Then find and use the following template available in the Developer Hub.

You will have to insert the Secret name, namespace, and choose a target OpenShift cluster. Then accept the action.

You should see a similar screen. The Secret has been successfully created.

Let’s verify if it exists on our OpenShift cluster:

Final Thoughts

Red Hat is steadily developing its Backstage-based product, adding new functionality in future versions. The ability to easily create and install custom plug-ins in the Developer Hub appears to be a key element in building an attractive platform for developers. This article focuses on demonstrating how to convert a standard Backstage plugin into a dynamic form supported by Red Hat Developer Hub.

The post Backstage Dynamic Plugins with Red Hat Developer Hub appeared first on Piotr's TechBlog.

Previously, I published several articles about Spring AI with examples of using different AI models. Therefore, I will not focus on the introduction to Spring AI. For example, you can read about integration between Spring AI and Azure AI in the following post. Please refer to the following article for a quick intro to the Spring AI project.

Source Code

Feel free to use my source code if you’d like to try it out yourself. To do that, you must clone my sample GitHub repository. Then you should only follow my instructions.

Prerequisites

Create the OpenShift Cluster

For this exercise, you will need a relatively large OpenShift cluster. At least one of the cluster’s nodes must have a GPU. I created a cluster on AWS with one node on a g4dn.12xlarge machine. On OpenShift, you can achieve this by creating the MachineSet object that creates nodes using the appropriate virtual machine available on AWS.

Install Required Operators

Next, install and configure several operators on the cluster. Begin with the “Node Feature Discovery” operator. On OpenShift, this operator enables automatic discovery of cluster nodes with features such as GPUs. After installing the operator, create the NodeFeatureDiscovery object. The default values set by the OpenShift console during object creation are sufficient.

The operator’s task is to mark the node with the detected GPU using the appropriate label. The label is feature.node.kubernetes.io/pci-10de.present=true. After configuring the operator, verify that the correct GPU has been detected.

$ oc get node -l feature.node.kubernetes.io/pci-10de.present=true
NAME                                        STATUS   ROLES    AGE   VERSION
ip-10-0-45-120.us-east-2.compute.internal   Ready    worker   15d   v1.31.6

Next, install the NVIDIA GPU Operator. This operator automatically installs, configures, and manages NVIDIA drivers and tools on nodes with NVIDIA graphics cards. This allows OpenShift to recognize the GPU as a resource that can be declared in pods. This will enable OpenShift to work with the “Node Feature Discovery” operator to label nodes with GPUs. The NVIDIA GPU operator uses the feature.node.kubernetes.io/pci-10de.present=true label to determine where to install the drivers. For this to happen, the ClusterPolicy object must be created. As before, you can use the default values generated by the OpenShift Console when creating this object.

The OpenShift AI feature for serving AI models requires the installation of OpenShift Serverless and OpenShift Service Mesh operators. The key solution here is KServe. KServe uses Knative to scale models on demand and integrates with Istio to secure model routing and versioning.

The final step in this phase is to install the OpenShift AI Operator and create the DataScienceCluster object. If the previous installations were successful, everything will be configured automatically after creating the DataScienceCluster object. For instance, OpenShift AI will make the Istio control plane and the Knative Serving component.

OpenShift AI creates several namespaces within a cluster. The most important is the redhat-ods-applications namespace, where most components comprising the entire solution are run.

$ oc get pod -n redhat-ods-applications
NAME                                                              READY   STATUS    RESTARTS   AGE
authorino-767bd64465-fq8bl                                        1/1     Running   0          15d
codeflare-operator-manager-5c69778b87-wxcwp                       1/1     Running   0          15d
data-science-pipelines-operator-controller-manager-6686587wcmkr   1/1     Running   0          15d
etcd-549d769449-hqzwt                                             1/1     Running   0          15d
kserve-controller-manager-85f9b8d66f-qpxbf                        1/1     Running   0          15d
kuberay-operator-8d77dcf84-qgsq5                                  1/1     Running   0          15d
kueue-controller-manager-7c895bd669-467nk                         1/1     Running   0          6h8m
modelmesh-controller-7f9dd5f848-ljlxp                             1/1     Running   0          15d
modelmesh-controller-7f9dd5f848-qqsl8                             1/1     Running   0          24d
modelmesh-controller-7f9dd5f848-txlhd                             1/1     Running   0          24d
notebook-controller-deployment-86f5b87585-p6nz5                   1/1     Running   0          15d
odh-model-controller-574ff4657-q75gr                              1/1     Running   0          15d
odh-notebook-controller-manager-9d754d5f-2ptk9                    1/1     Running   0          15d
rhods-dashboard-5b96595667-79tx6                                  2/2     Running   0          15d
rhods-dashboard-5b96595667-8m52g                                  2/2     Running   0          15d
rhods-dashboard-5b96595667-kx7p4                                  2/2     Running   0          15d
rhods-dashboard-5b96595667-nn2cf                                  2/2     Running   0          15d
rhods-dashboard-5b96595667-ttcht                                  2/2     Running   0          15d
trustyai-service-operator-controller-manager-bd9fbdb6d-kcd57      1/1     Running   0          15d

Configure and Use OpenShift AI

After installing OpenShift AI on a cluster, you can use its graphical UI. To access it, select “Red Hat OpenShift AI” from the menu at the top of the page.

After selecting the indicated option, you will be redirected to the following page. This page allows you to configure and use OpenShift AI on a cluster. The first step is to select a namespace on the cluster for the AI project. In my case, the namespace is ai.

To run an AI model on a cluster, choose how to serve it first. You can choose between a single-model serving platform and a multi-model serving platform. With the former, each model is deployed on its model server. Multiple models can be deployed on a single shared server with multi-model platforms. This article will use the first option: a single-model serving platform.

The next step is to create an acceleration profile. This profile should be created automatically after installing and configuring the NVIDIA GPU Operator. If, for some reason, it was not, you can easily create it manually. When creating this object, enter the nvidia.com/gpu value in the identifier field.

You can either click on the profile from the UI or create it using the YAML manifest.

apiVersion: dashboard.opendatahub.io/v1
kind: AcceleratorProfile
metadata:
  name: nvidia
  namespace: redhat-ods-applications
spec:
  displayName: nvidia
  enabled: true
  identifier: nvidia.com/gpu

Serve Model on OpenShift AI with vLLM

Create ServingRuntime Resource

In the previous step, we configured OpenShift AI to deploy the model with a single-model serving platform and a GPU accelerator. We will use KServe’s ModelCar functionality to deploy the model, which allows us to serve models directly from a container. This functionality is described in an article published on the Red Hat Developer blog. The article demonstrates how to build an image containing a model downloaded from the Hugging Face Hub. In turn, we will use images that have already been built and are available in the quay.io/repository/redhat-ai-services/modelcar-catalog repository. You can find ready-made images for AI models such as Granite and Llama.

To run a model on OpenShift AI in single-model serving runtime mode, you must define two objects: ServingRuntime and InferenceService. According to the OpenShift AI documentation, the ServingRuntime CR creates a serving runtime, an environment for deploying and managing a model. Here’s the ServingRuntime object that creates a runtime for the Llama 3.2 AI model. The annotation opendatahub.io/recommended-accelerators sets the name of the recommended accelerator to use with the runtime. Its value should be identical to the identifier field in the AcceleratorProfile object (1). The openshift.io/display-name annotation keeps the name with which the serving runtime is displayed (2). The spec.containers.image field indicates the runtime container image used by the serving runtime (3). This image differs depending on the type of accelerator used. Finally, the ServingRuntime object specifies that the single-model serving is used (4) and the vLLM model is supported by the runtime (5).

apiVersion: serving.kserve.io/v1alpha1
kind: ServingRuntime
metadata:
  annotations:
    opendatahub.io/recommended-accelerators: '["nvidia.com/gpu"]' # (1)
    openshift.io/display-name: vLLM ServingRuntime for KServe # (2)
  labels:
    opendatahub.io/dashboard: "true"
  name: llama-32-3b-instruct
spec:
  annotations:
    prometheus.io/path: /metrics 
    prometheus.io/port: "8080" 
  containers :
    - args:
        - --port=8080
        - --model=/mnt/models 
        - --served-model-name={{.Name}} 
      command: 
        - python
        - '-m'
        - vllm.entrypoints.openai.api_server
      env:
        - name: HF_HOME
          value: /tmp/hf_home
      # (3)
      image:
quay.io/modh/vllm@sha256:0d55419f3d168fd80868a36ac89815dded9e063937a8409b7edf3529771383f3
    name: kserve-container
    ports:
      - containerPort: 8080
        protocol: TCP
  multiModel: false # (4)
  supportedModelFormats: # (5) 
    - autoSelect: true
      name: vLLM

Create InterferenceService Resource

The InferenceService CRD creates a server or inference service that processes inference queries, passes them to the model, and returns the inference output. Here’s the InferenceService object related to the previously created llama-32-3b-instruct runtime (1). It must define some vLLM parameters to successfully run the model on the existing infrastructure and enable tool calling support on the Llama 3.2 model (2). The InferenceService object specifies the image containing the Llama 3.2 model, published in the the quay.io/redhat-ai-services/modelcar-catalog:llama-3.2-3b-instruct repository (3). Alternatively, you can create your image, publish it in the custom registry, and run it on OpenShift using InferenceService CR.

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  annotations:
    openshift.io/display-name: llama-32-3b-instruct
    serving.knative.openshift.io/enablePassthrough: 'true'
    serving.kserve.io/deploymentMode: Serverless
    sidecar.istio.io/inject: 'true'
    sidecar.istio.io/rewriteAppHTTPProbers: 'true'
  name: llama-32-3b-instruct # (1)
  labels:
    opendatahub.io/dashboard: 'true'
spec:
  predictor:
    maxReplicas: 1
    minReplicas: 1
    model: # (2)
      args:
        - '--dtype=half'
        - '--max_model_len=8192'
        - '--gpu_memory_utilization=.95'
        - '--enable-auto-tool-choice'
        - '--tool_call_parser=llama3_json'
      modelFormat:
        name: vLLM
      name: ''
      resources:
        limits:
          cpu: '8'
          memory: 10Gi
          nvidia.com/gpu: '1'
        requests:
          cpu: '4'
          memory: 8Gi
          nvidia.com/gpu: '1'
      runtime: llama-32-3b-instruct
      storageUri: 'oci://quay.io/redhat-ai-services/modelcar-catalog:llama-3.2-3b-instruct' # (3)

Deploy with OpenShift AI

You can also create the same configuration using the OpenShift AI UI. The diagram below shows the settings you need for Granite 3.2.

The OpenShift AI UI lists all the models running in a given AI project. You can check the endpoint where a particular model is available. In this case, two models are running in the AI project: Llama 3.2 and Granite 3.2. Both models are available internally on the cluster and externally via the Knative Route object.

Both models are automatically exposed on the node with the GPU. You can check the GPU resource reservations on a node using the oc describe command:

A single-model serving platform runs AI models as the Knative Service. You can use the oc get ksvc command to display a list of Knative services running in the ai namespace.

$ oc get ksvc -n ai
NAME                               URL                                                                                 LATESTCREATED                            LATESTREADY                              READY   REASON
granite-32-2b-instruct-predictor   https://granite-32-2b-instruct-predictor-ai.apps.piomin.ewyw.p1.openshiftapps.com   granite-32-2b-instruct-predictor-00007   granite-32-2b-instruct-predictor-00007   True    
llama-32-3b-instruct-predictor     https://llama-32-3b-instruct-predictor-ai.apps.piomin.ewyw.p1.openshiftapps.com     llama-32-3b-instruct-predictor-00002     llama-32-3b-instruct-predictor-00002     True 

Integrate Spring AI with vLLM

Dependencies and Properties

The vLLM runtime is compatible with the OpenAI REST API. To integrate our sample Spring Boot application with a model running on vLLM, we must use the standard Spring AI OpenAI starter. The app in the spring-ai-showcase repository has more functionality than what is tested in this article. In simplified terms, the list of dependencies needed for the app to communicate with the OpenAI API and the model running on OpenShift AI is below.

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>
<dependency>
  <groupId>org.springframework.ai</groupId>
  <artifactId>spring-ai-autoconfigure-model-openai</artifactId>
</dependency>

Although the model itself served on OpenShift AI does not require authorization with an API key the spring.ai.openai.api-key Spring AI parameter must be set. The endpoint’s address provided through the vLLM runtime must be specified in the spring.ai.openai.chat.base-url parameter. The default name of the model used must also be overwritten with the name under which the model was run on OpenShift AI. This name is for Llama 3.2 llama-32-3b-instruct. Below is a list of all the Spring Boot settings required for vLLM integration, which is available in the application-vllm.properties file.

spring.ai.openai.api-key = ${OPENAI_API_KEY:dummy}
spring.ai.openai.chat.base-url = https://llama-32-3b-instruct-ai.apps.piomin.ewyw.p1.openshiftapps.com
spring.ai.openai.chat.options.model = llama-32-3b-instruct

Implementation with Spring AI

The code below demonstrates how @RestController implements communication between the application and the target AI model. The @RestController class injects an auto-configured ChatClient.Builder to create an instance of ChatClient. The PersonController class implements a method for returning a list of persons from the GET /persons endpoint. The main goal is to generate a list of 10 objects with the fields defined in the Person class. The id field should be auto-incremented. The PromptTemplate object defines a message that will be sent to the chat model AI API. It doesn’t have to specify the exact fields that should be returned. This part is handled automatically by the Spring AI library after we invoke the entity() method on the ChatClient instance. The ParameterizedTypeReference object inside the entity method tells Spring AI to generate a list of objects.

@RestController
@RequestMapping("/persons")
public class PersonController {

    private final ChatClient chatClient;

    public PersonController(ChatClient.Builder chatClientBuilder,
                            ChatMemory chatMemory) {
        this.chatClient = chatClientBuilder
                .defaultAdvisors(
                        new PromptChatMemoryAdvisor(chatMemory),
                        new SimpleLoggerAdvisor())
                .build();
    }

    @GetMapping
    List<Person> findAll() {
        PromptTemplate pt = new PromptTemplate("""
                Return a current list of 10 persons if exists or generate a new list with random values.
                Each object should contain an auto-incremented id field.
                The age value should be a random number between 18 and 99.
                Do not include any explanations or additional text.
                Return data in RFC8259 compliant JSON format.
                """);

        return this.chatClient.prompt(pt.create())
                .call()
                .entity(new ParameterizedTypeReference<>() {});
    }

    @GetMapping("/{id}")
    Person findById(@PathVariable String id) {
        PromptTemplate pt = new PromptTemplate("""
                Find and return the object with id {id} in a current list of persons.
                """);
        Prompt p = pt.create(Map.of("id", id));
        return this.chatClient.prompt(p)
                .call()
                .entity(Person.class);
    }
}

The llama-32-3b-instruct model uses a “tool-calling” approach for API calls. You can read more about it in one of my Spring AI articles, which are available at this link. For instance, the class below implements the @Tool annotation, connecting to the database and searching it for a list of shares for individual companies. The key to using this tool is its description in the description field, which is then appropriately interpreted by the LLM model.

public class WalletTools {

    private WalletRepository walletRepository;

    public WalletTools(WalletRepository walletRepository) {
        this.walletRepository = walletRepository;
    }

    @Tool(description = "Number of shares for each company in my wallet")
    public List<Share> getNumberOfShares() {
        return (List<Share>) walletRepository.findAll();
    }
}

Then, the @Tool reference is set to the chat client when it interacts with the AI model. The AI model can call the tool’s method as required based on the tool’s description and the input prompt’s content.

@RestController
@RequestMapping("/wallet")
public class WalletController {

    private final ChatClient chatClient;
    private final StockTools stockTools;
    private final WalletTools walletTools;

    public WalletController(ChatClient.Builder chatClientBuilder,
                            StockTools stockTools,
                            WalletTools walletTools) {
        this.chatClient = chatClientBuilder
                .defaultAdvisors(new SimpleLoggerAdvisor())
                .build();
        this.stockTools = stockTools;
        this.walletTools = walletTools;
    }
    
    @GetMapping("/with-tools")
    String calculateWalletValueWithTools() {
        PromptTemplate pt = new PromptTemplate("""
        What’s the current value in dollars of my wallet based on the latest stock daily prices ?
        """);

        return this.chatClient.prompt(pt.create())
                .tools(stockTools, walletTools)
                .call()
                .content();
    }
    
}