LangChain4j Archives - Piotr's TechBlog

MCP with Quarkus LangChain4j

piotr.minkowski — Mon, 24 Nov 2025 07:45:15 +0000

This article shows how to use Quarkus LangChain4j support for MCP (Model Context Protocol) on both the server and client sides. You will learn how to serve tools and prompts on the server side and discover them in the Quarkus MCP client-side application. The Model Context Protocol is a standard for managing contextual interactions with AI models. It provides a standardized way to connect AI models to external data sources and tools. It can help with building complex workflows on top of LLMs.

This article is the second part of a series describing some of the Quarkus AI project’s most notable features. Before reading this article, I recommend checking out two previous parts of the tutorial:

Part 1: Getting Started with Quarkus LangChain4j and Chat Model
Part 2: AI Tool Calling with Quarkus LangChain4j

You can also compare Quarkus’ support for MCP with similar support on the Spring AI side. You can find the article I mentioned on my blog here.

Source Code

Feel free to use my source code if you’d like to try it out yourself. To do that, you must clone my sample GitHub repository. Then you should only follow my instructions.

Architecture for the Quarkus MCP Scenario

Let’s start with a diagram of our application architecture. Two Quarkus applications act as MCP servers. They connect to the in-memory database and use Quarkus LangChain4j MCP Server support to expose @Tool methods to the MCP client-side app. The client-side app communicates with the OpenAI model. It includes the tools exposed by the server-side apps in the user query to the AI model. The person-mcp-server app provides @Tool methods for searching persons in the database table. The account-mcp-server is doing the same for the persons’ accounts.

Build an MCP Server with Quarkus

Both MCP server applications are similar. They connect to the H2 database via the Quarkus Panache ORM extension. Both provide MCP API via Server-Sent Events (SSE) transport. Here’s a list of required Maven dependencies:

<dependencies>
  <dependency>
    <groupId>io.quarkiverse.mcp</groupId>
    <artifactId>quarkus-mcp-server-sse</artifactId>
  </dependency>
  <dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-hibernate-orm-panache</artifactId>
  </dependency>
  <dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-jdbc-h2</artifactId>
  </dependency>
  <dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-junit5</artifactId>
    <scope>test</scope>
  </dependency>
</dependencies>


  
    io.quarkiverse.mcp
    quarkus-mcp-server-sse
  
  
    io.quarkus
    quarkus-hibernate-orm-panache
  
  
    io.quarkus
    quarkus-jdbc-h2
  
  
    io.quarkus
    quarkus-junit5
    test

XML

Let’s start with the person-mcp-server application. Here’s the @Entity class for interacting with the person table. It uses Panache support to avoid the need for getter and setter declarations.

@Entity
public class Person extends PanacheEntityBase {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    public Long id;
    public String firstName;
    public String lastName;
    public int age;
    public String nationality;
    @Enumerated(EnumType.STRING)
    public Gender gender;

}

@Entity
public class Person extends PanacheEntityBase {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    public Long id;
    public String firstName;
    public String lastName;
    public int age;
    public String nationality;
    @Enumerated(EnumType.STRING)
    public Gender gender;

}

Java

The PersonRepository class contains a single method for searching persons by their nationality:

@ApplicationScoped
public class PersonRepository implements PanacheRepository<Person> {

    public List<Person> findByNationality(String nationality) {
        return find("nationality", nationality).list();
    }

}

@ApplicationScoped
public class PersonRepository implements PanacheRepository<Person> {

    public List<Person> findByNationality(String nationality) {
        return find("nationality", nationality).list();
    }

}

Java

Next, prepare the “tools service” that searches for a single person by ID or a list of people of a given nationality in the database. Each method must be annotated with @Tool and include a description in the description field. Quarkus LangChain4j does not allow a Java List to be returned, so we need to wrap it using a dedicated Persons object.

@ApplicationScoped
public class PersonTools {

    PersonRepository personRepository;

    public PersonTools(PersonRepository personRepository) {
        this.personRepository = personRepository;
    }

    @Tool(description = "Find person by ID")
    public Person getPersonById(
            @ToolArg(description = "Person ID") Long id) {
        return personRepository.findById(id);
    }

    @Tool(description = "Find all persons by nationality")
    public Persons getPersonsByNationality(
            @ToolArg(description = "Nationality") String nationality) {
        return new Persons(personRepository.findByNationality(nationality));
    }
}

@ApplicationScoped
public class PersonTools {

    PersonRepository personRepository;

    public PersonTools(PersonRepository personRepository) {
        this.personRepository = personRepository;
    }

    @Tool(description = "Find person by ID")
    public Person getPersonById(
            @ToolArg(description = "Person ID") Long id) {
        return personRepository.findById(id);
    }

    @Tool(description = "Find all persons by nationality")
    public Persons getPersonsByNationality(
            @ToolArg(description = "Nationality") String nationality) {
        return new Persons(personRepository.findByNationality(nationality));
    }
}

Java

Here’s our List wrapper:

public class Persons {

    private List<Person> persons;

    public Persons(List<Person> persons) {
        this.persons = persons;
    }

    public List<Person> getPersons() {
        return persons;
    }

    public void setPersons(List<Person> persons) {
        this.persons = persons;
    }
}

public class Persons {

    private List<Person> persons;

    public Persons(List<Person> persons) {
        this.persons = persons;
    }

    public List<Person> getPersons() {
        return persons;
    }

    public void setPersons(List<Person> persons) {
        this.persons = persons;
    }
}

Java

The implementation of the account-mcp-server application is essentially very similar. Here’s the @Entity class for interacting with the account table. It uses Panache support to avoid the need for getter and setter declarations.

@Entity
public class Account extends PanacheEntityBase {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    public Long id;
    public String number;
    public int balance;
    public Long personId;

}

@Entity
public class Account extends PanacheEntityBase {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    public Long id;
    public String number;
    public int balance;
    public Long personId;

}

Java

The AccountRepository class contains a single method for searching accounts by person ID:

@ApplicationScoped
public class AccountRepository implements PanacheRepository<Account> {

    public List<Account> findByPersonId(Long personId) {
        return find("personId", personId).list();
    }

}

@ApplicationScoped
public class AccountRepository implements PanacheRepository<Account> {

    public List<Account> findByPersonId(Long personId) {
        return find("personId", personId).list();
    }

}

Java

Once again, the list inside the “tools service” must be wrapped by the dedicated object. The single method annotated with @Tool returns a list of accounts assigned to a given person.

@ApplicationScoped
public class AccountTools {

    AccountRepository accountRepository;

    public AccountTools(AccountRepository accountRepository) {
        this.accountRepository = accountRepository;
    }

    @Tool(description = "Find all accounts by person ID")
    public Accounts getAccountsByPersonId(
            @ToolArg(description = "Person ID") Long personId) {
        return new Accounts(accountRepository.findByPersonId(personId));
    }

}

@ApplicationScoped
public class AccountTools {

    AccountRepository accountRepository;

    public AccountTools(AccountRepository accountRepository) {
        this.accountRepository = accountRepository;
    }

    @Tool(description = "Find all accounts by person ID")
    public Accounts getAccountsByPersonId(
            @ToolArg(description = "Person ID") Long personId) {
        return new Accounts(accountRepository.findByPersonId(personId));
    }

}

Java

The person-mcp-server starts on port 8082, while the account-mcp-server listens on port 8081. To change the default HTTP, use the quarkus.http.port property in your application.properties file.

Build an MCP Client with Quarkus

Our application interacts with the OpenAI chat model, so we must include the Quarkus LangChain4j OpenAI extension. In turn, to integrate the client-side application with MCP-compliant servers, we need to include the quarkus-langchain4j-mcp extension.

<dependencies>
  <dependency>
    <groupId>io.quarkus</groupId>
    <artifactId>quarkus-rest-jackson</artifactId>
  </dependency>
  <dependency>
    <groupId>io.quarkiverse.langchain4j</groupId>
    <artifactId>quarkus-langchain4j-openai</artifactId>
    <version>${quarkus.langchain4j.version}</version>
  </dependency>
  <dependency>
    <groupId>io.quarkiverse.langchain4j</groupId>
    <artifactId>quarkus-langchain4j-mcp</artifactId>
    <version>${quarkus.langchain4j.version}</version>
  </dependency>
</dependencies>


  
    io.quarkus
    quarkus-rest-jackson
  
  
    io.quarkiverse.langchain4j
    quarkus-langchain4j-openai
    ${quarkus.langchain4j.version}
  
  
    io.quarkiverse.langchain4j
    quarkus-langchain4j-mcp
    ${quarkus.langchain4j.version}

XML

The sample-client Quarkus app interacts with both the person-mcp-server app and the account-mcp-server app. Therefore, it defines two AI services. As with a standard AI application in Quarkus, those services must be annotated with @RegisterAiService. Then we define methods and prompt templates, also with annotations @UserMessage or @SystemMessage. If a given method is to use one of the MCP servers, it must be annotated with @McpToolBox. The name inside the annotation corresponds to the MCP server name set in the configuration properties. The PersonService AI service visible below uses the person-service MCP server.

@ApplicationScoped
@RegisterAiService
public interface PersonService {

    @SystemMessage("""
        You are a helpful assistant that generates realistic person data.
        Always respond with valid JSON format.
        """)
    @UserMessage("""
        Find persons with {nationality} nationality.
        Output **only valid JSON**, no explanations, no markdown, no ```json blocks.
        """)
    @McpToolBox("person-service")
    Persons findByNationality(String nationality);

    @SystemMessage("""
        You are a helpful assistant that generates realistic person data.
        Always respond with valid JSON format.
        """)
    @UserMessage("How many persons come from {nationality} ?")
    @McpToolBox("person-service")
    int countByNationality(String nationality);

}

@ApplicationScoped
@RegisterAiService
public interface PersonService {

    @SystemMessage("""
        You are a helpful assistant that generates realistic person data.
        Always respond with valid JSON format.
        """)
    @UserMessage("""
        Find persons with {nationality} nationality.
        Output **only valid JSON**, no explanations, no markdown, no ```json blocks.
        """)
    @McpToolBox("person-service")
    Persons findByNationality(String nationality);

    @SystemMessage("""
        You are a helpful assistant that generates realistic person data.
        Always respond with valid JSON format.
        """)
    @UserMessage("How many persons come from {nationality} ?")
    @McpToolBox("person-service")
    int countByNationality(String nationality);

}

Java

The AI service shown above corresponds to this configuration. You need to specify the MCP server’s name and address. As you remember, person-mcp-server listens on port 8082. The client application uses the name person-service here, and the standard endpoint to MCP SSE for Quarkus is /mcp/sse. To explore the solution itself, it is also worth enabling logging of MCP requests and responses.

quarkus.langchain4j.mcp.person-service.transport-type = http
quarkus.langchain4j.mcp.person-service.url = http://localhost:8082/mcp/sse
quarkus.langchain4j.mcp.person-service.log-requests = true
quarkus.langchain4j.mcp.person-service.log-responses = true

quarkus.langchain4j.mcp.person-service.transport-type = http
quarkus.langchain4j.mcp.person-service.url = http://localhost:8082/mcp/sse
quarkus.langchain4j.mcp.person-service.log-requests = true
quarkus.langchain4j.mcp.person-service.log-responses = true

Plaintext

Here is a similar implementation for the AccountService AI service. It interacts with the MCP server configured under the account-service name.

@ApplicationScoped
@RegisterAiService
public interface AccountService {

    @SystemMessage("""
        You are a helpful assistant that generates realistic data.
        Return a single number.
        """)
    @UserMessage("How many accounts has person with {personId} ID ?")
    @McpToolBox("account-service")
    int countByPersonId(int personId);

    @UserMessage("""
        How many accounts has person with {personId} ID ?
        Return person name, nationality and a total balance on his/her accounts.
        """)
    @McpToolBox("account-service")
    String balanceByPersonId(int personId);

}

@ApplicationScoped
@RegisterAiService
public interface AccountService {

    @SystemMessage("""
        You are a helpful assistant that generates realistic data.
        Return a single number.
        """)
    @UserMessage("How many accounts has person with {personId} ID ?")
    @McpToolBox("account-service")
    int countByPersonId(int personId);

    @UserMessage("""
        How many accounts has person with {personId} ID ?
        Return person name, nationality and a total balance on his/her accounts.
        """)
    @McpToolBox("account-service")
    String balanceByPersonId(int personId);

}

Java

Here’s the corresponding configuration for that service. No surprises.

quarkus.langchain4j.mcp.account-service.transport-type = http
quarkus.langchain4j.mcp.account-service.url = http://localhost:8081/mcp/sse
quarkus.langchain4j.mcp.account-service.log-requests = true
quarkus.langchain4j.mcp.account-service.log-responses = true

quarkus.langchain4j.mcp.account-service.transport-type = http
quarkus.langchain4j.mcp.account-service.url = http://localhost:8081/mcp/sse
quarkus.langchain4j.mcp.account-service.log-requests = true
quarkus.langchain4j.mcp.account-service.log-responses = true

Plaintext

Finally, we must provide some configuration to integrate our Quarkus application with Open AI chat model. It assumes that Open AI token is available as the OPEN_AI_TOKEN environment variable.

quarkus.langchain4j.chat-model.provider = openai
quarkus.langchain4j.log-requests = true
quarkus.langchain4j.log-responses = true
quarkus.langchain4j.openai.api-key = ${OPEN_AI_TOKEN}
quarkus.langchain4j.openai.timeout = 20s

quarkus.langchain4j.chat-model.provider = openai
quarkus.langchain4j.log-requests = true
quarkus.langchain4j.log-responses = true
quarkus.langchain4j.openai.api-key = ${OPEN_AI_TOKEN}
quarkus.langchain4j.openai.timeout = 20s

Plaintext

We can test individual AI services by calling endpoints provided by the client-side application. There are two endpoints GET /count-by-person-id/{personId} and GET /balance-by-person-id/{personId} that use LLM prompts to calculate number of persons and a total balances amount of all accounts belonging to a given person.

@Path("/accounts")
public class AccountResource {

    private final AccountService accountService;

    public AccountResource(AccountService accountService) {
        this.accountService = accountService;
    }

    @POST
    @Path("/count-by-person-id/{personId}")
    public int countByPersonId(int personId) {
        return accountService.countByPersonId(personId);
    }

    @POST
    @Path("/balance-by-person-id/{personId}")
    public String balanceByPersonId(int personId) {
        return accountService.balanceByPersonId(personId);
    }

}

@Path("/accounts")
public class AccountResource {

    private final AccountService accountService;

    public AccountResource(AccountService accountService) {
        this.accountService = accountService;
    }

    @POST
    @Path("/count-by-person-id/{personId}")
    public int countByPersonId(int personId) {
        return accountService.countByPersonId(personId);
    }

    @POST
    @Path("/balance-by-person-id/{personId}")
    public String balanceByPersonId(int personId) {
        return accountService.balanceByPersonId(personId);
    }

}

Java

MCP for Promps

MCP servers can also provide other functionalities beyond just tools. Let’s go back to the person-mcp-server app for a moment. To share a prompt message, you can create a class that defines methods returning the PromptMessage object. Then, we must annotate such methods with @Prompt, and their arguments with @PromptArg.

@ApplicationScoped
public class PersonPrompts {

    final String findByNationalityPrompt = """
        Find persons with {nationality} nationality.
        Output **only valid JSON**, no explanations, no markdown, no ```json blocks.
        """;

    @Prompt(description = "Find by nationality.")
    PromptMessage findByNationalityPrompt(@PromptArg(description = "The nationality") String nationality) {
        return PromptMessage.withUserRole(new TextContent(findByNationalityPrompt));
    }

}

@ApplicationScoped
public class PersonPrompts {

    final String findByNationalityPrompt = """
        Find persons with {nationality} nationality.
        Output **only valid JSON**, no explanations, no markdown, no ```json blocks.
        """;

    @Prompt(description = "Find by nationality.")
    PromptMessage findByNationalityPrompt(@PromptArg(description = "The nationality") String nationality) {
        return PromptMessage.withUserRole(new TextContent(findByNationalityPrompt));
    }

}

Java

Once we start the application, we can use Quarkus Dev UI to verify a list of provided tools and prompts.

Client-side integration with MCP prompts is a bit more complex than with tools. We must inject the McpClient to a resource controller to load a given prompt programmatically using its name.

@Path("/persons")
public class PersonResource {

    @McpClientName("person-service")
    McpClient mcpClient;
    
    // OTHET METHODS...
    
    @POST
    @Path("/nationality-with-prompt/{nationality}")
    public List<Person> findByNationalityWithPrompt(String nationality) {
        Persons p = personService.findByNationalityWithPrompt(loadPrompt(nationality), nationality);
        return p.getPersons();
    }
    
    private String loadPrompt(String nationality) {
        McpGetPromptResult prompt = mcpClient.getPrompt("findByNationalityPrompt", Map.of("nationality", nationality));
        return ((TextContent) prompt.messages().getFirst().content().toContent()).text();
    }
}

@Path("/persons")
public class PersonResource {

    @McpClientName("person-service")
    McpClient mcpClient;
    
    // OTHET METHODS...
    
    @POST
    @Path("/nationality-with-prompt/{nationality}")
    public List<Person> findByNationalityWithPrompt(String nationality) {
        Persons p = personService.findByNationalityWithPrompt(loadPrompt(nationality), nationality);
        return p.getPersons();
    }
    
    private String loadPrompt(String nationality) {
        McpGetPromptResult prompt = mcpClient.getPrompt("findByNationalityPrompt", Map.of("nationality", nationality));
        return ((TextContent) prompt.messages().getFirst().content().toContent()).text();
    }
}

Java

In this case, the Quarkus AI service should not define the @UserMessage on the entire method, but just as the method argument. Then a prompt message is loaded from the MCP server and filled with the nationality parameter value before sending to the AI model.

@ApplicationScoped
@RegisterAiService
public interface PersonService {

    // OTHER METHODS...
    
    @SystemMessage("""
        You are a helpful assistant that generates realistic person data.
        Always respond with valid JSON format.
        """)
    @McpToolBox("person-service")
    Persons findByNationalityWithPrompt(@UserMessage String userMessage, String nationality);

}

@ApplicationScoped
@RegisterAiService
public interface PersonService {

    // OTHER METHODS...
    
    @SystemMessage("""
        You are a helpful assistant that generates realistic person data.
        Always respond with valid JSON format.
        """)
    @McpToolBox("person-service")
    Persons findByNationalityWithPrompt(@UserMessage String userMessage, String nationality);

}

Java

Testing MCP Tools with Quarkus

Quarkus provides a dedicated module for testing MCP tools. We can use after including the following dependency in the Maven pom.xml:

<dependency>
  <groupId>io.quarkiverse.mcp</groupId>
  <artifactId>quarkus-mcp-server-test</artifactId>
  <scope>test</scope>
</dependency>


  io.quarkiverse.mcp
  quarkus-mcp-server-test
  test

XML

The following test verifies the MCP tool methods provided by the person-mcp-server application. The McpAssured class allows us to use SSE, streamable, and WebSocket test clients. To create a new client for SSE, invoke the newConnectedSseClient() static method. After that, we can use one of several available variants of the toolsCall(...) method to verify the response returned by a given @Tool.

@QuarkusTest
public class PersonToolsTest {

    ObjectMapper mapper = new ObjectMapper();

    @Test
    public void testGetPersonsByNationality() {
        McpAssured.McpSseTestClient client = McpAssured.newConnectedSseClient();
        client.when()
                .toolsCall("getPersonsByNationality", Map.of("nationality", "Denmark"),
                        r -> {
                            try {
                                Persons p = mapper.readValue(r.content().getFirst().asText().text(), Persons.class);
                                assertFalse(p.getPersons().isEmpty());
                            } catch (JsonProcessingException e) {
                                throw new RuntimeException(e);
                            }
                        })
                .thenAssertResults();
    }

    @Test
    public void testGetPersonById() {
        McpAssured.McpSseTestClient client = McpAssured.newConnectedSseClient();
        client.when()
                .toolsCall("getPersonById", Map.of("id", 10),
                        r -> {
                            try {
                                Person p = mapper.readValue(r.content().getFirst().asText().text(), Person.class);
                                assertNotNull(p);
                                assertNotNull(p.id);
                            } catch (JsonProcessingException e) {
                                throw new RuntimeException(e);
                            }
                        })
                .thenAssertResults();
    }
}

@QuarkusTest
public class PersonToolsTest {

    ObjectMapper mapper = new ObjectMapper();

    @Test
    public void testGetPersonsByNationality() {
        McpAssured.McpSseTestClient client = McpAssured.newConnectedSseClient();
        client.when()
                .toolsCall("getPersonsByNationality", Map.of("nationality", "Denmark"),
                        r -> {
                            try {
                                Persons p = mapper.readValue(r.content().getFirst().asText().text(), Persons.class);
                                assertFalse(p.getPersons().isEmpty());
                            } catch (JsonProcessingException e) {
                                throw new RuntimeException(e);
                            }
                        })
                .thenAssertResults();
    }

    @Test
    public void testGetPersonById() {
        McpAssured.McpSseTestClient client = McpAssured.newConnectedSseClient();
        client.when()
                .toolsCall("getPersonById", Map.of("id", 10),
                        r -> {
                            try {
                                Person p = mapper.readValue(r.content().getFirst().asText().text(), Person.class);
                                assertNotNull(p);
                                assertNotNull(p.id);
                            } catch (JsonProcessingException e) {
                                throw new RuntimeException(e);
                            }
                        })
                .thenAssertResults();
    }
}

Java

Running Quarkus Applications

Finally, let’s run all our quarkus applications. Go to the mcp/account-mcp-server directory and run the application in development mode:

$ cd mcp/account-mcp-server
$ mvn quarkus:dev

$ cd mcp/account-mcp-server
$ mvn quarkus:dev

ShellSession

Then do the same for the person-mcp-server application.

$ cd mcp/person-mcp-server
$ mvn quarkus:dev

$ cd mcp/person-mcp-server
$ mvn quarkus:dev

ShellSession

Before running the last sample-client application, export the OpenAI API token as the OPEN_AI_TOKEN environment variable.

$ cd mcp/sample-client
$ export OPEN_AI_TOKEN=<YOUR_OPENAI_TOKEN>
$ mvn quarkus:dev

$ cd mcp/sample-client
$ export OPEN_AI_TOKEN=<YOUR_OPENAI_TOKEN>
$ mvn quarkus:dev

ShellSession

We can verify a list of tools or prompts exposed by each MCP server application by visiting its Quarkus Dev UI console. It provides a dedicated “MCP Server tile.”

Here’s a list of tools provided by the person-mcp-server app via Quarkus Dev UI.

Then, we can switch to the sample-client Dev UI console. We can verify and test all interactions with the MCP servers from our client-side app.

Once all the sample applications are running, we can test the MCP communication by calling the HTTP endpoints exposed by the sample-client app. Both person-mcp-server and account-mcp-server load some test data on startup using the import.sql file. Here are the test API calls for all the REST endpoints.

$ curl -X POST http://localhost:8080/persons/nationality/Denmark
$ curl -X POST http://localhost:8080/persons/count-by-nationality/Denmark
$ curl -X POST http://localhost:8080/persons/nationality-with-prompt/Denmark
$ curl -X POST http://localhost:8080/accounts/count-by-person-id/2
$ curl -X POST http://localhost:8080/accounts/balance-by-person-id/2

$ curl -X POST http://localhost:8080/persons/nationality/Denmark
$ curl -X POST http://localhost:8080/persons/count-by-nationality/Denmark
$ curl -X POST http://localhost:8080/persons/nationality-with-prompt/Denmark
$ curl -X POST http://localhost:8080/accounts/count-by-person-id/2
$ curl -X POST http://localhost:8080/accounts/balance-by-person-id/2

ShellSession

Conclusion

With Quarkus, creating applications that use MCP is not difficult. If you understand the idea of tool calling in AI, understanding the MCP-based approach is not difficult for you. This article shows you how to connect your application to several MCP servers, implement tests to verify the elements shared by a given application using MCP, and support on the Quarkus Dev UI side.

The post MCP with Quarkus LangChain4j appeared first on Piotr's TechBlog.

AI Tool Calling with Quarkus LangChain4j

piotr.minkowski — Mon, 23 Jun 2025 05:19:14 +0000

This article will show you how to use Quarkus LangChain4j AI support with the most popular chat models for the “tool calling” feature. Tool calling (sometimes referred to as function calling) is a typical pattern in AI applications that enables a model to interact with APIs or tools, extending its capabilities. The most popular AI models are trained to know when to call a function. The Quarkus LangChain4j extension offers built-in support for tool calling. In this article, you will learn how to define tool methods to get data from the third-party APIs and the internal database.

This article is the second part of a series describing some of the Quarkus AI project’s most notable features. Before reading on, I recommend checking out my introduction to Quarkus LangChain4j, which is available here. The first part describes such features as prompts, structured output, and chat memory. There is also a similar tutorial series about Spring AI. You can compare Quarkus support for tool calling described here with a similar Spring AI support described in the following post.

Source Code

Feel free to use my source code if you’d like to try it out yourself. To do that, you must clone my sample GitHub repository. Then you should only follow my instructions.

Tool Calling Motivation

For ease of comparison, this article will implement an identical scenario to an analogous application written in Spring AI. You can find a GitHub sample repository with the Spring AI app here. As you know, the “tool calling” feature helps us solve a common AI model challenge related to internal or live data sources. If we want to augment a model with such data, our applications must allow it to interact with a set of APIs or tools. In our case, the internal database (H2) contains information about the structure of our stock wallet. The sample Quarkus application asks an AI model about the total value of the wallet based on daily stock prices or the highest value for the last few days. The model must retrieve the structure of our stock wallet and the latest stock prices.

Use Tool Calling with Quarkus LangChain4j

Create ShareTools

Let’s begin with the ShareTools implementation, which is responsible for getting a list of the wallet’s shares from a database. It defines a single method annotated with @Tool. The most crucial element here is to provide a clear description of the method within the @Tool annotation. It allows the AI model to understand the function’s responsibilities. The method returns the number of shares for each company in our portfolio. It is retrieved from the database through the Quarkus Panache ORM repository.

@ApplicationScoped
public class ShareTools {

    private ShareRepository shareRepository;

    public ShareTools(ShareRepository shareRepository) {
        this.shareRepository = shareRepository;
    }

    @Tool("Return number of shares for each company in my wallet")
    public List<Share> getNumberOfShares() {
        return shareRepository.findAll().list();
    }
}

@ApplicationScoped
public class ShareTools {

    private ShareRepository shareRepository;

    public ShareTools(ShareRepository shareRepository) {
        this.shareRepository = shareRepository;
    }

    @Tool("Return number of shares for each company in my wallet")
    public List<Share> getNumberOfShares() {
        return shareRepository.findAll().list();
    }
}

Java

The sample application launches an embedded, in-memory database and inserts test data into the stock table. Our wallet contains the most popular companies on the U.S. stock market, including Amazon, Meta, and Microsoft. Here’s a dataset inserted on application startup.

insert into share(id, company, quantity) values (1, 'AAPL', 100);
insert into share(id, company, quantity) values (2, 'AMZN', 300);
insert into share(id, company, quantity) values (3, 'META', 300);
insert into share(id, company, quantity) values (4, 'MSFT', 400);

insert into share(id, company, quantity) values (1, 'AAPL', 100);
insert into share(id, company, quantity) values (2, 'AMZN', 300);
insert into share(id, company, quantity) values (3, 'META', 300);
insert into share(id, company, quantity) values (4, 'MSFT', 400);

SQL

Create StockTools

The StockTools The class is responsible for interaction with the TwelveData stock API. It defines two methods. The getLatestStockPrices method returns only the latest close price for a specified company. It is a tool calling version of the method provided within the pl.piomin.services.functions.stock.StockService function. The second method is more complicated. It must return historical daily close prices for a defined number of days. Each price must be correlated with a quotation date.

@ApplicationScoped
public class StockTools {

    private Logger log;
    private StockDataClient stockDataClient;

    public StockTools(@RestClient StockDataClient stockDataClient, Logger log) {
        this.stockDataClient = stockDataClient;
        this.log = log;
    }
    
    @ConfigProperty(name = "STOCK_API_KEY", defaultValue = "none")
    String apiKey;

    @Tool("Return latest stock prices for a given company")
    public StockResponse getLatestStockPrices(String company) {
        log.infof("Get stock prices for: %s", company);
        StockData data = stockDataClient.getStockData(company, apiKey, "1min", 1);
        DailyStockData latestData = data.getValues().get(0);
        log.infof("Get stock prices (%s) -> %s", company, latestData.getClose());
        return new StockResponse(Float.parseFloat(latestData.getClose()));
    }

    @Tool("Return historical daily stock prices for a given company")
    public List<DailyShareQuote> getHistoricalStockPrices(String company, int days) {
        log.infof("Get historical stock prices: %s for %d days", company, days);
        StockData data = stockDataClient.getStockData(company, apiKey, "1min", days);
        return data.getValues().stream()
                .map(d -> new DailyShareQuote(company, Float.parseFloat(d.getClose()), d.getDatetime()))
                .toList();
    }

}

@ApplicationScoped
public class StockTools {

    private Logger log;
    private StockDataClient stockDataClient;

    public StockTools(@RestClient StockDataClient stockDataClient, Logger log) {
        this.stockDataClient = stockDataClient;
        this.log = log;
    }
    
    @ConfigProperty(name = "STOCK_API_KEY", defaultValue = "none")
    String apiKey;

    @Tool("Return latest stock prices for a given company")
    public StockResponse getLatestStockPrices(String company) {
        log.infof("Get stock prices for: %s", company);
        StockData data = stockDataClient.getStockData(company, apiKey, "1min", 1);
        DailyStockData latestData = data.getValues().get(0);
        log.infof("Get stock prices (%s) -> %s", company, latestData.getClose());
        return new StockResponse(Float.parseFloat(latestData.getClose()));
    }

    @Tool("Return historical daily stock prices for a given company")
    public List<DailyShareQuote> getHistoricalStockPrices(String company, int days) {
        log.infof("Get historical stock prices: %s for %d days", company, days);
        StockData data = stockDataClient.getStockData(company, apiKey, "1min", days);
        return data.getValues().stream()
                .map(d -> new DailyShareQuote(company, Float.parseFloat(d.getClose()), d.getDatetime()))
                .toList();
    }

}

Java

Here’s the DailyShareQuote Java record returned in the response list.

public record DailyShareQuote(String company, float price, String datetime) {
}

public record DailyShareQuote(String company, float price, String datetime) {
}

Java

Here’s a @RestClient responsible for calling the TwelveData stock API.

@RegisterRestClient(configKey = "stock-api")
public interface StockDataClient {

    @GET
    @Path("/time_series")
    StockData getStockData(@RestQuery String symbol,
                           @RestQuery String apikey,
                           @RestQuery String interval,
                           @RestQuery int outputsize);
}

@RegisterRestClient(configKey = "stock-api")
public interface StockDataClient {

    @GET
    @Path("/time_series")
    StockData getStockData(@RestQuery String symbol,
                           @RestQuery String apikey,
                           @RestQuery String interval,
                           @RestQuery int outputsize);
}

Java

For the demo, you can easily enable complete logging of both communication with the AI model through LangChain4j and with the stock API via @RestClient.

quarkus.langchain4j.log-requests = true
quarkus.langchain4j.log-responses = true
quarkus.rest-client.stock-api.url = https://api.twelvedata.com
quarkus.rest-client.logging.scope = request-response
quarkus.rest-client.stock-api.scope = all
%dev.quarkus.log.category."org.jboss.resteasy.reactive.client.logging".level = DEBUG

quarkus.langchain4j.log-requests = true
quarkus.langchain4j.log-responses = true
quarkus.rest-client.stock-api.url = https://api.twelvedata.com
quarkus.rest-client.logging.scope = request-response
quarkus.rest-client.stock-api.scope = all
%dev.quarkus.log.category."org.jboss.resteasy.reactive.client.logging".level = DEBUG

Plaintext

Quarkus LangChain4j Tool Calling Flow

You can easily register @Tools on your Quarkus AI service with the tools argument inside the @RegisterAiService annotation. The calculateWalletValueWithTools() method calculates the value of our stock wallet in dollars. It uses the latest daily stock prices for each company’s shares from the wallet. Since this method directly returns the response received from the AI model, it is essential to perform additional validation of the content received. For this purpose, a so-called guardrail should be implemented and set in place. We can easily achieve it with the @OutputGuardrails annotation. The calculateHighestWalletValue method calculates the value of our stock wallet in dollars for each day in the specified period determined by the days variable. Then it must return the day with the highest stock wallet value.

@RegisterAiService(tools = {StockTools.class, ShareTools.class})
public interface WalletAiService {

    @UserMessage("""
    What’s the current value in dollars of my wallet based on the latest stock daily prices ?
    
    Return subtotal value in dollars for each company in my wallet.
    In the end, return the total value in dollars wrapped by ***.
    """)
    @OutputGuardrails(WalletGuardrail.class)
    String calculateWalletValueWithTools();

    @UserMessage("""
    On which day during last {days} days my wallet had the highest value in dollars based on the historical daily stock prices ?
    """)
    String calculateHighestWalletValue(int days);
}

@RegisterAiService(tools = {StockTools.class, ShareTools.class})
public interface WalletAiService {

    @UserMessage("""
    What’s the current value in dollars of my wallet based on the latest stock daily prices ?
    
    Return subtotal value in dollars for each company in my wallet.
    In the end, return the total value in dollars wrapped by ***.
    """)
    @OutputGuardrails(WalletGuardrail.class)
    String calculateWalletValueWithTools();

    @UserMessage("""
    On which day during last {days} days my wallet had the highest value in dollars based on the historical daily stock prices ?
    """)
    String calculateHighestWalletValue(int days);
}

Java

Here’s the implementation of the guardrail that validates the response returned by the calculateWalletValueWithTools method. It verifies if the total value in dollars is wrapped by *** and starts with the $ sign.

@ApplicationScoped
public class WalletGuardrail implements OutputGuardrail {

    Pattern pattern = Pattern.compile("\\*\\*\\*(.*?)\\*\\*\\*");

    private Logger log;
    
    public WalletGuardrail(Logger log) {
        this.log = log;
    }

    @Override
    public OutputGuardrailResult validate(AiMessage responseFromLLM) {
        try {
            Matcher matcher = pattern.matcher(responseFromLLM.text());
            if (matcher.find()) {
                String amount = matcher.group(1);
                log.infof("Extracted amount: %s", amount);
                if (amount.startsWith("$")) {
                    return success();
                }
            }
        } catch (Exception e) {
            return reprompt("Invalid text format", e, "Make sure you return a valid requested text");
        }
        return failure("Total amount not found");
    }
}

@ApplicationScoped
public class WalletGuardrail implements OutputGuardrail {

    Pattern pattern = Pattern.compile("\\*\\*\\*(.*?)\\*\\*\\*");

    private Logger log;
    
    public WalletGuardrail(Logger log) {
        this.log = log;
    }

    @Override
    public OutputGuardrailResult validate(AiMessage responseFromLLM) {
        try {
            Matcher matcher = pattern.matcher(responseFromLLM.text());
            if (matcher.find()) {
                String amount = matcher.group(1);
                log.infof("Extracted amount: %s", amount);
                if (amount.startsWith("$")) {
                    return success();
                }
            }
        } catch (Exception e) {
            return reprompt("Invalid text format", e, "Make sure you return a valid requested text");
        }
        return failure("Total amount not found");
    }
}

Java

Here’s the REST endpoints implementation. It uses the WalletAiService bean to interact with the AI model. It exposes two endpoints: GET /wallet/with-tools and GET /wallet/highest-day/{days}.

@Path("/wallet")
@Produces(MediaType.TEXT_PLAIN)
public class WalletController {

    private final WalletAiService walletAiService;

    public WalletController(WalletAiService walletAiService) {
        this.walletAiService = walletAiService;
    }

    @GET
    @Path("/with-tools")
    public String calculateWalletValueWithTools() {
        return walletAiService.calculateWalletValueWithTools();
    }

    @GET
    @Path("/highest-day/{days}")
    public String calculateHighestWalletValue(int days) {
        return walletAiService.calculateHighestWalletValue(days);
    }

}

@Path("/wallet")
@Produces(MediaType.TEXT_PLAIN)
public class WalletController {

    private final WalletAiService walletAiService;

    public WalletController(WalletAiService walletAiService) {
        this.walletAiService = walletAiService;
    }

    @GET
    @Path("/with-tools")
    public String calculateWalletValueWithTools() {
        return walletAiService.calculateWalletValueWithTools();
    }

    @GET
    @Path("/highest-day/{days}")
    public String calculateHighestWalletValue(int days) {
        return walletAiService.calculateHighestWalletValue(days);
    }

}

Java

The following diagram illustrates the flow for the second use case, which returns the day with the highest stock wallet value. First, it must connect to the database and retrieve the stock wallet structure, which contains the number of shares for each company. Then, it must call the stock API for every company found in the wallet. So, finally, the method calculateHighestWalletValue should be called four times with different values of the company name parameter and a value of the days determined by the HTTP endpoint path variable. Once all the data is collected, the AI model calculates the highest wallet value and returns it together with the quotation date.

Automated Testing

Most of my repositories are automatically updated to the latest versions of libraries. After updating the library version, automated tests are run to verify that everything works as expected. To verify the correctness of today’s scenario, we will mock stock API calls while integrating with the actual OpenAI service. To mock API calls, you can use the quarkus-junit5-mockito extension.

<dependency>
  <groupId>io.quarkus</groupId>
  <artifactId>quarkus-junit5</artifactId>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>io.quarkus</groupId>
  <artifactId>quarkus-junit5-mockito</artifactId>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>io.rest-assured</groupId>
  <artifactId>rest-assured</artifactId>
  <scope>test</scope>
</dependency>


  io.quarkus
  quarkus-junit5
  test


  io.quarkus
  quarkus-junit5-mockito
  test


  io.rest-assured
  rest-assured
  test

XML

The following JUnit test verifies two endpoints exposed by WalletController. As you may remember, there is also an output guardrail set on the AI service called by the GET /wallet/with-tools endpoint.

@QuarkusTest
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
class WalletControllerTest {

    @InjectMock
    @RestClient
    StockDataClient stockDataClient;

    @BeforeEach
    void setUp() {
        // Mock the stock data responses
        StockData aaplStockData = createMockStockData("AAPL", "150.25");
        StockData amznStockData = createMockStockData("AMZN", "120.50");
        StockData metaStockData = createMockStockData("META", "250.75");
        StockData msftStockData = createMockStockData("MSFT", "300.00");

        // Mock the stock data client responses
        when(stockDataClient.getStockData(eq("AAPL"), anyString(), anyString(), anyInt()))
            .thenReturn(aaplStockData);
        when(stockDataClient.getStockData(eq("AMZN"), anyString(), anyString(), anyInt()))
            .thenReturn(amznStockData);
        when(stockDataClient.getStockData(eq("META"), anyString(), anyString(), anyInt()))
            .thenReturn(metaStockData);
        when(stockDataClient.getStockData(eq("MSFT"), anyString(), anyString(), anyInt()))
            .thenReturn(msftStockData);
    }

    private StockData createMockStockData(String symbol, String price) {
        DailyStockData dailyData = new DailyStockData();
        dailyData.setDatetime("2023-01-01");
        dailyData.setOpen(price);
        dailyData.setHigh(price);
        dailyData.setLow(price);
        dailyData.setClose(price);
        dailyData.setVolume("1000");

        StockData stockData = new StockData();
        stockData.setValues(List.of(dailyData));
        return stockData;
    }

    @Test
    @Order(1)
    void testCalculateWalletValueWithTools() {
        given()
          .when().get("/wallet/with-tools")
          .then().statusCode(200)
                 .contentType(ContentType.TEXT)
                 .body(notNullValue())
                 .body(not(emptyString()));
    }

    @Test
    @Order(2)
    void testCalculateHighestWalletValue() {
        given()
          .pathParam("days", 7)
          .when().get("/wallet/highest-day/{days}")
          .then().statusCode(200)
                 .contentType(ContentType.TEXT)
                 .body(notNullValue())
                 .body(not(emptyString()));
    }
}

@QuarkusTest
@TestMethodOrder(MethodOrderer.OrderAnnotation.class)
class WalletControllerTest {

    @InjectMock
    @RestClient
    StockDataClient stockDataClient;

    @BeforeEach
    void setUp() {
        // Mock the stock data responses
        StockData aaplStockData = createMockStockData("AAPL", "150.25");
        StockData amznStockData = createMockStockData("AMZN", "120.50");
        StockData metaStockData = createMockStockData("META", "250.75");
        StockData msftStockData = createMockStockData("MSFT", "300.00");

        // Mock the stock data client responses
        when(stockDataClient.getStockData(eq("AAPL"), anyString(), anyString(), anyInt()))
            .thenReturn(aaplStockData);
        when(stockDataClient.getStockData(eq("AMZN"), anyString(), anyString(), anyInt()))
            .thenReturn(amznStockData);
        when(stockDataClient.getStockData(eq("META"), anyString(), anyString(), anyInt()))
            .thenReturn(metaStockData);
        when(stockDataClient.getStockData(eq("MSFT"), anyString(), anyString(), anyInt()))
            .thenReturn(msftStockData);
    }

    private StockData createMockStockData(String symbol, String price) {
        DailyStockData dailyData = new DailyStockData();
        dailyData.setDatetime("2023-01-01");
        dailyData.setOpen(price);
        dailyData.setHigh(price);
        dailyData.setLow(price);
        dailyData.setClose(price);
        dailyData.setVolume("1000");

        StockData stockData = new StockData();
        stockData.setValues(List.of(dailyData));
        return stockData;
    }

    @Test
    @Order(1)
    void testCalculateWalletValueWithTools() {
        given()
          .when().get("/wallet/with-tools")
          .then().statusCode(200)
                 .contentType(ContentType.TEXT)
                 .body(notNullValue())
                 .body(not(emptyString()));
    }

    @Test
    @Order(2)
    void testCalculateHighestWalletValue() {
        given()
          .pathParam("days", 7)
          .when().get("/wallet/highest-day/{days}")
          .then().statusCode(200)
                 .contentType(ContentType.TEXT)
                 .body(notNullValue())
                 .body(not(emptyString()));
    }
}

Java

Tests can be automatically run, for example, by the CircleCI pipeline on each dependency update via the pull request.

Run the Application to Verify Tool Calling

Before starting the application, we must set environment variables with the AI model and stock API tokens.

$ export OPEN_AI_TOKEN=<YOUR_OPEN_AI_TOKEN>
$ export STOCK_API_KEY=<YOUR_STOCK_API_KEY>

$ export OPEN_AI_TOKEN=<YOUR_OPEN_AI_TOKEN>
$ export STOCK_API_KEY=<YOUR_STOCK_API_KEY>

ShellSession

Then, run the application in development mode with the following command:

mvn quarkus:dev

mvn quarkus:dev

ShellSession

Once the application is started, you can call the first endpoint. The GET /wallet/with-tools calculates the total least value of the stock wallet structure stored in the database.

curl http://localhost:8080/wallet/with-tools

curl http://localhost:8080/wallet/with-tools

ShellSession

You can see either the response from the chat AI model or the exception thrown after an unsuccessful validation using a guardrail. If LLM response validation fails, the REST endpoint returns the HTTP 500 code.

Here’s the successfully validated LLM response.

The sample Quarkus application logs the whole communication with the AI model. Here, you can see a first request containing a list of registered functions (tools) along with their descriptions.

Then we can call the GET /wallet/highest-day/{days} endpoint to return the day with the highest wallet value. Let’s calculate it for the last 7 days.

curl http://localhost:8080/wallet/highest-day/7

curl http://localhost:8080/wallet/highest-day/7

ShellSession

Here’s the response.

Finally, you can perform a similar test as before, but for the Mistral AI model. Before running the application, set your API token for Mistral AI and rename the default model to mistralai.

$ export MISTRAL_AI_TOKEN=<YOUR_MISTRAL_AI_TOKEN>
$ export AI_MODEL_PROVIDER=mistralai

$ export MISTRAL_AI_TOKEN=<YOUR_MISTRAL_AI_TOKEN>
$ export AI_MODEL_PROVIDER=mistralai

ShellSession

Then, run the sample Quarkus application with the following command and repeat the same “tool calling” tests as before.

mvn quarkus:dev -Pmistral-ai

mvn quarkus:dev -Pmistral-ai

ShellSession

Final Thoughts

Quarkus LangChain4j provides a seamless way to run tools in AI-powered conversations. You can register a tool by adding it as a part of the @RegisterAiService annotation. Also, you can easily add a guardrail on the selected AI service method. Tools are a vital part of agentic AI and the MCP concepts. It is therefore essential to understand it properly. You can expect more articles on Quarkus LangChain4j soon, including on MCP.

The post AI Tool Calling with Quarkus LangChain4j appeared first on Piotr's TechBlog.

Getting Started with Quarkus LangChain4j and Chat Model

piotr.minkowski — Wed, 18 Jun 2025 16:36:08 +0000

This article will teach you how to use the Quarkus LangChain4j project to build applications based on different chat models. The Quarkus AI Chat Model offers a portable and straightforward interface, enabling seamless interaction with these models. Our sample Quarkus application will switch between three popular chat models provided by OpenAI, Mistral AI, and Ollama. This article is the first in a series explaining AI concepts with Quarkus LangChain4j. Look for more on my blog in this area soon. The idea of this tutorial is very similar to the series on Spring AI. Therefore, you will be able to easily compare the two approaches, as the sample application will do the same thing as an analogous Spring Boot application.

If you like Quarkus, then you can find quite a few articles about it on my blog. Just go to the Quarkus category and find the topic you are interested in.

SourceCode

Feel free to use my source code if you’d like to try it out yourself. To do that, you must clone my sample GitHub repository. Then you should only follow my instructions.

Motivation

Whenever I create a new article or example related to AI, I like to define the problem I’m trying to solve. The problem this example solves is very trivial. I publish numerous small demo apps to explain complex technology concepts. These apps typically require data to display a demo output. Usually, I add demo data by myself or use a library like Datafaker to do it for me. This time, we can leverage the AI Chat Models API for that. Let’s begin!

The Quarkus-related topic I’m describing today, I also explained earlier for Spring Boot. For a comparison of the features offered by both frameworks for simple interaction with the AI chat model, you can read this article on Spring AI.

Dependencies

The sample application uses the current latest version of the Quarkus framework.


  
    
      io.quarkus.platform
      quarkus-bom
      ${quarkus.platform.version}
      pom
      import

XML

You can easily switch between multiple AI model implementations by activating a dedicated Maven profile. By default, the open-ai profile is active. It includes the quarkus-langchain4j-openai module in the Maven dependencies. You can also activate the mistral-ai and ollama profile. In that case, the quarkus-langchain4j-mistral-ai or quarkus-langchain4j-ollama module will be included instead of the LangChain4j OpenAI extension.


  
    open-ai
    
      true
    
    
      
        io.quarkiverse.langchain4j
        quarkus-langchain4j-openai
        ${quarkus-langchain4j.version}
      
    
  
  
    mistral-ai
    
      
        io.quarkiverse.langchain4j
        quarkus-langchain4j-mistral-ai
        ${quarkus-langchain4j.version}
      
    
  
  
    ollama
    
      
        io.quarkiverse.langchain4j
        quarkus-langchain4j-ollama
        ${quarkus-langchain4j.version}

XML

The sample Quarkus application is simple. It exposes some REST endpoints and communicates with a selected AI model to return an AI-generated response via each endpoint. So, you need to include only core Quarkus modules like quarkus-rest-jackson or quarkus-arc. To implement JUnit tests with REST API, it also includes the quarkus-junit5 and rest-assured modules in the test scope.


  
  
    io.quarkus
    quarkus-rest-jackson
  
  
    io.quarkus
    quarkus-arc
  

  
  
    io.quarkus
    quarkus-junit5
    test
  
  
    io.rest-assured
    rest-assured
    test

XML

Quarkus LangChain4j Chat Models Integration

Quarkus provides an innovative approach to interacting with AI chat models. First, you need to annotate your interface by defining AI-oriented methods with the @RegisterAiService annotation. Then you must add a proper description and input prompt inside the @SystemMessage and @UserMessage annotations. Here is the sample PersonAiService interaction, which defines two methods. The generatePersonList method aims to ask the AI model to generate a list of 10 unique persons in a form consistent with the input object structure. The getPersonById method must read the previously generated list from chat memory and return a person’s data with a specified id field.

@RegisterAiService
@ApplicationScoped
public interface PersonAiService {

    @SystemMessage("""
        You are a helpful assistant that generates realistic person data.
        Always respond with valid JSON format.
        """)
    @UserMessage("""
        Generate exactly 10 unique persons

        Requirements:
        - Each person must have a unique integer ID (like 1, 2, 3, etc.)
        - Use realistic first and last names per each nationality
        - Ages should be between 18 and 80
        - Return ONLY the JSON array, no additional text
        """)
    PersonResponse generatePersonList(@MemoryId int userId);

    @SystemMessage("""
        You are a helpful assistant that can recall generated person data from chat memory.
        """)
    @UserMessage("""
        In the previously generated list of persons for user {userId}, find and return the person with id {id}.
        
        Return ONLY the JSON object, no additional text.
        """)
    Person getPersonById(@MemoryId int userId, int id);

}

Java

There are a few more things to add regarding the code snippet above. The beans created by @RegisterAiService are @RequestScoped by default. The Quarkus LangChain4j documentation states that this is possible, allowing objects to be deleted from the chat memory. In the case seen above, the list of people is generated per user ID, which acts as the key by which we search the chat memory. To guarantee that the getPersonById method finds a list of persons generated per @MemoryId the PersonAiService interface must be annotated with @ApplicationScoped. The InMemoryChatMemoryStore implementation is enabled by default, so you don’t need to declare any additional beans to use it.

Quarkus LangChain4j can automatically map the LLM’s JSON response to the output POJO. However, until now, it has not been possible to map it directly to the output collection. Therefore, you must wrap the output list with the additional class, as shown below.

public class PersonResponse {

    private List<Person> persons;

    public List<Person> getPersons() {
        return persons;
    }

    public void setPersons(List<Person> persons) {
        this.persons = persons;
    }
}

Java

Here’s the Person class:

public class Person {

    private Integer id;
    private String firstName;
    private String lastName;
    private int age;
    private String nationality;
    private Gender gender;
    
    // GETTERS and SETTERS

}

Java

Finally, the last part of our implementation is REST endpoints. Here’s the REST controller that injects and uses PersonAiService to interact with the AI chat model. It exposes two endpoints: GET /api/{userId}/persons and GET /api/{userId}/persons/{id}. You can generate several lists of persons by specifying the userId path parameter.

@Path("/api")
@Produces(MediaType.APPLICATION_JSON)
@Consumes(MediaType.APPLICATION_JSON)
public class PersonController {

    private static final Logger LOG = Logger.getLogger(PersonController.class);

    PersonAiService personAiService;

    public PersonController(PersonAiService personAiService) {
        this.personAiService = personAiService;
    }

    @GET
    @Path("/{userId}/persons")
    public PersonResponse generatePersons(@PathParam("userId") int userId) {
        return personAiService.generatePersonList(userId);
    }

    @GET
    @Path("/{userId}/persons/{id}")
    public Person getPersonById(@PathParam("userId") int userId, @PathParam("id") int id) {
        return personAiService.getPersonById(userId, id);
    }

}

Java

Use Different AI Models with Quarkus LangChain4j

Configuration Properties

Here is a configuration defined within the application.properties file. Before proceeding, you must generate the OpenAI and Mistral AI API tokens and export them as environment variables. Additionally, you can enable logging of requests and responses in AI model communication. It is also worth increasing the default timeout for a single request from 10 seconds to a higher value, such as 20 seconds.

quarkus.langchain4j.chat-model.provider = ${AI_MODEL_PROVIDER:openai}
quarkus.langchain4j.log-requests = true
quarkus.langchain4j.log-responses = true

# OpenAI Configuration
quarkus.langchain4j.openai.api-key = ${OPEN_AI_TOKEN}
quarkus.langchain4j.openai.timeout = 20s

# Mistral AI Configuration
quarkus.langchain4j.mistralai.api-key = ${MISTRAL_AI_TOKEN}
quarkus.langchain4j.mistralai.timeout = 20s

# Ollama Configuration
quarkus.langchain4j.ollama.base-url = ${OLLAMA_BASE_URL:http://localhost:11434}

Plaintext

To run a sample Quarkus application and connect it with OpenAI, you must set the OPEN_AI_TOKEN environment variable. Since the open-ai Maven profile is activated by default, you don’t need to set anything else while running an app.

$ export OPEN_AI_TOKEN=<your_openai_token>
$ mvn quarkus:dev

ShellSession

Then, you can call the GET /api/{userId}/persons endpoint with different userId path variable values. Here are sample API requests and responses.

After that, you can call the GET /api/{userId}/persons/{id} endpoint to return a specified person found in the chat memory.

Switch Between AI Models

Then, you can repeat the same exercise with the Mistral AI model. You must set the AI_MODEL_PROVIDER to mistral, export its API token as the MISTRAL_AI_TOKEN environment variable, and enable the mistral-ai profile while running the app.

$ export AI_MODEL_PROVIDER=mistralai
$ export MISTRAL_AI_TOKEN=<your_mistralai_token>
$ mvn quarkus:dev -Pmistral-ai

ShellSession

The app should start successfully.

Once it happens, you can repeat the same sequence of requests as before for OpenAI.

$ curl http://localhost:8080/api/1/persons
$ curl http://localhost:8080/api/2/persons
$ curl http://localhost:8080/api/1/persons/1
$ curl http://localhost:8080/api/2/persons/1

ShellSession

You can check the request sent to the AI model in the application logs.

Here’s a log showing an AI chat model response:

Finally, you can run a test with ollama. By default, the LangChain4j extension for Ollama uses the llama3.2 model. You can change it by setting the quarkus.langchain4j.ollama.chat-model.model-id property in the application.properties file. Assuming that you use the llama3.3 model, here’s your configuration:

quarkus.langchain4j.ollama.base-url = ${OLLAMA_BASE_URL:http://localhost:11434}
quarkus.langchain4j.ollama.chat-model.model-id = llama3.3
quarkus.langchain4j.ollama.timeout = 60s

Plaintext

Before proceeding, you must run the llama3.3 model on your laptop. Of course, you can choose another, smaller model, because llama3.3 is 42 GB.

ollama run llama3.3

ShellSession

It can take a lot of time. However, a model is finally ready to use.

Once a model is running, you can set the AI_MODEL_PROVIDER environment variable to ollama and activate the ollama profile for the app:

$ export AI_MODEL_PROVIDER=ollama
$ mvn quarkus:dev -Pollama

ShellSession

This time, our application is connected to the llama3.3 model started with ollama:

With the Quarkus LangChain4j Ollama extension, you can take advantage of dev services support. It means that you don’t need to install and run Ollama on your laptop or run a model with ollama CLI. Quarkus will run Ollama as a Docker container and automatically run a selected AI model on it. In that case, you don’t need to set the quarkus.langchain4j.ollama.base-url property. Before switching to that option, let’s use a smaller AI model by setting the quarkus.langchain4j.ollama.chat-model.model-id = mistral property. Then start the app in the same way as before.

Final Thoughts

I must admit that the Quarkus LangChain4j extension is enjoyable to use. With a few simple annotations, you can configure your application to talk to the AI model of your choice correctly. In this article, I presented a straightforward example of integrating Quarkus with an AI chat model. However, we quickly reviewed features such as prompts, structured output, and chat memory. You can expect more articles in the Quarkus series with AI soon.

The post Getting Started with Quarkus LangChain4j and Chat Model appeared first on Piotr's TechBlog.