Today we will focus on Java libraries. Usually, we use several external libraries in our projects – even if we do not include them directly. For example, Spring Boot comes with a defined set of dependencies included by starters. Assuming we include e.g. spring-boot-starter-test we include libraries like mockito, junit-jupiter or hamcrest at the same time. Of course, these are well-known libraries for the community.

In fact, there are a lot of different Java libraries. Usually, I don’t need to use many of them (or even I need none of them) when working with the frameworks like Spring Boot or Quarkus. However, there are some very interesting libraries that may be useful everywhere. I’m writing about them because you might not hear about any of them. I’m going to introduce 5 of my favorite “useful & unknown” Java libraries. Let’s begin!

Source Code

If you would like to try it by yourself, you may always take a look at my source code. To do that you need to clone my GitHub repository. Yo can also find the example Then you should just follow my instructions.

Instancio

On the first fire will go Instancio. How do you generate test data in your unit tests? Instancio will help us with that. It aims to reduce the time and lines of code spent on manual data setup in unit tests. It instantiates and populates objects with random data, making our tests more dynamic. We can generate random data with Instancio but at the same, we can set custom data in a particular field.

Before we start with Instancio, let’s discuss our data model. Here’s the first class – Person:

public class Person {

   private Long id;
   private String name;
   private int age;
   private Gender gender;
   private Address address;

   // getters and setters ...

}

Our class contains three simple fields (id, name, age), a single enum Gender , and the instance of the Address class. Gender is just a simple enum containing MALE and FEMALE values. Here’s the implementation of the Address class:

public class Address {

   private String country;
   private String city;
   private String street;
   private int houseNumber;
   private int flatNumber;

   // getters and setters ...

}

Now, let’s create a test to check whether the Person service will successfully add and obtain objects from the store. We want to generate random data for all the fields except the id field, which is set by the service. Here’s our test:

@Test
void addAndGet() {
   Person person = Instancio.of(Person.class)
             .ignore(Select.field(Person::getId))
             .create();
   person = personService.addPerson(person);
   Assertions.assertNotNull(person.getId());
   person = personService.findById(person.getId());
   Assertions.assertNotNull(person);
   Assertions.assertNotNull(person.getAddress());
}

The values generated for my test run are visible below. As you see, the id field equals null. Other fields contain random values generated per the field type (String or int).

Person(id=null, name=ATDLCA, age=2619, gender=MALE, 
address=Address(country=FWOFRNT, city=AIRICCHGGG, street=ZZCIJDZ, houseNumber=5530, flatNumber=1671))

Let’s see how we can generate several objects with Instancio. Assuming we need 5 objects in the list for our test, we can do that in the following way. We will also set a constant value for the city fields inside the Address object. Then we would like to test the method for searching objects by the city name.

@Test
void addListAndGet() {
   final int numberOfObjects = 5;
   final String city = "Warsaw";
   List<Person> persons = Instancio.ofList(Person.class)
           .size(numberOfObjects)
           .set(Select.field(Address::getCity), city)
           .create();
   personService.addPersons(persons);
   persons = personService.findByCity(city);
   Assertions.assertEquals(numberOfObjects, persons.size());
}

Let’s take a look at the last example. The same as before, we are generating a list of objects – this time 100. We can easily specify the additional criteria for generated values. For example, I would like to set a value for the age field between 18 and 65.

@Test
void addGeneratorAndGet() {
   List<Person> persons = Instancio.ofList(Person.class)
            .size(100)
            .ignore(Select.field(Person::getId))
            .generate(Select.field(Person::getAge), 
                      gen -> gen.ints().range(18, 65))
            .create();
   personService.addPersons(persons);
   persons = personService.findAllGreaterThanAge(40);
   Assertions.assertTrue(persons.size() > 0);
}

That’s just a small set of customizations, that Instancio offers, for test data generation. You can read more about other options in their docs.

Datafaker

The next library we will discuss today is Datafaker. The purpose of this library is quite similar to the previous one. We need to generate random data. However, this time we need data that looks like real data. From my perspective, it is useful for demo presentations or examples running somewhere.

Datafaker creates fake data for your JVM programs within minutes, using our wide range of more than 100 data providers. This can be very helpful when generating test data to fill a database, generating data for a stress test, or anonymizing data from production services. Let’s include it in our dependencies.

<dependency>
  <groupId>net.datafaker</groupId>
  <artifactId>datafaker</artifactId>
  <version>1.7.0</version>
</dependency>

We will expand our sample model a little. Here’s a new class definition. The Contact class contains two fields email and phoneNumber. We will validate both these fields using the jakarta.validation module.

public class Contact {

   @Email
   private String email;
   @Pattern(regexp="\\d{2}-\\d{3}-\\d{2}-\\d{2}")
   private String phoneNumber;

   // getters and setters ...

}

Here’s a new version of our Person class that contains the Contant object instance:

public class Person {

   private Long id;
   private String name;
   private int age;
   private Gender gender;
   private Address address;
   @Valid
   private Contact contact;

   // getters and setters ...

}

Now, let’s generate fake data for the Person object. We can create localized data just by setting the Locale object in the Faker constructor (1). For me, it is Poland There are a lot of providers for the standard values. To set email we need to use the Internet provider (2). There is a dedicated provider for generating phone numbers (3), addresses (4), and person names (5). You can see a full list of available providers here. After creating test data, we can run the test that adds a new Person verified on the server side.

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class PersonsControllerTests {

   @Autowired
   private TestRestTemplate restTemplate;

   @Test
   void add() {
      Faker faker = new Faker(Locale.of("pl")); // (1)
      Contact contact = new Contact();
      contact.setEmail(faker.internet().emailAddress()); // (2)
      contact.setPhoneNumber(faker.phoneNumber().cellPhone()); // (3)
      Address address = new Address();
      address.setCity(faker.address().city()); // (4)
      address.setCountry(faker.address().country());
      address.setStreet(faker.address().streetName());
      int number = Integer
         .parseInt(faker.address().streetAddressNumber());
      address.setHouseNumber(number);
      number = Integer.parseInt(faker.address().buildingNumber());
      address.setFlatNumber(number);
      Person person = new Person();
      person.setName(faker.name().fullName()); // (5)
      person.setContact(contact);
      person.setAddress(address);
      person.setGender(Gender.valueOf(
         faker.gender().binaryTypes().toUpperCase()));
      person.setAge(faker.number().numberBetween(18, 65));

      person = restTemplate
         .postForObject("/persons", person, Person.class);
      Assertions.assertNotNull(person);
      Assertions.assertNotNull(person.getId());
   }

}

Here’s the data generated during my test. I think you can find one inconsistency here (the country field)

Person(id=null, name=Stefania Borkowski, age=51, gender=FEMALE, address=Address(country=Ekwador, city=Sępopol, street=al. Chudzik, houseNumber=882, flatNumber=318), contact=Contact{email='gilbert.augustyniak@gmail.com', phoneNumber='69-733-43-77'})

Sometimes you need to generate a more predictable random result. It’s possible to provide a seed value in the Faker constructor. When providing a seed, the instantiation of Fake objects will always happen in a predictable way, which can be handy for generating results multiple times. Here’s a new version of my Faker object declaration:

Faker faker = new Faker(Locale.of("pl"), new Random(0));

JPA Streamer

Our next library is related to JPA queries. If you like to use Java streams and you are building apps that interact with databases through JPA or Hibernate, the JPA Streamer library may be an interesting choice. It is a library for expressing JPA/Hibernate/Spring queries using standard Java streams. JPA Streamer instantly gives Java developers type-safe, expressive and intuitive means of obtaining data in database applications. Moreover, you can easily integrate it with Spring Boot and Quarkus. Firstly, let’s include JPA Streamer in our dependencies:

<dependency>
  <groupId>com.speedment.jpastreamer</groupId>
  <artifactId>jpastreamer-core</artifactId>
  <version>1.1.2</version>
</dependency>

If you want to integrate it with Spring Boot you need to add one additional dependency:

<dependency>
  <groupId>com.speedment.jpastreamer.integration.spring</groupId>
  <artifactId>spring-boot-jpastreamer-autoconfigure</artifactId>
  <version>1.1.2</version>
</dependency>

In order to test JPA Streamer, we need to create an example entities model.

@Entity
public class Employee {

   @Id
   @GeneratedValue(strategy = GenerationType.IDENTITY)
   private Integer id;
   private String name;
   private String position;
   private int salary;
   @ManyToOne(fetch = FetchType.LAZY)
   private Department department;
   @ManyToOne(fetch = FetchType.LAZY)
   private Organization organization;

   // getters and setters ...

}

There are also two other entities: Organization and Department. Here are their definitions:

@Entity
public class Department {
   @Id
   @GeneratedValue(strategy = GenerationType.IDENTITY)
   private Integer id;
   private String name;
   @OneToMany(mappedBy = "department")
   private Set<Employee> employees;
   @ManyToOne(fetch = FetchType.LAZY)
   private Organization organization;

   // getters and setters ...
}

@Entity
public class Organization {
   @Id
   @GeneratedValue(strategy = GenerationType.IDENTITY)
   private Integer id;
   private String name;
   @OneToMany(mappedBy = "organization")
   private Set<Department> departments;
   @OneToMany(mappedBy = "organization")
   private Set<Employee> employees;

   // getters and setters ...
}

Now, we can prepare some queries using the Java streams pattern. In the following fragment of code, we are searching an entity by id and then joining two relations. By default, it is LEFT JOIN, but we can customize it when calling the joining() method. In the following fragment of code, we join Department and Organization, which are in @ManyToOne a relationship with the Employee entity. Then we filter the result, convert the object to the DTO and pick the first result.

@GetMapping("/{id}")
public EmployeeWithDetailsDTO findById(@PathVariable("id") Integer id) {
   return streamer.stream(of(Employee.class)
           .joining(Employee$.department)
           .joining(Employee$.organization))
        .filter(Employee$.id.equal(id))
        .map(EmployeeWithDetailsDTO::new)
        .findFirst()
        .orElseThrow();
}

Of course, we can call many other Java stream methods. In the following fragment of code, we count the number of employees assigned to the particular department.

@GetMapping("/{id}/count-employees")
public long getNumberOfEmployees(@PathVariable("id") Integer id) {
   return streamer.stream(Department.class)
         .filter(Department$.id.equal(id))
         .map(Department::getEmployees)
         .mapToLong(Set::size)
         .sum();
}

If you are looking for a detailed explanation and more examples with JPA Streamer you can my article dedicated to that topic.

Blaze Persistence

Blaze Persistence is another library from the JPA and Hibernate area. It allows you to write complex queries with a consistent builder API with rich Criteria API for JPA providers. That’s not all. You can also use the Entity-View module dedicated to DTO mapping. Of course, you can easily integrate with Spring Boot or Quarkus. If you want to use all Blaze Persistence modules in your app it is worth adding the dependencyManagement section in your Maven pom.xml:

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>com.blazebit</groupId>
            <artifactId>blaze-persistence-bom</artifactId>
            <version>1.6.8</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>    
    </dependencies>
</dependencyManagement>

Personally, I’m using Blaze Persistence for DTO mapping. Thanks to the integration with Spring Boot we can replace Spring Data Projections with Blaze Persistence Entity-Views. It will be especially useful for more advanced mappings since Blaze Persistence offers more features and better performance for that. You can read a detailed comparison in the following article. If we want to integrate Blaze Persistence Entity-Views with Spring Data we should add the following dependencies:

<dependency>
  <groupId>com.blazebit</groupId>
  <artifactId>blaze-persistence-integration-spring-data-2.7</artifactId>
</dependency>
<dependency>
  <groupId>com.blazebit</groupId>
  <artifactId>blaze-persistence-integration-hibernate-5.6</artifactId>
  <scope>runtime</scope>
</dependency>
<dependency>
  <groupId>com.blazebit</groupId>
  <artifactId>blaze-persistence-entity-view-processor</artifactId>
</dependency>

Then, we need to create an interface with getters for mapped fields. It should be annotated with the @EntityView that refers to the target entity class. In the following example, we are mapping two entity fields firstName and lastName to the single fields inside the PersonDTO object. In order to map the entity’s primary key we should use the @IdMapping annotation.

@EntityView(Person.class)
public interface PersonView {

   @IdMapping
   Integer getId();
   void setId(Integer id);

   @Mapping("CONCAT(firstName,' ',lastName)")
   String getName();
   void setName(String name);

}

We can still take advantage of the Spring Data repository pattern. Our repository interface needs to extend the EntityViewRepository interface.

@Transactional(readOnly = true)
public interface PersonViewRepository 
    extends EntityViewRepository<PersonView, Integer> {

    PersonView findByAgeGreaterThan(int age);

}

We also need to provide some additional configuration and enable Blaze Persistence in the main or the configuration class:

@SpringBootApplication
@EnableBlazeRepositories
@EnableEntityViews
public class PersonApplication {

   public static void main(String[] args) {
      SpringApplication.run(PersonApplication.class, args);
   }

}

Hoverfly

Finally, the last of the Java libraries on my list – Hoverfly. To be more precise, we will use the Java version of the Hoverfly library documented here. It is a lightweight service virtualization tool that allows you to stub or simulate HTTP(S) services. Hoverfly Java is a native language binding that gives you an expressive API for managing Hoverfly in Java. It gives you a Hoverfly class which abstracts away the binary and API calls, a DSL for creating simulations, and a JUnit integration for using it within unit tests.

Ok, there are some other, similar libraries… but somehow I really like Hoverfly It is a simple, lightweight library that may perform tests in different modes like simulation, spying, capture, or diffing. You can use Java DSL to build request matchers to response mappings. Let’s include the latest version of Hoverfly in the Maven dependencies:

<dependency>
  <groupId>io.specto</groupId>
  <artifactId>hoverfly-java-junit5</artifactId>
  <version>0.14.3</version>
</dependency>

Let’s assume we have the following method in our Spring @RestController. Before returning a ping response for itself, it calls another service under the address http://callme-service:8080/callme/ping.

@GetMapping("/ping")
public String ping() {
   String response = restTemplate
     .getForObject("http://callme-service:8080/callme/ping", 
                   String.class);
   LOGGER.info("Calling: response={}", response);
   return "I'm caller-service " + version + ". Calling... " + response;
}

Now, we will create the test for our controller. In order to use Hoverfly to intercept outgoing traffic we register HoverflyExtension (1). Then we may the Hoverfly object to create a request mather and simulate an HTTP response (2). The simulated response body is I'm callme-service v1.

@SpringBootTest(properties = {"VERSION = v2"}, 
    webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ExtendWith(HoverflyExtension.class) // (1)
public class CallerCallmeTest {
    
   @Autowired
   TestRestTemplate restTemplate;

   @Test
   void callmeIntegration(Hoverfly hoverfly) {
      hoverfly.simulate(
            dsl(service("http://callme-service:8080")
               .get("/callme/ping")
               .willReturn(success().body("I'm callme-service v1.")))
      ); // (2)
      String response = restTemplate
         .getForObject("/caller/ping", String.class);
      assertEquals("I'm caller-service v2. Calling... I'm callme-service v1.", response);
   }
}

We can easily customize Hovefly behavior with the @HoverflyConfig annotation. By default, Hoverfly works in proxy mode. Assuming we want it to act as a web server we need to set the property webserver to true (1). After that, it will listen for requests on localhost and the port indicated by the proxyPort property. In the next step, we will also enable Spring Cloud @LoadBalancedClient to configure a static list of target URLs instead of dynamic discovery (2). Finally, we can create a Hoverfly test. This time we are intercepting traffic from the web server listening on the localhost:8080 (3).

@SpringBootTest(webEnvironment = 
   SpringBootTest.WebEnvironment.RANDOM_PORT)
@HoverflyCore(config = 
   @HoverflyConfig(logLevel = LogLevel.DEBUG, 
                    webServer = true, 
                    proxyPort = 8080)) // (1)
@ExtendWith(HoverflyExtension.class)
@LoadBalancerClient(name = "account-service", 
                    configuration = AccountServiceConf.class) // (2)
public class GatewayTests {

    @Autowired
    TestRestTemplate restTemplate;

    @Test
    public void findAccounts(Hoverfly hoverfly) {
        hoverfly.simulate(dsl(
            service("http://localhost:8080")
                .andDelay(200, TimeUnit.MILLISECONDS).forAll()
                .get(any())
                .willReturn(success("[{\"id\":\"1\",\"number\":\"1234567890\",\"balance\":5000}]", "application/json")))); // (3)

        ResponseEntity<String> response = restTemplate
                .getForEntity("/account/1", String.class);
        Assertions.assertEquals(200, response.getStatusCodeValue());
        Assertions.assertNotNull(response.getBody());
    }
}

Here’s the load balancer client configuration created just for the test purpose.

class AccountServiceInstanceListSuppler implements 
    ServiceInstanceListSupplier {

    private final String serviceId;

    AccountServiceInstanceListSuppler(String serviceId) {
        this.serviceId = serviceId;
    }

    @Override
    public String getServiceId() {
        return serviceId;
    }

    @Override
    public Flux<List<ServiceInstance>> get() {
        return Flux.just(Arrays
                .asList(new DefaultServiceInstance(serviceId + "1", 
                        serviceId, 
                        "localhost", 8080, false)));
    }
}

Final Thoughts

As you probably figured out, I used all that Java libraries with Spring Boot apps. Although Spring Boot comes with a defined set of external libraries, sometimes we may need some add-ons. The Java libraries I presented are usually created to solve a single, particular problem like e.g. test data generation. It’s totally fine from my perspective. I hope you will find at least one position from my list useful in your projects.

The post Useful & Unknown Java Libraries appeared first on Piotr's TechBlog.

Source code

The source code is available on GitHub. If you want to clone the repository or just give me a star go here

Building applications with Spring Cloud Kubernetes

Spring Cloud Kubernetes provides implementations of well known Spring Cloud components based on Kubernetes API. It includes a discovery client, load balancer, and property sources support. We should add the following Maven dependency to enable it in our project.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-kubernetes-all</artifactId>
</dependency>

Our application connects to the Mongo database, exposes REST API, and communicates with other applications over HTTP. Therefore we need to include some additional dependencies.

<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-mongodb</artifactId>
</dependency>

The overview of our system is visible in the picture below. We need to mock communication between applications and with Kubernetes API. We will also run an embedded in-memory Mongo database during tests. For more details about building microservices with Spring Cloud Kubernetes read the following article.

Testing API with Kubernetes MockServer

First, we need to include a Spring Boot Test starter, that contains basic dependencies used for JUnit tests implementation. Since our application is connected to Mongo and Kubernetes API, we should also mock them during the test. Here’s the full list of required dependencies.

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>de.flapdoodle.embed</groupId>
    <artifactId>de.flapdoodle.embed.mongo</artifactId>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>io.fabric8</groupId>
    <artifactId>kubernetes-server-mock</artifactId>
    <version>4.10.3</version>
    <scope>test</scope>
</dependency>

Let’s discuss what exactly is happening during our test.
(1) First, we are enabling fabric8 Kubernetes Client JUnit5 extension in CRUD mode. It means that we can create a Kubernetes object on the mocked server.
(2) Then the KubernetesClient is injected to the test by the JUnit5 extension.
(3) TestRestTemplate is able to call endpoints exposed by the application that is started during the test.
(4) We need to set the basic properties for KubernetesClient like a default namespace name, master URL.
(5) We are creating ConfigMap that contains application.properties file. ConfigMap with name employee is automatically read by the application employee.
(6) In the test method we are using TestRestTemplate to call REST endpoints. We are mocking Kubernetes API and running Mongo database in the embedded mode.

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@EnableKubernetesMockClient(crud = true) // (1)
@TestMethodOrder(MethodOrderer.Alphanumeric.class)
class EmployeeAPITest {

    static KubernetesClient client; // (2)

    @Autowired
    TestRestTemplate restTemplate; // (3)

    @BeforeAll
    static void init() {
        System.setProperty(Config.KUBERNETES_MASTER_SYSTEM_PROPERTY,
            client.getConfiguration().getMasterUrl());
        System.setProperty(Config.KUBERNETES_TRUST_CERT_SYSTEM_PROPERTY,
            "true");
        System.setProperty(
            Config.KUBERNETES_AUTH_TRYKUBECONFIG_SYSTEM_PROPERTY, "false");
        System.setProperty(
            Config.KUBERNETES_AUTH_TRYSERVICEACCOUNT_SYSTEM_PROPERTY, "false");
        System.setProperty(Config.KUBERNETES_HTTP2_DISABLE, "true");
        System.setProperty(Config.KUBERNETES_NAMESPACE_SYSTEM_PROPERTY,
            "default"); // (4)
        client.configMaps().inNamespace("default").createNew()
            .withNewMetadata().withName("employee").endMetadata()
            .addToData("application.properties",
                "spring.data.mongodb.uri=mongodb://localhost:27017/test")
            .done(); // (5)
    }

    @Test // (6)
    void addEmployeeTest() {
        Employee employee = new Employee(1L, 1L, "Test", 30, "test");
        employee = restTemplate.postForObject("/", employee, Employee.class);
        Assertions.assertNotNull(employee);
        Assertions.assertNotNull(employee.getId());
    }

    @Test
    void addAndThenFindEmployeeByIdTest() {
        Employee employee = new Employee(1L, 2L, "Test2", 20, "test2");
        employee = restTemplate.postForObject("/", employee, Employee.class);
        Assertions.assertNotNull(employee);
        Assertions.assertNotNull(employee.getId());
        employee = restTemplate
            .getForObject("/{id}", Employee.class, employee.getId());
        Assertions.assertNotNull(employee);
        Assertions.assertNotNull(employee.getId());
    }

    @Test
    void findAllEmployeesTest() {
        Employee[] employees =
            restTemplate.getForObject("/", Employee[].class);
        Assertions.assertEquals(2, employees.length);
    }

    @Test
    void findEmployeesByDepartmentTest() {
        Employee[] employees =
            restTemplate.getForObject("/department/1", Employee[].class);
        Assertions.assertEquals(1, employees.length);
    }

    @Test
    void findEmployeesByOrganizationTest() {
        Employee[] employees =
            restTemplate.getForObject("/organization/1", Employee[].class);
        Assertions.assertEquals(2, employees.length);
    }

}

Integration Testing on Kubernetes with Hoverfly

To test HTTP communication between applications we usually need to use an additional tool for mocking API. Hoverfly is an ideal solution for such a use case. It is a lightweight, open-source API simulation tool not only for REST-based applications. It allows you to write tests in Java and Python. In addition, it also supports JUnit5. You need to include the following dependencies to enable it in your project.

<dependency>
	<groupId>io.specto</groupId>
	<artifactId>hoverfly-java-junit5</artifactId>
	<version>0.13.0</version>
	<scope>test</scope>
</dependency>
<dependency>
	<groupId>io.specto</groupId>
	<artifactId>hoverfly-java</artifactId>
	<version>0.13.0</version>
	<scope>test</scope>
</dependency>

You can enable Hoverfly in your tests with @ExtendWith annotation. It automatically starts Hoverfly proxy during a test. Our main goal is to mock the Kubernetes client. To do that we still need to set some properties inside @BeforeAll method. The default URL used by KubernetesClient is kubernetes.default.svc. In the first step, we are mocking configmap endpoint and returning predefined Kubernetes ConfigMap with application.properties. The name of ConfigMap is the same as the application name. We are testing communication from the department application to the employee application.

@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ExtendWith(HoverflyExtension.class)
public class DepartmentAPIAdvancedTest {

    @Autowired
    KubernetesClient client;

    @BeforeAll
    static void setup(Hoverfly hoverfly) {
        System.setProperty(Config.KUBERNETES_TRUST_CERT_SYSTEM_PROPERTY, "true");
        System.setProperty(Config.KUBERNETES_AUTH_TRYKUBECONFIG_SYSTEM_PROPERTY, "false");
        System.setProperty(Config.KUBERNETES_AUTH_TRYSERVICEACCOUNT_SYSTEM_PROPERTY,
            "false");
        System.setProperty(Config.KUBERNETES_HTTP2_DISABLE, "true");
        System.setProperty(Config.KUBERNETES_NAMESPACE_SYSTEM_PROPERTY, "default");
        hoverfly.simulate(dsl(service("kubernetes.default.svc")
            .get("/api/v1/namespaces/default/configmaps/department")
            .willReturn(success().body(json(buildConfigMap())))));
    }

    private static ConfigMap buildConfigMap() {
        return new ConfigMapBuilder().withNewMetadata()
            .withName("department").withNamespace("default")
            .endMetadata()
            .addToData("application.properties",
                "spring.data.mongodb.uri=mongodb://localhost:27017/test")
            .build();
    }
	
    // TESTS ...
	
}

After application startup, we may use TestRestTemplate to call a test endpoint. The endpoint GET /organization/{organizationId}/with-employees retrieves data from the employee application. It finds the department by organization id and then finds all employees assigned to the department. We need to mock a target endpoint using Hoverfly. But before that, we are mocking Kubernetes APIs responsible for getting service and endpoint by name. The address and port returned by the mocked endpoints must be the same as the address of a target application endpoint.

@Autowired
TestRestTemplate restTemplate;

private final String EMPLOYEE_URL = "employee.default:8080";

@Test
void findByOrganizationWithEmployees(Hoverfly hoverfly) {
    Department department = new Department(1L, "Test");
    department = restTemplate.postForObject("/", department, Department.class);
    Assertions.assertNotNull(department);
    Assertions.assertNotNull(department.getId());

    hoverfly.simulate(
        dsl(service(prepareUrl())
            .get("/api/v1/namespaces/default/endpoints/employee")
            .willReturn(success().body(json(buildEndpoints())))),
        dsl(service(prepareUrl())
            .get("/api/v1/namespaces/default/services/employee")
            .willReturn(success().body(json(buildService())))),
        dsl(service(EMPLOYEE_URL)
            .get("/department/" + department.getId())
            .willReturn(success().body(json(buildEmployees())))));

    Department[] departments = restTemplate
        .getForObject("/organization/{organizationId}/with-employees", Department[].class, 1L);
    Assertions.assertEquals(1, departments.length);
    Assertions.assertEquals(1, departments[0].getEmployees().size());
}

private Service buildService() {
    return new ServiceBuilder().withNewMetadata().withName("employee")
            .withNamespace("default").withLabels(new HashMap<>())
            .withAnnotations(new HashMap<>()).endMetadata().withNewSpec().addNewPort()
            .withPort(8080).endPort().endSpec().build();
}

private Endpoints buildEndpoints() {
    return new EndpointsBuilder().withNewMetadata()
        .withName("employee").withNamespace("default")
        .endMetadata()
        .addNewSubset().addNewAddress()
        .withIp("employee.default").endAddress().addNewPort().withName("http")
        .withPort(8080).endPort().endSubset()
        .build();
}

private List<Employee> buildEmployees() {
    List<Employee> employees = new ArrayList<>();
    Employee employee = new Employee();
    employee.setId("abc123");
    employee.setAge(30);
    employee.setName("Test");
    employee.setPosition("test");
    employees.add(employee);
    return employees;
}

private String prepareUrl() {
    return client.getConfiguration().getMasterUrl()
        .replace("/", "")
        .replace("https:", "");
}

Conclusion

The approach described in this article allows you to create integration tests without running a Kubernetes instance. On the other hand, you could start a single-node Kubernetes instance like Microk8s and deploy your application there. You could as well use an existing cluster, and implement your tests with Arquillian Cube. It is able to communicate directly to the Kubernetes API.
Another key point is testing communication between applications. In my opinion, Hoverfly is the best tool for that. It is able to mock the whole traffic over HTTP in the single test. With Hoverfly, fabric8 and Spring Cloud you can improve your integration testing on Kubernetes.

The post Integration Testing on Kubernetes with JUnit5 appeared first on Piotr's TechBlog.

• Testing REST API with Hoverfly
• Testing Spring Boot Integration with Vault and Postgres using Testcontainers Framework

Today we will consider the system consisting of three microservices, where each microservice is developed by the different team. One of these microservices trip-management is integrating with two others: driver-management and passenger-management. The question is how to organize integration tests under these assumptions. In that case we can use one of the interesting features provided by Hoverfly – an ability to run it as a remote proxy. What does it mean in practice? It is illustrated in the picture below. The same external instance of Hoverfly proxy is shared between all microservices during JUnit tests. Microservice driver-management and passenger-management are testing their own methods exposed for use by trip-management, but all the requests are sent through Hoverfly remote instance acts as a proxy. Hoverfly will capture all the requests and responses sent during the tests. On the other hand trip-management is also testing its methods, but the communication with other microservices is simulated by the remote Hoverfly instance based on previously captured HTTP traffic.

We will use Docker for running remote instances of Hoverfly proxy. We will also use Docker images of microservices during the tests. That’s why we need the Testcontainers framework, which is responsible for running an application container before starting integration tests. So, the first step is to build a Docker image of driver-management and passenger-management microservices.

1. Building Docker Image

Assuming you have successfully installed Docker on your machine, and you have set environment variables DOCKER_HOST and DOCKER_CERT_PATH, you may use io.fabric:docker-maven-plugin for it. It is important to execute the build goal of that plugin just after package Maven phase, but before integration-test phase. Here’s the appropriate configuration inside Maven pom.xml.

<plugin>
   <groupId>io.fabric8</groupId>
   <artifactId>docker-maven-plugin</artifactId>
   <configuration>
      <images>
         <image>
            <name>piomin/driver-management</name>
            <alias>dockerfile</alias>
            <build>
               <dockerFileDir>${project.basedir}</dockerFileDir>
            </build>
         </image>
      </images>
   </configuration>
   <executions>
      <execution>
         <phase>pre-integration-test</phase>
         <goals>
            <goal>build</goal>
         </goals>
      </execution>
   </executions>
</plugin>

2. Application Integration Tests

Our integration tests should be run during the integration-test phase, so they must not be executed during test, before building application fat jar and Docker image. Here’s the appropriate configuration with maven-surefire-plugin.

<plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-surefire-plugin</artifactId>
   <configuration>
      <excludes>
         <exclude>pl.piomin.services.driver.contract.DriverControllerIntegrationTests</exclude>
      </excludes>
   </configuration>
   <executions>
      <execution>
         <id>integration-test</id>
         <goals>
            <goal>test</goal>
         </goals>
         <phase>integration-test</phase>
         <configuration>
            <excludes>
               <exclude>none</exclude>
            </excludes>
            <includes>
               <include>pl.piomin.services.driver.contract.DriverControllerIntegrationTests</include>
            </includes>
         </configuration>
      </execution>
   </executions>
</plugin>

3. Running Hoverfly

Before running any tests we need to start an instance of Hoverfly in proxy mode. To achieve it we use Hoverfly Docker image. Because Hoverfly has to forward requests to the downstream microservices by host name, we create Docker network and then run Hoverfly in this network.

$ docker network create tests
$ docker run -d --name hoverfly -p 8500:8500 -p 8888:8888 --network tests spectolabs/hoverfly

Hoverfly proxy is now available for me (I’m using Docker Toolbox) under address 192.168.99.100:8500. We can also take a look at the admin web console available under address http://192.168.99.100:8888. Under that address you can also access HTTP API, what is described later in the next section.

4. Including test dependencies

To enable Hoverfly and Testcontainers for our test we first need to include some dependencies to Maven pom.xml. Our sample applications are built on top of Spring Boot, so we also include the Spring Test project.

<dependency>
   <groupId>org.springframework.boot</groupId>
   <artifactId>spring-boot-starter-test</artifactId>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>org.testcontainers</groupId>
   <artifactId>testcontainers</artifactId>
   <version>1.10.6</version>
   <scope>test</scope>
</dependency>
<dependency>
   <groupId>io.specto</groupId>
   <artifactId>hoverfly-java</artifactId>
   <version>0.11.1</version>
   <scope>test</scope>
</dependency>

5. Building integration tests on the provider site

Now, we can finally proceed to JUnit test implementation. Here’s the full source code of the test for driver-management microservice, but some things need to be explained. Before running our test methods we first start a Docker container of application using Testcontainers. We use GenericContainer annotated with @ClassRule for that. Testcontainers provides api for interaction with containers, so we can easily set target Docker network and container hostname. We will also wait until the application container is ready for use by calling method waitingFor on GenericContainer.
The next step is to enable the Hoverfly rule for our test. We will run it in capture mode. By default Hoverfly is trying to start a local proxy instance, that’s why we provide remote addresses of an existing instance already started using Docker container.
The tests are pretty simple. We will call endpoints using Spring TestRestTemplate. Because the request must finally be proxied to the application container we use its hostname as the target address. The whole traffic is captured by Hoverfly.

public class DriverControllerIntegrationTests {

    private TestRestTemplate template = new TestRestTemplate();

    @ClassRule
    public static GenericContainer appContainer = new GenericContainer<>("piomin/driver-management")
            .withCreateContainerCmdModifier(cmd -> cmd.withName("driver-management").withHostName("driver-management"))
            .withNetworkMode("tests")
            .withNetworkAliases("driver-management")
            .withExposedPorts(8080)
            .waitingFor(Wait.forHttp("/drivers"));

    @ClassRule
    public static HoverflyRule hoverflyRule = HoverflyRule
            .inCaptureMode("driver.json", HoverflyConfig.remoteConfigs().host("192.168.99.100"))
            .printSimulationData();

    @Test
    public void testFindNearestDriver() {
        Driver driver = template.getForObject("http://driver-management:8080/drivers/{locationX}/{locationY}", Driver.class, 40, 20);
        Assert.assertNotNull(driver);
        driver = template.getForObject("http://driver-management:8080/drivers/{locationX}/{locationY}", Driver.class, 10, 20);
        Assert.assertNotNull(driver);
    }

    @Test
    public void testUpdateDriver() {
        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_JSON);
        DriverInput input = new DriverInput();
        input.setId(2L);
        input.setStatus(DriverStatus.UNAVAILABLE);
        HttpEntity<DriverInput> entity = new HttpEntity<>(input, headers);
        template.put("http://driver-management:8080/drivers", entity);
        input.setId(1L);
        input.setStatus(DriverStatus.AVAILABLE);
        entity = new HttpEntity<>(input, headers);
        template.put("http://driver-management:8080/drivers", entity);
    }

}

Now, you can execute the tests during application build using mvn clean verify command. The sample application source code is available on GitHub in repository sample-testing-microservices under branch remote.

6. Building integration tests on the consumer site

In the previous we have discussed the integration tests implemented on the consumer site. There are two microservices driver-management and passenger-management, that expose endpoints invoked by the third microservice trip-management. The traffic generated during the tests has already been captured by Hoverfly. It is a very important thing in that sample, because each time you will build the newest version of microservice Hoverfly is refreshing the structure of previously recorded requests. Now, if we run the tests for a consumer application (trip-management) it fully bases on the newest version of requests generated during tests by microservices on the provider site. You can check out the list of all requests captured by Hoverfly by calling endpoint http://192.168.99.100:8888/api/v2/simulation.
Here are the integration tests implemented inside trip-management. They also use a remote Hoverfly proxy instance. The only difference is in running mode, which is simulation. It tries to simulate requests sent to driver-management and passenger-management basing on the traffic captured by Hoverfly.

@SpringBootTest
@RunWith(SpringRunner.class)
@AutoConfigureMockMvc
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class TripIntegrationTests {

    ObjectMapper mapper = new ObjectMapper();

    @ClassRule
    public static HoverflyRule hoverflyRule = HoverflyRule
            .inSimulationMode(HoverflyConfig.remoteConfigs().host("192.168.99.100"))
            .printSimulationData();

    @Autowired
    MockMvc mockMvc;

    @Test
    public void test1CreateNewTrip() throws Exception {
        TripInput ti = new TripInput("test", 10, 20, "walker");
        mockMvc.perform(MockMvcRequestBuilders.post("/trips")
                .contentType(MediaType.APPLICATION_JSON_UTF8)
                .content(mapper.writeValueAsString(ti)))
                .andExpect(MockMvcResultMatchers.status().isOk())
                .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.any(Integer.class)))
                .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("NEW")))
                .andExpect(MockMvcResultMatchers.jsonPath("$.driverId", Matchers.any(Integer.class)));
    }

    @Test
    public void test2CancelTrip() throws Exception {
        mockMvc.perform(MockMvcRequestBuilders.put("/trips/cancel/1")
                .contentType(MediaType.APPLICATION_JSON_UTF8)
                .content(mapper.writeValueAsString(new Trip())))
                .andExpect(MockMvcResultMatchers.status().isOk())
                .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.any(Integer.class)))
                .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("IN_PROGRESS")))
                .andExpect(MockMvcResultMatchers.jsonPath("$.driverId", Matchers.any(Integer.class)));
    }

    @Test
    public void test3PayTrip() throws Exception {
        mockMvc.perform(MockMvcRequestBuilders.put("/trips/payment/1")
                .contentType(MediaType.APPLICATION_JSON_UTF8)
                .content(mapper.writeValueAsString(new Trip())))
                .andExpect(MockMvcResultMatchers.status().isOk())
                .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.any(Integer.class)))
                .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("PAYED")));
    }

}

Now, you can run command mvn clean verify on the root module. It runs the tests in the following order: driver-management, passenger-management and trip-management.

The post Microservices Integration Tests with Hoverfly and Testcontainers appeared first on Piotr's TechBlog.

• Teams coordination – with many independent teams managing their own microservices, it becomes very challenging to coordinate the overall process of software development and testing
• Complexity – there are many microservices that communicate with each other. We need to ensure that every one of them is working properly and is resistant to the slow responses or failures from other microservices
• Performance – since there are many independent services it is essential to test the whole architecture under traffic close to the production

Let’s discuss some interesting frameworks helping that may help you test microservices-based architecture.

Components tests with Hoverfly

Hoverfly simulation mode may be beneficial for building component tests. During component tests, we are verifying the whole microservice without communication over the network with other microservices or external data stores. The following picture shows how such a test is performed for our sample microservice.

Hoverfly provides simple DSL for creating simulations, and a JUnit integration for using it within JUnit tests. It may be orchestrated via JUnit @Rule. We are simulating two services and then overriding Ribbon properties to resolve an address of these services by client name. We should also disable communication with Eureka discovery by disabling registration after the application boot or fetching a list of services for a Ribbon client. Hoverfly simulates responses for PUT and GET methods exposed by passenger-management and driver-management microservices. A controller is the main component that implements business logic in our application. It stores data using an in-memory repository component and communicates with other microservices through @FeignClient interfaces. By testing three methods implemented by the controller we are testing the whole business logic implemented inside the trip-management service.

@SpringBootTest(properties = {
   "eureka.client.enabled=false",
   "ribbon.eureka.enable=false",
   "passenger-management.ribbon.listOfServers=passenger-management",
   "driver-management.ribbon.listOfServers=driver-management"
})
@RunWith(SpringRunner.class)
@AutoConfigureMockMvc
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class TripComponentTests {

    ObjectMapper mapper = new ObjectMapper();

    @Autowired
    MockMvc mockMvc;
    @ClassRule
    public static HoverflyRule rule = HoverflyRule.inSimulationMode(SimulationSource.dsl(
            HoverflyDsl.service("passenger-management:80")
                    .get(HoverflyMatchers.startsWith("/passengers/login/"))

                    .willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'John Walker'}")))
                    .put(HoverflyMatchers.startsWith("/passengers")).anyBody()
                    .willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'John Walker'}"))),
            HoverflyDsl.service("driver-management:80")
                    .get(HoverflyMatchers.startsWith("/drivers/"))
                    .willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'David Smith','currentLocationX': 15,'currentLocationY':25}")))
                    .put(HoverflyMatchers.startsWith("/drivers")).anyBody()
                    .willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'David Smith','currentLocationX': 15,'currentLocationY':25}")))
    )).printSimulationData();

    @Test
    public void test1CreateNewTrip() throws Exception {
        TripInput ti = new TripInput("test", 10, 20, "walker");
        mockMvc.perform(MockMvcRequestBuilders.post("/trips")
                .contentType(MediaType.APPLICATION_JSON_UTF8)
                .content(mapper.writeValueAsString(ti)))
                .andExpect(MockMvcResultMatchers.status().isOk())
                .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.any(Integer.class)))
                .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("NEW")))
                .andExpect(MockMvcResultMatchers.jsonPath("$.driverId", Matchers.any(Integer.class)));
    }

    @Test
    public void test2CancelTrip() throws Exception {
        mockMvc.perform(MockMvcRequestBuilders.put("/trips/cancel/1")
                .contentType(MediaType.APPLICATION_JSON_UTF8)
                .content(mapper.writeValueAsString(new Trip())))
                .andExpect(MockMvcResultMatchers.status().isOk())
                .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.any(Integer.class)))
                .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("IN_PROGRESS")))
                .andExpect(MockMvcResultMatchers.jsonPath("$.driverId", Matchers.any(Integer.class)));
    }

    @Test
    public void test3PayTrip() throws Exception {
        mockMvc.perform(MockMvcRequestBuilders.put("/trips/payment/1")
                .contentType(MediaType.APPLICATION_JSON_UTF8)
                .content(mapper.writeValueAsString(new Trip())))
                .andExpect(MockMvcResultMatchers.status().isOk())
                .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.any(Integer.class)))
                .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("PAYED")));
    }
}

The tests visible above verify only positive scenarios. What about testing some unexpected behavior like network delays or server errors? With Hoverfly we can easily simulate such behavior and define some negative scenarios. In the following fragment of code, I have defined three scenarios. In the first of them, the target service has been delayed 2 seconds. In order to simulate timeout on the client side I had to change the default readTimeout for the Ribbon load balancer and then disabled the Hystrix circuit breaker for Feign client. The second test simulates the HTTP 500 response status from the passenger-management service. The last scenario assumes an empty response from the method responsible for searching for the nearest driver.

@SpringBootTest(properties = {
        "eureka.client.enabled=false",
        "ribbon.eureka.enable=false",
        "passenger-management.ribbon.listOfServers=passenger-management",
        "driver-management.ribbon.listOfServers=driver-management",
        "feign.hystrix.enabled=false",
        "ribbon.ReadTimeout=500"
})
@RunWith(SpringRunner.class)
@AutoConfigureMockMvc
public class TripNegativeComponentTests {
    private ObjectMapper mapper = new ObjectMapper();
    @Autowired
    private MockMvc mockMvc;

    @ClassRule
    public static HoverflyRule rule = HoverflyRule.inSimulationMode(SimulationSource.dsl(
            HoverflyDsl.service("passenger-management:80")
                    .get("/passengers/login/test1")
                    .willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'John Smith'}")).withDelay(2000, TimeUnit.MILLISECONDS))
                    .get("/passengers/login/test2")
                    .willReturn(ResponseCreators.success(HttpBodyConverter.jsonWithSingleQuotes("{'id':1,'name':'John Smith'}")))
                    .get("/passengers/login/test3")
                    .willReturn(ResponseCreators.serverError()),
            HoverflyDsl.service("driver-management:80")
                    .get(HoverflyMatchers.startsWith("/drivers/"))
                    .willReturn(ResponseCreators.success().body("{}"))
            ));

    @Test
    public void testCreateTripWithTimeout() throws Exception {
        mockMvc.perform(MockMvcRequestBuilders.post("/trips").contentType(MediaType.APPLICATION_JSON).content(mapper.writeValueAsString(new TripInput("test", 15, 25, "test1"))))
                .andExpect(MockMvcResultMatchers.status().isOk())
                .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.nullValue()))
                .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("REJECTED")));
    }

    @Test
    public void testCreateTripWithError() throws Exception {
        mockMvc.perform(MockMvcRequestBuilders.post("/trips").contentType(MediaType.APPLICATION_JSON).content(mapper.writeValueAsString(new TripInput("test", 15, 25, "test3"))))
                .andExpect(MockMvcResultMatchers.status().isOk())
                .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.nullValue()))
                .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("REJECTED")));
    }

    @Test
    public void testCreateTripWithNoDrivers() throws Exception {
        mockMvc.perform(MockMvcRequestBuilders.post("/trips").contentType(MediaType.APPLICATION_JSON).content(mapper.writeValueAsString(new TripInput("test", 15, 25, "test2"))))
                .andExpect(MockMvcResultMatchers.status().isOk())
                .andExpect(MockMvcResultMatchers.jsonPath("$.id", Matchers.nullValue()))
                .andExpect(MockMvcResultMatchers.jsonPath("$.status", Matchers.is("REJECTED")));
    }
}

All the timeouts and errors in communication with external microservices are handled by the bean annotated with @ControllerAdvice. In such cases trip-management microservice should not return a server error response, but 200 OK with JSON response containing field status equals to REJECTED.

@ControllerAdvice
public class TripControllerErrorHandler extends ResponseEntityExceptionHandler {

    @ExceptionHandler({RetryableException.class, FeignException.class})
    protected ResponseEntity handleFeignTimeout(RuntimeException ex, WebRequest request) {
        Trip trip = new Trip();
        trip.setStatus(TripStatus.REJECTED);
        return handleExceptionInternal(ex, trip, null, HttpStatus.OK, request);
    }

}

Contract testing with Pact framework

The next type of test strategy usually implemented for microservices-based architecture is consumer-driven contract testing. In fact, there are some tools especially dedicated to such a type of test. One of them is Pact. Contract testing is a way to ensure that services can communicate with each other without implementing integration tests. A contract is signed between two sides of communication: the consumer and the provider. Pact assumes that contract code is generated and published on the consumer side, and then verified by the provider.

Pact provides tools that can store and share the contracts between consumers and providers. It is called Pact Broker. It exposes a simple RESTful API for publishing and retrieving pacts, and embedded web dashboard for navigating the API. We can easily run Pact Broker on the local machine using its Docker image.

We will begin by running Pact Broker. Pact Broker requires running an instance of PostgreSQL, so first we have to launch it using a Docker image, and then link our broker container with that container.

$ docker run -d --name postgres postgres \
  -p 5432:5432 \
  -e POSTGRES_USER=oauth \ 
  -e POSTGRES_PASSWORD=oauth123 \
  -e POSTGRES_DB=oauth

$ docker run -d --name pact-broker dius/pact-broker \
  --link postgres:postgres \
  -e PACT_BROKER_DATABASE_USERNAME=oauth \
  -e PACT_BROKER_DATABASE_PASSWORD=oauth123 \ 
  -e PACT_BROKER_DATABASE_HOST=postgres \
  -e PACT_BROKER_DATABASE_NAME=oauth \
  -p 9080:80

The next step is to implement contract tests on the consumer side. We will use the JVM implementation of the Pact library for that. It provides a PactProviderRuleMk2 object responsible for creating stubs of the provider service. We should annotate it with JUnit @Rule. Ribbon will forward all requests to passenger-management to the stub address – in that case localhost:8180. Pact JVM supports annotations and provides DSL for building test scenarios. The test method responsible for generating contract data should be annotated with @Pact. It is important to set fields state and provider because then the generated contract would be verified on the provider side using these names. Generated pacts are verified inside the same test class by the methods annotated with @PactVerification. Field fragment points to the name of the method responsible for generating pact inside the same test class. The contract is tested using PassengerManagementClient @FeignClient.

@RunWith(SpringRunner.class)
@SpringBootTest(properties = {
        "driver-management.ribbon.listOfServers=localhost:8190",
        "passenger-management.ribbon.listOfServers=localhost:8180",
        "ribbon.eureka.enabled=false",
        "eureka.client.enabled=false",
        "ribbon.ReadTimeout=5000"
})
public class PassengerManagementContractTests {
    @Rule
    public PactProviderRuleMk2 stubProvider = new PactProviderRuleMk2("passengerManagementProvider", "localhost", 8180, this);
    @Autowired
    private PassengerManagementClient passengerManagementClient;

    @Pact(state = "get-passenger", provider = "passengerManagementProvider", consumer = "passengerManagementClient")
    public RequestResponsePact callGetPassenger(PactDslWithProvider builder) {
        DslPart body = new PactDslJsonBody().integerType("id").stringType("name").numberType("balance").close();
        return builder.given("get-passenger").uponReceiving("test-get-passenger")
                .path("/passengers/login/test").method("GET").willRespondWith().status(200).body(body).toPact();
    }

    @Pact(state = "update-passenger", provider = "passengerManagementProvider", consumer = "passengerManagementClient")
    public RequestResponsePact callUpdatePassenger(PactDslWithProvider builder) {
        return builder.given("update-passenger").uponReceiving("test-update-passenger")
                .path("/passengers").method("PUT").bodyWithSingleQuotes("{'id':1,'amount':1000}", "application/json").willRespondWith().status(200)
                .bodyWithSingleQuotes("{'id':1,'name':'Adam Smith','balance':5000}", "application/json").toPact();
    }

    @Test
    @PactVerification(fragment = "callGetPassenger")
    public void verifyGetPassengerPact() {
        Passenger passenger = passengerManagementClient.getPassenger("test");
        Assert.assertNotNull(passenger);
        Assert.assertNotNull(passenger.getId());
    }

    @Test
    @PactVerification(fragment = "callUpdatePassenger")
    public void verifyUpdatePassengerPact() {
        Passenger passenger = passengerManagementClient.updatePassenger(new PassengerInput(1L, 1000));
        Assert.assertNotNull(passenger);
        Assert.assertNotNull(passenger.getId());
    }
}

Just running the tests is not enough. We also have to publish pacts generated during tests to Pact Broker. In order to achieve it we have to include the following Maven plugin to our pom.xml and then execute command mvn clean install pact:publish.

<plugin>
   <groupId>au.com.dius</groupId>
   <artifactId>pact-jvm-provider-maven_2.12</artifactId>
   <version>3.5.21</version>
   <configuration>
      <pactBrokerUrl>http://192.168.99.100:9080</pactBrokerUrl>
   </configuration>
</plugin>

Pact provides support for Spring on the provider side. Thanks to that we may use MockMvc controllers or inject properties from application.yml into the test class. Here’s the dependency declaration that has to be included to our pom.xml

<dependency>
   <groupId>au.com.dius</groupId>
   <artifactId>pact-jvm-provider-spring_2.12</artifactId>
   <version>3.5.21</version>
   <scope>test</scope>
</dependency>

Now, the contract is being verified on the provider side. We need to pass the provider name inside @Provider annotation and name of states for every verification test inside @State. These values have been during the tests on the consumer side inside @Pact annotation (fields state and provider).

@RunWith(SpringRestPactRunner.class)
@Provider("passengerManagementProvider")
@PactBroker
public class PassengerControllerContractTests {
    @InjectMocks
    private PassengerController controller = new PassengerController();
    @Mock
    private PassengerRepository repository;
    @TestTarget
    public final MockMvcTarget target = new MockMvcTarget();

    @Before
    public void before() {
        MockitoAnnotations.initMocks(this);
        target.setControllers(controller);
    }

    @State("get-passenger")
    public void testGetPassenger() {
        target.setRunTimes(3);
        Mockito.when(repository.findByLogin(Mockito.anyString()))
                .thenReturn(new Passenger(1L, "Adam Smith", "test", 4000))
                .thenReturn(new Passenger(3L, "Tom Hamilton", "hamilton", 400000))
                .thenReturn(new Passenger(5L, "John Scott", "scott", 222));
    }

    @State("update-passenger")
    public void testUpdatePassenger() {
        target.setRunTimes(1);
        Passenger passenger = new Passenger(1L, "Adam Smith", "test", 4000);
        Mockito.when(repository.findById(1L)).thenReturn(passenger);
        Mockito.when(repository.update(Mockito.any(Passenger.class)))
                .thenReturn(new Passenger(1L, "Adam Smith", "test", 5000));
    }
}

The Pact Broker host and port are injected from application.yml file.

pactbroker:
  host: "192.168.99.100"
  port: "8090"

Performance tests with Gatling

An important step of testing microservices before deploying them on production is performance testing. One of the interesting tools in this area is Gatling. It is a highly capable load-testing tool written in Scala. It means that we also have to use Scala DSL in order to build test scenarios. Let’s begin by adding the required library to pom.xml file.

<dependency>
   <groupId>io.gatling.highcharts</groupId>
   <artifactId>gatling-charts-highcharts</artifactId>
   <version>2.3.1</version>
</dependency>

Now, we may proceed to the test. In the scenario visible above we are testing two endpoints exposed by trip-management: POST /trips and PUT /trips/payment/${tripId}. In fact, this scenario verifies the whole functionality of our sample system, where we are setting up a trip and then paying for it after a finish.
Every test class using Gatling needs to extend the Simulation class. We are defining the scenario using the scenario method and then set its name. We may define multiple executions inside a single scenario. After every execution of the POST /trips method, the test save generated id returned by the service. Then it inserts that id into the URL used for calling method PUT /trips/payment/${tripId}. Every single test expects a response with 200 OK status.

Gatling provides two interesting features, which are worth mentioning. You can see how they are used in the following performance test. First of all, it is a feeder. It is used for polling records and injecting their content into the test. Feed rPassengers selects one of five defined logins randomly. The final test result may be verified using Assertions API. It is responsible for verifying global statistics like response time or number of failed requests matches expectations for a whole simulation. In the scenario visible below the criterium is max response time that needs to be lower 100 milliseconds.

class CreateAndPayTripPerformanceTest extends Simulation {

  val rPassengers = Iterator.continually(Map("passenger" -> List("walker","smith","hamilton","scott","holmes").lift(Random.nextInt(5)).get))

  val scn = scenario("CreateAndPayTrip").feed(rPassengers).repeat(100, "n") {
    exec(http("CreateTrip-API")
      .post("http://localhost:8090/trips")
      .header("Content-Type", "application/json")
      .body(StringBody("""{"destination":"test${n}","locationX":${n},"locationY":${n},"username":"${passenger}"}"""))
      .check(status.is(200), jsonPath("$.id").saveAs("tripId"))
    ).exec(http("PayTrip-API")
      .put("http://localhost:8090/trips/payment/${tripId}")
      .header("Content-Type", "application/json")
      .check(status.is(200))
    )
  }

  setUp(scn.inject(atOnceUsers(20))).maxDuration(FiniteDuration.apply(5, TimeUnit.MINUTES))
    .assertions(global.responseTime.max.lt(100))
}

In order to run a Gatling performance test you need to include the following Maven plugin to your pom.xml. You may run a single scenario or run multiple scenarios. After including the plugin you only need to execute the command mvn clean gatling:test.

<plugin>
   <groupId>io.gatling</groupId>
   <artifactId>gatling-maven-plugin</artifactId>
   <version>2.2.4</version>
   <configuration>
      <simulationClass>pl.piomin.performance.tests.CreateAndPayTripPerformanceTest</simulationClass>
   </configuration>
</plugin>

Here are some diagrams illustrating the results of performance tests for our microservice. Because the maximum response time has been greater than set inside assertion (100ms), the test has failed.

and …

Summary

The right selection of tools is not the most important element phase of microservices testing. However, the right tools can help you face the key challenges related to it. Hoverfly allows you to create full component tests that verifies if your microservice is able to handle delays or errors from downstream services. Pact helps you to organize a team by sharing and verifying contracts between independently developed microservices. Finally, Gatling can help you implement load tests for selected scenarios, in order to verify the end-to-end performance of your system.
The source code used as a demo for this article is available on GitHub: https://github.com/piomin/sample-testing-microservices.git. If you find this article interesting for you you may be also interested in some other articles related to this subject:

• Testing REST API with Hoverfly
• Performance Testing with Gatling
• Continuous Integration with Jenkins, Artifactory and Spring Cloud Contract

The post Testing Microservices: Tools and Frameworks appeared first on Piotr's TechBlog.

In one of my previous article Testing Java Microservices I described the competitive tool for testing – Spring Cloud Contract. In the article about Hoverfly I will use the same sample application based on Spring Boot, which I created for the needs of that previous article. Source code is available on GitHub in hoverfly branch. We have some microservices which interact between each other and basing on this sample I’m going to show how to use Hoverfly for component testing.

To enable testing with Hoverfly we have to include the following dependency in pom.xml file.

[code language=”xml”]
<dependency>
<groupId>io.specto</groupId>
<artifactId>hoverfly-java</artifactId>
<version>0.8.0</version>
<scope>test</scope>
</dependency>
[/code]

Hoverfly can be easily integrated with JUnit. We can orchestrate it using JUnit @ClassRule. Like I mentioned before we can switch between two different modes. In the code fragment below I decided two use mixed strategy inCaptureOrSimulationMode, where Hoverfly Rule is started in capture mode if the simulation file does not exist and in simulate mode if the file does exist. The default location of output JSON file is src/test/resources/hoverfly. By calling printSimulationData on HoverflyRule we are printing all simulation data on the console.

[code language=”java”]
@RunWith(SpringRunner.class)
@SpringBootTest(classes = { Application.class }, webEnvironment = WebEnvironment.DEFINED_PORT)
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class AccountApiFullTest {

protected Logger logger = Logger.getLogger(AccountApiFullTest.class.getName());

@Autowired
TestRestTemplate template;

@ClassRule
public static HoverflyRule hoverflyRule = HoverflyRule
.inCaptureOrSimulationMode("account.json", HoverflyConfig.configs().proxyLocalHost()).printSimulationData();

@Test
public void addAccountTest() {
Account a = new Account("1234567890", 1000, "1");
ResponseEntity<Account> r = template.postForEntity("/accounts", a, Account.class);
Assert.assertNotNull(r.getBody().getId());
logger.info("New account: " + r.getBody().getId());
}

@Test
public void findAccountByNumberTest() {
Account a = template.getForObject("/accounts/number/{number}", Account.class, "1234567890");
Assert.assertNotNull(a);
logger.info("Found account: " + a.getId());
}

@Test
public void findAccountByCustomerTest() {
Account[] a = template.getForObject("/accounts/customer/{customer}", Account[].class, "1");
Assert.assertTrue(a.length > 0);
logger.info("Found accounts: " + a);
}

}
[/code]

Now, let’s run our JUnit test class twice. During first attempt all requests are forwarded to the Spring @RestController which connects to embedded Mongo database. At the same time all requests and responses are recorded by Hoverfly and saved in the account.json file. This file fragment is visible below. During the second attempt all data is loaded from source file, there is no interaction with AccountController.

[code language=”java”]
"request" : {
"path" : {
"exactMatch" : "/accounts/number/1234567890"
},
"method" : {
"exactMatch" : "GET"
},
"destination" : {
"exactMatch" : "localhost:2222"
},
"scheme" : {
"exactMatch" : "http"
},
"query" : {
"exactMatch" : ""
},
"body" : {
"exactMatch" : ""
}
},
"response" : {
"status" : 200,
"body" : "{\"id\":\"5980642bc96045216447023b\",\"number\":\"1234567890\",\"balance\":1000,\"customerId\":\"1\"}",
"encodedBody" : false,
"templated" : false,
"headers" : {
"Content-Type" : [ "application/json;charset=UTF-8" ],
"Date" : [ "Tue, 01 Aug 2017 11:21:15 GMT" ],
"Hoverfly" : [ "Was-Here" ]
}
}
[/code]

Now, let’s take a look on customer-service tests. Inside GET /customer/{id} we are invoking method GET /accounts/customer/{customerId} from account-service. This method is simulating by Hoverfly with success response as you can see below.

[code language=”java”]
@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = WebEnvironment.DEFINED_PORT)
@FixMethodOrder(MethodSorters.NAME_ASCENDING)
public class CustomerControllerTest {

@Autowired
TestRestTemplate template;

@ClassRule
public static HoverflyRule hoverflyRule = HoverflyRule
.inSimulationMode(dsl(service("account-service:2222").get(startsWith("/accounts/customer/"))
.willReturn(success("[{\"id\":\"1\",\"number\":\"1234567890\"}]", "application/json"))))
.printSimulationData();

@Test
public void addCustomerTest() {
Customer c = new Customer("1234567890", "Jan Testowy", CustomerType.INDIVIDUAL);
c = template.postForObject("/customers", c, Customer.class);
}

@Test
public void findCustomerWithAccounts() {
Customer c = template.getForObject("/customers/pesel/{pesel}", Customer.class, "1234567890");
Customer cc = template.getForObject("/customers/{id}", Customer.class, c.getId());
Assert.assertTrue(cc.getAccounts().size() > 0);
}
}
[/code]

To run this test successfully we should override some properties from application.yml in src/test/resources/application.yml. Eureka discovery from Ribbon client should be disabled and the same for Hystrix in @FeignClient. Ribbon listOfServers property should has same value as service address inside HoverflyRule.

[code]
eureka:
client:
enabled: false

ribbon:
eureka:
enable: false
listOfServers: account-service:2222

feign:
hystrix:
enabled: false
[/code]

Here’s @FeignClient implementation for invoking API method from account-service.

[code language=”java”]
@FeignClient("account-service")
public interface AccountClient {

@RequestMapping(method = RequestMethod.GET, value = "/accounts/customer/{customerId}", consumes = {MediaType.APPLICATION_JSON_VALUE})
List<Account> getAccounts(@PathVariable("customerId") String customerId);

}
[/code]

When using simulation mode there is no need to start @SpringBootTest. Hoverfly has also some interesting capabilities like response templating, for example basing on path parameter, like in the fragment below.

[code language=”java”]
public class AccountApiTest {

TestRestTemplate template = new TestRestTemplate();

@ClassRule
public static HoverflyRule hoverflyRule = HoverflyRule.inSimulationMode(dsl(service("http://account-service")
.post("/accounts").anyBody().willReturn(success("{\"id\":\"1\"}", "application/json"))
.get(startsWith("/accounts/")).willReturn(success("{\"id\":\"{{Request.Path.[1]}}\",\"number\":\"123456789\"}", "application/json"))));

@Test
public void addAccountTest() {
Account a = new Account("1234567890", 1000, "1");
ResponseEntity<Account> r = template.postForEntity("http://account-service/accounts", a, Account.class);
System.out.println(r.getBody().getId());
}

@Test
public void findAccountByIdTest() {
Account a = template.getForObject("http://account-service/accounts/{id}", Account.class, new Random().nextInt(10));
Assert.assertNotNull(a.getId());
}

}
[/code]

We can simulate fixed method delay using DSL. Delay be set for all requests or for a particular HTTP method. Our delayed @ClassRule for CustomerControllerTest will now look like in the fragment below.

[code language=”java”]
@ClassRule
public static HoverflyRule hoverflyRule = HoverflyRule
.inSimulationMode(dsl(service("account-service:2222").andDelay(3000, TimeUnit.MILLISECONDS).forMethod("GET").get(startsWith("/accounts/customer/"))
.willReturn(success("[{\"id\":\"1\",\"number\":\"1234567890\"}]", "application/json"))));
[/code]

And now you can add ReadTimeout property into your Ribbon client configuration and run JUnit test again. You should receive the follwoing exception: java.net.SocketTimeoutException: Read timed out

[code]
ribbon:
eureka:
enable: false
ReadTimeout: 1000
listOfServers: account-service:2222
[/code]

Conclusion

In the post I showed you the most typical usage of Hoverfly library in microservices tests. However, this library is not dedicated to microservice testing as opposed to the Spring Cloud Contract previously described by me. For example, there is no mechanisms for sharing test stubs between different microservices like in Spring Cloud Contract (@AutoConfigureStubRunner). But there is an interesting feature for delaying responses thanks to which we can simulate some timeouts for Ribbon client or Hystrix fallback.

The post Testing REST API with Hoverfly appeared first on Piotr's TechBlog.