devkuma – Testcontainers

Introduction to Testcontainers

kc@example.com (kc kim) — Thu, 25 Aug 2022 10:42:43 +0900

Overview

The test environment is one of the most important parts of project setup. Testing in an environment that matches production as closely as possible is important.
However, it is not possible to build the same environment every time, and it is not easy to align the environment across all developers.

Ideally, each time test code runs, Docker would start containers for the test, the test would run, and then the containers would be stopped and removed afterward. Testcontainers makes this possible.

What is Testcontainers?

Testcontainers is a Java library that runs Docker containers for tests.
It lets you build infrastructure with Docker and supports a wide range of services. It also helps satisfy the Repeatable principle in FIRST.

Testcontainers Sites

Official documentation: https://www.testcontainers.org/
GitHub: https://github.com/testcontainers/testcontainers-java/

Why Use Testcontainers?

The test environment is one of the most important parts of project setup.

It can also be one of the most difficult and tedious tasks, but if it is set up well once, you can write clean test code later without worrying about the environment. At the same time, it is one of the areas that tends to take the most time during project setup and often involves many rounds of trial and error.

Testcontainers provides an environment where systems such as MySQL, PostgreSQL, Redis, Apache Kafka, Amazon SQS, and many others can be created as Docker containers and used for testing.

Without Testcontainers, you would need to create a real system environment and initialize test data every time you run tests. Also, if someone else accesses the system environment created for testing at the same time, another test could cause your test to fail. External modules could also make tests fail intermittently.

In other words, results can change depending on the situation or external influences. When this happens, it is very hard to find the failing area. This is often described as idempotency not being maintained, and idempotency is a very important concept in tests. Preserving idempotency has a large impact on overall test productivity.

What is idempotency?

It is the property that the result does not change even when an operation is applied multiple times. Put simply, a function or API should always produce the same result even if it is executed repeatedly.

Support for Various Modules

Testcontainers supports many modules. A few examples are listed below.

Using Testcontainers

This section briefly explains how to use it.

Add Libraries

To use Testcontainers with JUnit 5 in Gradle, add the following libraries.

testImplementation "org.junit.jupiter:junit-jupiter:5.8.1"
testImplementation "org.testcontainers:testcontainers:1.17.3"
testImplementation "org.testcontainers:junit-jupiter:1.17.3"

Write Test Code

@Testcontainers
public class RedisBackedCacheIntTest {

    private RedisBackedCache underTest;

    // container {
    @Container
    public GenericContainer redis = new GenericContainer(DockerImageName.parse("redis:5.0.3-alpine"))
        .withExposedPorts(6379);

    // }

    @BeforeEach
    public void setUp() {
        String address = redis.getHost();
        Integer port = redis.getFirstMappedPort();

        // Now we have an address and port for Redis, no matter where it is running
        underTest = new RedisBackedCache(address, port);
    }

    @Test
    public void testSimplePutAndGet() {
        underTest.put("test", "example");

        String retrieved = underTest.get("test");
        assertThat(retrieved).isEqualTo("example");
    }
}

`@Testcontainers`

To run a Redis container during the test, first add @Testcontainers to the test class.

@Testcontainers
public class RedisBackedCacheIntTest {

Create the Container

@Container
public GenericContainer redis = new GenericContainer(DockerImageName.parse("redis:5.0.3-alpine"))
    .withExposedPorts(6379);

The @Container annotation tells JUnit to notify this field about various events in the test lifecycle. Here, the container is a Testcontainers GenericContainer, configured to use a specific Redis image from Docker Hub and expose the specified port, 6379.

Check Whether the Code Can Communicate with the Container

Ask Testcontainers for the container address.

String address = redis.getHost();

Ask Testcontainers for the container port.

Integer port = redis.getFirstMappedPort();

Container Reuse (Option)

When using Testcontainers, a new Docker container is created every time tests run, which can take a long time.

To address this, Testcontainers provides a container reuse option, although it is still not a publicly supported feature. This relates to the Fast principle.

Requirements:

Configure the .testcontainers.properties file in the developer’s home directory.
```
testcontainers.reuse.enable=true
```

Specify the TC_REUSABLE=true option in the JDBC URL.

jdbc:tc:mysql:///test?TC_REUSABLE=true&TC_INITSCRIPT=file:src/test/resources/schema.ddl

Container reuse is possible only when both settings above are configured.

How to Run Testcontainers with Colima on macOS with Apple M1 Chips

kc@example.com (kc kim) — Wed, 23 Aug 2023 01:05:59 +0900

Docker changed the terms of service for Docker Desktop, so developers working at large companies can no longer use Docker Desktop for free. To use Docker for free on macOS, an alternative driver is needed. Several alternatives, such as hyperkit and VirtualBox, have been suggested, but they still do not seem easy to use on macOS with Apple M1 chips.

As of August 2023, the situation is as follows.

hyperkit: Does not work in an Apple M1 chip environment. Installation with Homebrew fails.
VirtualBox: The version that can run in an Apple M1 chip environment is still beta.

This article introduces the simplest way to run Testcontainers on Apple M1 with Colima, without using Docker Desktop.

Docker Setup

First, install Docker with Homebrew.

brew install docker

Colima Setup

Next, install Colima with Homebrew.

brew install colima

Then start Colima with the specified CPU and memory.

colima start --cpu 4 --memory 8

The command above runs the VM with 4 CPUs and 8 GiB of memory.

The Colima server becomes available at the following socket address.

unix:///Users/${HOME}/.colima/default/docker.sock.

You can connect to the Colima server with the Docker client.

You can check all contexts with the docker context ls command and switch to Colima with docker context use colima.

% docker context ls
NAME            DESCRIPTION                               DOCKER ENDPOINT                                  ERROR
colima *        colima                                    unix:///Users/user/.colima/default/docker.sock
default         Current DOCKER_HOST based configuration   unix:///var/run/docker.sock
desktop-linux                                             unix:///Users/user/.docker/run/docker.sock

To stop Colima, run the following command.

colima stop

Testcontainers Setup

Apple M1

If you use macOS on an Apple M1 chip, make sure that at least JNA (Java Native Access) version 5.7.0 or later is being used.

In IntelliJ IDEA, check whether the following library is included as a dependency.

If the version is too low, update it to 5.7.0.

Testcontainers Environment Variables

To run Testcontainers with Colima, the following environment variables are required.

DOCKER_HOST=unix://${HOME}/.colima/default/docker.sock
TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock
TESTCONTAINERS_RYUK_DISABLED=true

These environment variables are needed for the following reasons.

DOCKER_HOST: Specifies Colima’s socket address.
TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE: Specifies the socket address inside the container.
TESTCONTAINERS_RYUK_DISABLED: Disables Ryuk. Otherwise, containers may not stop and tests may time out.

In a Bash or Zsh shell, you can override the Docker host by declaring the environment variables as follows.
~/.bashrc or ~/.zshrc

export TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock
export DOCKER_HOST="unix://${HOME}/.colima/docker.sock"
export TESTCONTAINERS_RYUK_DISABLED=true

After changing the environment, restart IntelliJ IDEA so the changes are applied.

Testcontainers

kc@example.com (kc kim) — Mon, 01 Jan 0001 00:00:00 +0000

Kotest Testcontainers Extension

kc@example.com (kc kim) — Sun, 31 Mar 2024 15:11:00 +0900

Testcontainers

Testcontainers is a tool for building independent and reusable test environments. It supports container-based testing, so you can programmatically build the same environment as the test environment running on your local machine. Kotest can be used with Testcontainers to manage external resources needed for tests, helping you write more robust and reliable tests.

The main advantages of using Testcontainers are:

Consistent environment: Provides a consistent environment each time a test runs, making test results predictable.
Independence: Containerizes external resources so tests do not affect each other.
Ease of building test environments: Lets you programmatically build and manage test environments with code.
Support for various containers: Supports various kinds of containers, such as Docker and Kubernetes, so you can build many test scenarios.

The Testcontainers project provides lightweight temporary instances of common databases, Elasticsearch, Kafka, Selenium web browsers, or anything else that can run in a Docker container, making them ideal for use inside tests.

Kotest provides integration with Testcontainers through an additional module that offers several extensions, including special extensions for databases and Kafka and general container support for all supported Docker images.

Adding the Dependency

To use it, add the following dependency to your Gradle build file.

dependencies {
    testImplementation("io.kotest.extensions:kotest-extensions-testcontainers:${kotestVersion}")
}

Note: The group ID io.kotest.extensions differs from the main Kotest group ID, io.kotest.

For Maven projects, add the dependency as follows:

<dependency>
    <groupId>io.kotest.extensions</groupId>
    <artifactId>kotest-extensions-testcontainers</artifactId>
    <version>${kotest.version}</version>
    <scope>test</scope>
</dependency>

Databases

Using Testcontainers in Kotest is straightforward. Start the required container using the Testcontainers library inside the test code, run the test, and then clean up the container afterward.

For example, when performing database integration tests, it is common to start the database container at the beginning of the test and stop it when the test ends.

For this, Kotest lets you use beforeTest and afterTest to perform required work before and after test execution.

The following is example code using Kotest and Testcontainers together:

import io.kotest.core.spec.style.FreeSpec
import org.testcontainers.containers.PostgreSQLContainer
import org.testcontainers.junit.jupiter.Container
import org.testcontainers.junit.jupiter.Testcontainers

@Testcontainers
class PostgreSqlTest : FreeSpec() {

    companion object {
        @Container
        val postgresContainer: PostgreSQLContainer<Nothing> = PostgreSQLContainer<Nothing>()
            .withDatabaseName("test")
            .withUsername("test")
            .withPassword("test")
            .withExposedPorts(5432)
    }

    override fun beforeTest(testCase: TestCase) {
        postgresContainer.start()
        System.setProperty("DB_URL", postgresContainer.jdbcUrl)
        System.setProperty("DB_USERNAME", postgresContainer.username)
        System.setProperty("DB_PASSWORD", postgresContainer.password)
    }

    override fun afterTest(testCase: TestCase, result: TestResult) {
        postgresContainer.stop()
    }

    init {
        "Database test" {
            // write database test code
        }
    }
}

In the code above, a PostgreSQL container is used to perform database testing. The @Testcontainers annotation enables Testcontainers, and the @Container annotation defines the container. Then beforeTest and afterTest are overridden to start and stop the container before and after test execution.

devkuma – Testcontainers

Introduction to Testcontainers

Overview

What is Testcontainers?

Testcontainers Sites

Why Use Testcontainers?

Support for Various Modules

Using Testcontainers

Add Libraries

Write Test Code

@Testcontainers

Create the Container

Check Whether the Code Can Communicate with the Container

Container Reuse (Option)

How to Run Testcontainers with Colima on macOS with Apple M1 Chips

Docker Setup

Colima Setup

Testcontainers Setup

Apple M1

Testcontainers Environment Variables

Testcontainers

Kotest Testcontainers Extension

Testcontainers

Adding the Dependency

Databases

References

`@Testcontainers`