devkuma – Kotest

Kotest Framework

kc@example.com (kc kim) — Sun, 10 Mar 2024 13:28:00 +0900

Kotest Extensions

kc@example.com (kc kim) — Sun, 10 Mar 2024 13:03:00 +0900

Kotest Assertions

kc@example.com (kc kim) — Mon, 11 Mar 2024 12:42:00 +0900

Kotest Property Testing

kc@example.com (kc kim) — Sun, 17 Mar 2024 21:13:00 +0900

Kotest IntelliJ Plugin

kc@example.com (kc kim) — Sun, 25 Feb 2024 10:28:00 +0900

Kotest Property-based Testing

kc@example.com (kc kim) — Sun, 17 Mar 2024 21:48:00 +0900

Property-based Testing

Kotest is split into several independently usable subprojects. One of these subprojects is the property-based testing framework. As expected, you do not have to use Kotest as your test framework to benefit from its property testing support.

What Is Property Testing?

Developers usually write example-based tests. These are the familiar unit tests you know and use: provide some input values and expected results, and a test framework such as Kotest or JUnit checks whether the actual result matches the expectation.

One problem with this approach is that it is easy to miss errors because of edge cases the developer did not think of, or because the selected inputs do not provide enough coverage. With property testing, hundreds or thousands of values are fed into the same test, and those values are usually generated randomly by the property testing framework.

For example, a good property testing framework includes values such as negative infinity, empty lists, and strings containing non-ASCII characters. These are the kinds of cases we often forget when writing example-based tests.

Property testing was originally devised in frameworks such as QuickCheck as a way to test properties of an object, that is, properties that must be true for every input. In other words, invariants. As an example of an invariant, given two strings a and b, length(a + b) should always equal length(a) + length(b).

This is where the term property testing comes from.

However, property testing is not limited to things like monad laws or basic numeric functions. Any test that can benefit from a wide range of input values is a good candidate. For example, if you have a function that validates usernames and you want to test that valid emails are accepted, property testing is useful because it can generate thousands of combinations that strengthen the validation logic.

Getting Started

The property testing framework is supported on all targets.

JVM/Gradle

Add the following dependency to your build:

dependencies {
    testImplementation("io.kotest:kotest-property:$version")
}

JVM/Maven

Add the following dependency to your build:

<dependency>
   <groupId>io.kotest</groupId>
   <artifactId>kotest-property-jvm</artifactId>
   <version>${version}</version>
   <scope>test</scope>
</dependency>

Multiplatform

Add the following dependency to the commonTest source set:

kotlin {
  sourceSets {
    val commonTest by getting {
      dependencies {
        implementation("io.kotest:kotest-property:$version")
      }
    }
  }
}

You can also add dependencies to a specific target. For example, you can add a dependency used only by the jsTest source set for the JavaScript target.

kotlin {
   targets {
      js {
         browser()
         nodejs()
      }
   }
   sourceSets {
      val jsTest by getting {
         dependencies {
            implementation("io.kotest:kotest-property:$version")
         }
      }
   }
}

Next Steps

To generate input values for tests, Kotest uses the term generator. One generator per input argument is passed to the test function, and the test runs for the configured number of iterations.

Learn more about:

How test functions are used.
The different types of generators and their operations.
How to write custom generators.
How to specify configuration for tests, including seeds.

References

Property-based Testing | Kotest

Kotest Test Assertions

kc@example.com (kc kim) — Mon, 11 Mar 2024 12:42:00 +0900

Kotest is divided into several subprojects that can be used independently. One of these subprojects provides comprehensive assertion and matcher support. It can be used with the Kotest test framework or with other test frameworks such as JUnit or Spock.

The core feature of the assertion module is functions that test state. Kotest calls these types of state assertion functions matchers. There are core matchers and matchers for third-party libraries.

There are also many other utilities for writing tests, such as exception testing, functions that help test non-deterministic code, inspectors for collections, and soft assertions that group assertions.

Various Matchers

For example, to verify that a variable has an expected value, you can use the shouldBe function.

name shouldBe "devkuma"

There are general-purpose matchers such as shouldBe, as well as matchers for many specific scenarios, such as str.shouldHaveLength(10) for testing the length of a string and file.shouldBeDirectory() for testing whether a specific file points to a directory. These functions are provided in both infix and regular forms.

For example, assertions can usually be chained:

"substring".shouldContain("str")
           .shouldBeLowerCase()

myImageFile.shouldHaveExtension(".jpg")
           .shouldStartWith("https")

There are more than 350 matchers across several modules. Check all matchers here.

Clues

Sometimes a failed assertion may not contain enough information to tell exactly what went wrong.

For example, suppose you verify the following:

user.name shouldNotBe null

If this verification fails, it simply outputs:

<null> should not equal <null>

This is not particularly helpful. You can use clues to add additional context to failure messages.

Inspectors

Inspectors let you test elements in a collection and specify how many elements are expected to pass, such as all, none, or exactly k.

For example, you can check three elements in mylist as follows:

mylist.forExactly(3) {
    it.city shouldBe "Chicago"
}

Check inspectors here.

Custom Matchers

You can easily add your own matchers by extending the Matcher<T> interface. Here, T is the type you want to match. Custom matchers can be composed from existing matchers or used completely independently.

See the full working example.

References

Assertions | Kotest

Kotest External Extensions

kc@example.com (kc kim) — Sun, 10 Mar 2024 20:05:00 +0900

The extensions introduced here are separate library-based extensions, distinct from the Listeners included in io.kotest.core and described in the framework section.

Kotest Team Extensions

Project	Description
Allure	Provides output for the Allure framework.
HTML Reporter	Generates an HTML report of test results based on JUnit XML.
Instant	Overrides the `now` context in `java.time`.
JUnit XML	Provides output in JUnit XML format for integration with tools.
Ktor	Provides matchers for Ktor endpoints.
Koin	Kotlin DI framework.
Mockserver	Integrates a mock HTTP server with the test lifecycle.
Spring	Adds support for Spring beans in tests and Spring test annotations.
System	Provides utilities for integration with `System.*` functions.
Testcontainers	Runs Docker images as part of the test lifecycle.
Wiremock	Provides an HTTP server. See the project homepage here.

Third Party Extensions

Project	Description
Android	Toolbox for working with Kotest and Android.
FluentLenium	Integration between Kotest and FluentLenium.
H2 Database Extension	Integration between Kotest and the H2 database.
Http4k	Feature toolkit for Kotlin HTTP applications.
Kotless	Utilities for Kotest and Kotless.
KotlinFixture	Generates well-defined but random inputs by default.
LogCapture	Test library for asserting logging messages.
Micronaut	JVM-based full-stack Micronaut framework for building modular microservices that are easy to test.
Result4s	Result4s matchers.
Selfie	Self snapshot testing, including inline, disk, and memoized snapshots.
Sniffy	Network connection testing.
TestFiles	Structured file and directory creation for tests.

References

Extensions | Kotest

Kotest IntelliJ Plugin Introduction

kc@example.com (kc kim) — Sun, 25 Feb 2024 10:28:00 +0900

Kotest provides an IntelliJ plugin that is available from the JetBrains plugin marketplace, or by searching inside IntelliJ.

The plugin provides run icons for each test, a tool window for test navigation, duplicate test highlighting, assertion intentions, and more.

NOTE

The IntelliJ plugin requires Kotest 4.2 or later, and common tests in multiplatform projects are not run.

Gutter Icons

The IntelliJ plugin requires Kotest 4.2 or later, and common tests in multiplatform projects are not run.

Tests disabled with bang, or tests disabled with xfunctions such as xdescribe, show a disabled test icon in the gutter.

Running Tests

Running a spec from Gutter Icons runs all tests in that spec. Running a test runs that test and all nested tests.

Duplicate Test Highlighting

You cannot have two tests with the same name. The plugin highlights duplicate test names as errors.

When you right-click a package, you can run, debug, or run with coverage all tests in that package.

Intentions

The plugin includes several basic intentions. For example, you can quickly mark a test as disabled.

You can also highlight some text and mark it as something that should throw, or surround it with a soft assertion block.

References

IntelliJ Plugin | Kotest

Kotest Overview

kc@example.com (kc kim) — Sun, 25 Feb 2024 10:28:00 +0900

What Is Kotest?

Kotlin is a modern and powerful language that helps developers build stable and robust software. Effective testing is essential for maintaining quality and guaranteeing functionality, so Kotlin developers use powerful testing frameworks such as Kotest.

In Java environments, JUnit has long been widely used for testing. JUnit can also be used in Kotlin and is still common. JUnit is a testing framework implemented in Java, while Kotest is a testing framework implemented in pure Kotlin. With Kotest, you can use Kotlin syntax more naturally than with JUnit, which can reduce the amount of code. It also lets you write nested tests by including test cases inside other test cases. This provides a way to express tests in a more modular and structured form.

Kotest is a flexible and powerful testing framework written in Kotlin. It uses Kotlin’s language features to make test writing and management easier and more effective. The framework provides many test styles and configuration options so it can support a wide range of testing requirements.

The official site explains the details, but as a short introduction, one of the main features is that Kotest provides ten kinds of classes called Specs, and you can choose the Spec you want when writing tests. Each Spec is influenced by different languages and testing frameworks, so developers coming to Kotlin from another language can choose a test style familiar from their own background.

Kotest official site

Kotest also provides many features, assertions, and extensions, including experimental features.

This section summarizes Kotest’s basic syntax and also explains libraries that support Kotest.

What was Kotest originally?

Kotest is a testing framework for Kotlin. It was previously named KotlinTest, but from release 4.0 it was renamed to Kotest to avoid confusion with packages provided by JetBrains.

Kotest Benefits and Features

Kotest has many benefits and features.

Kotest Benefits

Kotest integrates closely with the Kotlin language and makes full use of Kotlin’s powerful features.
Kotest supports many testing styles, so developers can choose the style that fits their preference.
Kotest provides many features that improve test writing and management.

These features include assertions and matchers, mocking and stubbing, and test data management. Finally, Kotest provides an extensible plugin architecture, so users can extend the library as needed.

Kotest provides the tools and features needed to write tests in Kotlin-based projects, and it helps developers build stable, high-quality software.

Kotest Features

Uses the Kotest DSL
Lets you write simple and expressive tests with multiple TestStyles
Supports testing many parameter combinations with data-driven tests
Supports fine-grained test configuration for invocations, parallel processing, timeouts, test grouping, conditional disabling, and more
Provides nested tests
Provides dynamic tests, allowing tests to be added conditionally at runtime
Provides various callbacks for the test lifecycle

Introduction to Test-Driven Development

Kotest is a very useful tool for supporting test-driven development (TDD). TDD is a software development methodology in which tests are written first and code is then implemented based on those tests. This helps improve code quality and find bugs early.

Test-driven development (TDD)

Test-driven development is a software development methodology that repeats the process of writing test cases first and then writing code that passes those test cases. This methodology helps improve code quality and reduce bugs.

Kotest Supports TDD

Kotest provides the following features to support TDD:

Easy test writing: Kotest provides simple and intuitive APIs that make tests easy to write. You can test code more effectively by using various styles and matchers.
Automatic test execution: Kotest provides features that can run tests automatically whenever they change. This lets you run tests continuously and quickly check the impact of code changes.
Integration with test coverage tools: Kotest supports integration with code coverage tools, so you can measure test coverage and identify untested code. This helps improve test completeness.
Mock object and stub support: Kotest supports mock object and stub libraries such as MockK, allowing external dependencies to be replaced during tests. This makes tests more isolated and stable.

Core Principles of TDD

The core TDD principle “Red, Green, Refactor” describes the approach to follow during software development. It represents a series of steps: write tests, implement code, and improve code.

Red: The red state means a test is failing. Before implementing a new feature, you should create a red state. In Kotest, you write and run a test so that it fails. At this stage, you write code where the test does not exist yet or the expected result is not returned.
Green: The green state means a test is passing. You write code so that the failing test from the red state can pass. In Kotest, you modify the code so the failing test passes, then run the test again to reach the green state.
Refactor: The refactor step improves the code while preserving its behavior through tests. In the green state, the tests pass, but the code may still have room for improvement. In Kotest, you can refactor code to improve readability, maintainability, performance, and more. Tests should confirm that existing behavior has not changed because of the code changes.

By following the “Red, Green, Refactor” approach with Kotest, you can practice test-driven development more effectively during software development. You write code so that a failing test passes, then improve the code while using tests to maintain stability.

TDD helps improve software design, quality, and maintainability, and it can be applied effectively with test libraries such as Kotest. When developing software with TDD, Kotest helps you write and maintain tests more effectively.

Simple Kotest Examples

First, let’s look at a few simple examples.

Test with Style

You can write simple and expressive tests using one of the available styles:

class MyTests : StringSpec({
    "length should return size of string" {
        "hello".length shouldBe 5
    }
    "startsWith should test for a prefix" {
        "world" should startWith("wor")
    }
})

We will examine this code in more detail later, but briefly, you can see that it extends the StringSpec class. The test descriptions are written as strings, and validation is performed with shouldBe and should.

This shows that Kotest can use Kotlin syntax to write two nested tests.

Kotest lets you create tests in many styles, so you can choose the style that best fits your needs.

Checking Multiple Test Cases with Data-Driven Testing

With data-driven testing, you can easily handle a large number of input parameter combinations:

class StringSpecExample : StringSpec({
   "maximum of two numbers" {
      forAll(
         row(1, 5, 5),
         row(1, 0, 1),
         row(0, 0, 0)
      ) { a, b, max ->
         Math.max(a, b) shouldBe max
      }
   }
})

Fine-Tuning Test Execution

For each test or for all tests, you can specify invocation counts, parallel processing, and timeouts. You can also group tests by tag or disable them conditionally. You only need to configure this with config:

class MySpec : StringSpec({
   "should use config".config(timeout = 2.seconds, invocations = 10, threads = 2, tags = setOf(Database, Linux)) {
      // test here
   }
})

References

Introduction | Kotest

Kotest Spring Extension(SpringExtension)

kc@example.com (kc kim) — Sun, 31 Mar 2024 16:17:00 +0900

TIP

If you prefer looking at an example instead of reading documentation, see this sample project that uses [Spring WebFlux](https://github.com/kotest/kotest-examples-spring-webflux).

Spring Extension

Adding the Dependency

To use this extension, add the io.kotest.extensions:kotest-extensions-spring module to the test compile path. You can check the latest version on Maven Central.

First, to use Kotest with Spring, add the following dependency.

testImplementation("io.kotest.extensions:kotest-extensions-spring:1.0.0")

Note: The Maven group ID differs from the core test framework group, io.kotest.extensions.

Configuration

To apply SpringExtension to all tests, configure it through global settings.

class GlobalProjectConfig : AbstractProjectConfig() {
    override fun extensions() = listOf(SpringExtension)
}

To use the Spring extension, it must be enabled for all test classes or per test class.

To enable it globally, register SpringExtension in the project configuration:

class ProjectConfig : AbstractProjectConfig() {
   override fun extensions() = listOf(SpringExtension)
}

Sometimes you may want to use tests without enabling SpringExtension globally. In this case, you can skip global configuration and configure extensions only in the test classes that use SpringExtension.

To enable it per test class:

class SpringTest : FunSpec() {
    override fun extensions() = listOf(SpringExtension)
}

To tell Spring which configuration class to use, add the @ContextConfiguration annotation to the Spec class. This annotation should point to a class annotated with Spring’s @Configuration annotation. Alternatively, you can use @ActiveProfiles to point to a specific application context file.

NOTE

In Kotest 4.3 and earlier, the Spring extension was called `SpringListener`. This extension is now deprecated and replaced by `SpringExtension`. Usage is the same, but `SpringExtension` provides more features.

Constructor Injection

For constructor injection, assuming autoscan is enabled, Kotest automatically registers SpringAutowireConstructorExtension when the Spring module is added to the build. If autoscan is disabled, you must manually load the extension in project configuration.

This extension intercepts each call that creates a Spec instance and autowires beans declared in the primary constructor.

The following example is a test class that requires a service named UserService in its primary constructor. This service class is a normal Spring bean annotated with @Component.

@ContextConfiguration(classes = [(Components::class)])
class SpringAutowiredConstructorTest(service: UserService) : WordSpec() {
  init {
    "SpringExtension" should {
      "have autowired the service" {
        service.repository.findUser().name shouldBe "system_user"
      }
    }
  }
}

You can also use @SpringBootTest:

@SpringBootTest(classes = [FooService::class])
class SpringTest(fooService: FooService) : FunSpec() {
    override fun extensions() = listOf(SpringExtension)

    init {
        test("getFoo test") {
            val actual = fooService.getFoo()
            actual shouldBe Foo("id", "foo")
        }
    }
}

@Service
class FooService {
    fun getFoo(): Foo {
        return Foo("id", "foo")
    }
}

data class Foo(
    val id: String,
    val title: String,
)

Test Context

The Spring extension makes TestContextManager available through the coroutine context where the test runs. You can get a handle to this instance through the testContextManager() extension method.

This lets you obtain the test context that Spring is using.

class MySpec(service: UserService) : WordSpec() {
  init {
    "SpringExtension" should {
      "provide the test context manager" {
         println("The context is " + testContextManager().testContext)
      }
    }
  }
}

Test Method Callbacks

Spring has various test callbacks such as beforeTestMethod, based on the idea that tests are methods. This assumption fits legacy test frameworks such as JUnit, but not modern test frameworks like Kotest, where tests are functions.

Therefore, when using nested spec styles, you can customize when test method callbacks are executed. By default, this is at the leaf node. You can pass a SpringTestLifecycleMode argument to the extension to run them at the root node:

class ProjectConfig : AbstractProjectConfig() {
   override fun extensions() = listOf(SpringTestExtension(SpringTestLifecycleMode.Root))
}

Final Classes

When using final classes, you may receive the following warning from Kotest:

Using SpringListener on a final class. If any Spring annotation fails to work, try making this class open

If desired, you can disable this warning by setting the system property kotest.listener.spring.ignore.warning to true.

References

Kotest IntelliJ Plugin Test Explorer

kc@example.com (kc kim) — Sun, 31 Mar 2024 14:10:00 +0900

The plugin provides a tool window view that displays the test structure. This window describes the currently selected test file and includes the specs defined in that file and the tests contained in those specs. The tree layout reflects the test structure, making navigation easy.

The tool window includes lifecycle callback methods, such as before/after test, and included test factories when they are defined.

Clicking a spec, test, include, or callback jumps directly to that element in the source editor.

Any test disabled with the Bang prefix displays a different icon.

You can run tests or specs directly from this window: run, debug, or run with coverage. This window also displays all test modules and can run all tests in those modules.

Modules, callbacks, and includes can be filtered out if you do not want to see them. They are included by default.

References

Test Explorer | Kotest

Kotest Property Test Functions

kc@example.com (kc kim) — Mon, 18 Mar 2024 22:01:00 +0900

For All

First, forAll receives an n-arity function (a, ..., n) -> Boolean that tests a property. If the function returns true for every input value, the test passes.

import io.kotest.core.spec.style.StringSpec
import io.kotest.property.forAll

class PropertyExample: StringSpec({
    "String size" {
        forAll<String, String> { a, b ->
            (a + b).length == a.length + b.length
        }
    }
})

This function accepts type parameters for the argument types and supports arities up to 14. Kotest uses these type parameters to find generators that provide, or generate, random values of the appropriate type.

For example, forAll<String, Int, Boolean> { a, b, c -> } is a 3-arity property test where argument a is a random string, argument b is a random int, and argument c is a random boolean.

Check All

Second, checkAll receives an n-arity function (a, ..., n) -> Unit that can simply run assertions against the inputs. With this approach, the test is considered valid if no exception is thrown. The following rewrites the same example in the same way using checkAll.

import io.kotest.core.spec.style.StringSpec
import io.kotest.matchers.string.shouldHaveLength
import io.kotest.property.checkAll

class PropertyExample: StringSpec({
    "String size" {
        checkAll<String, String> { a, b ->
            a + b shouldHaveLength a.length + b.length
        }
    }
})

The second approach is more general-purpose than returning a boolean, while the first approach comes from the original Haskell library that inspired this library.

Iterations

By default, Kotest runs a property test 1000 times. You can easily customize this by specifying the number of iterations when calling the test method.

Suppose you want to run the test 10,000 times.

import io.kotest.core.spec.style.StringSpec
import io.kotest.property.checkAll

class PropertyExample: StringSpec({
    "a many iterations test" {
        checkAll<Double, Double>(10_000) { a, b ->
            // test here
        }
    }
})

Specifying Generators

In the previous example, Kotest automatically provided values based on the type parameters. It does this by finding a generator that creates values for the required type. For example, the automatically provided integer generator creates random integers from all possible values such as negative values, positive values, infinity, and zero.

This is suitable for basic tests, but you often want more control over the sample space. For example, you might want to test a function only for numbers in a specific range. In that case, you should specify generators manually.

import io.kotest.core.spec.style.StringSpec
import io.kotest.property.Arb
import io.kotest.property.arbitrary.int
import io.kotest.property.forAll

class PropertyExample4 : StringSpec({

    fun isDrinkingAge(a: Int): Boolean =
        a in 19..Int.MAX_VALUE

    "is allowed to drink in Chicago" {
        forAll(Arb.int(21..150)) { a ->
            isDrinkingAge(a) // assuming some function that calculates if we're old enough to drink
        }
    }
    "is allowed to drink in London" {
        forAll(Arb.int(18..150)) { a ->
            isDrinkingAge(a) // assuming some function that calculates if we're old enough to drink
        }
    }
})

You can see that two tests are created and each passes a generator with the appropriate int range to the forAll function.

See here for the list of built-in generators.

References

Property Test Functions | Kotest

Kotest Project Setup

kc@example.com (kc kim) — Mon, 26 Feb 2024 21:59:00 +0900

Adding Dependencies

To use Kotest, you first need to add the library to your project. The Kotest testing framework is supported on JVM, JavaScript, and Native. To use Kotest across multiple platforms, configure the steps for each platform as described below.

JVM/Gradle

Kotest in a JVM/Gradle project uses the JUnit Platform Gradle plugin. In Gradle 4.6 or later, you only need to add useJUnitPlatform() to tasks whose test type is Test and add the Kotest JUnit 5 launcher dependency.

When using Gradle + Groovy (build file: build.gradle):

test {
  useJUnitPlatform()
}

Or when using Gradle + Kotlin (build file: build.gradle.kts):

tasks.withType<Test>().configureEach {
   useJUnitPlatform()
}

Then add the Kotest dependency as follows:

repositories {
    mavenCentral()
}

dependencies {
    testImplementation("io.kotest:kotest-runner-junit5:${kotestVersion}")
    // Add dependencies for other modules if needed.
}

JVM/Maven

For a JVM/Maven project, the following configuration is required in the build file (pom.xml).

Configure the Surefire plugin for JUnit tests:

<plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-surefire-plugin</artifactId>
   <version>2.22.2</version>
</plugin>

Then add the Kotest JUnit 5 runner dependency:

<dependencies>
    <dependency>
        <groupId>io.kotest</groupId>
        <artifactId>kotest-runner-junit5</artifactId>
        <version>{kotestVersion}</version>
        <scope>test</scope>
    </dependency>
</dependencies>

Kotlin/JS

TIP

You can find a working multiplatform project configured for JVM, Native, and JavaScript, with unit and data-driven test examples, at the site below. - https://github.com/kotest/kotest-examples-multiplatform Add the Kotest multiplatform Gradle plugin to the build.

For example:

plugins {
  id("io.kotest.multiplatform") version "5.0.2"
}

Add the engine dependency to the commonTest dependency block:

kotlin {
  targets {
    js(IR) { // LEGACY or BOTH are unsupported
      browser() // to compile for the web
      nodejs() // to compile against node
    }
  }

  sourceSets {
    val commonTest by getting {
      dependencies {
        implementation("io.kotest:kotest-framework-engine:$version")
      }
    }
  }
}

CAUTION

Only the new IR compiler backend for Kotlin/JS is supported. If you compile JS with the legacy compiler backend, you cannot use Kotest for tests.

You can write tests using FunSpec, ShouldSpec, or StringSpec. Tests can be placed in the commonTest or jsTest source set. Use the gradle check command to run the tests.

The JavaScript test engine has limited functionality compared with the JVM test engine. The biggest limitation is that Kotlin does not expose annotations to JavaScript code at runtime, so annotation-based configuration does not work.

NOTE

Tests for JavaScript cannot be nested. This is because underlying JavaScript test runners, such as Mocha or Karma, do not support promises compatible with coroutines in parent tests, and every test scope in Kotest is a coroutine. Therefore, the supported specs are limited to `FunSpec`, `ShouldSpec`, and `StringSpec`.

INFO

The IntelliJ Kotest plugin does not support running regular, Native, or JS tests directly from the IDE using the green run icon. Only execution through Gradle is supported.

Kotlin/Native

In Kotlin/Native projects, as with Kotlin/JS projects, you need to add the Kotest multiplatform Gradle plugin.

plugins {
  id("io.kotest.multiplatform") version "5.0.2"
}

Add the engine dependency to the commonTest dependency block:

kotlin {
  targets {
    linuxX64() // can add any supported native targets such as linux, mac, windows etc
  }
}
sourceSets {
  val commonTest by getting {
    dependencies {
      implementation("io.kotest:kotest-framework-engine:$version")
    }
  }
}

Tests can be placed in commonTest or in a specific Native source set. Use the gradle check command to run tests.

The Native test engine has limited functionality compared with the JVM test engine. The biggest limitation is that Kotlin does not expose annotations to Native code at runtime, so annotation-based configuration does not work.

Note The IntelliJ Kotest plugin does not support running regular, Native, or JS tests from the IDE. You must use the Gradle check task.

Android

INFO

Kotest currently officially supports only JVM tests. With the following steps, Kotest can generally be used for unit and integration tests that do not require the Android framework, or that use mocking, in a module's `src/test` folder.

Kotest on Android uses the JUnit Platform Gradle plugin. To use it, configure the Android test options block in the build file, then add the Kotest JUnit 5 launcher dependency.

android.testOptions {
   unitTests.all {
      it.useJUnitPlatform()
   }
}

dependencies {
   testImplementation 'io.kotest:kotest-runner-junit5:{version}'
}

Kotest Version

Here, specify the Kotest version you want to use in ${kotestVersion}.

As of April 2024, the latest Kotest version is 5.8.1, released on March 11, 2024. https://central.sonatype.com/artifact/io.kotest/kotest-runner-junit5

What is the difference between kotest-runner-junit5 and kotest-runner-junit5-jvm?

There are two basic runners for running Kotest tests as a JUnit 5 test engine: one with `jvm` in the name and one without it. The difference between these two versions depends on where Kotest is used. - **kotest-runner-junit5**: - This module integrates the Kotest test engine with JUnit 5 so Kotest tests can be run as JUnit 5 tests. It is used in Kotlin Multiplatform projects and supports running Kotest with JUnit 5 across multiple platforms, such as JVM, JS, and Native. - **kotest-runner-junit5-jvm**: - This module is used to run JUnit 5 tests for code written in Kotlin on the JVM. It is specialized for the JVM platform and provides all configuration required to run Kotest tests with JUnit 5. Each module supports running Kotest tests as JUnit 5 tests according to its platform. This integration makes Kotest usable in many environments and ensures compatibility with JUnit 5, allowing Kotest to be integrated with existing testing frameworks.

Creating a Kotest Environment in IntelliJ IDEA

To run Kotest in IntelliJ IDEA, you need to install the Kotest plugin.

Go to [Setting]-[Plugins], search for kotest, and install it.

References

Setup | Kotest

Kotest Ktor Extension

kc@example.com (kc kim) — Sat, 13 Apr 2024 21:45:00 +0900

The kotest-assertions-ktor module provides response matchers for Ktor applications. If you use server-side test support, there are matchers for TestApplicationResponse, and if you use the Ktor HTTP client, there are matchers for HttpResponse.

To add Ktor matchers, add the following dependency to your project.

io.kotest.extensions:kotest-assertions-ktor:${version}

The following is an example of using matchers with server-side test support:

withTestApplication({ module(testing = true) }) {
   handleRequest(HttpMethod.Get, "/").apply {
      response shouldHaveStatus HttpStatusCode.OK
      response shouldNotHaveContent "failure"
      response.shouldHaveHeader(name = "Authorization", value = "Bearer")
      response.shouldNotHaveCookie(name = "Set-Cookie", cookieValue = "id=1234")
   }
}

And here is an example of using client support:

val client = HttpClient(CIO)
val response = client.post("http://mydomain.com/foo")
response.shouldHaveStatus(HttpStatusCode.OK)
response.shouldHaveHeader(name = "Authorization", value = "Bearer")

References

Ktor | Kotest

Kotest IntelliJ Plugin Properties

kc@example.com (kc kim) — Sun, 31 Mar 2024 14:21:00 +0900

When running tests through the IntelliJ runner, properties configured in gradle.properties or Gradle build files are not picked up because the runner is not configured to use Gradle.

To support runtime system properties, the Kotest framework always looks for key-value pairs in a kotest.properties file on the classpath, for example in src/main/resources.

All key-value pairs in this file are set as system properties before tests run.

TIP

All properties specified in the `kotest.properties` file work both from the command line through Gradle and in tests run through the IntelliJ plugin.

For example, add the following to a kotest.properties file on the classpath:

foo=bar

Then the following test passes:

class FooTest : DescribeSpec() {
  init {
    describe("after adding kotest.properties") {
      it("foo should be set") {
         System.getProperty("foo") shouldBe "bar"
      }
    }
  }
}

Common Use Case

If you do not use these features, it is common to disable Kotest’s classpath scanning to save startup time. To do this, add the following to the kotest.properties file:

kotest.framework.classpath.scanning.config.disable=true
kotest.framework.classpath.scanning.autoscan.disable=true

Naming the Properties File

If you do not want to name the file kotest.properties, or if you want to support different files depending on the environment, you can set the property file name with the system property kotest.properties.filename.

For example, if you start tests with kotest.properties.filename=cluster.prd.properties, a key-value file named cluster.prd.properties is loaded before the tests run.

References

Properties | Kotest

Kotest Generators

kc@example.com (kc kim) — Mon, 18 Mar 2024 21:58:00 +0900

Generated values are provided by instances of the sealed class Gen. You can think of Gen as being like an input stream, but for property test values. Each Gen provides a usually infinite stream of these values for a specific type.

Kotest has two types of generators: Arb, which generates arbitrary values, and Exhaustive, which generates a finite set of values from a closed space.

You can mix both types of generators in a property test. For example, you can test a function with every even number from 0 to 200, which is exhaustive, along with 100 random positive integers, which comes from an arb.

Some generators are available only on the JVM. See the full list here.

Arbitrary

Arbs generate two types of values: a hard-coded set of edge cases and an infinite stream of randomly selected samples.

Samples may repeat, and some values may never be generated. For example, if you generate 1000 integers between 0 and Int.MAX, it cannot return every possible value, and some values may be generated more than once. Likewise, if you generate 1000 random integers between 0 and 500, some values may appear more than once.

Common arbitrary values include numbers with or without ranges, strings from a Unicode set, arbitrary lists, data classes with arbitrary parameters, emails, code points, and characters.

In addition to random values, arbs can provide edge cases. One design feature of Kotest property testing is that some value types always include “common” edge cases that you probably want included in tests.

For example, when testing a function that receives an integer, you will usually want to test at least zero, positive numbers, and negative numbers. If only random values were provided, the probability of zero appearing would be fairly low, so Kotest always provides several edge cases for integers unless configured otherwise.

When running a test, the framework randomly alternates between samples and edge cases. The split is determined by a configuration value, whose default is 2% edge cases.

Not every arb has edge cases, but the most common arb types do. Here are some examples of edge cases used by certain arbs:

ints: 0, 1, -1, Int.MAX_VALUE, Int.MIN_VALUE
double: 0, 1, -1, Double.MAX_VALUE, Double.MIN_VALUE, Double.POSITIVE_INFINITY, Double.NEGATIVE_INFINITY, Double.NaN
strings: empty string, minimum-length string, minimum code point
lists: empty list, single-element list, list with duplicate elements
maps: empty map
nullable values: null

Exhaustive

Exhaustive generates every value in a specified space. It is useful when you want to make sure every value in that space is used. For example, with enum values, it is generally more useful to make sure each enum value is used than to randomly select from the enum values and risk missing some values while duplicating others.

Common exhaustives include small collections, enums, boolean values, powersets of lists or sets, predefined integer ranges, and predefined string ranges.

When an exhaustive has provided every value, it starts over, so an exhaustive can be used in tests that need input.

For example:

enum class Season { Winter, Fall, Spring, Summer }

forAll<Int, Season>(100) { a, season -> ... }

Since 100 iterations are requested here, each season value will be provided 25 times.

References

Generators | Kotest

Writing Basic Kotest Tests

kc@example.com (kc kim) — Mon, 26 Feb 2024 21:59:00 +0900

Writing Test Code

After Kotest installation and setup are complete, let’s look at how to write test code.

The language features provided by Kotlin make it possible to define tests with a more powerful and simple approach. The days when tests had to be defined as methods in Java files are gone.

In Kotest, a test is essentially just a TestContext -> Unit function that contains test logic. Any assertion statement, called a matcher in Kotest terminology, that throws an exception in this function is intercepted by the framework and used to mark the test as failed or successful.

Test functions are not defined manually. They are defined through the Kotest DSL, which provides several ways to create and nest these functions. The DSL is written by creating a class that extends a class implementing a specific testing style.

Create a test class in the desired path, then extend the Spec class for the testing style.

For example, to use the Fun Spec style, extend the FunSpec class and create a test function with the test keyword, a name, and the actual test function.

class MyFirstTestClass : FunSpec({

   test("my first test") {
      1 + 2 shouldBe 3
   }

})

There are two main ways to write the inside of test code: writing it with a DSL (Domain-Specific Language), and using an init {} block.

The following sections explain both methods.

Writing with the DSL

First, let’s explain how to write tests with a DSL (Domain-Specific Language).

Kotest provides a DSL for writing tests. Using this DSL lets you express tests more naturally.
Each testing style provides its own DSL, and you can use it to structure tests. For example, StringSpec, WordSpec, and FunSpec each provide a DSL that defines tests using strings, words, or functions.

These DSLs are convenient for structuring and expressing tests, and they improve readability and maintainability.

import io.kotest.core.spec.style.StringSpec

class MyStringSpec : StringSpec({
    "This is a test" {
        // Write test code
    }
})

Writing with an init Block

Kotlin’s init block can be used to initialize and configure tests. This is generally used when more flexibility is needed.

The init block is located inside the test class constructor and is initialized when the class is instantiated. Therefore, it can be shared by all methods in the test class.

import io.kotest.core.spec.style.StringSpec

class MyStringSpec : StringSpec() {
    init {
        "This is a test" {
            // Write test code
        }
    }
}

The DSL and init block are chosen according to different use cases and developer preferences. Use the DSL when you want to write more structured tests, and use an init block when specific setup is required.

Nested Tests

Most styles provide the ability to nest tests. The actual syntax differs by style, but the basic difference is the keyword used for the outer test.

For example, in Describe Spec, the outer test is created with the describe function, and the inner test is created with the it function. JavaScript and Ruby developers can recognize this style immediately because it is commonly used in those languages’ testing frameworks.

class NestedTestExamples : DescribeSpec({

   describe("an outer test") {

      it("an inner test") {
        1 + 2 shouldBe 3
      }

      it("an inner test too!") {
        3 + 4 shouldBe 7
      }
   }

})

In Kotest terminology, tests that can contain other tests are called test containers, and terminal or leaf-node tests are called test cases. Both can contain test logic and assertions.

Dynamic Tests

Tests are just functions, so they are evaluated at runtime.

This approach provides the significant advantage of allowing tests to be generated dynamically. Unlike traditional JVM testing frameworks, where tests are always methods and are declared at compile time, Kotest can add tests conditionally at runtime.

For example, you can add tests based on elements in a list.

class DynamicTests : FunSpec({

    listOf(
      "sam",
      "pam",
      "tim",
    ).forEach {
       test("$it should be a three letter name") {
           it.shouldHaveLength(3)
       }
    }
})

This creates three tests at runtime. It is equivalent to writing the following:

class DynamicTests : FunSpec({

   test("sam should be a three letter name") {
      "sam".shouldHaveLength(3)
   }

   test("pam should be a three letter name") {
      "pam".shouldHaveLength(3)
   }

   test("tim should be a three letter name") {
     "tim".shouldHaveLength(3)
   }
})

Lifecycle Callbacks

Kotest provides several callbacks that are invoked at various points during the test lifecycle. These callbacks are useful for resetting state, setting up resources used by tests, and releasing those resources.

As mentioned earlier, Kotest test functions are labeled as test containers or test cases, and the containing class is labeled as a spec. You can register callbacks that are invoked before or after a test function, container, test case, or the spec itself.

To register a callback, simply pass a function to one of the callback methods.

For example, you can add callbacks before and after a test case using function literals:

class Callbacks : FunSpec({

   beforeEach {
      println("Hello from $it")
   }

   test("sam should be a three letter name") {
      "sam".shouldHaveLength(3)
   }

   afterEach {
      println("Goodbye from $it")
   }
})

The order of callbacks in the file does not matter. For example, you can place the afterEach block first in the class if you want.

To extract common code, create a named function and reuse it across multiple files. For example, if you want to reset the database before every test in two or more files, you can do this:

val resetDatabase: BeforeTest = {
  // truncate all tables here
}

class ReusableCallbacks : FunSpec({

   beforeTest(resetDatabase)

   test("this test will have a sparkling clean database!") {
       // test logic here
   }
})

For more information about all callbacks and when they are invoked, see Lifecycle Hooks and Extensions.

Assertions and Matchers

Kotest provides many matchers that can validate test results. They let you compare expected and actual results to confirm whether a test succeeded. For example, in the sample above, result shouldBe 2 is an assertion that verifies whether result is equal to 2.

Many other matchers exist. The following is sample code that uses some major matchers with Kotest.

package com.devkuma.kotest.tutorial.assertions

import io.kotest.core.spec.style.FreeSpec
import io.kotest.matchers.collections.shouldContain
import io.kotest.matchers.collections.shouldContainAll
import io.kotest.matchers.collections.shouldContainExactly
import io.kotest.matchers.collections.shouldNotContain
import io.kotest.matchers.ints.shouldBeGreaterThan
import io.kotest.matchers.ints.shouldBeInRange
import io.kotest.matchers.ints.shouldBeLessThan
import io.kotest.matchers.shouldBe
import io.kotest.matchers.shouldNotBe
import io.kotest.matchers.string.shouldBeBlank
import io.kotest.matchers.string.shouldBeEqualIgnoringCase

class MatchersTest : FreeSpec({

    "Matchers" - {
        val number = 10
        "checks whether the value is the same as the expected value" {
            number shouldBe 10
        }

        "checks whether the value is different from the expected value" {
            number shouldNotBe 20
        }

        val blankString = ""
        "checks whether the string consists only of whitespace" {
            blankString.shouldBeBlank()
        }

        val numbers = listOf(1, 2, 3, 4, 5)

        "checks whether the collection contains the specified element" {
            numbers shouldContain 3
        }

        "checks whether the collection does not contain the specified element" {
            numbers shouldNotContain 6
        }

        "checks whether the collection contains all specified elements" {
            numbers shouldContainAll listOf(2, 4)
        }

        "checks whether the collection exactly matches the specified elements" {
            numbers.shouldContainExactly(1, 2, 3, 4, 5)
        }

        val value = 10
        "checks whether the value is in the specified range" {
            value shouldBeInRange 5..15
        }

        "checks whether the value is greater than the specified value" {
            value shouldBeGreaterThan 5
        }

        "checks whether the value is less than the specified value" {
            value shouldBeLessThan 15
        }

        val message = "Hello, Kotest"
        "checks equality ignoring case" {
            message shouldBeEqualIgnoringCase "HELLO, KOTEST"
        }
    }
})

There are many more matchers, and they will be explained in detail later.

Test Execution and Reports

To run tests, you can run them from the IDE or use Gradle or Maven. After running tests, you can check reports for the test results. Kotest records test results in detail and generates reports so developers can understand the state of their tests.

To run a test, run the relevant test class or file in the IDE, or run tests using Gradle or Maven. After test execution, you can check the result of each test case in report form. Kotest provides detailed reports for executed tests so you can understand whether tests succeeded or failed.

Running Tests in IntelliJ IDEA

In IntelliJ IDEA, select the test class or file and choose the Kotest tab on the left. The test cases will be displayed as shown below.

Select the displayed test case and press the Run button. You can then confirm that the test code runs as shown below.

Running Gradle Tests

The following explains how to run Kotest through Gradle.

Running the Test Command

Move to the project directory in a terminal or command window, then run the following command to execute the project’s tests with Gradle.

./gradlew test

When execution completes, the test results are printed in the terminal. If the tests pass, the BUILD SUCCESSFUL message is printed as shown below.

% ./gradlew test

BUILD SUCCESSFUL in 548ms
3 actionable tasks: 3 up-to-date

If a test fails, the BUILD FAILED message is printed.

 % ./gradlew test 

BUILD SUCCESSFUL in 28s
3 actionable tasks: 3 executed
user@AL02263852 kotest-tutorial % ./gradlew test

> Task :test

com.devkuma.kotest.tutorial.MyTest > My first test FAILED
    io.kotest.assertions.AssertionFailedError at MyTest.kt:9

58 tests completed, 1 failed, 8 skipped

> Task :test FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///Users/user/develop/kotlin-tutorial/kotest-tutorial/build/reports/tests/test/index.html

* Try:
> Run with --scan to get full insights.

BUILD FAILED in 26s
3 actionable tasks: 2 executed, 1 up-to-date

Checking the Test Report

After running tests, you can view the generated HTML test report in the build/reports/tests directory to examine the results in detail.

In the failed example above, the test report link is shown as follows.

> There were failing tests. See the report at: file:///Users/user/develop/kotlin-tutorial/kotest-tutorial/build/reports/tests/test/index.html

Open this file in a browser, and it will appear as shown below.

The number of executed tests and the number of failed tests are shown in detail.

Even when tests succeed, the file is created in the same location, although the command output does not print the report link.

As shown here, Kotest lets you perform tests easily and efficiently with simple setup and test code. In the next sections, we will cover more Kotest features and usage patterns.

References

Writing Tests | Kotest

Kotest Custom Matchers

kc@example.com (kc kim) — Sun, 25 Feb 2024 10:28:00 +0900

Custom Matchers

You can easily define your own matchers in Kotest.

You only need to extend the Matcher<T> interface. Here, T is the type you want to match. The Matcher interface defines one function, test, which returns an instance of MatcherResult.

interface Matcher<in T> {
   fun test(value: T): MatcherResult
}

The MatcherResult type defines three functions: a function that returns a boolean indicating whether the test passed, and two failure messages.

interface MatcherResult {
   fun passed(): Boolean
   fun failureMessage(): String
   fun negatedFailureMessage(): String
}

passed
- Indicates whether the assertion is satisfied, true, or not satisfied, false.
failureMessage
- The message shown to the user when the match condition fails.
- It is shown when an assertion fails and tells what needs to happen for it to succeed.
- It can generally include details and differences between expected and actual values.
negatedFailureMessage
- The message shown to the user when the match condition evaluates to true in negated mode.
- This is the message that should be displayed when a matcher fails while using negation.
- It generally indicates that the predicate was expected to fail.

Defining a Custom Matcher

The difference between the two failure messages becomes clearer with an example. Suppose you are writing a length matcher for strings to check whether a string has the required length. The desired syntax is something like str.shouldHaveLength(8).

Then the first message should be like "string had length 15 but we expected length 8". The second message should be like "string should not have length 8".

First, implement the matcher type:

import io.kotest.matchers.Matcher
import io.kotest.matchers.MatcherResult

fun haveLength(length: Int) = Matcher<String> { value ->
    MatcherResult(
        value.length == length,
        { "string had length ${value.length} but we expected length $length" },
        { "string should not have length $length" },
    )
}

The error messages are wrapped in function calls so they are not evaluated unless needed. This is important for error messages that take time to generate.

Then you can pass this matcher to the should and shouldNot prefix functions as follows:

"hello foo" should haveLength(9)
"hello bar" shouldNot haveLength(3)

Extension Variants

Here, we define an extension function that calls the matcher function and returns the original value for function chaining. This is how Kotest structures its built-in matchers, and Kotest tries to follow the shouldXYZ naming strategy. For example:

fun String.shouldHaveLength(length: Int): String {
    this should haveLength(length)
    return this
}

fun String.shouldNotHaveLength(length: Int): String {
    this shouldNot haveLength(length)
    return this
}

This lets you call it as follows:

"hello foo".shouldHaveLength(9)
"hello bar".shouldNotHaveLength(3)

Defining a Custom Matcher that Checks for Even Numbers

Next, let’s create a custom matcher that checks whether a list contains even numbers.

The following creates a custom matcher named containEvenNumbers and checks whether the list contains a number that leaves a remainder of 0 when divided by 2.

package com.devkuma.kotest.tutorial.assertions.custommatchers

import io.kotest.core.spec.style.FunSpec
import io.kotest.matchers.*
import io.kotest.matchers.should

fun containEvenNumbers() = Matcher<List<Int>> { value ->
    MatcherResult(
        value.any { it % 2 == 0 },
        { "List should contain even numbers" },
        { "List should not contain even numbers" }
    )
}

fun List<Int>.shouldContainEvenNumbers(): List<Int> {
    this should containEvenNumbers()
    return this
}

infix fun <T> T.should(matcher: (T) -> Unit) = matcher(this)

class ContainEvenNumbersCustomMatcher : FunSpec({
    test("should") {
        val list = listOf(1, 2, 3, 4, 5)
        list should containEvenNumbers()
    }

    test("shouldContainEvenNumbers") {
        val list = listOf(1, 2, 3, 4, 5)
        list.shouldContainEvenNumbers()
    }
})

Custom matchers can improve the readability of test code. However, keep in mind that poor naming can have the opposite effect.

References

Custom Matchers | Kotest

Kotest System Extensions

kc@example.com (kc kim) — Sun, 31 Mar 2024 16:13:00 +0900

System Extensions

If you need to test code that uses java.lang.System, Kotest provides extensions that can modify the system and restore it after each test. These extensions are available only on the JVM.

To use these extensions, add the dependency to your project:

io.kotest:kotest-extensions-jvm:${version}

CAUTION

These extensions do not support concurrent test execution. Due to JVM specifications, only one instance of these extensions can run at a time, for example because there can be only one environment map. If you try to run more than one instance at a time, the result is undefined.

System Environment

The system environment extension lets you simulate how the system environment works. In other words, it lets you simulate what you get from System.getenv().

Kotest provides several extension functions that provide a system environment for a specific scope:

withEnvironment("FooKey", "BarValue") {
    System.getenv("FooKey") shouldBe "BarValue" // System environment overridden!
}

INFO

To use `withEnvironment` on JDK 17, you must add `--add-opens=java.base/java.util=ALL-UNNAMED` to the arguments of the JVM running the tests. If you run tests with Gradle, add the following to `build.gradle.kts`: ```kotlin tasks.withType().configureEach { jvmArgs("--add-opens=java.base/java.util=ALL-UNNAMED") } ```

This extension can also use multiple values through a Map or a list of Pairs.

withEnvironment(mapOf("FooKey" to "BarValue", "BarKey" to "FooValue")) {
  // Use FooKey and BarKey
}

This function adds keys and values if they do not exist in the current environment, and overrides them if they do exist. Keys not touched by the function remain in the environment unchanged.

Instead of extension functions, you can also apply this functionality to a wider scope using the provided listeners. There are alternatives for the Spec/per-test level and for the project level.

import io.kotest.core.spec.style.FreeSpec
import io.kotest.extensions.system.SystemEnvironmentTestListener
import io.kotest.matchers.shouldBe

class MyTest : FreeSpec() {

    override fun listeners() = listOf(SystemEnvironmentTestListener("foo", "bar"))

    init {
        "MyTest" {
            System.getenv("foo") shouldBe "bar"
        }
    }
}

import io.kotest.core.config.AbstractProjectConfig
import io.kotest.extensions.system.SystemEnvironmentProjectListener

class ProjectConfig : AbstractProjectConfig() {
    override fun listeners() = listOf(SystemEnvironmentProjectListener("foo", "bar"))
}

System Property Extension

You can override system properties, System.getProperties(), in the same way as the environment extension:

withSystemProperty("foo", "bar") {
    System.getProperty("foo") shouldBe "bar"
}

It is also provided as a listener:

import io.kotest.core.spec.style.FreeSpec
import io.kotest.extensions.system.SystemPropertyTestListener
import io.kotest.matchers.shouldBe

class MyTest : FreeSpec() {

    override fun listeners() = listOf(SystemPropertyTestListener("foo", "bar"))

    init {
        "MyTest" {
            System.getProperty("foo") shouldBe "bar"
        }
    }
}

System Security Manager

Similarly, the system security manager extension lets you override the system security manager, System.getSecurityManager().

withSecurityManager(myManager) {
    // Usage of security manager
}

It is also provided as a listener:

import io.kotest.core.spec.style.FreeSpec
import io.kotest.extensions.system.SecurityManagerListener

class MyTest : FreeSpec() {

    override fun listeners() = listOf(SecurityManagerListener(myManager))

    init {
        // Use my security manager
    }
}

System Exit Extension

Sometimes you may want to test whether code calls System.exit. For this, you can use the system exit listener. The listener throws an exception when System.exit is called, so you can catch and verify it:

import io.kotest.assertions.throwables.shouldThrow
import io.kotest.core.spec.style.FreeSpec
import io.kotest.extensions.system.SpecSystemExitListener
import io.kotest.extensions.system.SystemExitException
import io.kotest.matchers.shouldBe

class MyTest : FreeSpec() {

    override fun listeners() = listOf(SpecSystemExitListener)

    init {
        "Catch exception" {
            val thrown: SystemExitException = shouldThrow<SystemExitException> {
                System.exit(22)
            }

            thrown.exitCode shouldBe 22
        }
    }
}

no-stdout/no-stderr Listeners

You may want to ensure that no debug messages are left behind, or that logging always uses a logger.

For this, Kotest provides NoSystemOutListener and NoSystemErrListener. These listeners do not allow messages to be printed directly to System.out or System.err, respectively:

    // In Project or in Spec
    override fun listeners() = listOf(NoSystemOutListener, NoSystemErrListener)

Locale/Timezone Listeners

Some code uses or is sensitive to the default Locale and Timezone. Instead of manipulating system defaults directly, let Kotest handle it.

withDefaultLocale(Locale.FRANCE) {
  println("My locale is now France! Très bien!")
}

withDefaultTimeZone(TimeZone.getTimeZone(ZoneId.of("America/Sao_Paulo"))) {
  println("My timezone is now America/Sao_Paulo! Muito bem!")
}

They are also provided as listeners:

// In Project or in Spec
override fun listeners() = listOf(
    LocaleTestListener(Locale.FRANCE),
    TimeZoneTestListener(TimeZone.getTimeZone(ZoneId.of("America/Sao_Paulo")))
)

References

System Extensions | Kotest

Kotest Generators List

kc@example.com (kc kim) — Mon, 18 Mar 2024 22:15:00 +0900

This page lists all current Kotest generators. There are two types of generators: Arbitrary generators and Exhaustive generators.

Most generators are available on all platforms. Some are JVM-only or JS-only.

Generators for Arrow are also provided as a separate module.

Generator	Description	JVM	JS	Native
Nulls
`arb.orNull()`	Generates arbitrary values from an arb instance mixed with null values. For example, `Arb.int().orNull()` can generate `1, -1, null, 8, 17`, and so on. An overloaded version controls how often null is generated.	✓	✓	✓
`arb.orNull(nullProbability)`	Generates arbitrary values from an arb instance mixed with null values using the configured probability.	✓	✓	✓
Booleans
`Arb.boolean()`	Returns an `Arb` that generates `Boolean` values.	✓	✓	✓
`Arb.booleanArray(length, content)`	Returns an `Arb` that generates `BoolArray`, where `length` generates the array length and `content` generates the array contents.	✓	✓	✓
`Exhaustive.boolean()`	Alternates between true and false.	✓	✓	✓
Chars
`Arb.char(range1, range2,...)`	Returns arbitrary characters generated from one or more specified ranges. Multiple ranges make it possible to fill specific non-contiguous character ranges.	✓	✓	✓
`Arb.char(List<CharRange>)`	Returns characters distributed across the full list of character ranges. For example, `Arb.char(listOf('A'..'C', 'X'..'Z'))` generates A, B, C, X, Y, and Z with equal probability.
`Arb.charArray(length, content)`	Returns an `Arb` that generates `CharArray`, where `length` generates the array length and `content` generates the array contents.	✓	✓	✓
Constants
`Arb.constant(t)`	Returns an `Arb` that always returns `t`.	✓	✓	✓
Bytes
`Arb.byte(min, max)`	Returns an `Arb` that generates `Byte` values from `min` to `max`, inclusive. Edge cases are `min`, -1, 0, 1, and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.positiveByte(min, max)`	Returns an `Arb` that generates positive `Byte` values from `min` to `max`, inclusive. Edge cases are 1 and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.negativeByte(min, max)`	Returns an `Arb` that generates negative `Byte` values from `min` to `max`, inclusive. Edge cases are `min` and -1, included only when they are in the provided range.	✓	✓	✓
`Arb.byteArray(length, content)`	Returns an `Arb` that generates `ByteArray`, where `length` generates the array length and `content` generates the array contents.	✓	✓	✓
`Arb.uByte(min, max)`	Returns an `Arb` that generates `UByte` values from `min` to `max`, inclusive. Edge cases are `min`, 1, and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.uByteArray(length, content)`	Returns an `Arb` that generates `UByteArray`, where `length` generates the array length and `content` generates the array contents.	✓	✓	✓
Shorts
`Arb.short(min, max)`	Returns an `Arb` that generates `Short` values from `min` to `max`, inclusive. Edge cases are `min`, -1, 0, 1, and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.positiveShort(min, max)`	Returns an `Arb` that generates positive `Short` values from `min` to `max`, inclusive. Edge cases are 1 and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.negativeShort(min, max)`	Returns an `Arb` that generates negative `Short` values from `min` to `max`, inclusive. Edge cases are `min` and -1, included only when they are in the provided range.	✓	✓	✓
`Arb.shortArray(length, content)`	Returns an `Arb` that generates `ShortArray`, where `length` generates the array length and `content` generates the array contents.	✓	✓	✓
`Arb.uShort(min, max)`	Returns an `Arb` that generates `UShort` values from `min` to `max`, inclusive. Edge cases are `min`, 1, and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.uShortArray(length, content)`	Returns an `Arb` that generates `UShortArray`, where `length` generates the array length and `content` generates the array contents.	✓	✓	✓
Ints
`Arb.int(min, max)`	Returns an `Arb` that generates `Int` values from `min` to `max`, inclusive. Edge cases are `min`, -1, 0, 1, and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.positiveInt(min, max)`	Returns an `Arb` that generates positive `Int` values from `min` to `max`, inclusive. Edge cases are 1 and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.nonNegativeInt(min, max)`	Returns an `Arb` that generates non-negative `Int` values from `min` to `max`, inclusive. Edge cases are 0, 1, and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.negativeInt(min, max)`	Returns an `Arb` that generates negative `Int` values from `min` to `max`, inclusive. Edge cases are `min` and -1, included only when they are in the provided range.	✓	✓	✓
`Arb.nonPositiveInt(min, max)`	Returns an `Arb` that generates non-positive `Int` values from `min` to `max`, inclusive. Edge cases are `min`, -1, and 0, included only when they are in the provided range.	✓	✓	✓
`Arb.intArray(length, content)`	Returns an `Arb` that generates `IntArray`, where `length` generates the array length and `content` generates the array contents.	✓	✓	✓
`Arb.uInt(min, max)`	Returns an `Arb` that generates `UInt` values from `min` to `max`, inclusive. Edge cases are `min`, 1, and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.uIntArray(length, content)`	Returns an `Arb` that generates `UIntArray`, where `length` generates the array length and `content` generates the array contents.	✓	✓	✓
`Exhaustive.ints(range)`	Returns every integer in the given range.	✓	✓	✓
`Arb.multiple(k, max)`	Generates multiples of `k` up to the maximum value. The edge case is `0`.	✓	✓	✓
`Arb.factor(k)`	Generates factors of `k`.	✓	✓	✓
Longs
`Arb.long(min, max)`	Returns an `Arb` that generates `Long` values from `min` to `max`, inclusive. Edge cases are `min`, -1, 0, 1, and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.positiveLong(min, max)`	Returns an `Arb` that generates positive `Long` values from `min` to `max`, inclusive. Edge cases are 1 and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.negativeLong(min, max)`	Returns an `Arb` that generates negative `Long` values from `min` to `max`, inclusive. Edge cases are `min` and -1, included only when they are in the provided range.	✓	✓	✓
`Arb.longArray(length, content)`	Returns an `Arb` that generates `LongArray`, where `length` generates the array length and `content` generates the array contents.	✓	✓	✓
`Arb.uLong(min, max)`	Returns an `Arb` that generates `ULong` values from `min` to `max`, inclusive. Edge cases are `min`, 1, and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.uLongArray(length, content)`	Returns an `Arb` that generates `ULongArray`, where `length` generates the array length and `content` generates the array contents.	✓	✓	✓
`Exhaustive.longs(range)`	Returns every long in the given range.	✓	✓	✓
Floats
`Arb.float(min, max)`	Returns an `Arb` that generates `Float` values from `min` to `max`, inclusive. Edge cases include `Float.NEGATIVE_INFINITY`, `min`, -1.0, -`Float.MIN_VALUE`, -0.0, 0.0, `Float.MIN_VALUE`, 1.0, `max`, `Float.POSITIVE_INFINITY`, and `Float.NaN` only when they are in the provided range.	✓	✓	✓
`Arb.positiveFloat(min, max)`	Returns an `Arb` that generates positive `Float` values from `min` to `max`, inclusive. `Float.MIN_VALUE`, 1.0, `max`, and `Float.POSITIVE_INFINITY` are included only when they are in the provided range.	✓	✓	✓
`Arb.negativeFloat(min, max)`	Returns an `Arb` that generates negative `Float` values from `min` to `max`, inclusive. Edge cases are `Float.NEGATIVE_INFINITY`, `min`, -1.0, and -`Float.MIN_VALUE`, included only when they are in the provided range.	✓	✓	✓
`Arb.numericFloat(min, max)`	Returns an `Arb` that generates numeric `Float` values from `min` to `max`, inclusive. Edge cases are `min`, -1.0, -`Float.MIN_VALUE`, -0.0, 0.0, `Float.MIN_VALUE`, 1.0, and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.floatArray(length, content)`	Returns an `Arb` that generates `FloatArray`, where `length` generates the array length and `content` generates the array contents.	✓	✓	✓
Doubles
`Arb.double(min, max)`	Returns an `Arb` that generates `Double` values from `min` to `max`, inclusive. Edge cases include `Double.NEGATIVE_INFINITY`, `min`, -1.0, -`Double.MIN_VALUE`, -0.0, 0.0, `Double.MIN_VALUE`, 1.0, `max`, `Double.POSITIVE_INFINITY`, and `Double.NaN` only when they are in the provided range.	✓	✓	✓
`Arb.positiveDouble(min, max)`	Returns an `Arb` that generates positive `Double` values from `min` to `max`, inclusive. `Double.MIN_VALUE`, 1.0, `max`, and `Double.POSITIVE_INFINITY` are included only when they are in the provided range.	✓	✓	✓
`Arb.negativeDouble(min, max)`	Returns an `Arb` that generates negative `Double` values from `min` to `max`, inclusive. Edge cases are `Double.NEGATIVE_INFINITY`, `min`, -1.0, and -`Double.MIN_VALUE`, included only when they are in the provided range.	✓	✓	✓
`Arb.numericDouble(min, max)`	Returns an `Arb` that generates numeric `Double` values from `min` to `max`, inclusive. Edge cases are `min`, -1.0, -`Double.MIN_VALUE`, -0.0, 0.0, `Double.MIN_VALUE`, 1.0, and `max`, included only when they are in the provided range.	✓	✓	✓
`Arb.doubleArray(length, content)`	Returns an `Arb` that generates `DoubleArray`, where `length` generates the array length and `content` generates the array contents.	✓	✓	✓
Enums
`Arb.enum<T>()`	Randomly selects constants from the given enum.	✓	✓	✓
`Exhaustive.enum<T>()`	Iterates all constants defined in the given enum.	✓	✓	✓
Regional
`localeArb()`	Generates locales in Java format, such as `en_US` or `ca_ES_VALENCIA`.	✓	✓	✓
`Arb.timezoneCodeThree()`	Generates time zones in three-letter format such as BST or EST. It does not include every possible time zone and is used only for data sampling.	✓	✓	✓
`Arb.geoLocation()`	Generates a `GeoLocation` object using random latitude and longitude points uniformly distributed across the globe.	✓	✓	✓
Strings
`Arb.string(range)`	Generates printable random strings with size randomly selected from the given range. If no range is specified, `(0..100)` is used. Edge cases include empty strings and Unicode strings.	✓	✓	✓
`Arb.stringPattern(pattern)`	Uses RgxGen to generate strings matching the given pattern.	✓
`Exhaustive.azstring(range)`	Returns every AZ string in the specified range. For example, if the range is 1..2, it includes a, b, c, …, yz, zz.	✓	✓	✓
`Arb.email(localPartGen, domainGen)`	Generates arbitrary email addresses whose local part and domain part are random strings generated by the specified generators. Defaults are provided for both.	✓	✓	✓
`Arb.emailLocalPart()`	Generates an arbitrary local email part.	✓	✓	✓
`Arb.uuid(type)`	Generates a random UUID of the specified type.	✓
`Arb.domain(tlds, labelArb)`	Generates arbitrary domains using a random TLD, defaulting to one of the top 120 TLDs, and a label generator that generates the domain parts.	✓	✓	✓
Builders
`Arb.bind(arbA, arbB, fn)`	Takes values from each of the two given arbs and passes them to the provided function to generate a value.	✓	✓	✓
`Arb.bind(arbA, arbB, arbC, fn)`	Takes values from each of the three given arbs and passes them to the provided function to generate a value.	✓	✓	✓
`Arb.bind(arbA, ...., fn)`	Takes values from each given arb and passes them to the provided function to generate a value.	✓	✓	✓
Combinatorics
`Arb.choice(arbs)`	Randomly selects one of the given arbs and uses it to generate the next element.	✓	✓	✓
`Arb.choose(pairs)`	Generates values based on weights. For example, `Arb.choose(1 to 'A', 2 to 'B')` generates ‘A’ 33% of the time and ‘B’ 66% of the time.	✓	✓	✓
`Arb.frequency(list)`	Alias for choose.	✓	✓	✓
`Arb.shuffle(list)`	Generates random permutations of a list. For example, `Arb.shuffle(listOf(1,2,3))` can generate `listOf(3,1,2)`, `listOf(1,3,2)`, and so on.	✓	✓	✓
`Arb.subsequence(list)`	Generates random subsequences of the given list starting at index 0, including the empty list. For example, `Arb.subsequence(listOf(1,2,3))` can generate `listOf(1)`, `listOf(1,2)`, and so on.	✓	✓	✓
Collections
`Arb.element(collection)`	Randomly selects one element from the given collection.	✓	✓	✓
`Arb.element(vararg T)`	Randomly selects one element from the vararg arguments.	✓	✓	✓
`Arb.list(gen, range)`	Generates lists whose values are created by the specified element generator. Each list size is randomly determined from the specified range.	✓	✓	✓
`Arb.set(gen, range)`	Generates sets whose values are created by the specified element generator. Each set size is randomly determined from the specified range. The slippage argument specifies how many attempts are made to generate each element before failing when the underlying arb cannot produce enough unique values for the requested size.	✓	✓	✓
`Arb.set(gen, range, slippage)`	Generates sets whose values are created by the specified element generator. Each set size is randomly determined from the specified range.	✓	✓	✓
`Arb<T>.chunked(range)`	Generates lists where each list is filled with elements from this receiver. Each chunk size is randomly selected from the given range.	✓	✓	✓
`Arb<T>.chunked(minSize, maxSize)`	Generates lists where each list is filled with elements from this receiver. Each chunk size is randomly selected from the given range parameters.	✓	✓	✓
`Exhaustive.collection(list)`	Enumerates each element of the list one by one.	✓	✓	✓
Maps
`Arb.map(Arb<Pair<K,V>>, minSize, maxSize)`	Generates random maps with sizes between `minSize` and `maxSize`, where each element is generated from the specified arbitrary pair generator.	✓	✓	✓
`Arb.map(Arb<K>, Arb<V>, minSize, maxSize)`	Generates random maps with sizes between `minSize` and `maxSize`, where each key is generated from the specified key arb and each value from the specified value arb.	✓	✓	✓
Tuples
`Arb.pair(arb1, arb2)`	Generates a `Pair` instance by taking each value of the pair from the two provided `Arb`s.	✓	✓	✓
`Arb.triple(arb1, arb2, arb3)`	Generates a `Triple` instance by taking each value of the triple from the three provided `Arb`s.	✓	✓	✓
Dates
`Arb.date(range)`	Generates arbitrary dates using years between the given range.		✓
`Arb.datetime(range)`	Generates arbitrary date-times using years between the given range.		✓
`Arb.localDateTime(range)`	Generates arbitrary `LocalDateTime` values using years between the given range.	✓
`Arb.localDate(range)`	Generates arbitrary `LocalDate` values using years between the given range.	✓
Durations
`Arb.duration(range)`	Generates arbitrary durations from the specified range.	✓	✓	✓
Kotlinx Date/Time	Requires the `io.kotest.extensions:kotest-property-datetime` module.
`Arb.date(yearRange)`	Randomly generates `LocalDate` values using years in the given range and other fields.	✓	✓	✓
`Arb.datetime(yearRange, hourRange, minuteRange, secondRange)`	Generates `LocalDateTime` values using all fields from the given ranges.	✓	✓	✓
`Arb.instant(range)`	Generates `Instant` values with epochs randomly generated from the given range.	✓	✓	✓
Networking
`Arb.ipAddressV4()`	Generates arbitrary IP addresses in a.b.c.d format, where each part is between 0 and 255.	✓	✓	✓

References

Generators List | Kotest

Kotest Testing Styles

kc@example.com (kc kim) — Tue, 27 Feb 2024 22:53:00 +0900

Testing Styles

Kotest provides several testing styles. This page introduces the 10 testing styles supported by Kotest.

Testing Style	Inspired by
Fun Spec	ScalaTest
String Spec	A Kotest original
Should Spec	A Kotest original
Describe Spec	Javascript frameworks and RSpec
Behavior Spec	BDD frameworks
Word Spec	ScalaTest
Free Spec	ScalaTest
Feature Spec	Cucumber
Expect Spec	A Kotest original
Annotation Spec	JUnit

Each style follows its own syntax, and you can choose the one that best fits the situation. Each testing style has its own characteristics, strengths, and trade-offs, so select the style that matches your project requirements.

The following sections explain each testing style and show basic usage through examples that extend one of these style classes.

FunSpec

FunSpec is a function-based testing style. Each test case is written as a function, and the test body is written inside that function.

With FunSpec, you create a test by calling the test function with a descriptive string argument and passing the test itself as a lambda. If you are not sure which style to choose, this is a good default.

The following example shows FunSpec:

package com.devkuma.kotest.tutorial.testingstyles

import io.kotest.core.spec.style.FunSpec
import io.kotest.matchers.shouldBe

class FunSpecTest : FunSpec({
    test("FunSpec example - addition test") {
        val result = 2 + 2
        result shouldBe 4
    }
})

In addition to the normal style, you can disable tests by using the xcontext and xtest variants.

class MyTests : FunSpec({
    context("this outer block is enabled") {
        xtest("this test is disabled") {
            // test here
        }
    }
    xcontext("this block is disabled") {
        test("disabled by inheritance from the parent") {
            // test here
        }
    }
})

StringSpec

StringSpec is a string-based testing style. It lets you write tests with syntax close to natural language. Each test case is usually written as a string, and the test body is written inside braces {}.

StringSpec keeps the syntax minimal. You only need to write a lambda expression after the string in the test code.

The following example shows StringSpec:

package com.devkuma.kotest.tutorial.testingstyles

import io.kotest.core.spec.style.StringSpec
import io.kotest.matchers.shouldBe

class StringSpecTest : StringSpec({
    "StringSpec example - addition test" {
        val result = 2 + 2
        result shouldBe 4
    }
})

You can also add configuration to a test.

class MyTests : StringSpec({
    "strings.length should return size of string".config(enabled = false, invocations = 3) {
        "hello".length shouldBe 5
    }
})

ShouldSpec

ShouldSpec is similar to FunSpec, but it uses the should keyword instead of test. It is commonly used to describe the behavior of a test case.

The following example shows ShouldSpec:

package com.devkuma.kotest.tutorial.testingstyles

import io.kotest.core.spec.style.ShouldSpec
import io.kotest.matchers.shouldBe

class ShouldSpecTest : ShouldSpec({
    should("ShouldSpec example - addition test") {
        val result = 2 + 2
        result shouldBe 4
    }
})

Tests can also be nested inside one or more context blocks:

class MyTests : ShouldSpec({
    context("String.length") {
        should("return the length of the string") {
            "sammy".length shouldBe 5
            "".length shouldBe 0
        }
    }
})

In addition to the normal style, you can disable tests by using the xcontext and xshould variants.

class MyTests : ShouldSpec({
    context("this outer block is enabled") {
        xshould("this test is disabled") {
            // test here
        }
    }
    xcontext("this block is disabled") {
        should("disabled by inheritance from the parent") {
            // test here
        }
    }
})

DescribeSpec

The DescribeSpec testing style uses the describe / it keywords, so it feels familiar to people with a Ruby or JavaScript background. It uses describe syntax to group test cases, and tests must be nested inside one or more description blocks. This style is mainly used to describe test cases.

The following example shows DescribeSpec:

package com.devkuma.kotest.tutorial.testingstyles

import io.kotest.core.spec.style.DescribeSpec
import io.kotest.matchers.shouldBe

class DescribeSpecTest : DescribeSpec({
    describe("DescribeSpec example") {
        it("addition test") {
            val result = 2 + 2
            result shouldBe 4
        }
    }
})

In addition to the normal style, you can disable tests by using the xdescribe and xit variants:

class MyTests : DescribeSpec({
    describe("this outer block is enabled") {
        xit("this test is disabled") {
            // test here
        }
    }
    xdescribe("this block is disabled") {
        it("disabled by inheritance from the parent") {
            // test here
        }
    }
})

BehaviorSpec

This is a behavior-driven development (BDD) testing style. Tests are written with given, when, and then blocks. Test cases are written with a focus on major features or behavior.

The following example shows BehaviorSpec:

package com.devkuma.kotest.tutorial.testingstyles

import io.kotest.core.spec.style.BehaviorSpec
import io.kotest.matchers.shouldBe

class BehaviorSpecTest : BehaviorSpec({
    Given("BehaviorSpec example") {
        When("addition test") {
            val result = 2 + 2
            Then("2 plus 2 should be 4") {
                result shouldBe 4
            }
        }
    }
})

NOTE

`when` is a Kotlin keyword, so it must be wrapped in backticks. If you do not like using backticks, you can use the title-case versions, such as `Given`, `When`, and `Then`.

You can also add more depth by using the And keyword with Given and When:

class MyTests : BehaviorSpec({
    given("a broomstick") {
        and("a witch") {
            `when`("The witch sits on it") {
                and("she laughs hysterically") {
                    then("She should be able to fly") {
                        // test code
                    }
                }
            }
        }
    }
})

In addition to the normal style, you can disable tests by using the xgiven, xwhen, and xthen variants:

class MyTests : BehaviorSpec({
    xgiven("this is disabled") {
        When("disabled by inheritance from the parent") {
            then("disabled by inheritance from its grandparent") {
                // disabled test
            }
        }
    }
    given("this is active") {
        When("this is active too") {
            xthen("this is disabled") {
               // disabled test
            }
        }
    }
})

WordSpec

WordSpec is a string-based testing style. It uses syntax similar to natural language. It uses the should keyword to nest tests after a context string. Each test case is written as a string, and the test body is written inside a should block.

The following example shows WordSpec:

package com.devkuma.kotest.tutorial.testingstyles

import io.kotest.core.spec.style.WordSpec
import io.kotest.matchers.shouldBe

class WordSpecTest : WordSpec({
    "WordSpec example - addition test" should {
        "2 plus 2 should be 4" {
            val result = 2 + 2
            result shouldBe 4
        }
    }
})

It also supports the when keyword when you want to add another level of nesting. Note that when is a Kotlin keyword, so you must use backticks or the uppercase variant.

class MyTests : WordSpec({
    "Hello" When {
        "asked for length" should {
            "return 5" {
                "Hello".length shouldBe 5
            }
        }
        "appended to Bob" should {
            "return Hello Bob" {
                "Hello " + "Bob" shouldBe "Hello Bob"
            }
        }
    }
})

FreeSpec

FreeSpec is a free-form testing style. It reduces dependencies between test cases and lets you write tests independently.

With FreeSpec, you can nest tests to any depth by using the - keyword for outer tests and using only the test name for the final test:

The following example shows FreeSpec:

package com.devkuma.kotest.tutorial.testingstyles

import io.kotest.core.spec.style.FreeSpec
import io.kotest.matchers.shouldBe

class FreeSpecTest : FreeSpec({
    "FreeSpec example - addition test" - {
        val result = 2 + 2
        "2 plus 2 should be 4" {
            result shouldBe 4
        }
    }
})

CAUTION

The innermost test must not use the `-` keyword after the test name.

FeatureSpec

FeatureSpec is a feature-based testing style. Each test case is written around a feature or scenario.

FeatureSpec lets you use features and scenarios, which will feel familiar to anyone who has used Cucumber. It is not exactly the same as Cucumber, but the keywords imitate that style.

The following example shows FeatureSpec:

package com.devkuma.kotest.tutorial.testingstyles

import io.kotest.core.spec.style.FeatureSpec
import io.kotest.matchers.shouldBe

class FeatureSpecTest : FeatureSpec({
    feature("FeatureSpec example") {
        scenario("addition test") {
            val result = 2 + 2
            result shouldBe 4
        }
    }
})

In addition to the normal style, you can disable tests by using the xfeature and xscenario variants:

class MyTests : FeatureSpec({
    feature("this outer block is enabled") {
        xscenario("this test is disabled") {
            // test here
        }
    }
    xfeature("this block is disabled") {
        scenario("disabled by inheritance from the parent") {
            // test here
        }
    }
})

ExpectSpec

This style writes test specifications with the expect syntax. It is mainly used to express expected results.

ExpectSpec is similar to FunSpec and ShouldSpec, but it uses the expect keyword.

The following example shows ExpectSpec:

package com.devkuma.kotest.tutorial.testingstyles

import io.kotest.core.spec.style.ExpectSpec
import io.kotest.matchers.shouldBe

class ExpectSpecTest : ExpectSpec({
    context("ExpectSpec example") {
        expect("addition test") {
            val result = 2 + 2
            2 + 2 shouldBe 4
        }
    }
})

Tests can also be nested inside one or more context blocks:

class MyTests : ExpectSpec({
    context("a calculator") {
        expect("simple addition") {
            // test here
        }
        expect("integer overflow") {
            // test here
        }
    }
})

In addition to the normal style, you can disable tests by using the xcontext and xexpect variants.

class MyTests : ExpectSpec({
    context("this outer block is enabled") {
        xexpect("this test is disabled") {
            // test here
        }
    }
    xcontext("this block is disabled") {
        expect("disabled by inheritance from the parent") {
            // test here
        }
    }
})

AnnotationSpec

This is an annotation-based testing style. Tests are written by using specific annotations.

If you are migrating from JUnit, AnnotationSpec is a spec that uses annotations like JUnit 4/5. You only need to add the @Test annotation to functions defined in the spec class.

As with JUnit, you can add annotations for before tests/specs and after tests/specs.

@BeforeAll / @BeforeClass
@BeforeEach / @Before
@AfterAll / @AfterClass
@AfterEach / @After

Use @Ignore to ignore a test.

NOTE

This spec usually does not provide a major advantage over using JUnit because you mostly only adjust imports, but it can make existing tests relatively easy to migrate.

The following example shows AnnotationSpec:

package com.devkuma.kotest.tutorial.testingstyles

import io.kotest.core.spec.style.AnnotationSpec
import io.kotest.matchers.shouldBe

class AnnotationSpecExample : AnnotationSpec() {

    @BeforeEach
    fun beforeTest() {
        println("Before each test")
    }

    @Test
    fun test1() {
        1 shouldBe 1
    }

    @Test
    fun test2() {
        3 shouldBe 3
    }
}

Structuring Tests

Kotest supports structuring tests with context blocks, which let you define test cases as a hierarchy or group. This helps you classify tests logically and manage related test cases together.

For example, the following code structures tests with test groups:

import io.kotest.core.spec.style.FunSpec

class ContextTest : FunSpec({

    context("Calculator tests") {
        test("Addition") {
            // Test logic for addition
        }

        test("Subtraction") {
            // Test logic for subtraction
        }
    }

    context("String tests") {
        test("Length") {
            // Test logic for string length
        }

        test("Concatenation") {
            // Test logic for string concatenation
        }
    }
})

In this example, tests are classified and separated with context blocks.

The context block can be used only in FunSpec, ShouldSpec, and ExpectSpec.

References

Testing Styles | Kotest

Kotest Generator Operations

kc@example.com (kc kim) — Mon, 01 Apr 2024 22:53:00 +0900

If you want to use an Arb only to return values, even outside a property test, call the following:

val arbA: Arb<A> = ...
val a = arbA.next() // use Random.Default
val a2 = arbA.next(rs) // pass in Random

Filter

If you have an Arb and want to create a new Arb that provides a subset of values, you can call filter on the source Arb. For example, one way to generate even numbers is to take an integer generator and filter out odd values.

val evens = Arb.int().filter { it.value % 2 == 0 }
val odds = Arb.int().filter { it.value % 2 == 1 }

Map

If you have an Arb and want to transform generated values, you can use map.

val integerStrings: Arb<String> = Arb.int().map { it.toString() }

FlatMap

If you have an arb whose emissions or edge cases depend on a previous arbitrary emission, you can use flatMap.

val dependentArbs: Arb<String> = Arb.of("foo", "bar").flatMap { prefix ->
   Arb.int(1..10).map { integer ->
      "${prefix}-${integer}"
   }
}

Merging

You can merge two generators so that result elements are sampled equally from both generators.

val merged = arbA.merge(arbB)

Therefore, in the following example, "A" or "B" has the same probability of appearing in each random sample:

val a = arbitrary { "a" }
val b = arbitrary { "b" }
val ab = a.merge(b)

println(ab.take(1000).groupingBy { it }.eachCount())
// {a=493, b=507}

If you are merging more than two arbitrary objects, Arb.choice or Arb.choose may be more appropriate. For example:

Arb.choice(arbA, arbB, arbC) can be used for uniform sampling among arbA, arbB, and arbC.
Or, for finer control over the frequency of each arbitrary, use Arb.choose(4 to arbA, 1 to arbB, 5 to arbC). In this example, arbA, arbB, and arbC are sampled with probabilities of 40%, 10%, and 50%, respectively.

Bind

Bind is useful when you want to apply several arbitrary values. You can see how bind constructs values for a data class.

data class Person(val name: String, val age: Int)

val personArb: Arb<Person> = Arb.bind(
   Arb.string(),
   Arb.int()
) { name, age -> Person(name, age) }

References

Generator Operations | Kotest

Kotest Testcontainers Extension

kc@example.com (kc kim) — Sun, 31 Mar 2024 15:11:00 +0900

Testcontainers

Testcontainers is a tool for building independent and reusable test environments. It supports container-based testing, so you can programmatically build the same environment as the test environment running on your local machine. Kotest can be used with Testcontainers to manage external resources needed for tests, helping you write more robust and reliable tests.

The main advantages of using Testcontainers are:

Consistent environment: Provides a consistent environment each time a test runs, making test results predictable.
Independence: Containerizes external resources so tests do not affect each other.
Ease of building test environments: Lets you programmatically build and manage test environments with code.
Support for various containers: Supports various kinds of containers, such as Docker and Kubernetes, so you can build many test scenarios.

The Testcontainers project provides lightweight temporary instances of common databases, Elasticsearch, Kafka, Selenium web browsers, or anything else that can run in a Docker container, making them ideal for use inside tests.

Kotest provides integration with Testcontainers through an additional module that offers several extensions, including special extensions for databases and Kafka and general container support for all supported Docker images.

Adding the Dependency

To use it, add the following dependency to your Gradle build file.

dependencies {
    testImplementation("io.kotest.extensions:kotest-extensions-testcontainers:${kotestVersion}")
}

Note: The group ID io.kotest.extensions differs from the main Kotest group ID, io.kotest.

For Maven projects, add the dependency as follows:

<dependency>
    <groupId>io.kotest.extensions</groupId>
    <artifactId>kotest-extensions-testcontainers</artifactId>
    <version>${kotest.version}</version>
    <scope>test</scope>
</dependency>

Databases

Using Testcontainers in Kotest is straightforward. Start the required container using the Testcontainers library inside the test code, run the test, and then clean up the container afterward.

For example, when performing database integration tests, it is common to start the database container at the beginning of the test and stop it when the test ends.

For this, Kotest lets you use beforeTest and afterTest to perform required work before and after test execution.

The following is example code using Kotest and Testcontainers together:

import io.kotest.core.spec.style.FreeSpec
import org.testcontainers.containers.PostgreSQLContainer
import org.testcontainers.junit.jupiter.Container
import org.testcontainers.junit.jupiter.Testcontainers

@Testcontainers
class PostgreSqlTest : FreeSpec() {

    companion object {
        @Container
        val postgresContainer: PostgreSQLContainer<Nothing> = PostgreSQLContainer<Nothing>()
            .withDatabaseName("test")
            .withUsername("test")
            .withPassword("test")
            .withExposedPorts(5432)
    }

    override fun beforeTest(testCase: TestCase) {
        postgresContainer.start()
        System.setProperty("DB_URL", postgresContainer.jdbcUrl)
        System.setProperty("DB_USERNAME", postgresContainer.username)
        System.setProperty("DB_PASSWORD", postgresContainer.password)
    }

    override fun afterTest(testCase: TestCase, result: TestResult) {
        postgresContainer.stop()
    }

    init {
        "Database test" {
            // write database test code
        }
    }
}

In the code above, a PostgreSQL container is used to perform database testing. The @Testcontainers annotation enables Testcontainers, and the @Container annotation defines the container. Then beforeTest and afterTest are overridden to start and stop the container before and after test execution.

References

Kotest Clues

kc@example.com (kc kim) — Sun, 17 Mar 2024 21:22:00 +0900

Clues

NOTE

Clues work only when using the Kotest assertion library.

As a rule of thumb, a failed test should look like a good bug report. It should tell you what went wrong, and ideally why it went wrong.

Sometimes a failed assertion’s error message contains enough information to tell what went wrong.

For example:

username shouldBe "devkuma"

You may get an error like this:

expected:<"devkuma"> but was:<"devkuma@devkuma.com">

If this fails, you may simply get:

<null> should not equal <null>

This is not very helpful. This is where withClue can be used.

The withClue and asClue helpers add extra context to assertions so failures can explain themselves:

For example, you can use withClue with a string message.

withClue("Name should be present") {
    user.name shouldNotBe null
}

This produces an error like:

Name should be present
<null> should not equal <null>

The error message is much better, but still not perfect. For example, if you need to check the database, knowing the user’s ID may help.

You can use asClue to add the user ID to the error message:

withClue({ "Name should be present (user_id=${user.id})" }) {
    user.name shouldNotBe null
}

You can also use the asClue extension function to convert any object into a clue message.

{ "Name should be present (user_id=${user.id})" }.asClue {
    user.name shouldNotBe null
}

This message is generated only when the test fails, so it is safe to use with expensive operations.

TIP

Test failures include the failed assertion, test name, clues, and stack trace. It is recommended to use clues this way so you can understand both what failed and why it failed. This makes tests easier to maintain, especially when reverse-engineering the test author's intent.

TIP

Whenever you would write a code comment above an assertion, it is recommended to use `asClue` or `withClue` instead. In test failures, especially in CI, clues are shown, but comments are not.

You can also use domain objects as clues:

data class HttpResponse(val status: Int, val body: String)

val response = HttpResponse(404, "the content")

response.asClue {
    it.status shouldBe 200
    it.body shouldBe "the content"
}

It prints as follows:

HttpResponse(status=404, body=the content)
Expected :200
Actual   :404

NOTE

Kotest treats all clues like `() -> Any?` as lazy clues, calculates them, and uses `.toString()` on the result value instead of calling `.toString()` on the function itself. In most cases, this does exactly what you need, but if the clue object implements `() -> Any?` and you want to use `clue.toString()`, it is recommended to manually wrap the clue as `{ clue.toString() }.asClue {... }`.

Nested Clues

Clues can be nested, and all of them can be seen in the failed assertion message:

{ "Verifying user_id=${user.name}" }.asClue {
  "email_confirmed should be false since we've just created the user".asClue {
    user.emailConfirmed shouldBe false
  }
  "login".asClue {
    user.login shouldBe "sksamuel"
  }
}

The failure may appear as follows:

Verifying user_id=42
email_confirmed should be false since we've just created the user
<true> should equal <false>

References

Clues | Kotest

Kotest Conditional Evaluation

kc@example.com (kc kim) — Wed, 28 Feb 2024 21:19:00 +0900

Config

Kotest supports disabling tests by setting configuration flags on tests. These configuration flags include enabled, enabledIf, and enabledOrReasonIf.

enabled flags

You can disable a test case by setting the enabled parameter of the config function to false. Conversely, set it to true to enable the test.

The following example shows how to set the enabled parameter:

import io.kotest.core.spec.style.StringSpec
import io.kotest.matchers.shouldBe

class ConfigEnabledTest : StringSpec({

    "always runs".config(enabled = true) {
        // This test always runs.
        1 + 1 shouldBe 2
    }

    "never runs".config(enabled = false) {
        // This test never runs.
        1 + 1 shouldBe 3
    }
})

When set to false, the test is not executed. When set to true, the test is executed.

The same mechanism can be used to run tests only under specific conditions. For example, you can use SystemUtils.IS_OS_LINUX from Apache Commons Lang to run a specific test only in a Linux environment.

"runs only on Linux".config(enabled = IS_OS_LINUX) {
    // This test runs only on Linux.
}

"runs only on macOS".config(enabled = IS_OS_MAC) {
    // This test runs only on macOS.
}

Enabled if

If you want to use a function that is evaluated every time a test is invoked, you can use enabledIf. This function has the form (TestCase) -> Boolean, and it can access the test when evaluating whether the test should be enabled or disabled at runtime.

For example, if you want to disable all tests that start with the word danger unless the environment is Linux, you can write the following:

import io.kotest.core.spec.style.StringSpec
import io.kotest.core.test.EnabledIf
import org.apache.commons.lang3.SystemUtils.IS_OS_LINUX

class ConfigEnabledIf : StringSpec({

    val disableDangerOnNotLinux: EnabledIf = { !it.name.testName.startsWith("danger") || IS_OS_LINUX }

    "danger will robinson".config(enabledIf = disableDangerOnNotLinux) {
        // test here
    }

    "very safe will".config(enabledIf = disableDangerOnNotLinux) {
        // test here
    }
})

Enabled or Reason If

The third variant of the enabled flag is enabledOrReasonIf, which can return the reason why a test is disabled. This variant has the form (TestCase) -> Enabled, where Enabled is a type that can include a skipped reason. The reason string is passed to the test report.

For example, the previous danger example can be rewritten as follows:

import io.kotest.core.spec.style.StringSpec
import io.kotest.core.test.Enabled
import io.kotest.core.test.TestCase
import org.apache.commons.lang3.SystemUtils

class ConfigEnabledOrReasonIfTest : StringSpec({

    val disableDangerOnFridays: (TestCase) -> Enabled = {
        if (it.name.testName.startsWith("danger") || SystemUtils.IS_OS_LINUX)
            Enabled.disabled("It's a linux, and we don't like danger!")
        else
            Enabled.enabled
    }

    "danger Will Robinson".config(enabledOrReasonIf = disableDangerOnFridays) {
        // test here
    }

    "safe Will Robinson".config(enabledOrReasonIf = disableDangerOnFridays) {
        // test here
    }
})

The following result is displayed in the console:

Process finished with exit code 0

It's a linux, and we don't like danger!

Focus

In Kotest, Focus is a feature that gives priority to specific tests when running them. Focus lets you run tests with attention on a specific test or test group. This helps developers concentrate on a specific area and debug or develop more efficiently.

Setting Focus is simple. Prefix a test function or test group with f:. The matching test or test group is then focused, which means only that test and all child tests defined inside its scope are executed, and the remaining tests are not executed.

For example, the following code uses Focus:

import io.kotest.core.spec.style.StringSpec
import io.kotest.matchers.shouldBe

class FocusTest: StringSpec({
    "this test is not focused" {
        // This test is not executed.
        1 + 1 shouldBe 2
    }

    // This test is focused and executed.
    "f: Focus example" {
        1 + 1 shouldBe 2
    }

    "this test is also not focused" {
        // This test is not executed.
        1 + 1 shouldBe 2
    }
})

In this code, only the "f: Focus example" test is focused and executed. The remaining tests are not executed. This helps developers concentrate only on the important area and work more efficiently.

Focus mode does not work for nested tests because nested tests are discovered only after the parent test has executed.

Bang

In Kotest, Bang (!) is used to mark a test so it is ignored and not executed. This is useful when a specific test should not run in the current situation. It is especially useful for temporarily disabling a test during development. You can re-enable the test later without deleting it entirely.

Bang is used before a test function or test group.

For example, the following code uses Bang:

import io.kotest.core.spec.style.StringSpec
import io.kotest.matchers.shouldBe

class BangTest : StringSpec({
    "!this test does not run" {
        // This test is not executed.
        1 + 1 shouldBe 2
    }

    "this test runs" {
        // This test is executed.
        1 + 1 shouldBe 2
    }
})

In this code, the test prefixed with !, "!this test does not run", is not executed. The other tests are executed.

You can use Bang this way to temporarily disable a specific test and re-enable it when needed.

X-Method

In Kotest, X-Method is metadata used to disable tests. If you prefix a specific test function or test group with x, that test is not executed. This lets you temporarily disable a specific test and re-enable it later.

For example, the following code uses X-Method:

import io.kotest.core.spec.style.DescribeSpec

class XMethodTest : DescribeSpec({

    xdescribe("this block and its children are now disabled") {
        it("this test does not run") {
            // disabled test
        }
    }
})

In this code, X-Method is used to disable a specific test group and test. xdescribe represents a disabled group and its tests. The remaining tests are executed.

You can use X-Method this way to temporarily disable a specific test or test group and re-enable it later when needed.

`@Ignored`

In Kotest, the @Ignored annotation is used to disable all test functions. When this annotation is used, the test functions are not executed. This lets you temporarily disable specific tests and re-enable them later.

For example, the following code uses the @Ignored annotation:

package com.devkuma.conditional.evaluation

import io.kotest.core.annotation.Ignored
import io.kotest.core.spec.style.StringSpec
import io.kotest.matchers.shouldBe

@Ignored
class IgnoredTest : StringSpec({
    "this test does not run" {
        // This test is not executed.
        1 + 1 shouldBe 3
    }
}) {
    init {
        error("boom") // This error does not occur because the spec is not created.
    }
}

In this code, the @Ignored annotation disables all tests.

You can use this annotation to temporarily disable a specific test function and re-enable it when needed.

`@EnabledIf`

In Kotest, the @EnabledIf annotation is used to enable test functions only when a specific condition is true. This annotation lets you execute a test only when a specific condition is satisfied.

To use the @EnabledIf annotation, you need to create a class that implements the EnabledCondition interface. This interface includes the isEnabled method, and that method determines a specific condition and returns true or false.

For example, the following code uses the @EnabledIf annotation to enable a test based on a specific condition:

package com.devkuma.conditional.evaluation

import io.kotest.core.annotation.EnabledCondition
import io.kotest.core.annotation.EnabledIf
import io.kotest.core.spec.Spec
import io.kotest.core.spec.style.StringSpec

import io.kotest.matchers.shouldBe
import kotlin.reflect.KClass

@EnabledIf(MyEnabledCondition::class)
class EnabledIfTest2 : StringSpec({
    "this test runs when a specific condition is true" {
        // This test runs when the condition specified by MyEnabledCondition is true.
        1 + 1 shouldBe 2
    }
})

object MyEnabledCondition : EnabledCondition {
    override fun enabled(kclass: KClass<out Spec>): Boolean {
        // Determine the specific condition here and return true or false.
        return true // Return true to enable the test when the condition is satisfied.
    }
}

In this code, the MyEnabledCondition object implements the EnabledCondition interface. The enabled method determines a specific condition and returns true or false. The @EnabledIf annotation is used to specify this condition. Therefore, the test runs only when the specific condition is true.

This lets you conditionally enable tests based on specific conditions and flexibly control when tests run.

Kotest Exceptions

kc@example.com (kc kim) — Sun, 17 Mar 2024 21:23:00 +0900

Exceptions

Kotest provides several assertions for verifying exceptions.

Use shouldThrow to assert that a block throws an exception of the specified type.

shouldThrow<IllegalArgumentException> {
    // code that should throw IllegalArgumentException
}

The exception can also be captured and inspected.

val exception = shouldThrow<IllegalArgumentException> {
    // code that should throw IllegalArgumentException
}

exception.message shouldBe "expected message"

shouldThrow<T> also passes when the thrown exception is a subtype of T. If the exact exception type must match, use shouldThrowExactly.

shouldThrowExactly<IllegalArgumentException> {
    // code that should throw exactly IllegalArgumentException
}

If only the fact that some exception is thrown matters, use shouldThrowAny.

shouldThrowAny {
    // code that should throw any Throwable
}

References

Exceptions | Kotest

Kotest Kafka (EmbeddedKafkaListener) Extension

kc@example.com (kc kim) — Sun, 10 Mar 2024 20:04:00 +0900

Embedded Kafka Extension

Kotest provides an extension that runs an embedded Kafka instance. This can be useful in situations where using a Kafka Docker image is problematic.

In general, when developing or testing message-queue-based systems with Apache Kafka, installing and configuring a real Kafka server can be cumbersome and time-consuming. EmbeddedKafkaListener solves this problem by letting you use Apache Kafka in embedded form in the test environment.

Adding the Dependency

To use the embedded Kafka extension, add the io.kotest.extensions:kotest-extensions-embedded-kafka module to the test compile path. You can check the latest version on Maven Central.

testImplementation("io.kotest.extensions:kotest-extensions-embedded-kafka:<version>")

Configuration

import io.kotest.core.spec.style.FunSpec
import io.kotest.extensions.embedded.kafka.embeddedKafkaListener

class EmbeddedKafkaListenerTest : FunSpec({
    listener(embeddedKafkaListener)
})

Or:

import io.kotest.core.spec.style.FunSpec
import io.kotest.extensions.embedded.kafka.embeddedKafkaListener

class EmbeddedKafkaListenerTest : FunSpec() {
    init {
        listener(embeddedKafkaListener)
    }
}

The broker starts when the spec is created and stops when the spec is complete.

With EmbeddedKafkaListener, an embedded Kafka server is started and stopped while tests run. This lets you test Kafka-based applications during test execution without depending on a real Kafka server.

NOTE: The underlying embedded Kafka library uses a global object for state. Do not start multiple Kafka instances at the same time.

Consumer / Producer

You can use convenient methods from the listener to create consumers and producers:

import io.kotest.assertions.nondeterministic.eventually
import io.kotest.core.spec.style.FunSpec
import io.kotest.extensions.embedded.kafka.embeddedKafkaListener
import io.kotest.matchers.shouldBe
import org.apache.kafka.clients.producer.ProducerRecord
import java.time.Duration
import java.time.temporal.ChronoUnit
import kotlin.time.Duration.Companion.seconds

class EmbeddedKafkaListenerTest : FunSpec({

    listener(embeddedKafkaListener)

    test("send / receive") {

        val producer = embeddedKafkaListener.stringStringProducer()
        producer.send(ProducerRecord("foo", "a"))
        producer.close()

        val consumer = embeddedKafkaListener.stringStringConsumer("foo")
        eventually(10.seconds) {
            consumer.poll(Duration.of(1, ChronoUnit.SECONDS)).first().value() shouldBe "a"
        }
        consumer.close()
    }
})

The stringStringProducer and stringStringConsumer methods return producers and consumers that accept strings for keys and values. There are similar methods for byte pairs.

Alternatively, you can create clients directly by accessing the host and port where the Kafka instance is deployed:

class EmbeddedKafkaListenerTest : FunSpec({

  listener(embeddedKafkaListener)
   
  val props = Properties().apply {
      put(CommonClientConfigs.BOOTSTRAP_SERVERS_CONFIG, "${embeddedKafkaListener.host}:${embeddedKafkaListener.port}")
  }
   
  val producer = KafkaProducer<String, String>(props)
   
})

Specifying a Custom Port

You can create a new instance of the listener that specifies a port and use that instance instead of the default instance.

import io.kotest.assertions.nondeterministic.eventually
import io.kotest.core.spec.style.FunSpec
import io.kotest.extensions.embedded.kafka.EmbeddedKafkaListener
import io.kotest.matchers.shouldBe
import org.apache.kafka.clients.producer.ProducerRecord
import java.time.Duration
import java.time.temporal.ChronoUnit
import kotlin.time.Duration.Companion.seconds

class EmbeddedKafkaCustomPortTest : FunSpec({

    val listener = EmbeddedKafkaListener(5678)
    listener(listener)

    test("send / receive") {

        val producer = listener.stringStringProducer()
        producer.send(ProducerRecord("foo", "a"))
        producer.close()

        val consumer = listener.stringStringConsumer("foo")
        eventually(10.seconds) {
            consumer.poll(Duration.of(1, ChronoUnit.SECONDS)).first().value() shouldBe "a"
        }
        consumer.close()
    }
})

You can also specify an additional ZooKeeper port.

val listener = EmbeddedKafkaListener(kafkaPort = 6005, zookeeperPort = 9005)

References

Kotest Isolation Modes

kc@example.com (kc kim) — Fri, 01 Mar 2024 10:01:00 +0900

Isolation Modes

Depending on how you configure Isolation Modes, test performance can differ, and you can also prevent test failures caused by instance reuse. For that reason, this is something you should understand clearly.

There are three ways to configure Isolation Mode. It is controlled by the IsolationMode enum, which provides InstancePerTest, SingleInstance, and InstancePerLeaf. Use whichever mode fits your situation.

If you want tests to run in new instances so that state shared between tests can be initialized, change the isolation mode.

You can do this by assigning a value to isolationMode with the following DSL:

class MyTest : WordSpec({
    isolationMode = IsolationMode.SingleInstance
    
    // tests here
})

Or you can override the fun isolationMode(): IsolationMode function:

class MyTest : WordSpec() {
    override fun isolationMode() = IsolationMode.SingleInstance

    init {
        // tests here
    }
}

DANGER

Kotest defaults to a single instance. This is the same as ScalaTest, which inspired this framework, and as Jest, Jasmine, and other JavaScript frameworks, but it differs from JUnit.

SingleInstance

SingleInstance, the default isolation mode, creates one instance of the Spec class and then runs each test case in order until all tests are complete.

This mode allows state to be shared between all tests, so it is useful when you want to keep dependencies between tests.

package com.devkuma.kotest.tutorial.isolation

import io.kotest.core.spec.style.WordSpec
import java.util.*

class SingleInstanceExample : WordSpec({

    val id = UUID.randomUUID()

    "a" should {
        println("a: $id")
        "b" {
            println("b: $id")
        }
        "c" {
            println("c: $id")
        }
    }
})

The following is the result of running the example:

a: 3823310d-43d3-47a0-953b-fa3afeb39307
b: 3823310d-43d3-47a0-953b-fa3afeb39307
c: 3823310d-43d3-47a0-953b-fa3afeb39307

The same ID is printed three times because the same instance is used for every test.

InstancePerTest

The next mode, IsolationMode.InstancePerTest, creates a new Spec for every test case, including internal contexts. In other words, the outer context is executed as a standalone test in its own Spec instance.

The example makes this clear.

package com.devkuma.kotest.tutorial.isolation

import io.kotest.core.spec.IsolationMode
import io.kotest.core.spec.style.WordSpec
import java.util.*

class InstancePerTestExample : WordSpec() {

    override fun isolationMode(): IsolationMode = IsolationMode.InstancePerTest

    val id = UUID.randomUUID()

    init {
        "a" should {
            println("a: $id")
            "b" {
                println("b: $id")
            }
            "c" {
                println("c: $id")
            }
        }
    }
}

The following is the result of running the example:

a: 19c3b379-a688-428c-b3e8-8cb507bdf083
a: ffaff5f6-5c6a-41a9-98b9-52f9ca134dfa
b: ffaff5f6-5c6a-41a9-98b9-52f9ca134dfa
a: 08e42544-def7-4859-8b2b-34bb50fe83b0
c: 08e42544-def7-4859-8b2b-34bb50fe83b0

The outer context, test "a", runs first, then runs again for test "b", and again for test "c". This is very useful when you want to reuse variables each time from a clean instance of the Spec class.

InstancePerLeaf

IsolationMode.InstancePerLeaf creates a new instance for every test case, including internal contexts, and runs them independently. This mode prevents state from being shared between test cases and guarantees test independence. However, keep in mind that it consumes more resources.

You can confirm this with the following example.

package com.devkuma.kotest.tutorial.isolation

import io.kotest.core.spec.IsolationMode
import io.kotest.core.spec.style.WordSpec
import java.util.*

class InstancePerLeafExample : WordSpec() {

    override fun isolationMode(): IsolationMode = IsolationMode.InstancePerLeaf

    private val id = UUID.randomUUID()

    init {
        "a" should {
            println("a: $id")
            "b" {
                println("b: $id")
            }
            "c" {
                println("c: $id")
            }
        }
    }
}

The following is the result of running the example:

a: 71830c57-6ee0-4b1f-8c32-812defac2b1e
b: 71830c57-6ee0-4b1f-8c32-812defac2b1e
a: 480ee65a-df4c-4a78-ac80-b6ae2c85ee20
c: 480ee65a-df4c-4a78-ac80-b6ae2c85ee20

Test "a" runs first, and test "b" runs in the same instance. Then a new Spec is created, test "a" runs again, and then test "c" runs.

Global Isolation Mode

You can also configure Isolation Mode globally through a system property.

object GlobalIsolationConfiguration : AbstractProjectConfig() {
    override val isolationMode = IsolationMode.SingleInstance
}

References

Isolation Modes | Kotest

Kotest Lifecycle Hooks

kc@example.com (kc kim) — Tue, 23 Apr 2024 00:37:00 +0900

Managing the Test Lifecycle

Managing the test lifecycle means performing the necessary work before and after each test case runs. Kotest provides beforeTest and afterTest blocks so you can manage the test lifecycle. With these blocks, you can perform setup or cleanup work before and after each test case.

Lifecycle Hooks

Sometimes you need to perform work before or after each test case runs. For example, you may need setup or cleanup work such as opening or closing a database connection, or measuring test execution time. Lifecycle hooks are used for this purpose.

Kotest provides many kinds of hooks that can be defined directly inside a spec. For more advanced use cases, such as distributable plugins or reusable hooks, you can use extensions.

At the end of this section, you can find the available hooks and when they are executed.

There are several ways to use hooks in Kotest.

Writing Hooks with DSL Methods

The first and simplest method is to use DSL methods available inside a spec to create and register a TestListener. For example, you can call beforeTest and afterTest directly as follows.

package com.devkuma.kotest.tutorial.lifecycle.ex1

import io.kotest.core.spec.style.WordSpec

class TestSpec : WordSpec({
    beforeTest {
        println("Starting a test $it")
    }

    afterTest { (test, result) ->
        println("Finished spec with result $result")
    }

    "this test" should {
        "be alive" {
            println("devkuma is alive!")
        }
    }
})

The following is the result of running the example:

Starting a test TestCase(descriptor=TestDescriptor... omitted ...
Starting a test TestCase(descriptor=TestDescriptor... omitted ...
Johnny5 is alive!
Finished spec with result Success(duration=6ms)
Finished spec with result Success(duration=51ms)

In the background, these DSL methods create an instance of TestListener that overrides the appropriate functions and registers it so the listener is executed.

You can use afterProject as a DSL method to create a project listener, but there is no beforeProject because the project has already started by the time the framework is detecting specs.

DSL Methods that Accept Functions

These DSL methods accept functions, so you can extract the logic into a function and reuse it in multiple places.

package com.devkuma.kotest.tutorial.lifecycle.ex2

import io.kotest.core.spec.style.WordSpec
import io.kotest.core.test.TestCase

val startTest: suspend (TestCase) -> Unit = {
    println("Starting a test $it")
}

class TestSpec : WordSpec({

    // used once
    beforeTest(startTest)

    "this test" should {
        "be alive" {
            println("devkuma is alive!")
        }
    }
})

class OtherSpec : WordSpec({

    // used twice
    beforeTest(startTest)

    "this test" should {
        "fail" {
            println("boom")
        }
    }
})

The following is the result of running the example:

Starting a test TestCase(descriptor=TestDescriptor... omitted ...
boom
Starting a test TestCase(descriptor=TestDescriptor... omitted ...
devkuma is alive!

Overriding Callback Functions in a Spec

The second method is to override callback functions in the spec. The example below is a variation of the first method.

package com.devkuma.kotest.tutorial.lifecycle.ex3

import io.kotest.core.spec.style.WordSpec
import io.kotest.core.test.TestCase
class TestSpec : WordSpec() {

    override suspend fun beforeTest(testCase: TestCase) {
        println("Starting a test $testCase")
    }

    init {
        "this test" should {
            "be alive" {
  println("devkuma is alive!")
            }
        }
    }
}

Lifecycle Callbacks

To understand all callbacks correctly, it is important to understand the possible TestType values:

Container: a container that can include other tests
Test: a leaf test that cannot include nested tests. In other words, a test that can run independently without invoking other tests
Dynamic: can be either a container or a test, and is used when tests are dynamically added by features such as property tests or data tests

Kotest provides several callback functions that run according to the test lifecycle.

The following table describes each callback used in Kotest:

Callback	Description
`beforeContainer`	Called before all tests in a test container are executed.
`afterContainer`	Called after all tests in a test container are executed.
`beforeEach`	Called before each test case runs. Use this when you want to perform separate setup for each test.
`afterEach`	Called after each test case runs. Use this when you want to perform separate cleanup for each test.
`beforeAny`	Called before all tests in a specific container or spec run. Use this for common setup for all tests in a container or spec.
`afterAny`	Called after all tests in a specific container or spec run. Use this for common cleanup for all tests in a container or spec.
`beforeTest`	Called before each test method runs. You can use this function to set up each test. It behaves the same as beforeAny.
`afterTest`	Called after each test method runs. You can use this function to clean up each test. It behaves the same as afterAny.
`beforeSpec`	Called before each spec runs. Use this for setup that applies to the whole spec. For example, you can open a database connection or initialize a specific resource. When `IsolationMode` is `SingleInstance`, it behaves the same as `prepareSpec`.
`afterSpec`	Called after each spec runs. Use this for cleanup that applies to the whole spec. For example, you can close a database connection or clean up a specific resource. When `IsolationMode` is `SingleInstance`, it behaves the same as `finalizeSpec`. If an exception occurs in this callback, subsequent `beforeSpec` and `AfterSpec` callbacks are skipped.
`prepareSpec`	Called before a spec runs, and used to perform custom initialization work. When `IsolationMode` is SingleInstance, it behaves the same as beforeSpec.
`finalizeSpec`	Called after a spec runs, and used to perform custom cleanup work. When `IsolationMode` is `SingleInstance`, it behaves the same as `afterSpec`.
`beforeInvocation`	Called before each test method or test case runs. If there is no invocation configuration, it behaves the same as beforeTest.
`afterInvocation`	Called after each test method or test case runs. If there is no invocation configuration, it behaves the same as afterTest.

These callbacks let you perform specific actions at various points, such as before and after each test or before and after each spec. Each callback is invoked at its corresponding point, and you can implement the logic you need.

beforeAny and beforeTest are simply different names for the same function, but they differ from beforeEach. beforeAny and beforeTest are called before TestType.Test, while beforeEach is called before both TestType.Container and TestType.Test. The same applies to afterAny, afterTest, and afterEach.

The following example shows when each callback function is executed:

package com.devkuma.kotest.tutorial.lifecycle.ex4

import io.kotest.core.spec.style.StringSpec
import io.kotest.matchers.shouldBe

class CallbackTest : StringSpec({

    // Callback called before each spec runs
    beforeSpec {
        println("beforeSpec: perform setup before the spec runs.")
        // Add setup work for the whole spec.
    }

    // Callback called after each spec runs
    afterSpec {
        println("afterSpec: perform cleanup after the spec runs.")
        // Add cleanup work for the whole spec.
    }

    // Callback called before each test runs
    beforeTest { testCase ->
        println("beforeTest: perform setup before ${testCase.name.testName} runs.")
        // Add common setup work for tests here.
    }

    // Callback called after each test runs
    afterTest { (testCase, testResult) ->
        println("afterTest: perform cleanup after ${testCase.name.testName} runs.")
        println("Result: $testResult")
        // Add common cleanup work for tests here.
    }

    // Callback called before all tests run
    beforeAny { testCase ->
        println("beforeAny: perform setup before ${testCase.name.testName} test runs.")
        // Add common setup work for all tests here.
    }

    // Callback called after all tests run
    afterAny { (testCase, testResult) ->
        println("afterAny: perform cleanup after ${testCase.name.testName} test runs.")
        println("Result: $testResult")
        // Add common cleanup work for all tests here.
    }

    // Callback called before each test runs
    beforeEach { testCase ->
        println("beforeEach: perform setup before ${testCase.name.testName} test runs.")
        // Add individual setup work for tests here.
    }

    // Callback called after each test runs
    afterEach { (testCase, testResult) ->
        println("afterEach: perform cleanup after ${testCase.name.testName} test runs.")
        println("Result: $testResult")
        // Add individual cleanup work for tests here.
    }

    // Test case 1
    "TestCase1" {
        println("Run TestCase1.")
        val result = 2 + 2
        result shouldBe 4
    }

    // Test case 2
    "TestCase2" {
        println("Run 'TestCase2'.")
        val list = mutableListOf<Int>()
        list.add(1)
        list.add(2)
        list.size shouldBe 2
    }
})

The following is the result of running the example:

beforeSpec: perform setup before the spec runs.
beforeEach: perform setup before TestCase1 test runs.
beforeTest: perform setup before TestCase1 runs.
beforeAny: perform setup before TestCase1 test runs.
Run TestCase1.
afterAny: perform cleanup after TestCase1 test runs. Result: Success(duration=36ms)
afterTest: perform cleanup after TestCase1 runs. Result: Success(duration=36ms)
afterEach: perform cleanup after TestCase1 test runs. Result: Success(duration=36ms)
beforeEach: perform setup before TestCase2 test runs.
beforeTest: perform setup before TestCase2 runs.
beforeAny: perform setup before TestCase2 test runs.
Run 'TestCase2'.
afterAny: perform cleanup after TestCase2 test runs. Result: Success(duration=1ms)
afterTest: perform cleanup after TestCase2 runs. Result: Success(duration=1ms)
afterEach: perform cleanup after TestCase2 test runs. Result: Success(duration=1ms)
afterSpec: perform cleanup after the spec runs.

References

Lifecycle hooks | Kotest

Kotest MockServer

kc@example.com (kc kim) — Sun, 31 Mar 2024 14:47:00 +0900

Kotest provides an extension for integration with the MockServer library.

NOTE

You need to add the `io.kotest.extensions:kotest-extensions-mockserver` module to your build.

MockServer

MockServer provides a virtual HTTP server. With this library, you can test as if communicating with a real server without using a separate mocking library such as mockk.

Kotest provides an extension for integration with the MockServer library.

Adding the Dependency

To use MockServer, add the io.kotest.extensions:kotest-extensions-mockserver module as a dependency. You can check the latest version on Maven Central.

testImplementation("io.kotest.extensions:kotest-extensions-mockserver:<version>")

Configuration

With a mock server, you can define an in-process HTTP server with hard-coded responses for the paths you want to test.

To use it in Kotest, attach a MockServerListener instance to the spec under test, and Kotest automatically controls its lifecycle.

Connecting the Mock Server

Then you only need to connect responses using MockServerClient.

For example:

class MyMockServerTest : FunSpec() {
  init {

      // this attaches the server to the lifeycle of the spec
      listener(MockServerListener(1080))

      // we can use the client to create routes. Here we are setting them up
      // before each test by using the beforeTest callback.
      beforeTest {
         MockServerClient("localhost", 1080).`when`(
            HttpRequest.request()
               .withMethod("POST")
               .withPath("/login")
               .withHeader("Content-Type", "application/json")
               .withBody("""{"username": "foo", "password": "bar"}""")
         ).respond(
            HttpResponse.response()
               .withStatusCode(202)
               .withHeader("X-Test", "foo")
         )
      }

      // this test will confirm the endpoint works
      test("login endpoint should accept username and password json") {

         // using the ktor client to send requests
         val client = HttpClient(CIO)
         val resp = client.post<io.ktor.client.statement.HttpResponse>("http://localhost:1080/login") {
            contentType(ContentType.Application.Json)
            body = """{"username": "foo", "password": "bar"}"""
         }

         // these handy matchers come from the kotest-assertions-ktor module
         resp.shouldHaveStatus(HttpStatusCode.Accepted)
         resp.shouldHaveHeader("X-Test", "foo")
      }
  }
}

The example above is, of course, only testing the mock itself, but it shows how a real test could be structured. For example, if you have an API client you want to test, you can use the mock server to configure API routes, then call methods on the API client and verify that the response is handled correctly.

References

MockServer | Kotest

Kotest Seeds

kc@example.com (kc kim) — Sun, 31 Mar 2024 13:18:00 +0900

Property Test Seeds

When a property test runs, values are generated using a random source created from a seed value. By default, this seed value is chosen randomly, using the default kotlin.random.Random instance. However, there are cases where you may want to fix or repeat this value.

To change the default used by all tests unless overridden by the options listed below, change the configuration value PropertyTesting.defaultSeed.

Specifying a Seed Manually

To set a seed manually, pass an instance of PropTestConfig to the property test method. You might do this after finding a test failure and wanting that value to keep running as a regression test in the future.

For example:

import io.kotest.core.spec.style.StringSpec
import io.kotest.property.PropTestConfig
import io.kotest.property.forAll

class PropertyExample : StringSpec({
    "String size" {
        forAll<String, String>(PropTestConfig(seed = 127305235)) { a, b ->
            (a + b).length == a.length + b.length
        }
    }
})

TIP

Whenever a property test fails, Kotest prints the seed that was used. You can duplicate the test and configure it to use this seed to create a permanent regression test for that value.

Re-running Failed Seeds

By default, when a property test fails, the seed used by that test is written to a file under ~/.kotest/seeds/<spec>/<testname>. Whenever the property test runs, Kotest detects whether this file exists and uses this seed instead of a random seed. The next time the test succeeds, the seed file is removed.

NOTE

Manually specified seeds always take precedence over failed seeds.

TIP

This feature can be disabled by setting `PropertyTesting.writeFailedSeed = false`.

Failing When a Seed Is Set

Some users may not want to specify seeds manually. They may want to use them only locally during development and not check them in. In that case, set PropertyTesting.failOnSeed = false on the server or set the environment variable kotest.proptest.seed.fail-if-set to false.

Then, if a seed is detected, the test suite fails.

References

Property Test Seeds | Kotest

Kotest Soft Assertions

kc@example.com (kc kim) — Sat, 16 Mar 2024 14:25:00 +0900

Soft Assertions

Soft assertions let several assertions run together and report all failures at once. They are useful when you want to see every failed condition instead of stopping at the first failure.

assertSoftly {
    foo shouldBe bar
    foo should contain(baz)
}

This is similar to JUnit’s assertAll.

You can also pass a subject to assertSoftly. Inside the block, assertions are applied to that subject.

assertSoftly(foo) {
    shouldNotEndWith("b")
    length shouldBe 3
}

Example:

import io.kotest.assertions.assertSoftly
import io.kotest.core.spec.style.FunSpec
import io.kotest.matchers.shouldBe

class SoftAssertionTest : FunSpec({
    test("soft assertions") {
        val person = Person("devkuma", 30)

        assertSoftly(person) {
            name shouldBe "devkuma"
            age shouldBe 30
        }
    }
})

data class Person(val name: String, val age: Int)

Soft assertions can also be configured through project configuration so that they are applied implicitly to all tests.

References

Soft Assertions | Kotest

Kotest Configuration

kc@example.com (kc kim) — Thu, 04 Apr 2024 22:53:00 +0900

Configuration

Kotest provides a way to specify several configuration options when running property tests. This is done by passing a PropTestConfig instance to the test method.

For example:

class PropertyExample: StringSpec({
   "String size" {
      forAll<String, String>(PropTestConfig(options here...)) { a,b ->
         (a + b).length == a.length + b.length
      }
   }
})

Seed

A commonly used configuration option is the seed used for the random source. This option is used when you want the same values to be repeated every time the test runs. You can use this option after finding a test failure and wanting that specific set of values to continue running in the future as a regression test.

For more details on how to use seeds, click here.

Min Failure

By default, Kotest does not allow failures. You may want to run a non-deterministic test several times and accept a small number of failures. You can specify this in the configuration.

class PropertyExample: StringSpec({
   "some flakey test" {
      forAll<String, String>(PropTestConfig(maxFailure = 3)) { a,b ->
         // max of 3 inputs can fail
      }
   }
})

PropTestListener

In property tests, you may need to perform setup and teardown for each test iteration. To do this, you can register a PropTestListener in PropTestConfig.

class PropertyExample: StringSpec({
   "some property test which require setup and tear down in each iteration" {
      forAll<String, String>(PropTestConfig(listeners = listOf(MyPropTestListener))) { a,b ->
         // some assertion
      }
   }
})

References

Configuration | Kotest

Kotest JUnit XML Format Reporter

kc@example.com (kc kim) — Sun, 31 Mar 2024 16:47:00 +0900

JUnit includes an XML report generator called the legacy XML report. Many tools integrate with this format, so it is very useful. However, this report has no concept of nested tests. Therefore, when used with nested testing styles in Kotest, parent tests are included as orphans.

To solve this problem, Kotest has its own implementation of the same format, and you can configure whether to include parent tests or collapse names.

NOTE

The build requires the `io.kotest:kotest-extensions-junitxml` module. Search [Maven Central](https://central.sonatype.com/artifact/io.kotest/kotest-extensions-junitxml) for the latest version.

To configure it in your project, add JunitXmlReporter using project configuration.

class MyConfig : AbstractProjectConfig() {
  override fun extensions(): List<Extension> = listOf(
    JunitXmlReporter(
      includeContainers = false, // don't write out status for all tests
      useTestPathAsName = true, // use the full test path (ie, includes parent test names)
      outputDir = "../target/junit-xml" // include to set output dir for maven
    )
  )
}

The reporter also needs to know where the build output folder is by setting a system property. Gradle also needs to know that it should not generate its own JUnit XML report. Configure this in Gradle’s test block.

tasks.named<Test>("test") {
  useJUnitPlatform()
  reports {
    junitXml.required.set(false)
  }
  systemProperty("gradle.build.dir", project.buildDir)
}

Parameters

The reporter has three parameters:

If includeContainers is true, all intermediate tests are included in the report as their own tests. The default is false.
If useTestPathAsName is true, the full test path is used as the name. In other words, the names of all parent tests are included in the name as a single string.
If outputDir is set, reports are generated in that folder. The default is test-results/test.

References

JUnit XML Format Reporter | Kotest

Kotest Inspectors

kc@example.com (kc kim) — Mon, 18 Mar 2024 22:15:00 +0900

Inspectors

Inspectors are extension functions for collections and arrays. They verify whether the given assertion is true for a specified number or group of elements.

For example, the following verifies that at least two values in xs are greater than zero.

xs.forAtLeast(2) {
    it shouldBeGreaterThan 0
}

The following verifies that no element in xs is greater than zero.

xs.forNone {
    it shouldBeGreaterThan 0
}

Kotest provides these inspectors:

Inspector	Description
`forAll`	The assertion must pass for every element.
`forNone`	The assertion must fail for every element.
`forOne`	The assertion must pass for exactly one element.
`forAtMostOne`	The assertion must pass for at most one element.
`forAtLeastOne`	The assertion must pass for at least one element.
`forAtLeast(k)`	The assertion must pass for at least `k` elements.
`forAtMost(k)`	The assertion must pass for at most `k` elements.
`forAny`	Alias for `forAtLeastOne`.
`forSome`	The assertion must pass for at least one element but not all elements.
`forExactly(k)`	The assertion must pass for exactly `k` elements.

Example:

import io.kotest.core.spec.style.FunSpec
import io.kotest.inspectors.forAll
import io.kotest.inspectors.forAtLeast
import io.kotest.inspectors.forNone
import io.kotest.matchers.ints.shouldBeGreaterThan

class NumbersTestWithInspectors : FunSpec({
    test("inspectors") {
        val numbers = listOf(1, 2, 3, 4, 5)

        numbers.forAll {
            it shouldBeGreaterThan 0
        }

        numbers.forAtLeast(2) {
            it shouldBeGreaterThan 3
        }

        numbers.forNone {
            it shouldBeGreaterThan 10
        }
    }
})

References

Inspectors | Kotest

Kotest Basic Extensions

kc@example.com (kc kim) — Mon, 12 Feb 2024 01:47:00 +0900

Extensions

Extensions are reusable lifecycle hooks. In fact, lifecycle hooks themselves are represented internally as instances of extensions. In the past, the term listener was used for simple interfaces and extension for advanced interfaces, but they are no longer distinguished and the two terms may be used interchangeably.

With extensions, you can perform additional actions during test execution or integrate custom functionality. Extensions let you control and extend tests more flexibly.

In general, extensions can define work that runs before or after tests, disable tests under specific conditions, or perform logging in a specific way. Extensions are provided as listeners, interceptors, interfaces, and other forms, so they can handle a variety of scenarios.

Basic Extension Usage

The basic usage is to create an implementation of the extension interface you need and register it on a test, spec, or the entire project through ProjectConfig.

For example, the following example creates a before/after spec listener and registers it on a spec:

package com.devkuma.kotest.tutorial.extensions.ex1

import io.kotest.core.listeners.AfterSpecListener
import io.kotest.core.listeners.BeforeSpecListener
import io.kotest.core.spec.Spec

class MyTestListener : BeforeSpecListener, AfterSpecListener {
    override suspend fun beforeSpec(spec: Spec) {
        println("beforeSpec")
    }
    override suspend fun afterSpec(spec: Spec) {
        println("afterSpec")
    }
}

To apply this extension, specify the listener with the extension function as follows:

package com.devkuma.kotest.tutorial.extensions.ex1

import io.kotest.core.spec.style.WordSpec

class TestSpec : WordSpec({
    extension(MyTestListener())
    
    "testSpec" should {
        println("testSpec")
    }
})

The following is the result of running the example:

beforeSpec
testSpec
afterSpec

All extensions registered inside a spec are used for every test in that Spec, including test factories and nested tests.

To apply an extension to all specs in the entire project, you can register it with the @AutoScan annotation.

The following example creates a before/after listener and applies it to all specs by adding @AutoScan:

package com.devkuma.kotest.tutorial.extensions.ex2

import io.kotest.core.listeners.AfterProjectListener
import io.kotest.core.listeners.BeforeProjectListener

@AutoScan
class ProjectListener : BeforeProjectListener, AfterProjectListener {
    override suspend fun beforeProject() {
        println("beforeProject")
    }

    override suspend fun afterProject() {
        println("afterProject")
    }
}

Unlike the previous example, the following test code does not explicitly specify an extension:

package com.devkuma.kotest.tutorial.extensions.ex2

import io.kotest.core.spec.style.FunSpec

class ProjectTest1 : FunSpec({
    test("test1") {
        println("test1")
    }
})

class ProjectTest2 : FunSpec({
    test("test2") {
        println("test2")
    }
})

The following is the result of running the example:

beforeProject
test1
test2
afterProject

CAUTION

Some extensions can only be registered at the project level. For example, registering a `BeforeProjectListener` inside a spec has no effect because the project has already started by the time that extension is discovered.

Simple Extensions

Simple Extensions in Kotest provide various listeners so users can perform desired work before and after test execution.

The table below lists the most basic extensions that handle test and spec lifecycle events. Most of them are the same as lifecycle hooks. For advanced extensions that can modify how the engine executes, see the advanced extensions section.

The role and usage of each listener are as follows:

Extension	Description
`BeforeContainerListener`	Called before a container, or test group, is executed. It is mainly used to set up an entire container. For example, you can initialize a specific database or configure external resources.
`AfterContainerListener`	Called after a container, or test group, is executed. It is mainly used for cleanup work needed after a container. For example, you can close database connections or release external resources.
`BeforeEachListener`	Called before each test runs. You can perform common setup needed before each test, such as initializing test data or setting a specific state.
`AfterEachListener`	Called after each test runs. You can perform common cleanup needed after each test, such as releasing resources used by the test or resetting state.
`BeforeTestListener`	Called before each test function runs. You can perform specific setup before each test function, such as initializing test data or configuring the test environment.
`AfterTestListener`	Called after each test function runs. You can perform cleanup after each test function, such as releasing resources used by the test or resetting state.
`BeforeInvocationListener`	Called before each parameterized test runs.
`AfterInvocationListener`	Called after each parameterized test runs.
`BeforeSpecListener`	Called before a test spec runs.
`AfterSpecListener`	Called after a test spec runs.
`PrepareSpecListener`	Called before a test spec runs, mainly to prepare for specific work.
`FinalizeSpecListener`	Called after a test spec runs, mainly to finish work such as releasing resources.
`BeforeProjectListener`	Called before all tests in the project run.
`AfterProjectListener`	Called after all tests in the project run.

These listeners help users perform the work they need before and after test execution, and each one is called at a specific point according to its role. This improves test flexibility and reusability.

Advanced Extensions

Advanced Extensions are powerful Kotest features that allow fine-grained control over the test execution process and custom logic.

The following describes the role and behavior of each advanced extension:

Extension	Description
`ConstructorExtension`	Changes the constructor of a test class and extends tests with constructor injection.
`TestCaseExtension`	Customizes and modifies the execution of each test case.
`SpecExtension`	Changes and extends the behavior of a test spec.
`SpecRefExtension`	Performs an extension for a specific test spec reference.
`DisplayNameFormatterExtension`	Formats and modifies the display name of a test.
`EnabledExtension`	Enables or disables tests.
`ProjectExtension`	Performs project-level extensions and controls global settings.
`SpecExecutionOrderExtension`	Adjusts and controls the execution order of test specs.
`TagExtension`	Adds and manages tags on tests.
`InstantiationErrorListener`	Handles errors that occur during test class instantiation.
`InstantiationListener`	Handles test class instantiation events.
`PostInstantiationExtension`	Performs additional work after instantiation.
`IgnoredSpecListener`	Handles ignored test specs.
`SpecFilter`	Filters test specs according to specific conditions.
`TestFilter`	Filters tests according to specific conditions.

You can use these advanced extensions to control test execution in detail and customize it as needed.

Using Extensions

System Out Listener

One practical example of an extension is Kotest’s NoSystemOutListener. This extension raises an error when output is written to standard output.

package com.devkuma.kotest.tutorial.extensions.ex3

import io.kotest.core.spec.style.DescribeSpec
import io.kotest.extensions.system.NoSystemOutListener

class MyTestSpec : DescribeSpec({

    listener(NoSystemOutListener)

    describe("No test should write to standard output.") {
        it("standard out output") {
            println("boom") // fails
        }
    }
})

The following is the result of running the example:

io.kotest.extensions.system.SystemOutWriteException
	at io.kotest.extensions.system.NoSystemOutListener$setup$1.error(NoSystemOutExtensions.kt:18)
	at io.kotest.extensions.system.NoSystemOutListener$setup$1.print(NoSystemOutExtensions.kt:23)
	at io.kotest.extensions.system.NoSystemOutListener$setup$1.print(NoSystemOutExtensions.kt:17)
	at java.base/java.io.PrintStream.println(PrintStream.java:1054)

... omitted ...

The test code includes a println function, so standard output is produced and an error occurs.

Timer Listener

Another example records the time taken by each test case.

You can do this with the beforeTest and afterTest functions:

package com.devkuma.kotest.tutorial.extensions.ex4

import io.kotest.core.listeners.AfterTestListener
import io.kotest.core.listeners.BeforeTestListener
import io.kotest.core.test.TestCase
import io.kotest.core.test.TestResult

object TimerListener : BeforeTestListener, AfterTestListener {

    private var started = 0L

    override suspend fun beforeTest(testCase: TestCase) {
        started = System.currentTimeMillis()
    }

    override suspend fun afterTest(testCase: TestCase, result: TestResult) {
        println("Duration of ${testCase.descriptor.parent.id} = " + (System.currentTimeMillis() - started))
    }
}

You can register it in test code as follows:

package com.devkuma.kotest.tutorial.extensions.ex4

import io.kotest.core.spec.style.FunSpec

class TimeTest : FunSpec({
    extensions(TimerListener)

    // tests here
    test("TimeTest") {
        println("TimeTest")
    }
})

The following is the result of running the example:

TimeTest
Duration of DescriptorId(value=com.devkuma.kotest.tutorial.extensions.exam4.TimeTest) = 8

Or you can register it for the entire project.

object MyConfig : AbstractProjectConfig() {
    override fun extensions(): List<Extension> = listOf(TimerListener)
}

References

Kotest HTML Reporter

kc@example.com (kc kim) — Sun, 31 Mar 2024 17:03:00 +0900

When using JUnit XML, tests with nested tests can generate XML results with output. Unfortunately, Gradle generates HTML reports using in-memory results that do not support nested tests, and it does not appear to be able to import results from another XML file.

To solve this problem, Kotest has a listener that can generate HTML reports based on XML reports generated by JUnit XML.

NOTE

The following module is required: `io.kotest:kotest-extensions-htmlreporter` in your build. Search [Maven Central](https://central.sonatype.com/artifact/io.kotest/kotest-extensions-htmlreporter) for the latest version.

To use it, simply add it as a listener through project configuration.

class ProjectConfig : AbstractProjectConfig() {

   override val specExecutionOrder = SpecExecutionOrder.Annotated

    override fun extensions(): List<Extension> = listOf(
        JunitXmlReporter(
            includeContainers = false,
            useTestPathAsName = true,
        ),
        HtmlReporter()
    )
}

Then add html.required.set(false) to the test task so Gradle does not generate its own HTML report.

tasks.test {
  useJUnitPlatform()
  reports {
    html.required.set(false)
    junitXml.required.set(false)
  }
  systemProperty("gradle.build.dir", project.buildDir)
}

Also add JunitXmlReporter. This creates the XML reports needed to generate the HTML report. No additional configuration is required; simply start generating the HTML report.

By default, reports are stored at /to/buildDir/reports/tests/test, but you can change this by modifying the outputDir parameter.

References

HTML Reporter | Kotest

Kotest Coroutines

kc@example.com (kc kim) — Sun, 31 Mar 2024 13:05:00 +0900

Testing Coroutine Dispatchers

TestDispatcher is a special CoroutineDispatcher provided by the kotlinx-coroutines-test module that lets developers control a virtual clock and skip delays.

TestDispatcher is a special CoroutineDispatcher provided by the kotlinx-coroutines-test module, which allows developers to control a virtual clock and skip delays.

TestDispatcher supports the following operations.

currentTime gets the current virtual time.
runCurrent() runs tasks scheduled at the current virtual time.
advanceUntilIdle() runs all queued tasks until there are no more pending tasks.
advanceTimeBy(timeDelta) runs queued tasks until the current virtual time has advanced by timeDelta.

To use TestDispatcher in a test, enable coroutineTestScope in the test configuration:

package com.devkuma.kotest.tutorial.coroutines.ex1

import io.kotest.core.spec.style.FunSpec

class TestDispatcherTest : FunSpec() {
    init {
        test("foo").config(coroutineTestScope = true) {
            // this test will run with a test dispatcher
        }
    }
}

Inside this test, you can retrieve a handle to the scheduler through an extension named testCoroutineScheduler. You can use this scheduler to manipulate time:

import io.kotest.core.test.testCoroutineScheduler

class TestDispatcherTest : FunSpec() {
   init {
      test("advance time").config(coroutineTestScope = true) {
        val duration = 1.days
        // launch a coroutine that would normally sleep for 1 day
        launch {
          delay(duration.inWholeMilliseconds)
        }
        // move the clock on and the delay in the above coroutine will finish immediately.
        testCoroutineScheduler.advanceTimeBy(duration.inWholeMilliseconds)
        val currentTime = testCoroutineScheduler.currentTime
      }
   }
}

You can enable the test dispatcher for all tests in a spec by setting coroutineTestScope to true at the spec level:

class TestDispatcherTest : FunSpec() {
   init {
      coroutineTestScope = true
      test("this test uses a test dispatcher") {
      }
      test("and so does this test!") {
      }
   }
}

Finally, you can enable the test dispatcher for all tests in a module by using ProjectConfig:

class ProjectConfig : AbstractProjectConfig() {
  override var coroutineTestScope = true
}

Coroutine Debugging

kotlinx-coroutines-debug is a module that provides debugging support for coroutines on the JVM. When enabled, a debug agent is installed by ByteBuddy and captures information when coroutines are created, started, suspended, and resumed.

Kotest provides a feature for enabling debugging per test. You can use it by enabling coroutineDebugProbe in the test configuration.

When this feature is enabled, all coroutines started inside the test are included in a coroutine dump after the test completes or as soon as an exception occurs.

class CoroutineDebugging : FunSpec() {
   init {
      test("foo").config(coroutineDebugProbes = true) {
         someMethodThatLaunchesACoroutine() // launches a new coroutine
      }
   }
}

The coroutine dump looks like this:

Coroutines dump 2021/11/27 22:17:43

Coroutine DeferredCoroutine{Active}@71f1906, state: CREATED
    (Coroutine creation stacktrace)
    at kotlin.coroutines.intrinsics.IntrinsicsKt__IntrinsicsJvmKt.createCoroutineUnintercepted(IntrinsicsJvm.kt:122)
    at kotlinx.coroutines.intrinsics.CancellableKt.startCoroutineCancellable(Cancellable.kt:30)
    at kotlinx.coroutines.BuildersKt__Builders_commonKt.async$default(Builders.common.kt:82)
    at kotlinx.coroutines.BuildersKt.async$default(Unknown Source)
    at com.sksamuel.kotest.engine.coroutines.Wibble$1.invokeSuspend(CoroutineDebugTest.kt:37)
    at com.sksamuel.kotest.engine.coroutines.Wibble$1.invoke(CoroutineDebugTest.kt)

Spec-level Configuration

You can enable coroutine debugging for all tests in a spec by overriding the coroutineDebugProbes setting inside the spec:

class CoroutineDebugging : FunSpec() {
  init {

    coroutineDebugProbes = true

    test("foo") {
      // debugging enabled here
    }

    test("bar") {
      // debugging enabled here
    }

  }
}

Project-wide Configuration

You can enable coroutine debugging for all tests in a project by using ProjectConfig:

class ProjectConfig : AbstractProjectConfig() {
  override val coroutineDebugProbes = true
}

References

Kotest Assertion Mode

kc@example.com (kc kim) — Sat, 16 Mar 2024 22:22:00 +0900

Assertion Mode

When using the Kotest framework with Kotest assertions, you can make the build fail or print a warning to stderr if a test runs without executing any assertions.

To do this, set assertionMode to AssertionMode.Error or AssertionMode.Warn inside the spec.

For example:

import io.kotest.core.spec.style.FunSpec
import io.kotest.core.test.AssertionMode

class MySpec : FunSpec() {
    init {
        assertions = AssertionMode.Error
        test("this test has no assertions") {
            val name = "devkuma"
            name.length == 7 // this isn't actually testing anything
        }
    }
}

Running this test prints the following result:

Test 'this test has no assertions' did not invoke any assertions

It can also be configured globally through project configuration or with the system property kotest.framework.assertion.mode.

NOTE

Assertion Mode works only with Kotest assertions and does not work with other assertion libraries.

References

Assertion Mode | Kotest

Kotest Allure Extension

kc@example.com (kc kim) — Sun, 14 Apr 2024 00:32:00 +0900

Allure is an open source framework designed for detailed interactive test reports. It works by generating report files and then using them to create the final HTML report. You can think of it as similar to traditional JUnit reports, but more advanced and detailed.

TIP

If you prefer looking at an example instead of reading documentation, there is a sample project [here](https://github.com/kotest/kotest-examples-allure).

There are two steps. The first step is generating raw data when tests run, and the second step is compiling that data into an interactive HTML report.

This module provides integration for using Allure with Kotest. To get started, add the following dependency to your Gradle build file.

io.kotest.extensions:kotest-extensions-allure:${kotest.version}

Note: The group ID is different from the default Kotest dependency, io.kotest. It is io.kotest.extensions.

Data Collection

Allure has data collectors for most test frameworks, and this module provides integration for Kotest. Once the module is added to the build, connect the AllureTestReporter class globally using project configuration.

class MyConfig : AbstractProjectConfig {
    override fun listeners() = listOf(AllureTestReporter())
}

Now, whenever tests run, Kotest writes test data in Allure JSON format.

Gradle Plugin

Now that the tests are complete, the data can be compiled into the final report.

You can do this manually using the Allure binary, or you can use the Allure Gradle plugin. To use the Gradle plugin, first add the plugin to the plugin block in your build.

plugins {
  ...
  id("io.qameta.allure") version "2.8.1"
}

Next, add the Allure configuration section to set the version and disable autoconfiguration, because Allure can only autoconfigure JUnit and Kotest handles this itself.

allure {
  autoconfigure = false
  version = "2.13.1"
}

Finally, run the Gradle task allureReport. A report is generated under ./build/reports/allure-report, where you can find the report’s index.html entry point.

Build Directory Settings

If you are not using the Gradle plugin, you need to tell Allure where the build directory is by setting the allure.results.directory system property in the test configuration. If you use the Gradle plugin, the plugin handles this automatically, so you can skip this step.

For example:

tasks.named<Test>("test") { // or "jvmTest" etc
   useJUnitPlatform()
   systemProperty("allure.results.directory", project.buildDir.toString() + "/allure-results")
}

Final Report

If everything completes successfully, after test execution and report generation you will see something like this:

References

Allure | Kotest

Kotest Shrinking

kc@example.com (kc kim) — Sat, 06 Apr 2024 18:59:00 +0900

Shrinking

In property-based testing, the first failure case found may contain a lot of complexity that is not actually responsible for making the test fail. Shrinking is the mechanism by which a property-based testing framework simplifies a failing case to find the smallest reproducible case. In Kotest, the way generators shrink failing cases is defined by implementations of the shrinking interface. Built-in generators usually have default shrinkers defined by the framework, and custom generators can specify custom shrinker implementations.

Shrinking Built-in Generators

Built-in generators, see the generators list, have default shrink functions defined by the framework. A shrink function takes the value that caused the test to fail as input and returns a list of new values that Kotest can apply to the test. The exact behavior depends on the data type. For example, for strings, shrinking can remove the first or last character; for integers, it can reduce the value or halve it. Shrinking behavior is also defined for edge cases such as empty strings or integer zero. Shrinking is performed when a test using these generators fails.

Arb.positiveInt().checkAll { i ->
    calculateProperty(i) shouldBe true
}

If the test fails for one of the generated inputs, the shrink result is displayed:

Property test failed for inputs

0) 1792716902

Caused by io.kotest.assertions.AssertionFailedError: expected:<1792716902> but was:<0> at
    PropertyBasedTest$1$1$3$1.invokeSuspend(PropertyBasedTest.kt:54)
    PropertyBasedTest$1$1$3$1.invoke(PropertyBasedTest.kt)
    PropertyBasedTest$1$1$3$1.invoke(PropertyBasedTest.kt)
    io.kotest.property.internal.ProptestKt$proptest$3$2.invokeSuspend(proptest.kt:45)

Attempting to shrink arg 1792716902
Shrink #1: 1 pass
Shrink #2: 597572300 fail
Shrink #3: 199190766 fail
Shrink #4: 66396922 fail
Shrink #5: 22132307 fail
Shrink #6: 7377435 fail
Shrink #7: 2459145 fail

[...]

Shrink #999: 29948 pass
Shrink #1000: 44922 pass
Shrink #1001: 59896 pass
Shrink #1002: 89839 fail
Shrink result (after 1002 shrinks) => 89839

Caused by io.kotest.assertions.AssertionFailedError: expected:<89839> but was:<0> at
    PropertyBasedTest$1$1$3$1.invokeSuspend(PropertyBasedTest.kt:54)
    PropertyBasedTest$1$1$3$1.invoke(PropertyBasedTest.kt)
    PropertyBasedTest$1$1$3$1.invoke(PropertyBasedTest.kt)
    io.kotest.property.internal.ShrinkfnsKt$shrinkfn$1$1$smallestA$1.invokeSuspend(shrinkfns.kt:19)

By default, Kotest shrinks 1000 times. This behavior is configurable. For example, if you want shrinking to continue without a limit:

Arb.positiveInt().checkAll(PropTestConfig(shrinkingMode = ShrinkingMode.Unbounded)) { i ->
    calculateProperty(i) shouldBe true
}

Shrinking Custom Generators

Custom generators do not have shrinkers defined by Kotest. Instead, you can implement a custom shrinker. The example below returns the coordinates next to the value itself.

data class Coordinate(val x: Int, val y: Int)

class CoordinateTest : FunSpec({
    context("Coordinate Transformations") {
        // Shrinker takes the four neighbouring coordinates
        val coordinateShrinker = Shrinker<Coordinate> { c ->
            listOf(
                Coordinate(c.x - 1, c.y),
                Coordinate(c.x, c.y - 1),
                Coordinate(c.x + 1, c.y),
                Coordinate(c.x, c.y + 1),
            )
        }
        val coordinateArb = arbitrary(coordinateShrinker) {
            Coordinate(Arb.nonNegativeInt().bind(), Arb.nonNegativeInt().bind())
        }

        test("Coordinates are always positive after transformation") {
            coordinateArb.checkAll {
                transform(it).x shouldBeGreaterThanOrEqualTo 0
                transform(it).y shouldBeGreaterThanOrEqualTo 0
            }
        }
    }
})

References

Shrinking | Kotest

Kotest Core Matchers

kc@example.com (kc kim) — Thu, 28 Mar 2024 00:46:00 +0900

These matchers are provided by the kotest-assertions-core module.

General
`obj.shouldBe(other)`	Checks that the given `obj` and `other` are equal.
`expr.shouldBeTrue()`	Checks that the expression is true. Equivalent to `expr.shouldBe(true)`.
`expr.shouldBeFalse()`	Checks that the expression is false. Equivalent to `expr.shouldBe(false)`.
`shouldThrow<T> { block }`	Checks that the block throws `T` or a subtype of `T`.
`shouldThrowExactly<T> { block }`	Checks that the block throws exactly `T`.
`shouldThrowAny { block }`	Checks that the block throws any type of `Throwable`.
`shouldThrowMessage(message) { block }`	Checks that the code block throws a `Throwable` with the given message.

Types
`obj.shouldBeSameInstanceAs(other)`	Compares objects by identity, that is, by the same reference.
`obj.shouldBeTypeOf<T>()`	Checks that the reference is exactly type `T`; subclasses fail.
`obj.shouldBeInstanceOf<T>()`	Checks that the reference is type `T` or a subclass of `T`.
`obj.shouldHaveAnnotation(annotationClass)`	Checks that the object has an annotation of the given type.
`obj.shouldBeNull()`	Checks that the reference is null.

Comparables
`comp.shouldBeLessThan(other)`	Uses `compareTo` to check that `comp` is less than `other`.
`comp.shouldBeLessThanOrEqualTo(other)`	Uses `compareTo` to check that `comp` is less than or equal to `other`.
`comp.shouldBeEqualComparingTo(other)`	Uses `compareTo` to check that `comp` equals `other`.
`comp.shouldBeEqualComparingTo(other, comparator)`	Uses `comparator.compare` to check that `comp` equals `other`.
`comp.shouldBeGreaterThan(other)`	Uses `compareTo` to check that `comp` is greater than `other`.
`comp.shouldBeGreaterThanOrEqualTo(other)`	Uses `compareTo` to check that `comp` is greater than or equal to `other`.

Iterator
`iterator.shouldBeEmpty()`	Checks that the iterator has no next value.
`iterator.shouldHaveNext()`	Checks that the iterator has a next value.

Maps
`map.shouldContain("key", "value")`	Checks that the map contains a mapping from `"key"` to `"value"`.
`map.shouldContainAll(other)`	Checks that the map contains all pairs from the given map.
`map.shouldContainExactly(other)`	Checks that the map contains exactly the pairs from the given map and no extra entries.
`map.shouldContainKey(key)`	Checks that the map contains the key with any value.
`map.shouldContainKeys(keys)`	Checks that the map contains mappings for all given keys.
`map.shouldContainValue(value)`	Checks that the map contains at least one mapping with the given value.
`map.shouldContainValues(values)`	Checks that the map contains all given values.
`map.shouldBeEmpty()`	Checks that the map is empty.
`map.shouldMatchAll("k1" to {it shouldBe "v1"}, "k2" to {it shouldBe "v2"}, ...)`	Asserts that all entries in the map can match the provided matchers; extra keys are ignored.
`map.shouldMatchExactly("k1" to {it shouldBe "v1"}, "k2" to {it shouldBe "v2"}, ...)`	Checks that the map entries match the provided matchers exactly.

Strings
`str.shouldBeBlank()`	Checks that the string is empty or contains only whitespace.
`str.shouldBeEmpty()`	Checks that the string has length 0.
`str.shouldBeLowerCase()`	Checks that the string is all lowercase.
`str.shouldBeUpperCase()`	Checks that the string is all uppercase.
`str.shouldContain("substr")`	Checks that the string contains the given substring. This matcher is case-sensitive.
`str.shouldContain(regex)`	Checks that the string contains the given regular expression.
`str.shouldContainADigit()`	Checks that the string contains at least one digit.
`str.shouldContainIgnoringCase(substring)`	Checks that the string contains the substring ignoring case.
`str.shouldContainOnlyDigits()`	Checks that the string contains only digits or is empty.
`str.shouldBeInteger([radix])`	Checks that the string contains an int and returns it.
`str.shouldContainOnlyOnce(substring)`	Checks that the string contains the substring exactly once.
`str.shouldEndWith(suffix)`	Checks that the string ends with the given suffix.
`str.shouldHaveLength(length)`	Checks that the string has the given length.
`str.shouldHaveLineCount(count)`	Checks that the string has the given number of lines.
`str.shouldHaveMaxLength(max)`	Checks that the string is not longer than the given maximum length.
`str.shouldHaveMinLength(min)`	Checks that the string is not shorter than the given minimum length.
`str.shouldHaveSameLengthAs(anotherString)`	Checks that the string has the same length as another string.
`str.shouldMatch(regex)`	Checks that the string fully matches the given regular expression.
`str.shouldStartWith("prefix")`	Checks that the string starts with the given prefix.
`str.shouldBeEqualIgnoringCase(other)`	Checks that the string equals another string ignoring case.
`str.shouldBeTruthy()`	Checks that the string is truthy: one of `true`, `yes`, `y`, or `1`.
`str.shouldBeFalsy()`	Checks that the string is falsy: one of `false`, `no`, `n`, or `0`.

Integers, Longs, Doubles, and Floats
`number.shouldBeBetween(x, y)`	Checks that the number is between `x` and `y`, inclusive.
`number.shouldBeLessThan(n)`	Checks that the number is less than `n`.
`number.shouldBeLessThanOrEqual(n)`	Checks that the number is less than or equal to `n`.
`number.shouldBeGreaterThan(n)`	Checks that the number is greater than `n`.
`number.shouldBeGreaterThanOrEqual(n)`	Checks that the number is greater than or equal to `n`.
`number.shouldBeInRange(range)`	Checks that the number is in the given range.
`number.shouldBeEven()`	Checks that the number is even.
`number.shouldBeOdd()`	Checks that the number is odd.
`number.shouldBeZero()`	Checks that the number is zero.
`double.shouldBe(value plusOrMinus(tolerance))`	Checks that the double equals the given value within tolerance.
`double.shouldBePositive()`	Checks that the double is positive.
`double.shouldBeNegative()`	Checks that the double is negative.
`double.shouldBePositiveInfinity()`	Checks that the double is positive infinity.
`double.shouldBeNegativeInfinity()`	Checks that the double is negative infinity.
`double.shouldBeNaN()`	Checks that the double is NaN.

BigDecimal
`bigDecimal.shouldHavePrecision(n)`	Checks that the BigDecimal precision equals `n`.
`bigDecimal.shouldHaveScale(n)`	Checks that the BigDecimal scale equals `n`.
`bigDecimal.shouldBePositive()`	Checks that the BigDecimal is positive.
`bigDecimal.shouldBeNegative()`	Checks that the BigDecimal is negative.
`bigDecimal.shouldBeZero()`	Checks that the BigDecimal is zero.
`bigDecimal.shouldBeLessThan(n)`	Checks that the BigDecimal is less than `n`.
`bigDecimal.shouldBeLessThanOrEquals(n)`	Checks that the BigDecimal is less than or equal to `n`.
`bigDecimal.shouldBeGreaterThan(n)`	Checks that the BigDecimal is greater than `n`.
`bigDecimal.shouldBeGreaterThanOrEquals(n)`	Checks that the BigDecimal is greater than or equal to `n`.
`bigDecimal.shouldBeInRange(r)`	Checks that the BigDecimal is in the given range.

Channels
`channel.shouldReceiveWithin(duration)`	Checks that the channel receives within the duration.
`channel.shouldReceiveNoElementsWithin(duration)`	Checks that the channel receives no elements within the duration.
`channel.shouldHaveSize(n)`	Checks that the channel receives exactly `n` elements before closing.
`channel.shouldReceiveAtLeast(n)`	Checks that the channel receives at least `n` elements.
`channel.shouldReceiveAtMost(n)`	Checks that the channel receives at most `n` elements before closing.
`channel.shouldBeClosed()`	Checks that the channel is closed.
`channel.shouldBeOpen()`	Checks that the channel is open.
`channel.shouldBeEmpty()`	Checks that the channel is empty.

URIs
`uri.shouldHaveAuthority(fragment)`	Checks that the URI has the given authority.
`uri.shouldHaveFragment(fragment)`	Checks that the URI has the given fragment.
`uri.shouldHaveHost(scheme)`	Checks that the URI has the given host.
`uri.shouldHaveParameter(scheme)`	Checks that the URI query string contains the given parameter.
`uri.shouldHavePath(scheme)`	Checks that the URI has the given path.
`uri.shouldHavePort(scheme)`	Checks that the URI has the given port.
`uri.shouldHaveQuery(fragment)`	Checks that the URI has the given query.
`uri.shouldHaveScheme(scheme)`	Checks that the URI has the given scheme.

Files
`file.shouldBeAbsolute()`	Checks that the file represents an absolute path.
`file.shouldBeADirectory()`	Checks that the file represents a directory.
`file.shouldBeAFile()`	Checks that the file represents a file.
`file.shouldBeCanonical()`	Checks that the file is in canonical form.
`file.shouldBeEmpty()`	Checks that the file exists and is empty.
`file.shouldBeExecutable()`	Checks that the current process can execute the file.
`file.shouldBeHidden()`	Checks that the file exists on disk and is hidden.
`file.shouldBeReadable()`	Checks that the current process can read the file.
`file.shouldBeRelative()`	Checks that the file represents a relative path.
`file.shouldBeSmaller(file)`	Checks that this file is smaller than the given file.
`file.shouldBeLarger(file)`	Checks that this file is larger than the given file.
`file.shouldBeWriteable()`	Checks that the current process can write the file.
`dir.shouldBeNonEmptyDirectory()`	Checks that the file is a directory and is not empty.
`dir.shouldContainFile(name)`	Checks that the directory contains a file with the given name.
`dir.shouldContainNFiles(name)`	Checks that the directory contains exactly `n` files.
`file.shouldExist()`	Checks that the file exists on disk as a directory or file.
`file.shouldHaveExtension(ext)`	Checks that the file ends with the given extension.
`file.shouldHaveFileSize(size)`	Checks that the file has the given size.
`file.shouldHaveName(name)`	Checks that the file name matches the given name.
`file.shouldHavePath(path)`	Checks that the file path matches the given path.
`file.shouldStartWithPath(prefix)`	Checks that the file path starts with the given prefix.
`dir.shouldContainFileDeep(name)`	Checks that the directory or one of its subdirectories contains a file with the given name.
`dir.shouldContainFiles(name1, name2, ..., nameN)`	Checks that the directory contains all files with the given names.
`file.shouldBeSymbolicLink()`	Checks that the file is a symbolic link.
`file.shouldHaveParent(name)`	Checks that the file has a parent with the given name.

Date and time
`date.shouldHaveSameYearAs(otherDate)`	Checks that the date has the same year as the given date.
`date.shouldHaveSameMonthAs(otherDate)`	Checks that the date has the same month as the given date.
`date.shouldBeBefore(otherDate)`	Checks that the date is before the given date.
`date.shouldBeAfter(otherDate)`	Checks that the date is after the given date.
`date.shouldBeWithin(duration, otherDate)`	Checks that the date is within the duration of the given date.
`date.shouldBetween(firstDate, secondDate)`	Checks that the date is between the first and second dates.
`date.shouldHaveYear(year)`	Checks that the date has the given year.
`date.shouldHaveMonth(month)`	Checks that the date has the given month.
`date.shouldHaveDayOfYear(day)`	Checks that the date has the given day of year.
`date.shouldHaveDayOfMonth(day)`	Checks that the date has the given day of month.
`date.shouldHaveDayOfWeek(day)`	Checks that the date has the given day of week.
`date.shouldHaveHour(hour)`	Checks that the date has the given hour.
`date.shouldHaveMinute(minute)`	Checks that the date has the given minute.
`date.shouldHaveSecond(second)`	Checks that the date has the given second.
`date.shouldHaveNano(nano)`	Checks that the date has the given nanosecond.
`zonedDateTime.shouldBeToday()`	Checks that the `ZonedDateTime` is today.
`zonedDateTime.shouldHaveSameInstantAs(other: ZonedDateTime)`	Checks that the `ZonedDateTime` is the same instant as another `ZonedDateTime`.
`offsetDateTime.shouldBeToday()`	Checks that the `OffsetDateTime` is today.
`offsetDateTime.shouldHaveSameInstantAs(other: OffsetDateTime)`	Checks that the `OffsetDateTime` is the same instant as another `OffsetDateTime`.
`time.shouldBeBefore(otherTime)`	Checks that the time is before the given time.
`time.shouldBeAfter(otherTime)`	Checks that the time is after the given time.
`time.shouldBeBetween(firstTime, secondTime)`	Checks that the time is between `firstTime` and `secondTime`.
`instant.shouldBeAfter(anotherInstant)`	Checks that the instant occurs after another instant.
`instant.shouldBeBefore(anotherInstant)`	Checks that the instant occurs before another instant.
`instant.shouldBeBetween(fromInstant, toInstant)`	Checks that the instant is between `fromInstant` and `toInstant`.
`instant.shouldBeCloseTo(anotherInstant, duration)`	Checks that the instant is close to another instant within the duration.

Concurrent, futures, and threads
`shouldCompleteWithin(timeout, unit, function)`	Checks that the function completes within the given duration.
`shouldTimeout(timeout, unit, function)`	Checks that the function does not complete within the given duration.
`shouldTimeout(duration, suspensableFunction)`	Checks that the suspendable function does not complete within the given duration.
`future.shouldBeCancelled()`	Checks that the future was cancelled.
`future.shouldBeCompleted()`	Checks that the future completed.
`future.shouldBeCompletedExceptionally()`	Checks that the future completed exceptionally.
`future.shouldCompleteExceptionallyWith(throwable)`	Checks that the future completed with the given exception.
`thread.shouldBeBlocked()`	Checks that the thread is currently blocked.
`thread.shouldBeDaemon()`	Checks that the thread is a daemon thread.
`thread.shouldBeAlive()`	Checks that the thread is alive.
`thread.shouldBeTerminating()`	Checks that the thread has terminated.

Throwables / Exceptions
`throwable.shouldHaveMessage(message)`	Checks that the throwable message equals the given message.
`throwable.shouldHaveCause()`	Checks that the throwable has a cause.
`throwable.shouldHaveCause { block }`	Checks that the throwable has a cause and passes it to the block.
`throwable.shouldHaveCauseInstanceOf<T>()`	Checks that the throwable has a cause that is type `T` or a subtype of `T`.
`throwable.shouldHaveCauseOfType<T>()`	Checks that the throwable has a cause that is exactly type `T`.

Result and Optional
`result.shouldBeSuccess()`	Checks that the result is success.
`result.shouldBeSuccess(value)`	Checks that the result is success and has the given value.
`result.shouldBeSuccess(block)`	Assumes the result is success and runs the block with the result value.
`result.shouldBeFailure()`	Checks that the result is failure.
`result.shouldBeFailureOfType<Type : Throwable>()`	Checks that the result is failure with the given exception class.
`result.shouldBeFailure(block)`	Assumes the result is failure and runs the block with the exception.
`optional.shouldBePresent()`	Checks that the `Optional` is present.
`optional.shouldBePresent { value -> .. }`	Checks that the `Optional` is present and runs the block with its value.
`optional.shouldBeEmpty()`	Checks that the `Optional` is empty.

Reflection
`kclass.shouldHaveAnnotations()`	Checks that the class has annotations.
`kclass.shouldHaveAnnotations(n)`	Checks that the class has exactly `n` annotations.
`kclass.shouldBeAnnotatedWith<T>()`	Checks that the class is annotated with the given type.
`kclass.shouldHaveFunction(name)`	Checks that the class has a function with the given name.
`kclass.shouldHaveMemberProperty(name)`	Checks that the class has a member property with the given name.
`kclass.shouldBeSubtypeOf<T>()`	Checks that the class is a subtype of `T`.
`kclass.shouldBeSupertypeOf<T>()`	Checks that the class is a supertype of `T`.
`kclass.shouldBeData()`	Checks that the class is a data class.
`kclass.shouldBeSealed()`	Checks that the class is a sealed class.
`kclass.shouldBeCompanion()`	Checks that the class is a companion object.
`kclass.shouldHavePrimaryConstructor()`	Checks that the class has a primary constructor.
`kclass.shouldHaveVisibility(visibility)`	Checks that the class has the given visibility.
`kfunction.shouldHaveReturnType<T>()`	Checks that the function returns the given type.
`kfunction.shouldBeInline()`	Checks that the function is inline.
`kfunction.shouldBeInfix()`	Checks that the function is infix.
`kproperty.shouldBeOfType<T>()`	Checks that the property has the given type.
`kproperty.shouldBeConst()`	Checks that the property is `const`.
`kproperty.shouldBeLateInit()`	Checks that the property is a `lateinit var`.
`kcallable.shouldHaveVisibility(visibility)`	Checks that the member has the given visibility.
`kcallable.shouldBeFinal()`	Checks that the member is final.
`kcallable.shouldBeOpen()`	Checks that the member is open.
`kcallable.shouldBeAbstract()`	Checks that the member is abstract.
`kcallable.shouldBeSuspendable()`	Checks that the member is suspendable.
`kcallable.shouldAcceptParameters(parameters)`	Checks that the member can be called with the given parameters.
`kcallable.shouldHaveParametersWithName(parameters)`	Checks that the member has parameters with the given names.
`ktype.shouldBeOfType<T>()`	Checks that the `KType` is `T`.

Statistic
`collection.shouldHaveMean(mean)`	Checks that the collection has the given mean with default precision 4.
`collection.shouldHaveMean(mean, precision)`	Checks that the collection has the given mean with the given precision.
`collection.shouldHaveVariance(mean)`	Checks that the collection has the given variance with default precision 4.
`collection.shouldHaveVariance(mean, precision)`	Checks that the collection has the given variance with the given precision.
`collection.shouldHaveStandardDeviation(mean)`	Checks that the collection has the given standard deviation with default precision 4.
`collection.shouldHaveStandardDeviation(mean, precision)`	Checks that the collection has the given standard deviation with the given precision.

Regex
`regex.shouldBeRegex(anotherRegex)`	Checks that the regex equals another regex by comparing the pattern and options.
`regex.shouldHavePattern(regexPattern)`	Checks that the regex has the given pattern.
`regex.shouldHaveExactRegexOptions(regexOptions)`	Checks that the regex has exactly the given options.
`regex.shouldIncludeRegexOption(regexOption)`	Checks that the regex includes the given option.
`regex.shouldIncludeRegexOptions(regexOptions)`	Checks that the regex includes the given options.

Selective Matchers
`any.shouldBeEqualToUsingFields(other: T, vararg properties: KProperty<*>)`	Checks equality against `other` using only the given properties.
`any.shouldBeEqualToIgnoringFields(other: T, vararg properties: KProperty<*>)`	Checks equality against `other` while ignoring the given properties.

Field by Field Comparison Matchers
`any.shouldBeEqualToComparingFields(other: T)`	Checks equality against `other` by comparing fields instead of using `equals`, ignoring private fields.
`any.shouldBeEqualToComparingFields(other: T, ignorePrivateFields: Boolean)`	Checks equality against `other` by comparing fields instead of using `equals`, including private fields when configured.
~~`any.shouldBeEqualToComparingFieldsExcept(...)`~~	Deprecated. Use `shouldBeEqualToComparingFields` or `shouldBeEqualToIgnoringFields` instead.

Resource Matchers
`str shouldMatchResource "/path/to/test_resource.txt"`	Checks that the string equals the given resource, ignoring line separator differences.
`str shouldNotMatchResource "/path/to/test_resource.txt"`	Checks that the string does not equal the given resource, ignoring line separator differences.
`str.shouldMatchResource("/path/to/test_resource.txt", ::providedMatcher)`	Uses `providedMatcher` to check that the string matches the given resource.
`str.shouldNotMatchResource("/path/to/test_resource.txt", ::providedMatcher)`	Uses `providedMatcher` to check that the string does not match the given resource.
`byteArray shouldMatchResource "/path/to/test_resource.bin"`	Checks that the byte array equals the given resource.
`byteArray shouldNotMatchResource "/path/to/test_resource.bin"`	Checks that the byte array does not equal the given resource.
`byteArray.shouldMatchResource("/path/to/test_resource.bin", ::providedMatcher)`	Uses `providedMatcher` to check that the byte array matches the given resource.
`byteArray.shouldNotMatchResource("/path/to/test_resource.bin", ::providedMatcher)`	Uses `providedMatcher` to check that the byte array does not match the given resource.

References

Core Matchers | Kotest

Kotest Testing Exceptions

kc@example.com (kc kim) — Mon, 04 Mar 2024 01:11:00 +0900

Testing Exceptions in Kotest

With Kotest, you can easily test exceptions:

package com.devkuma.kotest.tutorial.exceptions

import io.kotest.assertions.throwables.shouldNotThrow
import io.kotest.assertions.throwables.shouldThrow
import io.kotest.core.spec.style.FreeSpec
import io.kotest.matchers.shouldBe

class ExceptionTest : FreeSpec({

    "exception test" - {
        "when an exception is thrown" {
            // Run code that throws an exception and catch that exception.
            val errorProneFunction: () -> Unit = {
                throw IllegalArgumentException("Exception occurred!")
            }
            val exception = shouldThrow<IllegalArgumentException> {
                errorProneFunction()
            }

            // The test passes when the expected exception is thrown.
            exception.message shouldBe "Exception occurred!"
        }

        "when no exception is thrown" {
            // Run code that does not throw an exception and verify that no exception is thrown.
            val safeFunction: () -> Unit = {}
            val exception = shouldNotThrow<IllegalArgumentException> {
                safeFunction()
            }

            // The test passes when no exception is thrown.
            exception shouldBe null
        }
    }
})

References

Testing Exceptions | Kotest

Kotest Current Instant Listeners Extension

kc@example.com (kc kim) — Sat, 13 Apr 2024 23:34:00 +0900

TIP

Since Kotest 5.6.0, current instant listeners are in the `io.kotest:kotest-extensions-now:${kotest-version}` artifact. To use the features mentioned below, add it as a dependency.

Current Instant Listeners

For various reasons, such as setting the creation date of an entity, you may need to use the now static function from the java.time classes.

data class MyEntity(
    creationDate: LocalDateTime = LocalDateTime.now()
)

But what should you do when you want to test that value? Every time now is called, the value changes.

For this, Kotest provides ConstantNowListener and the withConstantNow function.

While the code runs, the value you want to test will always be now.

val foreverNow = LocalDateTime.now()

withConstantNow(foreverNow) {
    LocalDateTime.now() shouldBe foreverNow
    delay(10) // Code is taking a small amount of time to execute, but `now` changed!
    LocalDateTime.now() shouldBe foreverNow
}

Or you can use a listener for all tests:

override fun listeners() = listOf(
    ConstantNowTestListener(foreverNow)
)

CAUTION

`withContantNow` and `ConstantNowTestListener` are very sensitive to race conditions. When you use them, you mock the static `now` method globally for the entire JVM instance, so results may be inconsistent if tests run in parallel.

References

Current Instant Listeners | Kotest

Kotest Statistics

kc@example.com (kc kim) — Sat, 06 Apr 2024 19:00:00 +0900

Statistics

Sometimes you may want to know what kinds of values Kotest generates to check that a generator is configured the way you want. Property test statistics are designed for this need.

The collect function is the starting point for statistics and is used to count categories of values. Use it by calling the function inside a property test with the category you want to increment.

For example, suppose you want to collect statistics for the RoundingMode values used by BigDecimal. Call checkAll as usual and pass the rounding mode to the collect function.

checkAll(Arb.enum<RoundingMode>(), Arb.bigDecimal()) { mode, decimal ->
  collect(mode)
  // test here
}

When the test completes, Kotest prints the test name and the count/percentage for each category:

Statistics: [collecting stats] (1000 iterations, 1 args)

HALF_DOWN                                                     142 (14%)
HALF_UP                                                       141 (14%)
CEILING                                                       132 (13%)
FLOOR                                                         122 (12%)
UP                                                            119 (12%)
UNNECESSARY                                                   119 (12%)
HALF_EVEN                                                     118 (12%)
DOWN                                                          107 (11%)

The categories you use do not have to be enums. They can be any object, and if you want more control, you can wrap them conditionally.

checkAll(Arb.int()) { k ->
  when {
    k % 2 == 0 -> collect("EVEN")
    else -> collect("ODD")
  }
  // test here
}

Labels

Sometimes you may need orthogonal sets of statistics. For example, in a simple numeric test, you may want to check both that a certain percentage is even and that a certain percentage is negative. One way is to use EVEN_POS, EVEN_NEG, ODD_POS, and ODD_NEG.

checkAll(Arb.int()) { k ->
  when {
    k > 0 && k % 2 == 0 -> collect("EVEN_POS")
    k % 2 == 0 -> collect("EVEN_NEG")
    k > 0 -> collect("ODD_POS")
    else -> collect("ODD_NEG")
  }
  // test here
}

This gives one output set:

EVEN_POS                                                       142 (27%)
EVEN_NEG                                                       141 (23%)
ODD_POS                                                        132 (24%)
ODD_NEG                                                        122 (26%)

However, as combinations increase, this becomes hard to manage, so Kotest supports labelled statistics. You can think of these as separate statistics sets. To use labels, simply pass the label name as the first argument to the collect method.

checkAll(Arb.int()) { k ->
  when {
    k % 2 == 0 -> collect("even_odd", "EVEN")
    else -> collect("even_odd", "ODD")
  }
  when {
    k > 0 -> collect("pos_neg", "POS")
    else -> collect("pos_neg", "NEG")
  }
  // test here
}

Kotest now prints several statistics sets with the label name in the title:

Statistics: [collecting labelled stats] (1000 iterations, 1 args) [even_odd]

ODD                                                           520 (52%)
EVEN                                                          480 (48%)


Statistics: [collecting labelled stats] (1000 iterations, 1 args) [pos_neg]

NEG                                                           527 (53%)
POS                                                           473 (47%)

Report Mode

By default, statistics for all property tests are printed. There are four modes that can be configured with the global configuration object PropertyTesting.

The possible options are:

Mode	Function
`PropertyTesting.statisticsReportMode = StatisticsReportMode.OFF`	Disables all statistics reporting.
`PropertyTesting.statisticsReportMode = StatisticsReportMode.ALL`	Enables all statistics reporting.
`PropertyTesting.statisticsReportMode = StatisticsReportMode.SUCCESS`	Outputs statistics only on successful tests.
`PropertyTesting.statisticsReportMode = StatisticsReportMode.FAILED`	Outputs statistics only on failed tests.

Checking Statistics Coverage

To programmatically assert that certain values are being generated, you can specify constraints that must be satisfied.

For example, in the previous rounding example, you can use withCoveragePercentages to verify that at least 10% of inputs cover HALF_DOWN and 10% cover FLOOR.

withCoveragePercentages(mapOf(RoundingMode.HALF_DOWN to 10.0, RoundingMode.FLOOR to 10.0)) {
  checkAll(Arb.enum<RoundingMode>(), Arb.bigDecimal()) { mode, decimal ->
    collect(mode)
    // use the mode / decimal
  }
}

To check absolute counts instead of percentages, use withCoverageCounts.

withCoverageCounts(mapOf(RoundingMode.HALF_DOWN to 75, RoundingMode.FLOOR to 75)) {
  checkAll(Arb.enum<RoundingMode>(), Arb.bigDecimal()) { mode, decimal ->
    collect(mode)
    // use the mode / decimal
  }
}

Custom Reports

You can customize the report format or generate reports from raw data by using your own instance of a statistics reporter. This is configured through the global configuration object PropertyTesting.

For example:

object MyStatisticsReporter : object : StatisticsReporter { ... }
PropertyTesting.statisticsReporter = MyStatisticsReporter

References

Statistics | Kotest

Kotest Collection Matchers

kc@example.com (kc kim) — Thu, 28 Mar 2024 01:03:00 +0900

Also see inspectors, which are useful for testing several elements in a collection.

Collections
`collection.shouldBeEmpty()`	Checks that the collection has zero elements.
`collection.shouldBeUnique()`	Checks that all elements in the collection are distinct by natural equality.
`collection.shouldBeUnique(comparator)`	Checks that all elements are unique using the given `comparator`.
`collection.shouldContain(element)`	Checks that the collection contains the specified element.
`collection.shouldContainAll(e1, e2, ..., en)`	Checks that the collection contains all listed elements, regardless of order.
`collection.shouldContainDuplicates()`	Checks that the collection contains at least one duplicated element.
`collection.shouldContainExactly()`	Checks that the collection contains only the given values in order and no other values.
`collection.shouldContainExactlyInAnyOrder()`	Checks that the collection contains exactly the given values in any order and nothing else.
`collection.shouldContainAllInAnyOrder()`	Checks that the collection contains all given values in any order and no other values.
`collection.shouldContainNoNulls()`	Checks that the collection has no null elements, or is empty.
`collection.shouldContainNull()`	Checks that the collection contains at least one null element.
`collection.shouldContainOnlyNulls()`	Checks that the collection contains only null elements, or is empty.
`collection.shouldContainAllIgnoringFields()`	Checks that all listed elements are in the collection while ignoring one or more fields.
`collection.shouldHaveSingleElement(element)`	Checks that the collection contains exactly one element and that it is the specified element.
`collection.shouldHaveSingleElement { block }`	Checks that the collection contains a single element matching the given predicate.
`collection.shouldHaveSize(length)`	Checks that the collection has exactly the given length.
`collection.shouldBeSingleton()`	Checks that the collection contains exactly one element.
`collection.shouldBeSingleton { block }`	Checks that the collection contains exactly one element, then runs the block with that element.
`collection.shouldHaveLowerBound(element)`	Checks that the given element is less than or equal to every element in the collection. Works only for elements that implement `Comparable`.
`collection.shouldHaveUpperBound(element)`	Checks that the given element is greater than or equal to every element in the collection. Works only for elements that implement `Comparable`.
`collection.shouldBeSmallerThan(col)`	Checks that the collection is smaller than another collection.
`collection.shouldBeLargerThan(col)`	Checks that the collection is larger than another collection.
`collection.shouldBeSameSizeAs(col)`	Checks that the collection has the same size as another collection.
`collection.shouldHaveAtLeastSize(n)`	Checks that the collection size is at least `n`.
`collection.shouldHaveAtMostSize(n)`	Checks that the collection size is at most `n`.
`list.shouldBeSorted()`	Checks that the list is sorted.
`list.shouldBeSortedBy { transform }`	Checks that the list is sorted by the values after applying the transform.
`list.shouldContainInOrder(other)`	Checks that this list contains the given list in order. Other elements may appear before or after it.
`list.shouldExistInOrder({ element }, ...)`	Checks that the list contains elements matching the predicates in order. Other elements may appear around or between matching elements.
`list.shouldHaveElementAt(index, element)`	Checks that this list contains the given element at the given position.
`list.shouldStartWith(lst)`	Checks that this list starts with the elements of the given list in order.
`list.shouldEndWith(lst)`	Checks that this list ends with the elements of the given list in order.
`iterable.shouldMatchEach(assertions)`	Iterates this list and the assertions, and checks that each element passes the associated assertion. Fails if the collection sizes differ.
`iterable.shouldMatchInOrder(assertions)`	Checks that this iterable has a contiguous subsequence matching the assertions in order.
`iterable.shouldMatchInOrderSubset(assertions)`	Checks that this iterable has a subsequence matching the assertions in order, with gaps allowed.
`value.shouldBeOneOf(collection)`	Checks that the specific instance is contained in the collection.
`collection.shouldContainAnyOf(collection)`	Checks that the collection contains at least one element from the given collection.
`value.shouldBeIn(collection)`	Checks that the object is contained in the collection by value, not by reference.

References

Collection Matchers | Kotest

Kotest Data-Driven Testing

kc@example.com (kc kim) — Mon, 04 Mar 2024 01:11:00 +0900

Data-Driven Testing

Data-Driven Testing is a way to run the same test code repeatedly against several data sets, similar to parameterized tests in JUnit 5. Instead of writing several tests for different input data, you can provide multiple inputs to a single test case and verify many cases.

Required module

To use data-driven testing, first add the `kotest-framework-datatest` module to your build. **Add dependency** To use Data-Driven Testing, first add the related dependency. ```kotlin dependencies { testImplementation("io.kotest:kotest-framework-datatest:") } ``` > This module has been supported since version 4.6.0, so check your version.

When writing logic-based tests, one or two specific code paths that work in a particular scenario may be enough. In other cases, tests are more example-based, and testing many parameter combinations can be useful.

In these situations, data-driven testing, also called table-driven testing, is an easy technique for avoiding tedious boilerplate.

Kotest provides first-class support for data-driven testing built into the framework. It automatically creates test case entries based on the input values you provide.

Getting Started with Data-Driven Testing

Let’s look at data-driven testing with a simple example.

Testing Pythagorean Triples

We will write a test for a Pythagorean triple function that returns true when the input values form a valid triangle, where a squared plus b squared equals c squared.

fun isPythagTriple(a: Int, b: Int, c: Int): Boolean = a * a + b * b == c * c

Since each row needs at least one element, and here it needs three, start by defining a data class that holds a single row of values. In this case, it contains two inputs and the expected result.

data class PythagTriple(val a: Int, val b: Int, val c: Int)

Use instances of this data class to generate tests, and pass them to the withData function, which accepts a lambda containing the test logic for each row.

For example:

package com.devkuma.kotest.tutorial.datadriven

import io.kotest.core.spec.style.FunSpec
import io.kotest.datatest.withData
import io.kotest.matchers.shouldBe

class PythagTripleTest : FunSpec({
    context("Pythag triples tests") {
        withData(
            PythagTriple(3, 4, 5),
            PythagTriple(6, 8, 10),
            PythagTriple(8, 15, 17),
            PythagTriple(7, 24, 25)
        ) { (a, b, c) ->
            isPythagTriple(a, b, c) shouldBe true
        }
    }
})

Because a data class is used, the input row can be destructured into member properties. When this runs, four test cases are created, one for each input row.

Kotest automatically creates a test case for each input row, as if you had manually written a separate test case for every row.

The test name is generated from the data class itself, but it can be customized.

If a specific input row has an error, the test fails and Kotest prints the failed value. For example, if you change the previous example to include the row PythagTriple(5, 4, 3), that test is shown as failed.

The error message includes the error and the input row details:

expected:<true> but was:<false>
Expected :true
Actual   :false

In detail, when a=5, b=4, c=3, 5 * 5 + 4 * 4 = 41 and 3 * 3 = 9 are not equal, so false is returned and the test fails.

In the previous example, the withData call was wrapped in a parent test, so an extra context, context("Pythag triples tests"), appears when the test results are shown. The syntax depends on the spec style being used. Here, a FunSpec with a context block was used as the container. In practice, data tests can be nested inside containers as much as needed.

However, this is optional, and data tests can also be defined at the root level.

For example:

class PythagTripleTest : FunSpec({
    withData(
        PythagTriple(3, 4, 5),
        PythagTriple(6, 8, 10),
        //PythagTriple(5, 4, 3),
        PythagTriple(8, 15, 17),
        PythagTriple(7, 24, 25)
    ) { (a, b, c) ->
        isPythagTriple(a, b, c) shouldBe true
    }
})

CAUTION

Data tests can only be defined in root or container scopes. They cannot be defined inside leaf scopes.

Testing Different Values

Another example of testing data of various types is shown below:

package com.devkuma.kotest.tutorial.datadriven

import io.kotest.core.spec.style.FunSpec
import io.kotest.datatest.withData
import io.kotest.matchers.shouldBe

class WithDataTest : FunSpec({

    context("verify string length") {
        withData(
            ValueExpected("Hello", 5), // first data set: string "Hello"
            ValueExpected("World", 5)  // second data set: string "World"
        ) { (str, expected) ->
            str.length shouldBe expected // verify that the string length matches the expected value for each data set
        }
    }

    context("verify set size") {
        withData(
            ValueExpected(setOf(1, 2, 3), 3),   // first data set: integer set {1, 2, 3}
            ValueExpected(setOf(4, 5, 6, 7), 4) // second data set: integer set {4, 5, 6, 7}
        ) { (set, expected) ->
            set.size shouldBe expected // verify that the set size matches the expected value for each data set
        }
    }

    context("verify map size") {
        withData(
            ValueExpected(mapOf("a" to 1, "b" to 2), 2),             // first data set: map {"a" -> 1, "b" -> 2}
            ValueExpected(mapOf("x" to 10, "y" to 20, "z" to 30), 3) // second data set: map {"x" -> 10, "y" -> 20, "z" -> 30}
        ) { (map, expected) ->
            map.size shouldBe expected // verify that the map size matches the expected value for each data set
        }
    }
})

data class ValueExpected<T>(
    val value: T,
    val expected: Int,
)

Using the forAll Function

You can also perform data-driven testing by using the forAll function for data verification.

package com.devkuma.kotest.tutorial.datadriven

import io.kotest.core.spec.style.StringSpec
import io.kotest.data.forAll
import io.kotest.data.row
import io.kotest.matchers.shouldBe

class ForAllTest : StringSpec({

    "verify string length" {
        forAll(
            row("Hello", 5),
            row("World", 5)
        ) { str, expected ->
            str.length shouldBe expected
        }
    }

    "verify set size" {
        forAll(
            row(setOf(1, 2, 3), 3),
            row(setOf(4, 5, 6, 7), 4)
        ) { set, expected ->
            set.size shouldBe expected
        }
    }

    "verify map size" {
        forAll(
            row(mapOf("a" to 1, "b" to 2), 2),
            row(mapOf("x" to 10, "y" to 20, "z" to 30), 3)
        ) { map, expected ->
            map.size shouldBe expected
        }
    }
})

In the example above, the forAll function is used to generate integer data and run the test repeatedly with that data.

Callbacks

To use before and after callbacks in data-driven tests, you can use the standard beforeTest and afterTest. Every test created with data-driven testing behaves the same as a normal test, so all standard callbacks work as if each test had been written directly.

For example:

package com.devkuma.kotest.tutorial.datadriven

import io.kotest.core.spec.style.FunSpec
import io.kotest.datatest.withData

class CallbackTest : FunSpec({

    beforeTest {
        println("beforeTest")
    }

    context("callback test") {
        withData("X", "Y", "Z") { a ->
            println(a)
        }
    }
})

The following is the result of running the example:

beforeTest
beforeTest
X
beforeTest
Y
beforeTest
Z

Data Test Names

By default, the name of each test is simply the toString() value of the input row. This usually works well for JVM data classes, but the input row must be stable.

However, if you are not using a stable data class, are running on a non-JVM target, or simply want to customize names, you can specify how test names are generated.

Stable Names

When Kotest creates tests, it needs stable test names while the test suite runs. Test names are used as the basis for identifiers when reporting test status to Gradle or IntelliJ. If a name is not stable, the ID can change and cause errors where tests do not appear or seem unfinished.

Kotest uses the input class’s toString() only when it determines that the value is stable. Otherwise, it uses the class name.

You can force Kotest to use toString() for test names by annotating the type with @IsStableType. Then toString() is used regardless.

Alternatively, you can fully customize the display name of the test.

Using mapOf

In Kotest, you can specify test names by passing a map to the withData function, where the key is the test name and the value is the input value for that row.

package com.devkuma.kotest.tutorial.datadriven

import io.kotest.core.spec.style.FunSpec
import io.kotest.datatest.withData
import io.kotest.matchers.shouldBe

class PythagTripleMapOfTest : FunSpec({
    
    context("Pythag triples tests") {
        withData(
            mapOf(
                "3, 4, 5" to PythagTriple(3, 4, 5),
                "6, 8, 10" to PythagTriple(6, 8, 10),
                "8, 15, 17" to PythagTriple(8, 15, 17),
                "7, 24, 25" to PythagTriple(7, 24, 25)
            )
        ) { (a, b, c) ->
            a * a + b * b shouldBe c * c
        }
    }
})

Test Name Function

Alternatively, you can pass a function to withData that receives a row as input and returns a test name. Depending on how generous Kotlin type inference feels, you may need to specify a type parameter on the withData function.

package com.devkuma.kotest.tutorial.datadriven

import io.kotest.core.spec.style.FunSpec
import io.kotest.datatest.withData
import io.kotest.matchers.shouldBe

class PythagTripleNameTest : FunSpec({

    context("Pythag triples tests") {
        withData<PythagTriple>(
            nameFn = { "${it.a}, ${it.b}, ${it.c}" },
            PythagTriple(3, 4, 5),
            PythagTriple(6, 8, 10),
            PythagTriple(8, 15, 17),
            PythagTriple(7, 24, 25)
        ) { (a, b, c) ->
            a * a + b * b shouldBe c * c
        }
    }
})

The output of this example is now a little clearer:

WithDataTestName

Another alternative is to implement the WithDataTestName interface. If this interface is provided, toString() is not used. Instead, the interface’s dataTestName() function is called for each row.

package com.devkuma.kotest.tutorial.datadriven

import io.kotest.datatest.WithDataTestName

data class PythagTriple(val a: Int, val b: Int, val c: Int) : WithDataTestName {
    override fun dataTestName() = "wibble $a, $b, $c wobble"
}

The output of this example becomes a little simpler:

Nested Data Tests

Kotest data tests are very flexible and can be nested with unlimited data test configurations. Each additional nesting level creates another nested layer in the test output, producing a Cartesian join of all inputs.

What is a Cartesian join?

This is a term used in relational databases. When joining two or more tables, it means combining every row of each table with every row of the other tables.

For example, the following code snippet has two nested layers.

package com.devkuma.kotest.tutorial.datadriven

import io.kotest.core.spec.style.FunSpec
import io.kotest.datatest.withData

class NestedDataTest : FunSpec({

    context("each service should support all http methods") {
        val services = listOf(
            "http://internal.foo",
            "http://internal.bar",
            "http://public.baz",
        )

        val methods = listOf("GET", "POST", "PUT")

        withData(services) { service ->
            withData(methods) { method ->
                // test service against method
            }
        }
    }
})

The following is the same example, but with custom test names added at the second level:

package com.devkuma.kotest.tutorial.datadriven

import io.kotest.core.spec.style.FunSpec
import io.kotest.datatest.withData

class NestedDataTest2 : FunSpec({

    context("each service should support all http methods") {

        val services = listOf(
            "http://internal.foo",
            "http://internal.bar",
            "http://public.baz",
        )

        val methods = listOf("GET", "POST", "PUT")

        withData(services) { service ->
            withData<String>({ "should support HTTP $it" }, methods) { method ->
                // test service against method
            }
        }
    }
})

Other Data Testing

The previous sections described tests using the withData function. This section explains basic ways to use data in Kotest.

Creating Test Data

Creating test data plays an important role in ensuring test reliability and completeness. Kotest lets you create test data in various ways. For example, you can define data directly inside a test function, or define data as member variables inside a test case class.

package com.devkuma.kotest.tutorial.datadriven

import io.kotest.core.spec.style.StringSpec
import io.kotest.matchers.shouldBe

class DataTest : StringSpec({

    val testData = listOf(1, 2, 3, 4, 5)

    "test using predefined test data" {
        testData.size shouldBe 5
    }

    "test using data generated from a range" {
        val generatedData = (1..10).toList()
        generatedData.size shouldBe 10
    }
})

In the example above, the member variable testData defines test data. It also uses generatedData to create data from a range.

Setting Up and Cleaning Up Test Data

Setting up the data needed before a test runs and cleaning it up after the test completes is important for test stability. In Kotest, you can use beforeTest and afterTest blocks to set up and clean up test data.

package com.devkuma.kotest.tutorial.datadriven

import io.kotest.core.spec.style.StringSpec

class DataSetupAndCleanupTest : StringSpec({

    val database = setupDatabase()

    beforeTest {
        database.connect()
    }

    afterTest {
        database.disconnect()
    }

    "Test with database connection" {
        // test logic using the database connection
    }
})

fun setupDatabase(): Database {
    // Setup database and return
}

In the example above, the beforeTest block connects to the database, and the afterTest block disconnects from it.

Creating, managing, and cleaning up test data are important elements for securing test reliability. By managing test data effectively with Kotest, you can run stable and robust tests and improve software quality.

References

Kotest Koin Extension

kc@example.com (kc kim) — Sat, 13 Apr 2024 10:33:00 +0900

The Koin DI Framework can be used with Kotest through the KoinExtension extension.

To use the extension in your project, add the dependency to your project:

io.kotest.extensions:kotest-extensions-koin:${version}

Once the dependency is added, you can use Koin in tests easily.

class KotestAndKoin : FunSpec(), KoinTest {

    override fun extensions() = listOf(KoinExtension(myKoinModule))

    val userService by inject<UserService>()

    init {
        test("use userService") {
            userService.getUser().username shouldBe "LeoColman"
        }
    }
}

By default, the extension starts and stops the Koin context between leaf tests. If you use a nested spec style such as DescribeSpec and want to keep the Koin context across all leaves of a root test, for example to share mock declarations between tests, you can specify the lifecycle mode as KoinLifecycleMode.Root in the KoinExtension constructor.

class KotestAndKoin : DescribeSpec(), KoinTest {

    override fun extensions() = listOf(KoinExtension(module = myKoinModule, mode = KoinLifecycleMode.Root))

    val userService by inject<UserService>()

    init {
        describe("use userService") {
            it("inside a leaf test") {
                userService.getUser().username shouldBe "LeoColman"
            }
            it("this shares the same context") {
                userService.getUser().username shouldBe "LeoColman"
            }
        }
    }
}

References

Koin | Kotest

Kotest JSON Matchers

kc@example.com (kc kim) — Wed, 10 Apr 2024 18:26:00 +0900

Overview

To use JSON matchers, add testImplementation("io.kotest.extensions:kotest-assertions-json:<version>") to your build.

Basic Matchers

Matcher	Description	Target
`shouldBeValidJson`	Checks that the given string parses as valid JSON.	Multiplatform
`shouldBeJsonObject`	Checks that the string is a valid JSON object.	Multiplatform
`shouldBeJsonArray`	Checks that the string is a valid JSON array.	Multiplatform

Content-Based Matching

See JSON content matching for more details.

Matcher	Description	Target
`shouldEqualJson`	Checks that a string matches the specified JSON structure.	Multiplatform
`shouldEqualSpecifiedJson`	Checks that a string matches the specified JSON structure while allowing unspecified extra properties.	Multiplatform
`shouldContainJsonKey`	Checks that a string is JSON and contains the specified JSON path.	JVM
`shouldContainJsonKeyValue`	Checks that a string is JSON and contains the specified JSON path with the specified value.	JVM
`shouldMatchJsonResource`	Checks that a string matches the JSON content of the specified test resource.	JVM

Schema Validation

Matcher	Description	Target
`shouldMatchSchema`	Checks that a `String` or `kotlinx.serialization.JsonElement` matches a `JsonSchema`. See below for schema configuration.	Multiplatform

JSON Content Matching

This module is available for JVM and JS targets.

shouldEqualJson

json.shouldEqualJson(other) checks that the JSON structure on the left is the same as the one on the right.

This matcher allows different formatting and different key order.

For example, the following two JSON strings are considered equal:

{
   "name": "sam",
   "location": "chicago",
   "age" : 41
}

and

{ "age" : 41, "name": "sam", "location": "chicago" }

The opposite of this matcher is shouldNotEqualJson, which fails if the two JSON strings are considered equal.

compareJsonOptions

shouldEqualJson supports an additional parameter of type CompareJsonOptions, which provides flags for toggling JSON comparison behavior.

Usage:
Options can be specified inline:

a.shouldEqualJson(b, compareJsonOptions { arrayOrder = ArrayOrder.Strict })

Another option is to define the comparison behavior as desired:

val myOptions = compareJsonOptions {
   typeCoercion = TypeCoercion.Enabled
   arrayOrder = ArrayOrder.Lenient
}

infix fun String.lenientShouldEqualJson(other: String) = this.shouldEqualJson(other, myOptions)

"[1, 2]" lenientShouldEqualJson "[2, 1]" // This will pass

Parameters

Name	Purpose	Possible values	Default
`PropertyOrder`	Determines whether JSON object property order is considered during comparison.	`PropertyOrder.Strict`, `PropertyOrder.Lenient`	`PropertyOrder.Lenient`; property order does not matter.
`ArrayOrder`	Determines whether JSON array element order is considered during comparison.	`ArrayOrder.Strict`, `ArrayOrder.Lenient`	`ArrayOrder.Strict`; element order matters.
`FieldComparison`	Determines whether comparison fails when `actualJSON` contains additional properties compared with `expected`.	`FieldComparison.Strict`, `FieldComparison.Lenient`	`FieldComparison.Strict`; additional properties make the values unequal.
`NumberFormat`	Determines whether number comparison is strict about number format, for example whether `100.0` and `100` are considered equal.	`NumberFormat.Strict`, `NumberFormat.Lenient`	`NumberFormat.Lenient`; number format does not matter.
`TypeCoercion`	Determines whether types are coerced, for example when a string contains a number or boolean value.	`TypeCoercion.Enabled`, `TypeCoercion.Disabled`	`TypeCoercion.Disabled`; types are not coerced.

Target: Multiplatform

shouldEqualSpecifiedJson

This is an alias for shouldEqualJson, except the default option for FieldComparison is set to FieldComparison.Lenient.

val a = """ { "a": true, "date": "2019-11-03" } """
val b = """ { "a": true } """

// this would pass
a shouldEqualSpecifiedJson b

// this would fail
a shouldEqualJson b

Target: Multiplatform

shouldContainJsonKey

json.shouldContainJsonKey("$.json.path") checks that a JSON string contains the specified JSON path.

The opposite matcher is shouldNotContainJsonKey, which fails if the JSON string contains the specified JSON path.

Target: JVM

shouldContainJsonKeyValue

str.shouldContainJsonKeyValue("$.json.path", value) checks that a JSON string contains a JSON path with the specified value.

The opposite matcher is shouldNotContainJsonKeyValue, which fails if the JSON path contains the specified value.

Target: JVM

shouldMatchJsonResource

json.shouldMatchJsonResource("/file.json") checks that the JSON is the same as the existing test resource /file.json, ignoring property order and formatting.

Target: JVM

JSON Schema Matchers

Matcher	Description	Target
`shouldMatchSchema`	Checks that a `String` or `kotlinx.serialization.JsonElement` matches a `JsonSchema`. See below for schema configuration.	Multiplatform

A subset of JSON Schema can be defined by parsing a text schema.

Example:

val parsedSchema = parseSchema(
  """
  {
  "$id": "https://example.com/geographical-location.schema.json",  // will  be ignored
  "$schema": "https://json-schema.org/draft/2020-12/schema",       // will be ignored
  "title": "Longitude and Latitude Values",                        // will be ignored
  "description": "A geographical coordinate.",                     // will be ignored
  "required": [ "latitude", "longitude" ],
  "type": "object",
  "properties": {
    "latitude": {
      "type": "number",
      "minimum": -90,
      "maximum": 90
    },
    "longitude": {
      "type": "number",
      "minimum": -180,
      "maximum": 180
    }
  }
}
  """
)

Or use Kotest’s built-in DSL:

val addressSchema = jsonSchema {
  obj {   // object is reserved, obj was chosen over jsonObject for brevity but could be changed ofc, or jsonObject could be added as alternative.
    withProperty("street", required = true) { string() }
    withProperty("zipCode", required = true) {
      integer {
        beEven() and beInRange(10000..99999)   // supports constructing a matcher that will be used to test values
      }
    }
    additionalProperties = false   // triggers failure if other properties are defined in actual
  }
}

val personSchema = jsonSchema {
  obj {
    withProperty("name", required = true) { string() }
    withProperty("address") { addressSchema() } // Schemas can re-use other schemas 🎉
  }
}

Building Schemas

Arrays

Arrays are used for ordered elements. In JSON, each element in an array can have a different type.

Length (minItems and maxItems)

Array length can be specified with the minItems and maxItems keywords. Each keyword value must be a non-negative number; the defaults are 0 and Int.MAX_VALUE.

val lengthBoundedSchema = jsonSchema {
  array(minItems = 0, maxItems = 1) { number() }
}

Uniqueness

A schema can check that every item in an array is unique. Just set the uniqueItems keyword to true.

val uniqueArray = jsonSchema {
  array(uniqueItems = true) { number() }
}

⚠️ Kotest currently supports only a subset of JSON Schema. The following items are not currently supported:

$defs and $refs
Recursive schemas
Parsing of schema composition
string.format
array.prefixItems,
array.contains,
array.items = false
array.maxContains
array.minContains
array.uniqueItems
enum

Validation

Once a schema is defined, it can be used to validate a String or kotlinx.serialization.JsonElement:

"{}" shouldMatchSchema personSchema

// fails with:
// $.name => Expected string, but was undefined

""" { "name": "Emil", "age": 34 } """
// Passes, since address isn't required and `additionalProperties` are allowed

devkuma – Kotest

Kotest Framework

Kotest Extensions

Kotest Assertions

Kotest Property Testing

Kotest IntelliJ Plugin

Kotest Property-based Testing

Property-based Testing

What Is Property Testing?

Getting Started

JVM/Gradle

JVM/Maven

Multiplatform

Next Steps

References

Kotest Test Assertions

Various Matchers

Clues

Inspectors

Custom Matchers

References

Kotest External Extensions

Kotest Team Extensions

Third Party Extensions

References

Kotest IntelliJ Plugin Introduction

Gutter Icons

Running Tests

Duplicate Test Highlighting

Context Menu Run/Debug

Intentions

References

Kotest Overview

What Is Kotest?

Kotest Benefits and Features

Kotest Benefits

Kotest Features

Introduction to Test-Driven Development

Kotest Supports TDD

Core Principles of TDD

Simple Kotest Examples

Test with Style

Checking Multiple Test Cases with Data-Driven Testing

Fine-Tuning Test Execution

References

Kotest Spring Extension(SpringExtension)

Spring Extension

Adding the Dependency

Configuration

Constructor Injection

Test Context

Test Method Callbacks

Final Classes

References

Kotest IntelliJ Plugin Test Explorer

References

Kotest Property Test Functions

For All

Check All

Iterations

Specifying Generators

References

Kotest Project Setup

Adding Dependencies

JVM/Gradle

JVM/Maven

Kotlin/JS

Kotlin/Native

Android

Kotest Version

Creating a Kotest Environment in IntelliJ IDEA

References

Kotest Ktor Extension

References

Kotest IntelliJ Plugin Properties

Common Use Case

Naming the Properties File

References

Kotest Generators

Arbitrary