Grouping Kotest Tests with Tags

Creating and Defining Tags

Sometimes you do not want to run every test. In that case, Kotest provides tags that let you decide at runtime which tests should run.

Creating Tags by Extending Tag

A tag is an object that extends the io.kotest.core.Tag class.

For example, to group tests by operating system, you can define the following tags:

import io.kotest.core.Tag

class Windows : Tag()
class Linux : Tag()

Defining Tags with the NamedTag Class

Alternatively, you can define tags by using the NamedTag class.

For example:

val linux = NamedTag("Linux")

When using the NamedTag class, follow these rules:

• A tag must not be null or blank.
• A tag must not contain whitespace.
• A tag must not contain ISO control characters.
• A tag must not contain the following characters:
  • !: exclamation mark
  • (: left parenthesis
  • ): right parenthesis
  • &: ampersand
  • |: pipe

Marking Tests

You can mark test cases with generated tags by using the config function:

import io.kotest.core.spec.style.StringSpec

class TagConfigTest : StringSpec() {
    init {
        "should run on Windows".config(tags = setOf(Windows)) {
            // ...
        }

        "should run on Linux".config(tags = setOf(Linux)) {
            // ...
        }

        "should run on Windows and Linux".config(tags = setOf(Windows, Linux)) {
            // ...
        }
    }
}

Running with Tags

You can also control which tests run by invoking the test runner with the system property kotest.tags. The expression to pass is a simple boolean expression using boolean operators, &, |, and !. You can also group expressions with parentheses.

For example:

Tag1 & (Tag2 | Tag3)

When running tests, enter the simple name of the tag object, excluding the package. Be careful with uppercase and lowercase letters. If two tag objects have the same simple name in different namespaces, they are treated as the same tag.

For example, to run only tests tagged with Linux and not run tests tagged with Database, invoke Gradle as follows:

gradle test -Dkotest.tags="Linux & !Database"

By passing the system property kotest.tags, only tests that are Linux and not Database are executed.

Tags can also be included or excluded at runtime, for example when running project configuration instead of properties, through RuntimeTagExpressionExtension:

RuntimeTagExpressionExtension.expression = "Linux & !Database"

Tag Expression Operators

Operators in descending precedence:

Tagging All Tests

You can also add tags to all tests in a spec by using the tags function on the spec itself.

For example, you can specify tags with the tags function as follows.

import io.kotest.core.spec.style.FunSpec

class TagFuncTest : FunSpec({

    tags(Linux, MySql)

    test("my test") { } // automatically marked with the above tags
})

Tagging Specs

There are two annotations that can be added to a spec class: @Tags and @RequiresTag. Both can take one or more tag names as arguments.

@Tags

The first tag annotation, @Tags, is applied to all tests in the class. If a specific tag is explicitly excluded and the tests are not run, the spec is not instantiated.

The basic way to use Tags is as follows:

For example, you may want to avoid running some slow tests.

import io.kotest.core.spec.style.FreeSpec
import io.kotest.core.annotation.Tag

@Tags("fast")
class MyFastTests : FreeSpec({
    "test 1" {
        // test content
    }
})

@Tags("slow")
class MySlowTests : FreeSpec({
    "test 2" {
        // test content
    }
})

If you set kotest.tags=fast, only tests tagged as fast run.

You can use multiple tags together to run tests that match specific conditions.

@Tags("fast", "smoke")
class MyFastSmokeTests : FreeSpec({
    "test 3" {
        // test content
    }
})

The code above represents a test tagged with both fast and smoke. In this case, you can use the kotest.tags system to run tests while considering one or both tags.

The following example shows how commands in the system property kotest.tags behave when tags are specified through @Tags, config, and the tags function:

@Tags("Linux")
class MyTestClass : FunSpec({

  tags(UnitTest)

  beforeSpec { println("Before") }

  test("A").config(tags = setOf(Mysql)) {}
  test("B").config(tags = setOf(Postgres)) {}
  test("C") {}
})

@RequiresTag

The second tag annotation, @RequiresTag, only checks whether all referenced tags exist, and skips the spec if they are missing.

For example, the following spec is skipped and not instantiated unless the Linux and Mysql tags are specified at runtime.

@RequiresTag("Linux", "Mysql")
class MyTestClass : FunSpec()

Tag Inheritance

By default, the @Tags annotation is considered only on the immediately preceding Spec where the annotation is applied. However, specs can also inherit tags from superclasses and superinterfaces. To enable this, set tagInheritance = true in the project configuration.

Gradle System Property Passing Configuration

Gradle configuration requires special attention.

To use system properties, such as -Dx=y, you need to configure Gradle so they are propagated to test execution, and you need to add test configuration:

Groovy:

test {
    //... Other configurations ...
    systemProperties = System.properties
}

Kotlin Gradle DSL:

val test by tasks.getting(Test::class) {
    // ... Other configurations ...
    systemProperties = System.getProperties().asIterable().associate { it.key.toString() to it.value }
}

Or you can configure it to accept only -Dkotest.tags=<tags> and pass that through.

val test by tasks.getting(Test::class) {
    // ... Other configurations ...
    systemProperties.putAll(System.getProperties().filter { it.key == "kotest.tags" }.map { it.key.toString() to it.value.toString() }.toMap())
}

This allows the JVM to read the system property correctly.

Tags are useful for selectively running or excluding tests according to specific environments or conditions, and they help manage and organize tests more efficiently.

References

• Grouping Tests with Tags | Kotest