Kotest Generators

Kotest generators are tools used to support property-based testing.
Tags:

Generated values are provided by instances of the sealed class Gen. You can think of Gen as being like an input stream, but for property test values. Each Gen provides a usually infinite stream of these values for a specific type.

Kotest has two types of generators: Arb, which generates arbitrary values, and Exhaustive, which generates a finite set of values from a closed space.

You can mix both types of generators in a property test. For example, you can test a function with every even number from 0 to 200, which is exhaustive, along with 100 random positive integers, which comes from an arb.

Some generators are available only on the JVM. See the full list here.

Arbitrary

Arbs generate two types of values: a hard-coded set of edge cases and an infinite stream of randomly selected samples.

Samples may repeat, and some values may never be generated. For example, if you generate 1000 integers between 0 and Int.MAX, it cannot return every possible value, and some values may be generated more than once. Likewise, if you generate 1000 random integers between 0 and 500, some values may appear more than once.

Common arbitrary values include numbers with or without ranges, strings from a Unicode set, arbitrary lists, data classes with arbitrary parameters, emails, code points, and characters.

In addition to random values, arbs can provide edge cases. One design feature of Kotest property testing is that some value types always include “common” edge cases that you probably want included in tests.

For example, when testing a function that receives an integer, you will usually want to test at least zero, positive numbers, and negative numbers. If only random values were provided, the probability of zero appearing would be fairly low, so Kotest always provides several edge cases for integers unless configured otherwise.

When running a test, the framework randomly alternates between samples and edge cases. The split is determined by a configuration value, whose default is 2% edge cases.

Not every arb has edge cases, but the most common arb types do. Here are some examples of edge cases used by certain arbs:

  • ints: 0, 1, -1, Int.MAX_VALUE, Int.MIN_VALUE
  • double: 0, 1, -1, Double.MAX_VALUE, Double.MIN_VALUE, Double.POSITIVE_INFINITY, Double.NEGATIVE_INFINITY, Double.NaN
  • strings: empty string, minimum-length string, minimum code point
  • lists: empty list, single-element list, list with duplicate elements
  • maps: empty map
  • nullable values: null

Exhaustive

Exhaustive generates every value in a specified space. It is useful when you want to make sure every value in that space is used. For example, with enum values, it is generally more useful to make sure each enum value is used than to randomly select from the enum values and risk missing some values while duplicating others.

Common exhaustives include small collections, enums, boolean values, powersets of lists or sets, predefined integer ranges, and predefined string ranges.

When an exhaustive has provided every value, it starts over, so an exhaustive can be used in tests that need input.

For example:

enum class Season { Winter, Fall, Spring, Summer }

forAll<Int, Season>(100) { a, season -> ... }

Since 100 iterations are requested here, each season value will be provided 25 times.

References