Skip to content

Listeners

It is a common requirement to execute code before or after tests or specs. For instance, to start (or reset) and shutdown an embedded database required by a test.

Kotest provides multiple callbacks into the test lifecycle which are used for this purpose.

TestListener

The main interface is TestListener. The following sections describe the callbacks available on this interface.

Callback Description
beforeContainer Invoked directly before each test with type TestType.Container is executed, with the TestCase instance as a parameter. If the test is marked as ignored / disabled / inactive, then this callback won't be invoked.
afterContainer Invoked immediately after a TestCase with type TestType.Container has finished, with the TestResult of that test. If a test case was skipped (ignored / disabled / inactive) then this callback will not be invoked for that particular test case.

The callback will execute even if the test fails.
beforeEach Invoked directly before each test with type TestType.Test is executed, with the TestCase instance as a parameter. If the test is marked as ignored / disabled / inactive, then this callback won't be invoked.
afterEach Invoked immediately after a TestCase with type TestType.Test has finished, with the TestResult of that test. If a test case was skipped (ignored / disabled / inactive) then this callback will not be invoked for that particular test case.

The callback will execute even if the test fails.
beforeAny Invoked directly before each test with any TestType is executed, with the TestCase instance as a parameter. If the test is marked as ignored / disabled / inactive, then this callback won't be invoked.
afterAny Invoked immediately after a TestCase with any TestType has finished, with the TestResult of that test. If a test case was skipped (ignored / disabled / inactive) then this callback will not be invoked for that particular test case.

The callback will execute even if the test fails.
beforeTest Invoked directly before each test is executed with the TestCase instance as a parameter. If the test is marked as ignored / disabled / inactive, then this callback won't be invoked.

This callback has the same behavior as beforeAny.
afterTest Invoked immediately after a TestCase has finished with the TestResult of that test. If a test case was skipped (ignored / disabled / inactive) then this callback will not be invoked for that particular test case.

The callback will execute even if the test fails.

This callback has the same behavior as afterAny.
beforeSpec Invoked after the Engine instantiates a spec to be used as part of a test execution.

The callback is provided with the Spec instance that the test will be executed under.

If a spec is instantiated multiple times - for example, if InstancePerTest or InstancePerLeaf isolation modes are used, then this callback will be invoked for each instance created, just before the first test (or only test) is executed for that spec.

This callback should be used if you need to perform setup each time a new spec instance is created.

If you simply need to perform setup once per class file, then use prepareSpec. This callback runs before any beforeTest functions are invoked.

When running in the default SingleInstance isolation mode, then this callback and prepareSpec are functionally the same since all tests will run in the same spec instance.
afterSpec Is invoked after the TestCases that are part of a particular spec instance have completed.

If a spec is instantiated multiple times - for example, if InstancePerTest or InstancePerLeaf isolation modes are used, then this callback will be invoked for each instantiated spec, after the tests that are applicable to that spec instance have returned.

This callback should be used if you need to perform cleanup after each individual spec instance. If you need to perform cleanup once per class file, then use finalizeSpec.

This callback runs after any afterTest callbacks have been invoked.

When running in the default SingleInstance isolation mode, then this callback and finalizeSpec are functionally the same since all tests will run in the same spec instance.
prepareSpec Called once per spec, when the engine is preparing to execute the tests for that spec. The KClass instance of the spec is provided as a parameter.

Regardless of how many times the spec is instantiated, for example, if InstancePerTest or InstancePerLeaf isolation modes are used, this callback will only be invoked once. If there are no active tests in a spec, then this callback will still be invoked.

When running in the default SingleInstance isolation mode, then this callback and beforeSpec are functionally the same since all tests will run in the same spec instance.
finalizeSpec Called once per Spec, after all tests have completed for that spec.

Regardless of how many times the spec is instantiated, for example, if InstancePerTest or InstancePerLeaf isolation modes are used, this callback will only be invoked once.

The results parameter contains every TestCase, along with the result of that test, including tests that were ignored (which will have a TestResult that has TestStatus.Ignored).

When running in the default SingleInstance isolation mode, then this callback and afterSpec are functionally the same since all tests will run in the same spec instance.
beforeInvocation Invoked before each 'run' of a test, with a flag indicating the iteration number. This callback is useful if you have set a test to have multiple invocations via config and want to do some setup / teardown between runs.

If you are running a test with the default single invocation then this callback is effectively the same as beforeTest.

Note: If you have set multiple invocations _and multiple threads, then these callbacks will be invoked concurrently._
afterInvocation Invoked after each 'run' of a test, with a flag indicating the iteration number. This callback is useful if you have set a test to have multiple invocations via config and want to do some setup / teardown between runs.

If you are running a test with the default single invocation then this callback is effectively the same as afterTest.

Note: If you have set multiple invocations _and multiple threads, then these callbacks will be invoked concurrently._

ProjectListener

The other listener interface is ProjectListener which defines two callbacks.

Callback Description
beforeProject This callback is invoked once before any spec discovery and execution takes place.
afterProject This callback is invoked once after all specs have completed. This callback is executed even if there are errors in specs / test cases.

How to use a Listener

There are several ways to use the methods in a listener.

DSL Methods

The first and simplest, is to use the DSL methods available inside a Spec which create and register a TestListener for you. For example, we can invoke beforeTest or afterTest (and others) directly alongside our tests.

class TestSpec : WordSpec({
   beforeTest {
     println("Starting a test $it")
   }
   afterTest { (test, result) ->
     println("Finished spec with result $result")
   }
   "this test" should {
      "be alive" {
        println("Johnny5 is alive!")
      }
   }
})

Behind the scenes, these DSL methods will create an instance of TestListener, overriding the appropriate functions, and ensuring that this test listener is registered to run.

You can use afterProject as a DSL method which will create an instance of ProjectListener, but there is no beforeProject because by the time the framework is at this stage of detecting a spec, the project has already started!

DSL methods with functions

Since these DSL methods accept functions, we can pull out logic to a function and re-use it in several places. The BeforeTest type used on the function definition is an alias to suspend (TestCase) -> Unit to keep things simple. There are aliases for the types of each of the callbacks.

val startTest: BeforeTest = {
   println("Starting a test $it")
}

class TestSpec : WordSpec({

   // used once
   beforeTest(startTest)

   "this test" should {
      "be alive" {
         println("Johnny5 is alive!")
      }
   }
})

class OtherSpec : WordSpec({

   // used twice
   beforeTest(startTest)

   "this test" should {
      "fail" {
         fail("boom")
      }
   }
})

Overriding callback functions in a Spec

The second, related, method is to override the callback functions in the Spec. This is essentially just a variation on the first method.

class TestSpec : WordSpec() {
    override fun beforeTest(testCase: TestCase) {
        println("Starting a test $testCase")
    }

    init {
        "this test" should {
            "be alive" {
                println("Johnny5 is alive!")
            }
        }
    }
}

Standalone Listener instances

The next method is to create a standalone implementation of the TestListener or ProjectListener interface and register it.

This is useful for TestListeners if you want to reuse a listener that has several dependant functions (such as starting and stopping a resource).

class MyTestListener : TestListener {
   override suspend fun beforeSpec(spec:Spec) {
      // power up kafka
   }
   override suspend fun afterSpec(spec: Spec) {
      // shutdown kafka
   }
}


class TestSpec : WordSpec({
    listener(MyTestListener())
    // tests here
})

Any listeners registered directly inside a Spec will be used for all tests in that spec (including test factories and nested tests).

Maybe you want a TestListener to run for every spec in the entire project. To do that, you can either mark the listener with @AutoScan, or you can register the listener via project config. For more information on this see ProjectConfig.

Instances of ProjectListener must be registered using @AutoScan or using project config level, since the beforeProject callback needs to be registered and executed any spec discovery begins.

An example of @Autoscan on a project listener:

@AutoScan
object MyProjectListener : ProjectListener {
  override suspend fun beforeProject() {
    println("Project starting")
  }
  override suspend fun afterProject() {
    println("Project complete")
  }
}

Real Examples

System Out Listener

A real example of a listener in example, is our NoSystemOutListener which throws an error if any output is written to standard out.

class MyTestSpec : DescribeSpec({

    listener(NoSystemOutListener)

    describe("All these tests should not write to standard out") {
        it("silence in the court") {
          println("boom") // failure
        }
    }
})

Timer Listener

Another example would be if we wanted to log the time taken for each test case. We can do this by using the beforeTest and afterTest functions as follows:

object TimerListener : TestListener {

  var started = 0L

  override fun beforeTest(testCase: TestCase): Unit {
    started = System.currentTimeMillis()
  }

  override fun afterTest(testCase: TestCase, result: TestResult): Unit {
    println("Duration of ${testCase.description} = " + (System.currentTimeMillis() - started))
  }
}

Then we can register like so:

class MyTestClass : FunSpec({
  listeners(TimerListener)
  // tests here
})

Or we could register it project wide:

object MyConfig : AbstractProjectConfig() {
    override fun listeners(): List<Listener> = listOf(TimerListener)
}

These functions will now be invoked for every test case inside the MyTestClass test class. Maybe you want this listener to run for every test in the entire project. To do that, you would register the listener with the project config singleton. For more information on this see ProjectConfig.