Testing Kotlin Lambda Invocations without Mocking

In this post I want to share a simple technique for verifying the invocation of a Kotlin lambda function without mocking it.

Let's imagine we have a class for loading data:

class DataLoader {
    ....

    fun load(query: String, fetcher: (String) -> Data): Data {
        return findCachedData(query) ?: fetcher(query).also { fetchedData ->
            cacheData(query, fetchedData)
        }
    }
}


data class Data(val result: String)

fun findCachedData(query: String): Data? {....}
fun cacheData(query: String, data: Data) {....}

When consumer calls the load function, Data from the cache will be returned immediately if cached entry can be found for the given query, otherwise the fetcher function provided by the user will be invoked with its result being cached.

val data1 = dataLoader.load("a") { query ->
    // this should be invoked the first time
    api.fetchDataByQuery(query)
}

val data2 = dataLoader.load("a") { query ->
    // this should NOT be invoked the second time as cached Data is available
    api.fetchDataByQuery(query)
}

Since the execution of the fetcher might be expensive, we want to make sure it's only invoked when no Data is available in the cache for the given query.

How do we test this behavior?

Mocks

Mocking is a common way to help verify that the code under test has (or has not) interacted with a dependency.

We can mock the (String) -> Data lambda expression as if it's an interface, and verify whether the invoke(...) function has been invoked.

Here's how our test might look like with mockito-kotlin:

private val dataLoader = DataLoader(...)

private val testData = Data("result")

@Test
fun `fetcher is not executed when data associated with the query exists in cache`() {
    val mockFetcher = mock<(String) -> Data> {
        on { invoke(any()) }.doReturn(testData)
    }

    dataLoader.load("query", mockFetcher)

    // fetcher should be invoked the first time
    verify(mockFetcher, times(1)).invoke(any())

    clearInvocations(mockFetcher)

    dataLoader.load("query", mockFetcher)

    // fetcher should NOT be invoked for the same query the second time
    verify(mockFetcher, times(0)).invoke(any())
}

Here's what we've done to test the behavior mentioned earlier:

Create a mock of the type (String) -> Data.
Stub the invoke(query: String): Data function and return a testData (the actual value is not important in this test).
Call dataLoader.load("query", mockFetcher) for the first time.
Verify that the invoke(query: String): Data function on the mockFetcher was invoked exactly 1 time, as no existing data associated with query exists in the cache yet.
Clear any previous recorded invocations on the mockFetcher to avoid stubbing again.
Call dataLoader.load("query", mockFetcher) for the second time.
Verify that the invoke(query: String): Data function on the mockFetcher was NOT invoked this time as cached Data for the query exists.

The same test looks similar with MockK:

private val dataLoader = DataLoader(...)

private val testData = Data("result")

@Test
fun `fetcher is not executed when data associated with the query exists in cache`() {
    val mockFetcher = mockk<(String) -> Data>()
    every { mockFetcher.invoke(any()) } returns testData

    dataLoader.load("query", mockFetcher)

    // fetcher should be invoked the first time
    verify(exactly = 1) { mockFetcher(any()) }

    clearMocks(mockFetcher)

    dataLoader.load("query", mockFetcher)

    // fetcher should NOT be invoked for the same query the second time
    verify(exactly = 1) { mockFetcher(any()) }
}

Fakes

The approach above seems reasonable, but having to use mocks in unit tests is usually a code smell.

The necessity to use mocks in unit tests is often caused by tight coupling between the unit under test and its dependencies. For example, when a class directly depends on a type from a third-party library or something that does IO operation, unit testing the class without mocking means the tests will likely be indeterministic and slow - the opposite of what we want with unit tests.

However, mocking comes with costs:

Mocking adds noise and cognitive load, especially when there are multiple / nested mocks that need to be stubbed in subtle ways to setup the specific test scenarios.
Mocking can make refactoring harder as tests with mocks verify both the external behavior and the internal workings of the class under test, coupling test code and production code.
Relying on a mocking framework for testing discourages developers from following good design principles such as dependency inversion.

As such we generally want to avoid using mocks in unit tests and only use them sparingly in higher-level integration tests when necessary.

Now if we look at the fetcher lambda again, there's nothing complex or indeterministic about its implementation that requires mocking. In fact, the implementation is always provided by the call-side as a trailing lambda:

dataLoader.load("a") {
    // perform any side-effects...
    ....
    // return data
    Data("result")
}

Mocking the fetcher here provides no benefit aside from being able to verify the invocation using the mocking framework.

We can easily achieve the same with a Fake implementation of the lambda:

private val dataLoader = DataLoader(...)

private val testData = Data("result")

@Test
fun `fetcher is not executed when data associated with the query exists in cache`() {
    var invokeCount = 0
    val testFetcher = { _: String ->
        invokeCount++
        testData
    }

    dataLoader.load("query", testFetcher)

    assertThat(invokeCount).isEqualTo(1)

    dataLoader.load("query", testFetcher)

    // invokeCount should NOT increment
    assertThat(invokeCount).isEqualTo(1)
}

Instead of mocking the fetcher: (String) -> Data lambda, we track the number of invocations with a invokeCount variable that gets incremented whenever the fetcher lambda is invoked. Now we're able to verify the lambda invocations with regular assertions.

The testFetcher implementation records the number of invocations on the lambda and returns a Data. This looks like something we want to reuse in multiple tests.

We need a TestFetcher class with a var for tracking the invocation count, but since the fetcher parameter is a Kotlin function type rather than a regular interface with a fun fetch(query: String): Data, what super type should this class implement?

It turns out that a class can implement a function type just like a regular interface:

class TestFetcher(val expectedData: Data) : (String) -> Data {
    var invokeCount = 0

    override fun invoke(query: String): Data {
        invokeCount++
        return expectedData
    }
}

With this fake fetcher implementation in place, our test has now become:

private val dataLoader = DataLoader(...)

private val testData = Data("result")

@Test
fun `fetcher is not executed when data associated with the query exists in cache`() {
    val testFetcher = TestFetcher(expectedData = testData)

    dataLoader.load("query", testFetcher)

    assertThat(testFetcher.invokeCount).isEqualTo(1)

    dataLoader.load("query", testFetcher)

    // invokeCount should NOT increment
    assertThat(testFetcher.invokeCount).isEqualTo(1)
}

We can also implement a different Fetcher that throws an exception:

class FailedTestFetcher(val expectedException: Exception) : (String) -> Data {
    var invokeCount = 0

    override fun invoke(query: String): Data {
        invokeCount++
        throw expectedException
    }
}

Now we can test that an exception thrown by the fetcher is propagated to the call-side:

private val dataLoader = DataLoader(...)

@Test
fun `fetcher exception is propagated`() {
    val testFetcher = FailedTestFetcher(expectedException = IOException())

    assertThrows(IOException::class.java) {
        dataLoader.load("query", testFetcher)
    }

    assertThat(testFetcher.invokeCount).isEqualTo(1)
}

Note that currently suspend function is not allowed as super types but this will be supported in the upcoming Kotlin 1.4.

// doesn't compile
class Fetcher : suspend (String) -> Data { 
    override suspend fun invoke(query: String): Data = ....
}

A real-world example

I recently wrote a new in-memory caching library for dropbox/store. One of the supported features is cache loader which is an API for getting cached value by key and using the provided loader: () -> Value lambda to compute and cache the value automatically if none exists.

interface Cache<in Key : Any, Value : Any> {
....
    /**
     * Returns the value associated with [key] in this cache if exists,
     * otherwise gets the value by invoking [loader], associates the value with [key] in the cache,
     * and returns the cached value.
     *
     * Note that [loader] is executed on the caller's thread. When called from multiple threads
     * concurrently, if an unexpired value for the [key] is present by the time the [loader] returns
     * the new value, the existing value won't be replaced by the new value.
     * Instead the existing value will be returned.
     *
     * Any exceptions thrown by the [loader] will be propagated to the caller of this function.
     */
    fun get(key: Key, loader: () -> Value): Value
....
}

This is similar to our DataLoader example above. We can verify the loader lambda invocations with the same approach:

@Test
fun `get(key, loader) returns existing value when an unexpired entry with the associated key exists before executing the loader`() {
    val cache = Cache.Builder.newBuilder()
        .expireAfterAccess(expiryDuration, TimeUnit.NANOSECONDS)
        .build<Long, String>()

    cache.put(1, "dog")

    // create a new `() -> Value` lambda
    val loader = createLoader("cat")

    val value = cache.get(1, loader)

    // loader should not have been invoked as "dog" is already in the cache.
    assertThat(loader.invokeCount)
        .isEqualTo(0)

    assertThat(value)
        .isEqualTo("dog")
}

To support adding different variants of fake Loader implementation for some of the more complex concurrency tests, we have a TestLoader class that takes val block: () -> Value as a constructor parameter, and a bunch of top-level functions for creating different Loader implementations for different needs.

private class TestLoader<Value>(private val block: () -> Value) : () -> Value {
    var invokeCount = 0
    override operator fun invoke(): Value {
        invokeCount++
        return block()
    }
}

// a regular loader that returns computed value immediately
private fun <Value> createLoader(computedValue: Value) =
    TestLoader { computedValue }

// a loader that throws an exception immediately
private fun createFailingLoader(exception: Exception) =
    TestLoader { throw exception }

// a loader that takes certain duration (virtual) to execute and return the computed value
private fun <Value> createSlowLoader(
    computedValue: Value,
    executionTime: Long
) = TestLoader {
    runBlocking { delay(executionTime) }
    computedValue
}

The complete test suite can be found here.

Summary

I hope you find this technique useful! Here's the TLDR:

You don't need help from a mocking framework for testing Kotlin lambda invocations - it's easy to track the invocation count in the lambda implementation provided
You don't need to wrap a function inside an interface to be able to provide a fake implementation in tests - a class can implement a function type directly just like implementing a regular interface.

Thanks to Eyal Guthmann for suggesting the use of invocation counter in my cache rewrite PR which inspired me to write this article.

Featured in Kotlin Weekly #185 and Android Weekly #401.

Blog

Testing Kotlin Lambda Invocations without Mocking

Yang

Mocks

Fakes

A real-world example

Summary

Join Our Newsletter. No Spam, Only the good stuff.

Related