Don’t Use @Transactional in Tests

How to not ruin your Spring Boot application test suite

Photo: Braulio Cassule and Jesuelson Dacosta

Follow me on Github and Twitter. Unicorn and Heart the article to help reach more audience.

Source Code

You can find the sample code used for this article here. The sample code has 3 branches, the first with @Transactional, the second without, and the third using an appropriate replacement.

If you need to run the project, you need PostgreSQL installed on your computer. Then configure the environment variables below:

• DATABASE_HOST: localhost

• DATABASE_NAME: avoid-transactional

• DATABASE_USERNAME: postgres

• DATABASE_PASSWORD: yourpassword

You can of course provide different values depending on your local database configuration.

I recommend you to create two databases, one for production and another for tests. So you can run the tests without interfering with manual production testing using a tool like cURL or Postman. You’ll need to change the DATABASE_NAME variable in-between.

The Problem

When we are doing integration tests with Spring Boot — the ones that test several layers of your code and are often slow — we tend to use the @Transactional annotation to guarantee that data is cleaned up after the test execution, as we want to guarantee a deterministic test suite.

@SpringBootTest
@Transactional
internal class CartItemsControllerTests {

    ...

    @Test
    fun getAllCartItems() {
        val cart = Cart()
        cart.addProduct(product, 3)
        cartsRepository.save(cart)

        mockMvc.perform(get("/carts/{id}/items", cart.id))
            .andExpect(status().isOk)
            .andExpect(jsonPath("$.[*].id").value(hasItem(cart.items[0].id.toString())))
            .andExpect(jsonPath("$.[*].product.id").value(hasItem(product.id.toString())))
            .andExpect(jsonPath("$.[*].product.name").value(hasItem(product.name)))
            .andExpect(jsonPath("$.[*].quantity").value(3))
    }
}

You can also annotate your test class with @Transactional, instead of annotating all test methods, of course.

The problem with creating tests like this is that it is highly susceptible to ruin your test suite with a lot of false negatives. A false negative is a test that runs successfully when it’s supposed to fail due to a bug in production code.

Having a lot of false negatives can render your test suite useless, for two reasons:

• The confidence developers have towards the test suite will decrease, and people will start to get fearful of doing changes because they may break something.

• A lot of bugs will go to production without developers noticing unless they start to do manual tests, which is counter-productive.

A false negative is a test that runs successfully, when it’s supposed to fail when the production code has a bug.

If you run all the tests in with-transactional git branch, they will run successfully:

There are 2 false negatives there

But let’s then try to execute the application and call the endpoints using tools like cURL or Postman.

Lazy Loading Does Not Work

If you try to create a cart and then add an item to it, you will receive a 500 HTTP error. You can use the pre-generated product with the ID *2bdb8e93–3832–47b6–8bbc-d6b5db277989 *for the cart item:

The exception thrown in the console log was a LazyInitializationException:

org.hibernate.**LazyInitializationException**: failed to lazily initialize a collection of role: com.example.avoidtransactional.domain.model.Cart._items, could not initialize proxy - no Session

This exception was thrown while Hibernate tried to lazily fetch Cart._items list. Lazy loading does not work when the database session is closed, which happens right after you retrieve an entity with the repository.

val product =  productsRepository.findByIdOrNull(cartItem.productId)
if (product == null) {
    return ResponseEntity.notFound().build()
}

val cart = cartsRepository.findByIdOrNull(cartId)
if (cart == null) {
    return ResponseEntity.notFound().build()
}

cart.addProduct(product, cartItem.quantity)
cartsRepository.save(cart) // with @Transactional there's no need of this line

The problem happens in the 11th line. The Cart.addProduct() method internally manipulates a lazy collection of cart items, which raises the exception.

If you remove the @Transactional annotation from CartItemsController class, the tests within it will fail with the same LazyInitializationException.

org.springframework.web.util.**NestedServletException**: Request processing failed; nested exception is org.hibernate.**LazyInitializationException**: failed to lazily initialize a collection of role: com.example.avoidtransactional.domain.model.Cart._items, could not initialize proxy - no Session

You can change the git branch to without-transactional, where I removed @Transactional from all tests. The two tests that still run successfully are passing because they don’t eagerly fetch collections.

After removing @Transactional, the false negatives are revealed

Why all the tests were not failing before? Well, it turns out that @Transactional wraps the contained code within a transaction — which maintains the database session opened across all code contained in the transaction, which is needed for successful lazy loading of collections.

The recommended way to deal with collections in JPA/Hibernate is to eagerly fetch them manually while maintaining the collection lazy, to fetch just what you need and to avoid the N-M performance problem. But for brevity, I changed the fetch mode to EAGER in without-transactional-replacement branch.

Entities Are Saved Automatically

Another subtle problem with using @Transactional in tests is that the auto-commit feature is enabled for the tested code. Meaning that entities fetched within a @Transactional code are automatically saved at the end.

The test of the method below should still pass, even though we’re not explicitly calling save() in the 16th line. Just be sure that you’re in with-transactional branch.

@RestController
@RequestMapping("carts/{cart_id}/items")
class CartItemsController(
    private val cartsRepository: CartsRepository,
    private val productsRepository: ProductsRepository) {

    @PostMapping
    fun addItemToCart(
        @PathVariable("cart_id") cartId: UUID,
        @RequestBody cartItem: CartItemInputModel
    ) : ResponseEntity<Any> {

        ...

        cart.addProduct(product, cartItem.quantity)
        //cartsRepository.save(cart) // with @Transactional there's no need of this line

        val savedProductItem = cart.items.first { x -> x.product.id == cartItem.productId }
        return ResponseEntity.status(HttpStatus.CREATED)
            .body(mapToCartItemViewModel(savedProductItem))
    }
}

But if you send a request to add an item to a cart, the new item won’t be saved into the database.

Hard to use the database to troubleshoot test failures

Another problem with using @Transactional is that you can’t use the database to investigate the failure reasons, as the data is never actually saved into the database because transactions are always rollback’d at the end of the test method execution.

So it would be convenient to not wrap our tests under transactions to let the data be saved, while only cleaning the database before a test method execution.

Proposed Solution

We can’t just remove @Transactional without doing anything else, as we need to guarantee that all tests are independent of each other (see Martin Fowler article on non-determinism in tests)

If you change the git branch to without-transactional-replacement you’ll see that the tests are being annotated with a JUnit 5 extension that cleans all the data before a test method execution.

@SpringBootTest
@ExtendWith(PostgresDbCleanerExtension::class)
internal class CartsControllerTests {
    @Autowired
    lateinit var cartsRepository: CartsRepository

    @Autowired
    lateinit var webApplicationContext: WebApplicationContext

    ...
}

PostgresDbCleanerExtension extends the interface BeforeEachCallback that implements the beforeEach method, which is called before each test method is executed to clean the database.

class PostgresDbCleanerExtension : BeforeEachCallback {
    companion object {
        private val LOGGER = LoggerFactory.getLogger(PostgresDbCleanerExtension::class.java)

        private val TABLES_TO_IGNORE = listOf(
            TableData("databasechangelog"),
            TableData("databasechangeloglock")
        )
    }

    @Throws(Exception::class)
    override fun beforeEach(context: ExtensionContext) {
        val dataSource = getDataSourceFromYamlProperties("application.yml")
        cleanDatabase(dataSource)
    }

    ...

    private fun cleanDatabase(dataSource: DataSource) {
        try {
            dataSource.connection.use { connection ->
                connection.autoCommit = false
                val tablesToClean = loadTablesToClean(connection)
                cleanTablesData(tablesToClean, connection)
                connection.commit()
            }
        } catch (e: SQLException) {
            LOGGER.error(String.format("Failed to clean database due to error: \"%s\"", e.message))
            e.printStackTrace()
        }
    }
}

After the database properties are loaded from application.yml and bound to a javax.sql.DataSource, we use to retrieve all database tables with the class java.sql.DatabaseMetaData:

@Throws(SQLException::class)
private fun loadTablesToClean(connection: Connection): List<TableData> {
    val databaseMetaData = connection.metaData
    val resultSet = databaseMetaData.getTables(
        connection.catalog, null, null, arrayOf("TABLE"))

    val tablesToClean = mutableListOf<TableData>()
    while (resultSet.next()) {
        val table = TableData(
            schema = resultSet.getString("TABLE_SCHEM"),
            name = resultSet.getString("TABLE_NAME")
        )

        if (!TABLES_TO_IGNORE.contains(table)) {
            tablesToClean.add(table)
        }
    }

    return tablesToClean
}

And clean all of them with the TRUNCATE query, except the ones contained in the TABLES_TO_IGNORE list, which are the ones related to database migration history:

@Throws(SQLException::class)
private fun cleanTablesData(tablesNames: List<TableData>, connection: Connection) {
    if (tablesNames.isEmpty()) {
        return
    }
    val stringBuilder = StringBuilder("TRUNCATE ")
    for (i in tablesNames.indices) {
        if (i == 0) {
            stringBuilder.append(tablesNames[i].fullyQualifiedTableName)
        } else {
            stringBuilder
                .append(", ")
                .append(tablesNames[i].fullyQualifiedTableName)
        }
    }
    connection.prepareStatement(stringBuilder.toString())
        .execute()
}

Using TRUNCATE is a bit slower, you could also disable foreign key constraint check and delete all the data using the DELETE command which is faster.

The method we used with TRUNCATE is slower probably because it reclaims disk space immediately and clears all junk data.

Solving the production bugs

You’ll also notice that we replaced lazy loading fetch to eager loading so we can solve the production bugs related to LazyInitializationException.

Now that we got rid of @Transactional, and we discovered the false negatives in our test suite, let’s fix the actual bugs so that tests can pass.

I changed fetch to FetchType.EAGER in the problematic Cart._items property. But you should really prefer to eagerly fetch manually as described in this article by Vlad Mihalcea.

@Entity
@Table(name = "carts")
class Cart : DomainEntity() {
    @OneToMany(fetch = FetchType.EAGER, cascade = [CascadeType.ALL], orphanRemoval = true)
    private val _items = mutableListOf<ProductItem>()

    ...

}

If you run the tests in without-transactional-replacement branch. They should all pass.

All tests passing without false negatives

Conclusion

We saw in this article why using @Transactional in our test suite is not a good idea.

@Transactional enables undesired behaviors that produce false negatives in our test suite, which can impact the confidence developers have to make changes and render the test suite useless.

Using @Transactional in tests can also make troubleshooting difficult, as no data is really saved in the database.

Always prefer to clean the data manually, instead of relying on transactions, and prefer to do so before the tests execute, as you’ll want to know why a test failed by analyzing the database.

Thank you for reading this article, please Heart and Unicorn to help the article reach more audience.

Don’t forget to follow me on Github and Twitter!

References

• https://miensol.pl/clear-database-in-spring-boot-tests/

• https://vladmihalcea.com/the-best-way-to-handle-the-lazyinitializationexception/

• https://martinfowler.com/articles/nonDeterminism.html

• https://www.marcobehler.com/guides/spring-transaction-management-transactional-in-depth