Test behaviour, not implementation

Introduction

Last year I spent a lot of time with writing and also practicing how to write good unit/integration tests in AEM (Adobe Experience Manager). Now I would like to share with you what I have learned so far. What I have learned is not only AEM related, you can apply it to any programming language or framework.

Before that time I had some "experience" with unit testing. I wrote several unit tests so that I can say "I have experience with it". But to be honest I didn't like to write it.

Even though I knew what benefits tests brings to the product what we are building, what brings to the team members and to me, I didn't care. Typical excuses were:

"I don't have time or we don't have time for it"
"I can't test or mock that"
"You can't write test, pick up some new feature or fix bug"

and on the end nobody pushed me to write it. Sadly, writing tests wasn't part of development process.

Now when I'm thinking a little bit, I didn't know how to write tests. Let's face it, writing tests is not easy, like many other things when you don't have experience with it.

Luckily that kind of things has changed at some point, and I would like to try convince all of you who are still thinking like "old" me.

I would like that we all start thinking more about Quality and not Quantity.

Writing tests as part of development process

Most of us works in "Agile" way (whatever that means) and use DDD (Deadline-driven development) methodology (I could write about that in separate post). This usually means that there is no time for writing tests. This need's to be changed, developers and all other technical team members should convince all other team members that writing tests should be part of development process. Writing tests should be part of any estimation. Period. Why?

There are a lot of benefits, but I will point out most important of them:

Bugs prevention
Better code quality
Provides some kind of documentation
Time saving
Money saving
Feeling "safe"

Now let's see typical "disadvantages":

Time consuming
Money consuming
Tests are slow to write
Tests are slow to run
Changing implementation require changing tests

Probably you have notice that I have mention time and money as advantage and disadvantage. Depending if you are thinking in short term, then yes it's waist of time and money, but If you think in long term then it's not true. Actually on the end it saves your time and money.

A lot of people thinks that they are waist of time and money. I think because they don't see some visual outcome of them, like how they see it when you build some feature. Try to think like this, with tests you can prevent a lot of bugs and a lot of ping pongs between Developers and QAs. Very often we have change requests during development and it happens that we implemented something in wrong way. New request just doesn't fit anymore to existing implementation. That means we need to refactor our old code or reimplement it from scratch. Here tests provides you some safe feel because you know if you broke behaviour or not. Another example could be big, never ending projects where several different teams have worked before you. Probably that project has poorly written documentation, you need to deal with legacy code and implement new features on top of it. Having tests is gold here. Also, a lot of projects starts like MVP which turns out to some core / base project with several subprojects. Not having test coverage here is total nonsense.

Last 3 disadvantages are also not true.

Tests are slow to write
- yes, If you don't know how to write it and if you don't have experience
- practice
Tests are slow to run
- again yes, if you don't know how to write it
Changing implementation require changing tests
- yes, because you are testing wrong things
- test behaviour not implementation

You don't believe me?

Take 1h of your time and watch the talk "TDD, Where Did It All Go Wrong" from Ian Cooper. For me this was eye opener. Before this talk I read few books about testing and I was not so convinced. In my opinion this is definitely the best talk about it.

tl; dr;

Test behaviour / requirements, not implementation
- with this kind of approach you will eliminate previously mentioned disadvantages
Test the public API of a module, not classes, methods or technical details
Unit test shouldn't be focused on classes, methods it should be focused on module, user stories
Test gives you promise what should be expected result / behaviour, so when you are refactoring an implementation, use tests to make sure that the implementation still yields the expected results
Write tests to cover the use cases or stories
Use the "Given When Then" model
Avoid mocks

This testing approach helps you to build right product. But negative point could be that it doesn't help you to build product right. Other downside is that you don't see exactly what is wrong when test is failing.

So classic unit testing approach push you to write more clean and quality code than "behaviour testing". In my opinion strict code reviews and static code analysis tools are better approach to achieve the same result. Second downside for me is really minor thing, since with debugging you can quickly find out what is happening.

I hope that you are still follow me and that I'm start changing a little bit your thinking about testing.

Now let's stop with theory and let's see how it works in practice.

Testing in AEM

Because last few years I'm working with AEM, I will show you how to test behaviours in your AEM projects. The same things you can apply in any other programming languages or frameworks. Depending on testing library support, this can be easier or harder to achieve.

As an example let say we need to implement Product Details API which is consumed by client side. To build Product Details API lets say in Spring you will probably create several classes like Product Controller, Service, Repository, DTO and so on. In AEM world this means you need to create Sling Servlet, OSGi Service, Sling Model and some DTO classes.

Product Details acceptance criteria:

show product details information (id, name, description, category id, images and variants)
product variants need's to be available for specific country
product variants are available from specific date
product variants need's to be sorted by sort order
name and description of product need's to be localized (depending on market), fallback is English

Implementation what you will see here is not perfect, it's simplified and hardcoded. In real world this is more complex. But here implementation is not important, instead we should focus how to test requirements of this API.

I will add here just 3 most important classes, other implementations you can see on Github

ProductDetails Sling Servlet

for handling request
it does some request validation
it use ProductDetailsService to get all information about requested product

package com.mkovacek.aem.core.servlets.products;

import com.mkovacek.aem.core.models.products.ProductDetailsModel;
import com.mkovacek.aem.core.records.response.Response;
import com.mkovacek.aem.core.services.products.ProductDetailsService;
import com.mkovacek.aem.core.services.response.ResponseService;

import lombok.extern.slf4j.Slf4j;

import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.servlets.HttpConstants;
import org.apache.sling.api.servlets.SlingSafeMethodsServlet;
import org.apache.sling.servlets.annotations.SlingServletResourceTypes;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;

import javax.servlet.Servlet;
import javax.servlet.ServletException;

import java.io.IOException;

@Slf4j
@Component(service = Servlet.class)
@SlingServletResourceTypes(
    resourceTypes = ProductDetailsServlet.RESOURCE_TYPE,
    selectors = ProductDetailsServlet.ALLOWED_SELECTOR,
    extensions = ProductDetailsServlet.JSON,
    methods = HttpConstants.METHOD_GET)
public class ProductDetailsServlet extends SlingSafeMethodsServlet {

    public static final String ALLOWED_SELECTOR = "productdetails";
    static final String RESOURCE_TYPE = "demo/components/productdetails";
    static final String JSON = "json";

    @Reference
    private transient ResponseService responseService;

    @Reference
    private transient ProductDetailsService productDetailsService;

    @Override
    public void doGet(final SlingHttpServletRequest request, final SlingHttpServletResponse response) throws ServletException, IOException {
        try {
            this.responseService.setJsonContentType(response);
            final String selector = request.getRequestPathInfo().getSelectorString();
            final String productId = this.responseService.getSuffix(request);

            if (this.responseService.areSelectorsValid(selector, ALLOWED_SELECTOR) && StringUtils.isNotBlank(productId)) {
                final Resource resource = request.getResource();
                final Response<ProductDetailsModel> data = this.productDetailsService.getProductDetails(productId, resource);
                this.responseService.sendOk(response, data);
            } else {
                this.responseService.sendBadRequest(response);
            }
        } catch (final Exception e) {
            log.error("Exception during handling request", e);
            this.responseService.sendInternalServerError(response);
        }
    }

}

ProductDetails OSGi Service

it's searching for requested product in repository / database
it's doing some product validation
maps product resource to ProductDetails model
returns product details

package com.mkovacek.aem.core.services.products.impl;

import com.day.cq.wcm.api.PageManager;
import com.mkovacek.aem.core.models.products.ProductDetailsModel;
import com.mkovacek.aem.core.records.response.Response;
import com.mkovacek.aem.core.records.response.Status;
import com.mkovacek.aem.core.services.products.ProductDetailsService;
import com.mkovacek.aem.core.services.resourceresolver.ResourceResolverService;

import lombok.extern.slf4j.Slf4j;

import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;

import java.util.Locale;
import java.util.Optional;

@Slf4j
@Component(service = ProductDetailsService.class, immediate = true)
public class ProductDetailsServiceImpl implements ProductDetailsService {

    private static final String PIM_READER = "pimReader";
    private static final Response<ProductDetailsModel> notFoundResponse = new Response<>(new Status(true, "Product Details not found"), null);
    private static final Response<ProductDetailsModel> errorResponse = new Response<>(new Status(false, "Error during fetching product details"), null);

    @Reference
    private ResourceResolverService resourceResolverService;

    @Override
    public Response<ProductDetailsModel> getProductDetails(final String id, final Resource resource) {
        try (final ResourceResolver resourceResolver = this.resourceResolverService.getResourceResolver(PIM_READER)) {
            final Locale locale = resourceResolver.adaptTo(PageManager.class).getContainingPage(resource).getLanguage(false);
            //usually this would be implemented with query
            final String productPath = StringUtils.join("/var/commerce/products/demo/", id);
            return Optional.ofNullable(resourceResolver.getResource(productPath))
                       .map(productResource -> productResource.adaptTo(ProductDetailsModel.class))
                       .map(productDetailsModel -> productDetailsModel.setLocale(locale))
                       .filter(ProductDetailsModel::isValid)
                       .map(productDetailsModel -> new Response<>(new Status(true), productDetailsModel))
                       .orElse(notFoundResponse);
        } catch (final Exception e) {
            log.error("Exception during fetching product details", e);
        }
        return errorResponse;
    }

}

ProductDetails Sling Model

representation of product resource in repository / database
used as response in JSON format

package com.mkovacek.aem.core.models.products;

import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.mkovacek.aem.core.services.products.ProductLocalizationService;
import com.mkovacek.aem.core.services.products.ProductValidatorService;

import lombok.Getter;
import lombok.extern.slf4j.Slf4j;

import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ValueMap;
import org.apache.sling.models.annotations.Default;
import org.apache.sling.models.annotations.DefaultInjectionStrategy;
import org.apache.sling.models.annotations.Model;
import org.apache.sling.models.annotations.injectorspecific.ChildResource;
import org.apache.sling.models.annotations.injectorspecific.OSGiService;
import org.apache.sling.models.annotations.injectorspecific.Self;
import org.apache.sling.models.annotations.injectorspecific.ValueMapValue;

import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import java.util.Locale;

@Slf4j
@Model(adaptables = {Resource.class}, defaultInjectionStrategy = DefaultInjectionStrategy.OPTIONAL)
public class ProductDetailsModel {

    @ValueMapValue
    @Default(values = StringUtils.EMPTY)
    @Getter
    private String id;

    @ValueMapValue
    @Default(values = StringUtils.EMPTY)
    @Getter
    private String categoryId;

    @ChildResource
    @Getter
    private List<ImageModel> images;

    @ChildResource
    private List<VariantsModel> variants;

    @Self
    private ValueMap valueMap;

    @OSGiService
    private ProductLocalizationService productLocalizationService;

    @OSGiService
    private ProductValidatorService productValidatorService;

    @Getter
    @JsonProperty("variants")
    private List<VariantsModel> validVariants = new ArrayList<>();

    @Getter
    private String name = StringUtils.EMPTY;

    @Getter
    private String description = StringUtils.EMPTY;

    @JsonIgnore
    public boolean isValid() {
        return !this.validVariants.isEmpty();
    }

    @JsonIgnore
    public ProductDetailsModel setLocale(final Locale locale) {
        this.setLocalizedValues(locale);
        this.validateAndSortVariants(locale);
        return this;
    }

    private void setLocalizedValues(final Locale locale) {
        this.name = this.productLocalizationService.getLocalizedProductDetail(this.valueMap, "name.", locale);
        this.description = this.productLocalizationService.getLocalizedProductDetail(this.valueMap, "description.", locale);
    }

    private void validateAndSortVariants(final Locale locale) {
        this.validVariants = this.productValidatorService.getValidVariants(this.variants, locale);
        this.validVariants.sort(Comparator.comparing(VariantsModel::getSortOrder));
    }

}

Except those 3 classes I need to create several more:

ImageModel, VariantsModel
BlobStorageService, ProductValidatorService, ProductLocalizationService, ResourceResolverService, ResponseService
Response and Status records

You saw that we have a lot of classes to build this user story. Usually what would developer test here are OSGi services. I'm not saying this is a bad approach, but for that you will need more time, and every time when you will refactor your code or add some new stuff, it's very likely that you will need to change your tests as well.

Instead of that let's test only Servlet because this is public API of this user story. So what we need to test in Servlet? First of all, we need to cover all requirments from acceptance criteria, additionaly we can cover some technical details of servlet implementation.

Test libraries in AEM

At the moment in my opinion the best library what you can use are AEM Mocks. AEM Mocks supports most common mock implementations of AEM APIs + contains Apache Sling and OSGi mock implementations. For other not implemented mocks you will need to implement it by yourself or use Mockito. Besides those two I will use Junit 5.

Some tips before we start:

Try to have Test classes clean as possible, it should contains just tests.
Move mocks in separate classes
Create some Util classes with common helper methods, if you repeat yourself in multiple places
Use @BeforeAll / AfterAll, @BeforeEach / AfterEach, Junit 5 annotations to not repeat yourself in every test method and to speed up your tests
Create common AEM context in separate class if you repeat yourself in several test classes
Don't programmatically create complex resources in AEM context, instead export it from real AEM instance as JSON resource and load it into AEM context.
Use ResourceResolverMock type whenever is possible to speed up your tests

ProductDetailsServletTest

You will see that this test class is more or less clean and it focused only on tests. There is no mocking here, separated mock example you can see here. I'm using @BeforeAll and @BeforeEach to do some common setup, like setting up market pages/resources and common request informations. Also I needed some helper class to easier register all necessary classes into AEM context. All resources are exported as JSON from real AEM instance and imported into AEM context so that we test on real data.

In this test class I'm testing technical details and requirements

technical details
- request validation
requirements
- response for non existing product id
- product details in different markets to cover localization
- product variants validation for specific markets
- product variants availability from specific date
- product varinats sorting

package com.mkovacek.aem.core.servlets.products;

import com.day.cq.wcm.api.Page;
import com.mkovacek.aem.core.context.AppAemContextBuilder;
import com.mkovacek.aem.core.context.constants.TestConstants;
import com.mkovacek.aem.core.context.utils.ResourceUtil;
import com.mkovacek.aem.core.services.blobstorage.impl.BlobStorageServiceImpl;
import com.mkovacek.aem.core.services.products.impl.ProductDetailsServiceImpl;
import com.mkovacek.aem.core.services.products.impl.ProductLocalizationServiceImpl;
import com.mkovacek.aem.core.services.products.impl.ProductValidatorServiceImpl;
import com.mkovacek.aem.core.services.resourceresolver.impl.ResourceResolverServiceImpl;
import com.mkovacek.aem.core.services.response.impl.ResponseServiceImpl;

import io.wcm.testing.mock.aem.junit5.AemContext;
import io.wcm.testing.mock.aem.junit5.AemContextExtension;

import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.resource.ResourceResolverFactory;
import org.apache.sling.testing.mock.sling.servlet.MockRequestPathInfo;
import org.apache.sling.testing.resourceresolver.MockResourceResolverFactory;
import org.junit.jupiter.api.*;
import org.junit.jupiter.api.extension.ExtendWith;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletResponse;

import java.io.IOException;
import java.util.Collections;

import static org.junit.jupiter.api.Assertions.assertAll;
import static org.junit.jupiter.api.Assertions.assertEquals;

@ExtendWith(AemContextExtension.class)
class ProductDetailsServletTest {

    private static final AemContext context = new AppAemContextBuilder()
                                                  .loadResource(TestConstants.HR_HR_LANDING_PAGE_JSON, TestConstants.HR_HR_LANDING_PAGE_PATH)
                                                  .loadResource(TestConstants.DE_AT_LANDING_PAGE_JSON, TestConstants.DE_AT_LANDING_PAGE_PATH)
                                                  .loadResource(TestConstants.FR_FR_LANDING_PAGE_JSON, TestConstants.FR_FR_LANDING_PAGE_PATH)
                                                  .loadResource(TestConstants.PRODUCTS_JSON, TestConstants.PRODUCTS_PATH)
                                                  .registerService(ResourceResolverFactory.class, new MockResourceResolverFactory())
                                                  .registerInjectActivateService(new ResourceResolverServiceImpl())
                                                  .registerInjectActivateService(new ResponseServiceImpl())
                                                  .registerInjectActivateService(new BlobStorageServiceImpl(), Collections.singletonMap("productImagesFolderPath", "https://dummyurl.com/images/products/"))
                                                  .registerInjectActivateService(new ProductValidatorServiceImpl())
                                                  .registerInjectActivateService(new ProductLocalizationServiceImpl())
                                                  .registerInjectActivateService(new ResponseServiceImpl())
                                                  .registerInjectActivateService(new ProductDetailsServiceImpl())
                                                  .build();

    private static final MockRequestPathInfo requestPathInfo = context.requestPathInfo();
    private final ProductDetailsServlet servlet = context.registerInjectActivateService(new ProductDetailsServlet());
    private static final String CONTENT_RESOURCE_PATH = "root/productdetails";
    private static String NOT_FOUND_RESPONSE;
    private static String BAD_REQUEST_RESPONSE;

    @BeforeAll
    static void setUpBeforeAllTests() throws IOException {
        context.addModelsForPackage(TestConstants.SLING_MODELS_PACKAGES);
        requestPathInfo.setExtension("json");
        NOT_FOUND_RESPONSE = ResourceUtil.getExpectedResult(ProductDetailsServlet.class, "responses/not-found-response.json");
        BAD_REQUEST_RESPONSE = ResourceUtil.getExpectedResult(ProductDetailsServlet.class, "responses/bad-request-response.json");
    }

    @BeforeEach
    void setupBeforeEachTest() {
        context.response().resetBuffer();
        requestPathInfo.setSelectorString(ProductDetailsServlet.ALLOWED_SELECTOR);
        requestPathInfo.setSuffix("123456789");
        final Page page = context.pageManager().getPage(TestConstants.HR_HR_LANDING_PAGE_PATH);
        context.request().setResource(page.getContentResource(CONTENT_RESOURCE_PATH));
    }

    @Test
    @DisplayName("GIVEN landing page (en-HR) WHEN servlet is called with not valid selector THEN it returns bad request response in JSON format")
    void testNotValidSelector() throws ServletException, IOException {
        requestPathInfo.setSelectorString(ProductDetailsServlet.ALLOWED_SELECTOR + ".test");
        this.servlet.doGet(context.request(), context.response());

        assertAll(
            () -> assertEquals(HttpServletResponse.SC_BAD_REQUEST, context.response().getStatus()),
            () -> assertEquals(BAD_REQUEST_RESPONSE, context.response().getOutputAsString())
        );
    }

    @Test
    @DisplayName("GIVEN landing page (en-HR) WHEN servlet is called without productId suffix THEN it returns bad request response in JSON format")
    void testNoProductId() throws ServletException, IOException {
        requestPathInfo.setSuffix(StringUtils.EMPTY);
        this.servlet.doGet(context.request(), context.response());

        assertAll(
            () -> assertEquals(HttpServletResponse.SC_BAD_REQUEST, context.response().getStatus()),
            () -> assertEquals(BAD_REQUEST_RESPONSE, context.response().getOutputAsString())
        );
    }

    @Test
    @DisplayName("GIVEN landing page (en-HR) WHEN servlet is called with not existing productId THEN it returns not found response in JSON format")
    void testNotExistingProductId() throws ServletException, IOException {
        requestPathInfo.setSuffix("123abc");
        this.servlet.doGet(context.request(), context.response());

        assertAll(
            () -> assertEquals(HttpServletResponse.SC_OK, context.response().getStatus()),
            () -> assertEquals(NOT_FOUND_RESPONSE, context.response().getOutputAsString())
        );
    }

    @Test
    @DisplayName("GIVEN landing page (en-HR) WHEN servlet is called with existing productId THEN it returns an expected localized (fallback) product details response in JSON format")
    void testProductDetailsInCroatianMarket() throws ServletException, IOException {
        this.servlet.doGet(context.request(), context.response());
        final String expectedProductDetails = ResourceUtil.getExpectedResult(this.getClass(), "responses/product-123456789-hr-HR.json");

        assertAll(
            () -> assertEquals(HttpServletResponse.SC_OK, context.response().getStatus()),
            () -> assertEquals(expectedProductDetails, context.response().getOutputAsString())
        );
    }

    @Test
    @DisplayName("GIVEN landing page (de-AT) WHEN servlet is called with existing productId THEN it returns an expected localized product details response in JSON format")
    void testProductDetailsInAustrianMarket() throws ServletException, IOException {
        this.setPageResource(TestConstants.DE_AT_LANDING_PAGE_PATH);
        this.servlet.doGet(context.request(), context.response());
        final String expectedProductDetails = ResourceUtil.getExpectedResult(this.getClass(), "responses/product-123456789-at-DE.json");

        assertAll(
            () -> assertEquals(HttpServletResponse.SC_OK, context.response().getStatus()),
            () -> assertEquals(expectedProductDetails, context.response().getOutputAsString())
        );
    }

    @Test
    @DisplayName("GIVEN landing page (fr-FR) WHEN servlet is called with existing productId which is not valid for French market THEN it returns not found response in JSON format")
    void testProductDetailsInFrenchMarket() throws ServletException, IOException {
        this.setPageResource(TestConstants.FR_FR_LANDING_PAGE_PATH);
        this.servlet.doGet(context.request(), context.response());

        assertAll(
            () -> assertEquals(HttpServletResponse.SC_OK, context.response().getStatus()),
            () -> assertEquals(NOT_FOUND_RESPONSE, context.response().getOutputAsString())
        );
    }

    private void setPageResource(final String path) {
        final Page page = context.pageManager().getPage(path);
        context.request().setResource(page.getContentResource(CONTENT_RESOURCE_PATH));
    }
}

With this testing approach I have covered 87% of lines of code. Other 13% what is not covered is catching exceptions.

Other good examples for testing in AEM would be components. For every component you have requirements. To achieve those requirements you will probaly create several classes like OSGi service, some Utils, Records and those requirements you will publicly exposed through Sling model to view layer. Ideal candidats for testing.

Sum up

If you don't write tests, start writing it
Test requirements not implementation
Developers should have time for writing tests

Blog