TheLazy.Dev
Portfolio GitHub

Blog

circleci |cicd |android |testing

Android App: Instrumented and Unit Tests Automation Using CircleCI

roselinebassey

Roseline Bassey

Posted on May 10, 2023

Android App: Instrumented and Unit Tests Automation Using CircleCI

As an Android developer, testing is an integral part of the app development process. By running tests you can verify your app’s functional behaviour, correctness, and usability before you release it to the public. CircleCI is a CI platform that supports running Android emulators from within an Android machine image. We will use CircleCI to set up automated testing for our project. Automated testing involves using tools that perform tests on the project for you, which is faster, repeatable, and gives you more actionable feedback about your app earlier in the development process.

This article describes how to automate instrumentation and unit tests for an open source Android project using CircleCI. I assume you already know how to create an Android app and write tests for your app. The cloned project and working pipeline can be found here.

Instrumentation Testing

Instrumented tests, also known as instrumentation tests, are tests that run on Android devices, whether physical or emulated. It involves testing the overall behaviour of an application by testing the interaction between different components. It checks how the application performs when different components work together, including user interfaces, APIs, and external dependencies. This type of testing helps identify issues that may arise due to interactions between components, such as compatibility issues or performance problems.

Unit Testing

Unit testing involves testing individual components or units of code in isolation to verify their functionality, usually a single class or function. In unit testing, each unit of code is tested separately from the rest of the codebase to ensure that it behaves as expected.

CircleCI

CircleCI is a hosted and automated solution for continuous integration (CI) builds. It uses an in app configuration file that uses YAML syntax. CircleCI is easier to set up and is hosted in the cloud. After learning about instrumentation tests, unit testing, and CircleCI, we can now learn how testing in CI/CD works.

Unit Testing and Instrumentation Testing in CI/CD

As a DevOps principle, it is a good practice to automate testing as part of the CI process to test your code against any bug. Automated tests can be triggered at various stages of your CircleCI pipeline, including development, staging, and/or production. An effective automated test runs immediately after changes are committed to the version control system's (for example, GitHub) staging or development repository. If the test meets the success criteria, then the committed change(s) may proceed to the next stage or environment. If the test failed, the committed changes are blocked from proceeding until those issues are resolved. By automating tests, teams can have confidence that code changes have been thoroughly tested and have met the success and quality standards, thereby, protecting the production environment against disruptions.

Setting up Automated Android Testing on CircleCI

Firstly, we will set up our config.ymlfile to run unit tests. I'll demonstrate two approaches for running instrumentation tests on CircleCI: using the No-orb and Orb examples. The No-orb example is a more complex process because you would have to write the steps yourself, while the CircleCI Android orb example is an easy process with many steps already written for you. There's no need to worry, the No-orb example is not as difficult as it seems because CircleCi offers clear documentation that will make setup simple. Please take note that you should only use one of the examples in your config.yml file. The two instances given here are merely for demonstration. Let's get started.

Create a circleci.yml File

At the root of your project’s folder, create a .circleci folder. Inside the folder create a config.yml file. Our workflow will be in this order:

  • Unit Tests
  • Instrumentation Tests

Running Unit Test

We will call our job unit-tests. The working_directory specifies the location of the project files. The docker image section specifies the Docker image to use. In this case, it uses the cimg/android:2023.02 image.

Here’s a full look at the unit test job:



jobs:
version: 2.
unit-tests:
    working_directory: ~/project
    docker:
      - image: cimg/android:2023.02
    steps:
      - checkout
      - run:
          name: Run Unit Tests
          command: |
            ./gradlew clean test
        - store_test_results:
            path: app/build/test-results
Enter fullscreen mode Exit fullscreen mode

The steps section includes three steps: first CircleCI checks out the repository where the code is stored. Then, run unit tests using the ./gradlew clean test command, and store the test results at the path specified.

Running Instrumentation Tests

As I mentioned earlier, there are different approaches to doing this. We will cover two examples or approaches. The first example is the No-orb example.

Using the No-orb Example

Our instrumented test requires an emulator to run on a machine image. Below is a breakdown of the instrumentation tests job. The full code is provided below. We will add each step to the instrumented test job in the circleci.yml file.



jobs:
instrumented_test:
    machine:
      image: android:202102-01
    resource_class: large
Enter fullscreen mode Exit fullscreen mode

So in the instrumented test job, we specify the image: android:202102-01 as the machine image and set the resource_class of the machine as "large" to improve build time. Under the step section we have the following steps:



steps:
      - checkout
      - run:
          name: Create avd
          command: |
            SYSTEM_IMAGES="system-images;android-29;default;x86"
            sdkmanager "$SYSTEM_IMAGES"
            echo "no" | avdmanager --verbose create avd -n test -k "$SYSTEM_IMAGES"
Enter fullscreen mode Exit fullscreen mode

The - checkout checks out the code from the repository while the command under - Create avd installs the Android system image required for the emulator to run, and creates a new virtual device named "test".



- run:
          name: Launch emulator
          command: |
            emulator -avd test -delay-adb -verbose -no-window -gpu swiftshader_indirect -no-snapshot -noaudio -no-boot-anim
          background: true
Enter fullscreen mode Exit fullscreen mode

The Launch emulator command launches the emulator and waits for it to start up. It runs in the background (background: true) so that other steps can execute in parallel.



 - run:
          name: Generate cache key
          command: |
            find . -name 'build.gradle' | sort | xargs cat |
            shasum | awk '{print $1}' > /tmp/gradle_cache_seed
Enter fullscreen mode Exit fullscreen mode

The Generate cache key command generates a cache key based on the contents of the build.gradle files in the project. This key is used to restore the Gradle cache later, to speed up the build process.



  - restore_cache:
          key: gradle-v1-{{ arch }}-{{ checksum "/tmp/gradle_cache_seed" }}
Enter fullscreen mode Exit fullscreen mode

restore_cache step attempts to restore the Gradle cache from a previous build, using the cache key generated in the previous step.



- run:
          # run in parallel with the emulator starting up, to optimize build time
          name: Run assembleDebugAndroidTest task
          command: |
            ./gradlew assembleDebugAndroidTest
Enter fullscreen mode Exit fullscreen mode

assembleDebugAndroidTest command compiles and build the debug version of the Android test APK and runs in parallel with the emulator starting up, to optimize build time.

This step can be modified to suit your project needs. I modified the above☝️ step to this:👇



 - run:
          # run in parallel with the emulator starting up, to optimize build time
          name: Run compileDemoBasicDebugJavaWithJavac task
          command: |
            ./gradlew compileDemoBasicDebugJavaWithJavac
Enter fullscreen mode Exit fullscreen mode

This step compiles the Java code in the project using the javac compiler. This runs in parallel with the emulator starting up, to optimize build time.



  - run:
          name: Wait for emulator to start
          command: |
            circle-android wait-for-boot
Enter fullscreen mode Exit fullscreen mode

As the name implies, the Wait for emulator to startstep waits for the emulator to finish booting up and become available for testing.



   - run:
          name: Disable emulator animations
          command: |
            adb shell settings put global window_animation_scale 0.0
            adb shell settings put global transition_animation_scale 0.0
            adb shell settings put global animator_duration_scale 0.0
Enter fullscreen mode Exit fullscreen mode

The Disable emulator animations step disables the emulator animations to speed up the testing process.



 - run:
          name: Run UI tests (with retry)
          command: |
            MAX_TRIES=2
            run_with_retry() {
               n=1
               until [ $n -gt $MAX_TRIES ]
               do
                  echo "Starting test attempt $n"
                  ./gradlew connectedAndroidTest && break
                  n=$[$n+1]
                  sleep 5
               done
               if [ $n -gt $MAX_TRIES ]; then
                 echo "Max tries reached ($MAX_TRIES)"
                 exit 1
               fi
            }
            run_with_retry
Enter fullscreen mode Exit fullscreen mode

Run UI tests (with retry) runs the instrumented tests on the emulator, with a retry mechanism in case of failures. The connectedAndroidTest command is the main command which runs the tests.



 - save_cache:
          key: gradle-v1-{{ arch }}-{{ checksum "/tmp/gradle_cache_seed" }}
          paths:
            - ~/.gradle/caches
            - ~/.gradle/wrapper
Enter fullscreen mode Exit fullscreen mode

save_cache saves the Gradle cache for future builds, using the cache key generated earlier.

Let's take a look at the whole config.yml file, which includes both the unit tests and the instrumentation tests jobs:



jobs:
version: 2.
unit-tests:
    working_directory: ~/project
    docker:
      - image: cimg/android:2023.02
    steps:
      - checkout
      - run:
          name: Run Unit Tests
          command: |
            ./gradlew clean test

      - store_test_results:
          path: app\build\test-results

instrumented_test:
    machine:
      image: android:202102-01
    resource_class: large
    steps:
      - checkout
      - run:
          name: Create avd
          command: |
            SYSTEM_IMAGES="system-images;android-29;default;x86"
            sdkmanager "$SYSTEM_IMAGES"
            echo "no" | avdmanager --verbose create avd -n test -k "$SYSTEM_IMAGES"
      - run:
          name: Launch emulator
          command: |
            emulator -avd test -delay-adb -verbose -no-window -gpu swiftshader_indirect -no-snapshot -noaudio -no-boot-anim
          background: true
      - run:
          name: Generate cache key
          command: |
            find . -name 'build.gradle' | sort | xargs cat |
            shasum | awk '{print $1}' > /tmp/gradle_cache_seed
      - restore_cache:
          key: gradle-v1-{{ arch }}-{{ checksum "/tmp/gradle_cache_seed" }}
      - run:
          # run in parallel with the emulator starting up, to optimize build time
          name: Run assembleDebugAndroidTest task
          command: |
            ./gradlew assembleDebugAndroidTest
      - run:
          name: Wait for emulator to start
          command: |
            circle-android wait-for-boot
      - run:
          name: Disable emulator animations
          command: |
            adb shell settings put global window_animation_scale 0.0
            adb shell settings put global transition_animation_scale 0.0
            adb shell settings put global animator_duration_scale 0.0
      - run:
          name: Run UI tests (with retry)
          command: |
            MAX_TRIES=2
            run_with_retry() {
               n=1
               until [ $n -gt $MAX_TRIES ]
               do
                  echo "Starting test attempt $n"
                  ./gradlew connectedAndroidTest && break
                  n=$[$n+1]
                  sleep 5
               done
               if [ $n -gt $MAX_TRIES ]; then
                 echo "Max tries reached ($MAX_TRIES)"
                 exit 1
               fi
            }
            run_with_retry
      - save_cache:
          key: gradle-v1-{{ arch }}-{{ checksum "/tmp/gradle_cache_seed" }}
          paths:
            - ~/.gradle/caches
            - ~/.gradle/wrapper
      - store_test_results:
          path: app\build\outputs\androidTest-results
Enter fullscreen mode Exit fullscreen mode

Using Android Orb Example

Since CircleCi has already done some of the work, this step is a bit shorter than the first example. So, you can use this instead of the No-orb example and it'll still function the same.




version: 2.1
orbs:
  android: circleci/android@2.1.2
jobs: 
  android-test:
    executor:
      name: android/android-machine
      tag: "202102-01"
      resource-class: large
    #run instrumentation tests

    steps:
      - checkout
      - run:
          name: installing emulator and Running Instrumentation tests
          command:  |
            sdkmanager "platform-tools" "platforms;android-29" "build-tools;30.0.0" "emulator"
            sdkmanager "system-images;android-29;google_apis;x86"
            echo no | avdmanager create avd -n test-emulator -k  "system-images;android-29;google_apis;x86"
            emulator -avd test-emulator -noaudio -no-boot-anim -gpu off -no-window &
            adb wait-for-device shell 'while [[ -z $(getprop sys.boot_completed) ]]; do sleep 1; done;'
            adb shell wm dismiss-keyguard
            sleep 1
            adb shell settings put global window_animation_scale 0
            adb shell settings put global transition_animation_scale 0
            adb shell settings put global animator_duration_scale 0
            ./gradlew connectedAndroidTest
Enter fullscreen mode Exit fullscreen mode

It's time to see our CI test in action on CircleCI.

How to Setup CircleCI

Make sure you have created at the root of your project’s folder, a .circleci folder, and inside of the folder you have a config.yml file. Before moving on to CircleCI, let's add a build job at the beginning of the config.yml file and define the workflow of our build pipeline at the end of the file. Using the no-orb example, the final config.yml file will look like this: 



version: 2.1
jobs:
  build:
    working_directory: ~/project
    docker:
      - image: cimg/android:2023.02
    steps:
      - checkout
      - restore_cache:
          keys:
            - v1-dependencies-{{ checksum "build.gradle" }}
            - v1-dependencies-
      - run:
          name: Install dependencies
          command: ./gradlew androidDependencies

      - save_cache:
          paths:
            - ~/.gradle
            - ~/.android
          key: v1-dependencies-{{ checksum "build.gradle" }}

      - run:
          name: Build project
          command: ./gradlew clean assemble

unit tests:
    working_directory: ~/project
    docker:
      - image: cimg/android:2023.02
    steps:
      - checkout
      - run:
          name: Run Local UnitTests
          command: |
            ./gradlew clean test

      - store_test_results:
          path: app\build\test-results


  instrumented_test:
    machine:
      image: android:202102-01
    resource_class: large
    steps:
      - checkout
      - run:
          name: Create avd
          command: |
            SYSTEM_IMAGES="system-images;android-29;default;x86"
            sdkmanager "$SYSTEM_IMAGES"
            echo "no" | avdmanager --verbose create avd -n test -k "$SYSTEM_IMAGES"
      - run:
          name: Launch emulator
          command: |
            emulator -avd test -delay-adb -verbose -no-window -gpu swiftshader_indirect -no-snapshot -noaudio -no-boot-anim
          background: true
      - run:
          name: Generate cache key
          command: |
            find . -name 'build.gradle' | sort | xargs cat |
            shasum | awk '{print $1}' > /tmp/gradle_cache_seed
      - restore_cache:
          key: gradle-v1-{{ arch }}-{{ checksum "/tmp/gradle_cache_seed" }}
      - run:
          # run in parallel with the emulator starting up, to optimize build time
          name: Run compileDemoBasicDebugJavaWithJavac task
          command: |
            ./gradlew compileDemoBasicDebugJavaWithJavac
      - run:
          name: Wait for emulator to start
          command: |
            circle-android wait-for-boot
      - run:
          name: Disable emulator animations
          command: |
            adb shell settings put global window_animation_scale 0.0
            adb shell settings put global transition_animation_scale 0.0
            adb shell settings put global animator_duration_scale 0.0
      - run:
          name: Run UI tests (with retry)
          command: |
            MAX_TRIES=2
            run_with_retry() {
               n=1
               until [ $n -gt $MAX_TRIES ]
               do
                  echo "Starting test attempt $n"
                  ./gradlew connectedAndroidTest && break
                  n=$[$n+1]
                  sleep 5
               done
               if [ $n -gt $MAX_TRIES ]; then
                 echo "Max tries reached ($MAX_TRIES)"
                 exit 1
               fi
            }
            run_with_retry
      - save_cache:
          key: gradle-v1-{{ arch }}-{{ checksum "/tmp/gradle_cache_seed" }}
          paths:
            - ~/.gradle/caches
            - ~/.gradle/wrapper
      - store_test_results:
          path: app\build\outputs\androidTest-results

workflows:
  build_and_deploy:
    jobs:
      - build
      - instrumented_test:
          requires:
            - build
      - unit tests:
          requires:
            - build
Enter fullscreen mode Exit fullscreen mode

Next, login to CircleCI using your GitHub, BitBucket, or GitLab account. In the CircleCI web, click on projects at the sidebar and then select which project on your version control you want CircleCI to build on. In my case, I selected the androidlibraryproject.

CircleCI build dashboard
In the “Select your config.yml file” modal, select Fastest, and a pop-up will appear like so:

CircleCI
Under fastest select to choose your preferred project and the branch where the project is located, then click Set Up Project to start building your project on CircleCI. You can monitor the build process on your dashboard. On a successful build, you'll have a green build like below:

CircleCI Dashboard

Note that the instrumentation tests job may take a few more minutes than other jobs to build. The time taken is solely dependent on your project. It took almost five minutes for the instrumentation test to finish building in my case.

CircleCi dashboard showing build time

Recap

In this article, you learned how to automate testing for your Android project using CircleCI. How to set up unit tests and also learned two approaches to setting up instrumentation tests on CirCleCI. You've learned the benefits of automating testing in CI and you've seen the CI test in action on CircleCI.

💖 💪 🙅 🚩
roselinebassey
Roseline Bassey

Posted on May 10, 2023

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Android App: Instrumented and Unit Tests Automation Using CircleCI
circleci Android App: Instrumented and Unit Tests Automation Using CircleCI

May 10, 2023