Testing REST APIs in Go: A Guide to Unit and Integration Testing with Go's Standard Testing Library

Introduction

This article is going to take you through how to use unit test and integration test to improve you development experience as you create rest apis in golang.

Unit tests are designed to verify the functionality of the smallest, individual parts of an application, often focusing on a single function or method. These tests are conducted in isolation from other parts of the code to ensure each component works as expected on its own.
Integration tests, on the other hand, assess how different modules or components of the application work together. In this article, we’ll focus on integration testing for our Go application, specifically checking that it interacts correctly with a PostgreSQL database by successfully making and executing SQL queries.

This article assumes that you are familiar with golang and how to create rest api in golang the main focus will be on creating test for your routes (unit tests) and testing your sql query functions (integration tests) for reference visit the github to have a look at the project.

Setting Up

Assuming you have setup your project similar to the one linked above you will have a folder structure similar to this

test_project
|__cmd
   |__api
      |__api.go
   |__main.go
|__db
   |___seed.go
|__internal
   |___db
       |___db.go
   |___services
       |___records
           |___routes_test.go
           |___routes.go
           |___store_test.go
           |___store.go
       |___user
           |___routes_test.go
           |___routes.go
           |___store_test.go
           |___store.go
|__test_data
|__docker-compose.yml
|__Dockerfile
|__Makefile

Testing in golang is easy compored to other language you may have encountered becase of the inbuilt testing package that provides tools needed to write tests.
Test files are named with _test.go this suffix allows for go to target this files for execution when running the command go test.

The entrypoint for our project is the main.go file located in the cmd folder

// main.go

package main

import (
    "log"

    "finance-crud-app/cmd/api"
    "finance-crud-app/internal/db"

    "github.com/gorilla/mux"
    "github.com/jmoiron/sqlx"
    _ "github.com/lib/pq"
)

type Server struct {
    db  *sqlx.DB
    mux *mux.Router
}

func NewServer(db *sqlx.DB, mux *mux.Router) *Server {
    return &Server{
        db:  db,
        mux: mux,
    }
}

func main() {

    connStr := "postgres://postgres:Password123@localhost:5432/crud_db?sslmode=disable"

    dbconn, err := db.NewPGStorage(connStr)
    if err != nil {
        log.Fatal(err)
    }
    defer dbconn.Close()

    server := api.NewAPIServer(":8085", dbconn)
    if err := server.Run(); err != nil {
        log.Fatal(err)
    }
}

From the code you can see we are creating a new api server by passing a database connection and port number. After creating the server we run it on the stated port.

The NewAPIServer command comes from the api.go file which

// api.go
package api

import (
    "finance-crud-app/internal/services/records"
    "finance-crud-app/internal/services/user"
    "log"
    "net/http"

    "github.com/gorilla/mux"
    "github.com/jmoiron/sqlx"
)

type APIServer struct {
    addr string
    db   *sqlx.DB
}

func NewAPIServer(addr string, db *sqlx.DB) *APIServer {
    return &APIServer{
        addr: addr,
        db:   db,
    }
}

func (s *APIServer) Run() error {
    router := mux.NewRouter()
    subrouter := router.PathPrefix("/api/v1").Subrouter()

    userStore := user.NewStore(s.db)
    userHandler := user.NewHandler(userStore)
    userHandler.RegisterRoutes(subrouter)

    recordsStore := records.NewStore(s.db)
    recordsHandler := records.NewHandler(recordsStore, userStore)
    recordsHandler.RegisterRoutes(subrouter)

    log.Println("Listening on", s.addr)

    return http.ListenAndServe(s.addr, router)
}

For this api we are using mux as our http router.

Integration Test

We have a user Store struct that handles sql queries related to the user entity.

// store.go
package user

import (
    "errors"
    "finance-crud-app/internal/types"
    "fmt"
    "log"

    "github.com/jmoiron/sqlx"
)

var (
    CreateUserError   = errors.New("cannot create user")
    RetrieveUserError = errors.New("cannot retrieve user")
    DeleteUserError   = errors.New("cannot delete user")
)

type Store struct {
    db *sqlx.DB
}

func NewStore(db *sqlx.DB) *Store {
    return &Store{db: db}
}

func (s *Store) CreateUser(user types.User) (user_id int, err error) {
    query := `
    INSERT INTO users
    (firstName, lastName, email, password)
    VALUES ($1, $2, $3, $4)
    RETURNING id`

    var userId int
    err = s.db.QueryRow(query, user.FirstName, user.LastName, user.Email, user.Password).Scan(&userId)
    if err != nil {
        return -1, CreateUserError
    }

    return userId, nil
}

func (s *Store) GetUserByEmail(email string) (types.User, error) {
    var user types.User

    err := s.db.Get(&user, "SELECT * FROM users WHERE email = $1", email)
    if err != nil {
        return types.User{}, RetrieveUserError
    }

    if user.ID == 0 {
        log.Fatalf("user not found")
        return types.User{}, RetrieveUserError
    }

    return user, nil
}

func (s *Store) GetUserByID(id int) (*types.User, error) {
    var user types.User
    err := s.db.Get(&user, "SELECT * FROM users WHERE id = $1", id)
    if err != nil {
        return nil, RetrieveUserError
    }

    if user.ID == 0 {
        return nil, fmt.Errorf("user not found")
    }

    return &user, nil
}

func (s *Store) DeleteUser(email string) error {

    user, err := s.GetUserByEmail(email)
    if err != nil {
        return DeleteUserError
    }
    // delete user records first
    _, err = s.db.Exec("DELETE FROM records WHERE userid = $1", user.ID)
    if err != nil {
        return DeleteUserError
    }

    _, err = s.db.Exec("DELETE FROM users WHERE email = $1", email)
    if err != nil {
        return DeleteUserError
    }
    return nil
}

In the file above we have 3 pointer receiver methods:

CreateUser
GetUserByEmail
GetUserById

For these methods to perform their function they have to interact with an external systems which ,in this case that is Postgres DB .

To test this methods we will first create a store_test.go file. In go we usually name our test files after the file we are targetting to test and add the suffix _test.go .

// store_test.go

package user_test

import (
    "finance-crud-app/internal/db"
    "finance-crud-app/internal/services/user"
    "finance-crud-app/internal/types"
    "log"
    "os"
    "testing"

    "github.com/jmoiron/sqlx"
    _ "github.com/lib/pq"
)

var (
    userTestStore *user.Store
    testDB        *sqlx.DB
)

func TestMain(m *testing.M) {
    // database
    ConnStr := "postgres://postgres:Password123@localhost:5432/crud_db?sslmode=disable"
    testDB, err := db.NewPGStorage(ConnStr)
    if err != nil {
        log.Fatalf("could not connect %v", err)
    }
    defer testDB.Close()
    userTestStore = user.NewStore(testDB)

    code := m.Run()
    os.Exit(code)
}

func TestCreateUser(t *testing.T) {
    test_data := map[string]struct {
        user   types.User
        result any
    }{
        "should PASS valid user email used": {
            user: types.User{
                FirstName: "testfirsjjlkjt-1",
                LastName:  "testlastkjh-1",
                Email:     "validuser@email.com",
                Password:  "00000000",
            },
            result: nil,
        },
        "should FAIL invalid user email used": {
            user: types.User{
                FirstName: "testFirstName1",
                LastName:  "testLastName1",
                Email:     "test1@email.com",
                Password:  "800890",
            },
            result: user.CreateUserError,
        },
    }

    for name, tc := range test_data {
        t.Run(name, func(t *testing.T) {
            value, got := userTestStore.CreateUser(tc.user)
            if got != tc.result {
                t.Errorf("test fail expected %v got %v instead and value %v", tc.result, got, value)
            }
        })
    }

    t.Cleanup(func() {
        err := userTestStore.DeleteUser("validuser@email.com")
        if err != nil {
            t.Errorf("could not delete user %v got error %v", "validuser@email.com", err)
        }
    })
}

func TestGetUserByEmail(t *testing.T) {
    test_data := map[string]struct {
        email  string
        result any
    }{
        "should pass valid user email address used": {
            email:  "test1@email.com",
            result: nil,
        },
        "should fail invalid user email address used": {
            email:  "validuser@email.com",
            result: user.RetrieveUserError,
        },
    }

    for name, tc := range test_data {
        got, err := userTestStore.GetUserByEmail(tc.email)
        if err != tc.result {
            t.Errorf("test fail expected %v instead got %v", name, got)
        }
    }
}

func TestGetUserById(t *testing.T) {
    testUserId, err := userTestStore.CreateUser(types.User{
        FirstName: "userbyid",
        LastName:  "userbylast",
        Email:     "unique_email",
        Password:  "unique_password",
    })
    if err != nil {
        log.Panicf("got %v when creating testuser", testUserId)
    }

    test_data := map[string]struct {
        user_id int
        result  any
    }{
        "should pass valid user id used": {
            user_id: testUserId,
            result:  nil,
        },
        "should fail invalid user id used": {
            user_id: 0,
            result:  user.RetrieveUserError,
        },
    }

    for name, tc := range test_data {
        t.Run(name, func(t *testing.T) {
            _, got := userTestStore.GetUserByID(tc.user_id)
            if got != tc.result {
                t.Errorf("error retrieving user by id got %v want %v", got, tc.result)
            }
        })
    }

    t.Cleanup(func() {
        err := userTestStore.DeleteUser("unique_email")
        if err != nil {
            t.Errorf("could not delete user %v got error %v", "unique_email", err)
        }
    })
}

func TestDeleteUser(t *testing.T) {
    testUserId, err := userTestStore.CreateUser(types.User{
        FirstName: "userbyid",
        LastName:  "userbylast",
        Email:     "delete_user@email.com",
        Password:  "unique_password",
    })
    if err != nil {
        log.Panicf("got %v when creating testuser", testUserId)
    }

    test_data := map[string]struct {
        user_email string
        result     error
    }{
        "should pass user email address used": {
            user_email: "delete_user@email.com",
            result:     nil,
        },
    }

    for name, tc := range test_data {
        t.Run(name, func(t *testing.T) {
            err = userTestStore.DeleteUser(tc.user_email)
            if err != tc.result {
                t.Errorf("error deletig user got %v instead of %v", err, tc.result)
            }
        })
    }

    t.Cleanup(func() {
        err := userTestStore.DeleteUser("delete_user@email.com")
        if err != nil {
            log.Printf("could not delete user %v got error %v", "delete_user@email.com", err)
        }
    })
}

Lets go through the file looking at what each section does.

The first action is to declare the variables userTestStore and testDB. These variables will be used to store pointers to the user store and db respectively. The reason we have declared them in the global file scope is because we want all functions in the test file to have access to the pointers.

The TestMain function allows us to do some setting up actions before the main test are run. We are initially connecting to the postgres store and saving the pointer into our global variable.
We have used that pointer to create a userTestStore that we will use to execute the sql queries we are trying to connect.

defer testDB.Close() closes the database connection after the test has completed

code := m.Run() runs the rest of the test function before returning and exiting.

TestCreateUser function will handle the testing of the create_user function. Our goal is to test if the function will create the user if a unique email is passed and the function should not be able to create a user if a non-unique email has already been used to create another user.

First we create the test data that we will use to test both case scenarios.

test_data := map[string]struct {
        user   types.User
        result any
    }{
        "should PASS valid user email used": {
            user: types.User{
                FirstName: "testfirsjjlkjt-1",
                LastName:  "testlastkjh-1",
                Email:     "validuser@email.com",
                Password:  "00000000",
            },
            result: nil,
        },
        "should FAIL invalid user email used": {
            user: types.User{
                FirstName: "testFirstName1",
                LastName:  "testLastName1",
                Email:     "test1@email.com",
                Password:  "800890",
            },
            result: user.CreateUserError,
        },
    }

I will loop through the map executing create_user function with the test date as parameters and compare if the returned value is the same the result we expect

for name, tc := range test_data {
        t.Run(name, func(t *testing.T) {
            value, got := userTestStore.CreateUser(tc.user)
            if got != tc.result {
                t.Errorf("test fail expected %v got %v instead and value %v", tc.result, got, value)
            }
        })
    }

For cases where the returned result is not the same as the expected result then our test will fail

The last part of this function is using the inbuilt testing package function Cleanup. This function registered a function that will be called when all the function in the test have already been executed. In our example case here we are using the function to clear up user data that was used during this test function execution.

Unit Tests

For our unit tests we are going to test the route handlers for our api. In this case the routes related to the user entity. Observe below.

package user

import (
    "finance-crud-app/internal/services/auth"
    "finance-crud-app/internal/types"
    "finance-crud-app/internal/utils"
    "fmt"
    "net/http"
    "strconv"

    "github.com/go-playground/validator/v10"
    "github.com/gorilla/mux"
)

type Handler struct {
    store types.UserStore
}

func NewHandler(store types.UserStore) *Handler {
    return &Handler{store: store}
}

func (h *Handler) RegisterRoutes(router *mux.Router) {
    router.HandleFunc("/login", h.handleLogin).Methods("POST")
    router.HandleFunc("/register", h.handleRegister).Methods("POST")

    // secured routes
    router.HandleFunc("/users/{userID}", auth.JWTAuthMiddleWare(h.handleGetUser, h.store)).Methods(http.MethodGet)
}

func (h *Handler) handleLogin(w http.ResponseWriter, r *http.Request) {
    var user types.LoginUserPayload
    if err := utils.ParseJSON(r, &user); err != nil {
        utils.WriteError(w, http.StatusBadRequest, fmt.Errorf("invalid payload: %v", err))
        return
    }

    if err := utils.Validate.Struct(user); err != nil {
        errors := err.(validator.ValidationErrors)
        utils.WriteError(w, http.StatusBadRequest, fmt.Errorf("invalid payload: %v", errors))
        return
    }

    u, err := h.store.GetUserByEmail(user.Email)
    if err != nil {
        utils.WriteError(w, http.StatusBadRequest, fmt.Errorf("invalid credentials"))
        return
    }

    if !auth.ComparePasswords(u.Password, []byte(user.Password)) {
        utils.WriteError(w, http.StatusBadRequest, fmt.Errorf("invalid credentials"))
        return
    }

    token, err := auth.CreateJWT(u.ID)
    if err != nil {
        utils.WriteError(w, http.StatusBadRequest, fmt.Errorf("error %v", err))
        return
    }

    utils.WriteJSON(w, http.StatusAccepted, map[string]string{"token": token})
}

func (h *Handler) handleRegister(w http.ResponseWriter, r *http.Request) {
    var user types.RegisterUserPayload
    if err := utils.ParseJSON(r, &user); err != nil {
        utils.WriteError(w, http.StatusBadRequest, err)
        return
    }

    if err := utils.Validate.Struct(user); err != nil {
        errors := err.(validator.ValidationErrors)
        utils.WriteError(w, http.StatusBadRequest, fmt.Errorf("invalid payload: %v", errors))
        return
    }

    hashedPassword, err := auth.HashPassword(user.Password)
    if err != nil {
        utils.WriteError(w, http.StatusInternalServerError, err)
        return
    }

    userId, err := h.store.CreateUser(types.User{
        FirstName: user.FirstName,
        LastName:  user.LastName,
        Email:     user.Email,
        Password:  hashedPassword,
    })

    if err != nil {
        utils.WriteError(w, http.StatusInternalServerError, err)
        return
    }
    utils.WriteJSON(w, http.StatusCreated, map[string]int{"user_id": userId})
}

func (h *Handler) handleGetUser(w http.ResponseWriter, r *http.Request) {
    vars := mux.Vars(r)
    str, ok := vars["userID"]
    if !ok {
        utils.WriteError(w, http.StatusBadRequest, fmt.Errorf("missing user ID"))
        return
    }

    userID, err := strconv.Atoi(str)
    if err != nil {
        utils.WriteError(w, http.StatusBadRequest, fmt.Errorf("invalid user ID"))
        return
    }

    user, err := h.store.GetUserByID(userID)
    if err != nil {
        utils.WriteError(w, http.StatusInternalServerError, err)
        return
    }

    utils.WriteJSON(w, http.StatusOK, user)
}

We have 3 function here that we would want to test

HandleLogin
HandleRegister
HandleGetUser

HandleGetUser

The handleGetUser function in this handler retrieves user details based on a user ID provided in the HTTP request URL. It starts by extracting the userID from the request path variables using the mux router. If the userID is missing or invalid (non-integer), it responds with a 400 Bad Request error. Once validated, the function calls the GetUserByID method on the data store to retrieve user information. If an error occurs during retrieval, it returns a 500 Internal Server Error. On success, it responds with a 200 OK status, sending the user details as JSON in the response body.

As stated before for you to for as to test the handler functions we need to create a routes_test.go. See mine below

package user

import (
    "finance-crud-app/internal/types"
    "net/http"
    "net/http/httptest"
    "testing"

    "github.com/gorilla/mux"
)

type mockUserStore struct{}

func (m *mockUserStore) DeleteUser(email string) error {
    return nil
}

func (m *mockUserStore) GetUserByEmail(email string) (types.User, error) {
    return types.User{}, nil
}

func (m *mockUserStore) CreateUser(u types.User) (userId int, err error) {
    return 1, nil
}

func (m *mockUserStore) GetUserByID(id int) (*types.User, error) {
    return &types.User{}, nil
}

func TestGetUserHandler(t *testing.T) {
    userStore := &mockUserStore{}
    handler := NewHandler(userStore)

    t.Run("should fail to get user with user_id that is not a number", func(t *testing.T) {
        req, err := http.NewRequest(http.MethodGet, "/user/abc", nil)
        if err != nil {
            t.Fatal(err)
        }

        rr := httptest.NewRecorder()
        router := mux.NewRouter()

        router.HandleFunc("/user/{userID}", handler.handleGetUser).Methods(http.MethodGet)

        router.ServeHTTP(rr, req)

        if rr.Code != http.StatusBadRequest {
            t.Errorf("expected status code %d, got %d", http.StatusBadRequest, rr.Code)
        }
    })

    t.Run("should pass to get user with numeric id", func(t *testing.T) {
        req, err := http.NewRequest(http.MethodGet, "/user/23", nil)
        if err != nil {
            t.Fatal(err)
        }

        rr := httptest.NewRecorder()
        router := mux.NewRouter()

        router.HandleFunc("/user/{userID}", handler.handleGetUser).Methods(http.MethodGet)
        router.ServeHTTP(rr, req)

        if rr.Code != http.StatusOK {
            t.Errorf("expected status code %d, got %d", http.StatusBadGateway, rr.Code)
        }
    })
}

Our New Handler function requires a user store as a parameter for it to create a handler struct.
Since we do not need actual store we create a mock struct and create receiver functions that mock the function of the actual struct. We do this because we are handling the store function tests separetly therefore we dont need to test that part of the code in the handler tests.

The test function TestGetUserHandler test two case scenarios, the first is attempting to retrieve a user without providing the user id

t.Run("should fail to get user with user_id that is not a number", func(t *testing.T) {
        req, err := http.NewRequest(http.MethodGet, "/user/abc", nil)
        if err != nil {
            t.Fatal(err)
        }

        rr := httptest.NewRecorder()
        router := mux.NewRouter()

        router.HandleFunc("/user/{userID}", handler.handleGetUser).Methods(http.MethodGet)

        router.ServeHTTP(rr, req)

        if rr.Code != http.StatusBadRequest {
            t.Errorf("expected status code %d, got %d", http.StatusBadRequest, rr.Code)
        }
    })

The test is expected to pass if the http request responds with a 400 status code.

The second test case scenario is cases where we are retrieving, user information by using the correct url containing a valid user id. In this test case we expected a response with 200 status code. If not that test will have failed.

t.Run("should pass to get user with numeric id", func(t *testing.T) {
        req, err := http.NewRequest(http.MethodGet, "/user/23", nil)
        if err != nil {
            t.Fatal(err)
        }

        rr := httptest.NewRecorder()
        router := mux.NewRouter()

        router.HandleFunc("/user/{userID}", handler.handleGetUser).Methods(http.MethodGet)
        router.ServeHTTP(rr, req)

        if rr.Code != http.StatusOK {
            t.Errorf("expected status code %d, got %d", http.StatusBadGateway, rr.Code)
        }
    })

Conclusion

We have managed to implement unit test in our project by creating test for our route handlers. We have seen how to use mocks to only test a small unit of code. We have been able to developed integration test for our function that interact with Postgresql DB.
If you would want some hands on time with the project code clone the repo from github here

Blog