Testable Go Code

Maybe you are new to testing, maybe you are new to Golang, or maybe you are curious about how to test your code in golang. In this article i'm going to take some "bad" go code and add tests to it, in the process we will need to also improve the code itself to make it testable, so even if you don't write tests, you can make adjustments to your own code.

Summary

What are tests and it's objectives?
What are the different types of tests?
Now into a real world code and testing it
Closing thoughts
References

What are tests and it's objectives?

Code testing exists with the objective to make sure that your code behaves the same after you add and remove code, with the passing of time, or even modify the code itself. They don't guarantee that your code is bug-free, neither that you don't have unknown behavior, they just make sure that given inputs it will do what you expect. Don't think that just because you wrote tests that your code is perfect and there is no error within, they don't replace QA, logging and tracing. Use them as guides, but don't blindly follow them as rule books on how your code behaves

What are the different types of tests?

If you are an experienced developer you can guess some times, but in this article i'm going to show only three of them for the sake of time and explanations.

Type testing
Unit testing
Integration testing

Type testing

This kind of test is the most common type of them. They are the red line bellow the code or a compilation error. Any language with a decent type system is capable of doing this kind of test for you. For free. Really.

// src/main.rs
fn main() {
    let a = 1 + "a";
}

// cargo build --release
   Compiling something v0.1.0 (/private/tmp/something)
error[E0277]: cannot add `&str` to `{integer}`
 --> src/main.rs:2:15

I did not write any kind of test, but there was a implicit test there.

Unit testing

A very common and prolific kind of test. The focus of unit testing is to check pure functions, not creating any side effects. If you run the same function 100 times, the same expected result should happen 100 times.

// sum.go
package math

// Sum takes a slice of integers and returns their sum.
func Sum(numbers []int) int {
    sum := 0
    for _, num := range numbers {
        sum += num
    }
    return sum
}

// sum_test.go
package math_test

import "testing"

func TestSum(t *testing.T) {
    input := int[]{1, 2, 3}

    result := Sum(input)
    expected := 6

    if result != expected {
        t.Errorf("Expected %d, but got %d for input %v", expected, result, input)
    }
}

Integration testing

A more specific kind of test. The object of the test is to integrate different parts of the system, parts that can fail and you can't control. Be like accessing a external source like a database, or using some system call like reading a file or writing to one. It is common the creation of mocks, stubs and/or spies to avoid the flaky nature of the execution.

// file_writer.go
package file

import (
    "io/ioutil"
    "os"
)

// WriteToFile writes content to a file with the given filename.
func WriteToFile(filename, content string) error {
    return ioutil.WriteFile(filename, []byte(content), 0644)
}


// file_writer_test.go

package file

import (
    "io/ioutil"
    "os"
    "testing"
)

func TestWriteToFile(t *testing.T) {
    // Define a test filename and content.
    filename := "testfile.txt"
    content := "This is a test file."

    // Clean up the file after the test.
    defer func() {
        err := os.Remove(filename)
        if err != nil {
            t.Errorf("Error deleting test file: %v", err)
        }
    }()

    // Call the function to write to the file.
    err := WriteToFile(filename, content)
    if err != nil {
        t.Fatalf("Error writing to file: %v", err)
    }

    // Read the file to verify its content.
    fileContent, err := ioutil.ReadFile(filename)
    if err != nil {
        t.Fatalf("Error reading from file: %v", err)
    }

    // Check if the content matches the expected content.
    if string(fileContent) != content {
        t.Errorf("File content doesn't match. Expected: %s, Got: %s", content, string(fileContent))
    }
}

Now into a real world code and testing it

For our execution code let's use the code below

// pkg/database.go
package pkg

import (
    "log"

    "github.com/jmoiron/sqlx"
    _ "github.com/lib/pq"
)

var connection *sqlx.DB

func InitConnection() {
    db, err := sqlx.Connect("postgres", "user=postgres password=postgres dbname=lab sslmode=disable")
    if err != nil {
        log.Fatalln("failed to connect", err)
    }

    if _, err := db.Exec(getSql()); err != nil {
        log.Fatalln("failed to execute sql", err)
    }

    connection = db
}

// not the best way to do this, but it works for this context
func getSql() string {
    return `
        create table if not exists posts
        (
            id         serial primary key,
            title      varchar(255) not null,
            body       text         not null,
            created_at timestamp default current_timestamp,
            updated_at timestamp default current_timestamp
        );

        create table if not exists comments
        (
            id         serial primary key,
            post_id    int  not null references posts (id) on delete cascade,
            body       text not null,
            created_at timestamp default current_timestamp
        );
    `
}

// pkg/service.go
package pkg

type Post struct {
    ID        int       `db:"id" json:"id"`
    Title     string    `db:"title" json:"title"`
    Body      string    `db:"body" json:"body"`
    CreatedAt string    `db:"created_at" json:"created_at"`
    UpdatedAt string    `db:"updated_at" json:"updated_at"`
    Comments  []Comment `json:"comments"`
}

type Comment struct {
    ID        int    `db:"id" json:"id"`
    Body      string `db:"body" json:"body"`
    PostID    int    `db:"post_id" json:"-"`
    CreatedAt string `db:"created_at" json:"created_at"`
}

func GetPostsWithComments() ([]Post, error) {
    var posts []Post

    if err := connection.Select(&posts, "select * from posts"); err != nil {
        return nil, err
    }

    for i := range posts {
        if err := connection.Select(&posts[i].Comments, "select * from comments where post_id = $1", posts[i].ID); err != nil {
            return nil, err
        }
    }

    return posts, nil
}

// main.go
package main

import (
    "encoding/json"
    "fmt"
    "log"

    "github.com/stneto1/better-go-article/pkg"
)

func main() {
    pkg.InitConnection()

    posts, err := pkg.GetPostsWithComments()
    if err != nil {
        log.Fatalln("failed to get posts", err)
    }

    jsonData, err := json.MarshalIndent(posts, "", "  ")
    if err != nil {
        log.Fatalln("failed to marshal json", err)
    }

    fmt.Println(string(jsonData))
}

The code above does:

Initializes the global connection
Fetch all posts with their comments
Print as pretty json for the terminal

Simples and straightforward. Now let's add tests to the main code, GetPostsWithComments.

// pkg/service_test.go
package pkg_test

import (
    "testing"

    "github.com/go-playground/assert/v2"
    "github.com/stneto1/better-go-article/pkg"
)

func TestGetPostsWithComments(t *testing.T) {
    posts, err := pkg.GetPostsWithComments()

    assert.Equal(t, err, nil)
    assert.Equal(t, len(posts), 0)
}

After running the code an error should appear, as the test was ran but the global connection was not initialized.

package pkg_test

import (
    "testing"

    "github.com/go-playground/assert/v2"
    "github.com/stneto1/better-go-article/pkg"
)

func TestGetPostsWithComments(t *testing.T) {
    pkg.InitConnection() // remember to call before each test

    posts, err := pkg.GetPostsWithComments()

    assert.Equal(t, err, nil)
    assert.Equal(t, len(posts), 0)
}

Now you run the tests and they all pass, nice job. But if you modify the database, the test may fail in the future, as there is no guarantee the state of the database when the tests run.

What are the issues with the current code:

Real DB connection
Global connection
N+1 queries
Connection was not closed
Leaky abstraction → Output model = DB model

Let's fix some of those issues

// pkg/database.go
func InitConnection() *sqlx.DB {
    db, err := sqlx.Connect("postgres", "user=postgres password=postgres dbname=lab sslmode=disable")
    if err != nil {
        log.Fatalln("failed to connect", err)
    }

    if _, err := db.Exec(getSql()); err != nil {
        log.Fatalln("failed to execute sql", err)
    }

    return db
}

//pkg/service.go
func GetPostsWithComments(conn *sqlx.DB) ([]Post, error) {
    // redacted
}


// main.go
func main() {
    conn := pkg.InitConnection()
    defer conn.Close()

    posts, err := pkg.GetPostsWithComments(conn)
    // redacted
}

// pkg/service_test.go

package pkg_test

import (
    "testing"

    "github.com/go-playground/assert/v2"
    "github.com/stneto1/better-go-article/pkg"
)

func TestGetPostsWithComments(t *testing.T) {
    conn := pkg.InitConnection()
    defer conn.Close()

    posts, err := pkg.GetPostsWithComments(conn)

    // redacted
}

Now the function GetPostsWithComments gets its connection as a parameter, so we can test a more controlled connection. We can also close connection after running the tests.

Real DB connection
~~Global connection~~
N+1 queries
~~Connection was not closed~~
Leaky abstraction → Output model = DB model

For the next issue, a real database connection.

We need a postgres connection for our current tests, if you run in an environment that does not have said connection, we can't run our tests. Let's improve it then. Look at the database connection type *sqlx.DB, not a postgres specific connection, so as long as we provide a valid sqlx connection, we can be sure that our tests can execute easily. And for that we can use sqlite, both in memory or writing to a single file, as SQL for the most part is a spec language, we can swap sqlite and postgres depending on where we are running.

// pkg/database.go

// redacted

func InitTempDB() *sqlx.DB {
    // This commented line is to create a temporary database in /tmp,
    // in case you want to access the file itself
    // db, err := sqlx.Connect("sqlite3", fmt.Sprintf("file:/tmp/%s.db", ulid.MustNew(ulid.Now(), nil).String()))
    db, err := sqlx.Connect("sqlite3", ":memory:")
    if err != nil {
        log.Fatalln("failed to connect", err)
    }

    if _, err := db.Exec(getSql()); err != nil {
        log.Fatalln("failed to execute sql", err)
    }

    return db
}

// pkg/service_test.go
package pkg_test

import (
    "testing"

    "github.com/go-playground/assert/v2"
    "github.com/stneto1/better-go-article/pkg"
)

func TestGetPostsWithComments(t *testing.T) {
    conn := pkg.InitTempDB()
    defer conn.Close()

    posts, err := pkg.GetPostsWithComments(conn)

    assert.Equal(t, err, nil)
    assert.Equal(t, len(posts), 0)
}

Our main code does not change, as we need an actual connection to execute our app in production, but for tests we can use in memory sqlite to have a "clean" database for each test.

~~Real DB connection~~
N+1 queries
Leaky abstraction → Output model = DB model

Now let's fix out last issues

We still make n+1 queries to our database, one query for the posts, and n queries for the comments. For that let's create a business struct.

// pkg/service.go

// redacted

type postsWithCommentsRow struct {
    PostID           int    `db:"posts_id"`
    PostTitle        string `db:"posts_title"`
    PostBody         string `db:"posts_body"`
    PostCreatedAt    string `db:"posts_created_at"`
    CommentID        int    `db:"comments_id"`
    CommentBody      string `db:"comments_body"`
    CommentCreatedAt string `db:"comments_created_at"`
}

func GetPostsWithComments(conn *sqlx.DB) ([]Post, error) {
    var rawPosts []postsWithCommentsRow

    if err := conn.Select(&rawPosts, `
        select posts.id       as posts_id,
            posts.title         as posts_title,
            posts.body          as posts_body,
            posts.created_at    as posts_created_at,
            comments.id         as comments_id,
            comments.body       as comments_body,
            comments.created_at as comments_created_at
        from posts
                left join comments on posts.id = comments.post_id
        order by posts.id;
    `); err != nil {
        return nil, err
    }

    posts := make([]Post, 0)

OuterLoop:
    for _, rawPost := range rawPosts {
        post := Post{
            ID:        rawPost.PostID,
            Title:     rawPost.PostTitle,
            Body:      rawPost.PostBody,
            CreatedAt: rawPost.PostCreatedAt,
        }

        for _, post := range posts {
            if post.ID == rawPost.PostID {
                continue OuterLoop
            }
        }

        posts = append(posts, post)
    }

    for _, rawPost := range rawPosts {
        for i, post := range posts {
            if post.ID == rawPost.PostID {
                comment := Comment{
                    ID:        rawPost.CommentID,
                    Body:      rawPost.CommentBody,
                    CreatedAt: rawPost.CommentCreatedAt,
                }

                posts[i].Comments = append(posts[i].Comments, comment)
            }
        }
    }

    return posts, nil
}

Our code increased quite a bit. But what did it changed? First of all, now we make only one query to the database. We also manually map query result into a business rule structure in the Post and Comments structs. We also separated the database struct postsWithCommentsRow from our external structs.

~~N+1 queries~~
~~Leaky abstraction → Output model = DB model~~

And just like that, we fixed out last two issues.

Closing thoughts

After understanding a little more about tests, we took some go code, wrote some tests and improved the code. So what now? Even if you don't like go, you can still understand the concepts from what we discussed here and use in your own code. Testing is one of the skills that you learn with passing the time and acquiring more experience.

References

https://stock.adobe.com/search?k=testing&asset_id=95004747

Blog

NYXa