Golden Testing Helm Charts

We love tests at Twingate. When working on the Twingate’s helm charts repository I wanted to incorporate testing like we do with other code. For a long time our release process requires a long process of manual testing - every change to the chart’s templates required a manual process of testing the chart with various inputs to make sure it still works as expected. Enter golden tests as a way to automate this process…

What are Golden Tests? #

Golden tests, also known as snapshot testing, involves comparing the current output of your code with a “golden” reference or a previously stored correct version.

It’s a particularly useful method in situations where the output is complex and can change over time, like configuration files or templates.

Gold Testing a Helm Chart #

The concept is simple:

We maintain a directory of different helm input files (different permutations of values.yaml that we want to test).
For each such file we also maintain the expected output from that file. The expected output for running our chart with foo.yaml as input is stored in foo.golden.yaml
When we run our tests the simply render the chart’s result for every input file and compare the output with the content of the golden copy - if they’re different we fail.
When we want to update the golden copy we can run the test with an -update flag in which case, instead of comparing, it will write the output to the golden file. We can review the diff before commiting the test changes.

Here’s the code to run our tests:

package golden

import (
 "flag"
 "fmt"
 "os"
 "path/filepath"
 "regexp"
 "strings"
 "testing"

 "github.com/gruntwork-io/terratest/modules/helm"
 "gopkg.in/yaml.v2"
)

var update = flag.Bool("update", false, "update golden test output files")

type ChartYaml struct {
 Name string `yaml:"name"`
 Version string `yaml:"version"`
}

func GetChartYaml(t *testing.T) ChartYaml {
 chartYamlFile, err := os.ReadFile("./my-chart/Chart.yaml")
 if err != nil {
 t.Fatalf("Error reading Chart.yaml: %v", err)
 }

 var chartYaml ChartYaml

 if err := yaml.Unmarshal(chartYamlFile, &chartYaml); err != nil {
 t.Fatalf("Error unmarshaling YAML data: %v", err)
 }

 return chartYaml
}

func TestHelmRender(t *testing.T) {
 files, err := os.ReadDir("./test/golden")
 if err != nil {
 t.Fatal(err)
 }

 chartYaml := GetChartYaml(t)

 for _, f := range files {
 if !f.IsDir() && strings.HasSuffix(f.Name(), ".yaml") && !strings.HasSuffix(f.Name(), ".golden.yaml") {
 // Render this values.yaml file
 output, error := helm.RenderTemplateE(t,
 &helm.Options{
 ValuesFiles: []string{"test/golden/" + f.Name()},
 },
 "./my-chart",
 "test",
 []string{},
 )

 goldenFile := "test/golden/" + strings.TrimSuffix(f.Name(), filepath.Ext(".yaml")) + ".golden.yaml"

 // If error, we write the error to the golden snapshot
 if error != nil {
 output = fmt.Sprintf("%v\n", error)
 } else {
 // Replace `cn.chart` helper value with a stable value for testing
 // because we don't want to have to update all snapshots whenever Chart version changes
 regex := regexp.MustCompile(fmt.Sprintf("%s-%s", chartYaml.Name, chartYaml.Version))
 bytes := regex.ReplaceAll([]byte(output), []byte(fmt.Sprintf("%s-major.minor.patch-test", chartYaml.Name)))
 output = fmt.Sprintf("%s\n", string(bytes))
 }

 // If we were called with `-update` param - write `output` to our golden snapshot file`
 if *update {
 err := os.WriteFile(goldenFile, []byte(output), 0644)
 if err != nil {
 t.Fatal(err)
 }
 }

 // Read golden snapshot file and make sure its content is identical to output
 expected, err := os.ReadFile(goldenFile)
 if err != nil {
 t.Fatal(err)
 }

 if string(expected) != output {
 t.Fatalf("Expected %s, but got %s\n. Update golden files by running `go test -v ./... -update`", string(expected), output)
 }
 }
 }
}

You’d run this simply by running go test -v ./... or go test -v ./... -update if you want to update the golden copies.

An example of how we use this in Twingate can be found on the helm-charts and Kubernetes Operator repositories on GitHub.

Blog

Golden Testing Helm Charts

Eran Kampf

What are Golden Tests? #

Gold Testing a Helm Chart #

Join Our Newsletter. No Spam, Only the good stuff.

Related