Handling errors concurrently in Go with ErrGroup

Preface

Concurrency is one of Go's strong points and I love working with the paradigm that the Go team has built.

It is a big topic with lots to talk about. I recommend reading through the Effective Go documentation about concurrency in Go to learn about goroutines, channels, and how they all work together.

Error handling is also done differently in Go than other languages, thanks to multiple return values. I recommend reading their blog post on error handling.

ErrGroup

If you don't need to do any further work off of the errors, use an ErrGroup!

An ErrGroup is essentially a wrapped sync.WaitGroup to catch errors out of the started goroutines.

WaitGroup

Here is an example without errors using a WaitGroup (from godoc):

package main

import (
    "sync"
)

type httpPkg struct{}

func (httpPkg) Get(url string) {}

var http httpPkg

func main() {
    var wg sync.WaitGroup
    var urls = []string{
        "http://www.golang.org/",
        "http://www.google.com/",
        "http://www.somename.com/",
    }
    for _, url := range urls {
        // Increment the WaitGroup counter.
        wg.Add(1)
        // Launch a goroutine to fetch the URL.
        go func(url string) {
            // Decrement the counter when the goroutine completes.
            defer wg.Done()
            // Fetch the URL.
            http.Get(url)
        }(url)
    }
    // Wait for all HTTP fetches to complete.
    wg.Wait()
  fmt.Println("Successfully fetched all URLs.")
}

To use a WaitGroup, first create the group:

var wg sync.WaitGroup

Next, for every goroutine, add that number to the group:

wg.Add(1)

Then whenever a goroutine is done, tell the group:

defer wg.Done()

The defer keyword:

It defers the execution of the statement following the keyword until the surrounding function returns.

Read more about it in the tour of go.

Finally, wait for the group to complete:

wg.Wait()

In this case, there are no errors that can occur. Let's look at how it changes if we needed to catch errors, using an ErrGroup.

ErrGroup

Here is the same example as above but using an ErrGroup (from godoc):

package main

import (
    "fmt"
    "net/http"

    "golang.org/x/sync/errgroup"
)

func main() {
    g := new(errgroup.Group)
    var urls = []string{
        "http://www.golang.org/",
        "http://www.google.com/",
        "http://www.somename.com/",
    }
    for _, url := range urls {
        // Launch a goroutine to fetch the URL.
        url := url // https://golang.org/doc/faq#closures_and_goroutines
        g.Go(func() error {
            // Fetch the URL.
            resp, err := http.Get(url)
            if err == nil {
                resp.Body.Close()
            }
            return err
        })
    }
    // Wait for all HTTP fetches to complete.
    if err := g.Wait(); err == nil {
        fmt.Println("Successfully fetched all URLs.")
    }
}

It looks very similar, here are the differences:

First, create the group:

var wg sync.WaitGroup

// VVV BECOMES VVV

g := new(errgroup.Group)

Next, instead of adding every goroutine to the group, call g.Go with the function to be a goroutine. The only requirement is it must have the following signature: func() error. Also, since the ErrGroup will handle when goroutines are completed, there is no need to call wg.Done().

go func(arg string) {
            // Decrement the counter when the goroutine completes.
            defer wg.Done()
            // ... work that can return error here
}(arg)

// VVV BECOMES VVV

g.Go(func() error {
  // ... work that can return error here
})

Finally, wait for the group to finish and handle the errors as needed:

wg.Wait()

// VVV BECOMES VVV

if err := g.Wait(); err == nil {
        fmt.Println("Successfully fetched all URLs.")
}

ErrGroups provide lots of opportunities on handling errors in goroutines.

That being said, ErrGroup is just another tool in the toolbox that should be used when the use case fits.
If some more complex decisions and work needs to be made based off of the errors, a channel is probably better fit.

What do you think?

Blog

Handling errors concurrently in Go with ErrGroup

Boston Cartwright

Preface

ErrGroup

WaitGroup

ErrGroup

Join Our Newsletter. No Spam, Only the good stuff.

Related