Go has the feature that concurrency can be easily described by goroutine (Though it is not always easy to get it to worked correctly).

The problem at this point is error handling, which is a major issue in actual application implementation.

• If an error occurs in one of the goroutines, we want to suspend the other goroutines.
• Wait until each goroutine finishes processing, and handle each error individually.

I think that these requirements often occur.
Of course, it is possible to implement these processes on your own, but there are libraries that are suitable for each of these requirements, and we would like to introduce them here along with sample notation.

Cases and usage for which errgroup is suitable

The golang.org/x/sync/errgroup provided as an official Go experimental package focuses primarily on handling single errors and is suitable for use similar to the standard sync package or for implementing processes that require interruption on error.

Continue to the end even if an error occurs

Group structure and pass the actual function to the .Go() method to start the goroutine, and similarly, .Wait() can be used to wait for the goroutine and retrieve errors. Unlike the sync package, the number of goroutine launches is handled internally, so you don't need to be particularly aware of it. In this usage, it is like a convenient sync package.

In the following implementation, the first goroutine displays 1 and returns an error after 100 ms of waiting, and the second displays 2 and returns an error after 200 ms of waiting.

Since the specification of errgroup is that it only returns the first error that occurs, the second and subsequent errors are lost and cannot be handled.

And since goroutine waits until all processing is completed, both signals 1 and 2, which are signals that processing has been completed, are displayed, but only errgroup: err1, which was returned first, is displayed for errors.

func errGroup() {
    var eg errgroup.Group

    eg.Go(func() error {
        time.Sleep(100 * time.Millisecond)
        fmt.Println("1")
        return fmt.Errorf("errgroup: err1")
    })
    eg.Go(func() error {
        time.Sleep(200 * time.Millisecond)
        fmt.Println("2")
        return fmt.Errorf("errgroup: err2")
    })

    if err := eg.Wait(); err != nil {
        fmt.Println(err.Error())
    }
}

$ go run main.go 
1
2
errgroup: err1

Interruption of other goroutine

In the previous example, it was a simple wait + retrieval of prior errors, similar to the standard sync implementation, but it is also possible to suspend a goroutine that has not generated any errors using context. In practical use, this mechanism may be useful, for example, in cases where multiple APIs are being queried simultaneously, but if any one of them fails, the whole process will be interrupted.

The following implementation incorporates context handling into the previous process to allow interruption. 100ms or 200ms wait and display 1 or 2 as a sign of completion is not changed, but ctx.Done() is also waiting at the same time, so if cancel() is invoked, it will be forced to interrupt at that point.

When initialized with errgroup.WithContext(), it uses context.WithCancel(ctx) internally to return the context to the caller. Furthermore, it keeps a cancel function, which is triggered when an error occurs.

https://cs.opensource.google/go/x/sync/+/7fc1605a:errgroup/errgroup.go;l=45

When executed, the first goroutine prints 1 and returns an error after 100ms of waiting. Since cancel() is executed at the same time, the second goroutine aborts execution before the end of 200ms, resulting in a context canceled.

func errGroupCtx() {
    ctx := context.Background()
    eg, ctx := errgroup.WithContext(ctx)

    eg.Go(func() error {
        select {
        case <-time.After(100 * time.Millisecond):
            fmt.Println("1")
            return fmt.Errorf("errgroupctx: err1")
        case <-ctx.Done():
            fmt.Println("errgroupctx: ctx done1", ctx.Err())
            return ctx.Err()
        }
    })
    eg.Go(func() error {
        select {
        case <-time.After(200 * time.Millisecond):
            fmt.Println("2")
            return nil
        case <-ctx.Done():
            fmt.Println("errgroupctx: ctx done2", ctx.Err())
            return ctx.Err()
        }
    })

    if err := eg.Wait(); err != nil {
        fmt.Println(err.Error())
    }
}

$ go run main.go
1
errgroupctx: ctx done2 context canceled
errgroupctx: err1

Cases and usage for which go-multierror is suitable

Unlike errgroup, github.com/hashicorp/go-multierror cannot be interrupted by context. However, it is useful for use cases where you need to check all errors and handle them carefully, because you can keep all errors.

Summarize of all errors that occur in goroutine.

go-multierror.Group type is used to control goroutine in the same way as for errgroup.

The following code returns an error after waiting 100ms or 200ms as in the previous example, but unlike the errgroup example, the second error is also logged. To retrieve the second error, refer to the .Errors member and get by the range statement.

The go-multierror.Error type returned by .Wait(), which is a goroutine Wait(), can be returned as a normal error because it implements the error interface. The .Error() method is also kindly designed to output the error in a somewhat human readable format (formatting can also be specified, though this is omitted this time).

import (
    multierror "github.com/hashicorp/go-multierror"
)

func goMultierror() {
    var meg multierror.Group

    meg.Go(func() error {
        time.Sleep(100 * time.Millisecond)
        return fmt.Errorf("multierror: err1")
    })
    meg.Go(func() error {
        time.Sleep(200 * time.Millisecond)
        return fmt.Errorf("multierror: err2")
    })

    merr := meg.Wait()
    for _, err := range merr.Errors {
        fmt.Println(err.Error())
    }
    fmt.Println(merr.Error())
}

multierror: err1
multierror: err2
2 errors occurred:
        * multierror: err1
        * multierror: err2

To handle multiple errors at once

Although it is out of the scope of concurrency processing, it is possible to simply append errors like a slice instead of the standard error wrapping and keep the errors in an easy-to-handle format.

Even if the error to be appended is an original type, it can be handled with errors.As errors.Is of the standard package if the Unwrap method is implemented (of course, the same applies to errors returned from goroutine).

Furthermore, if a function returns a go-multierror.Error type as an error type, you can slice and dice it out by making a type assertion.

type ErrX struct {
    err error
    msg string
}

func (ex *ErrX) Error() string {
    return ex.msg
}

func (ex *ErrX) Unwrap() error {
    return ex.err
}

type ErrY struct {
    err error
    msg string
}

func (ey *ErrY) Error() string {
    return ey.msg
}

func (ey *ErrY) Unwrap() error {
    return ey.err
}

func goMultierrorAppend() {
    var err error

    errX := &ErrX{err: nil, msg: "multierror-append: err1"}
    errY := &ErrY{err: nil, msg: "multierror-append: err2"}
    err = multierror.Append(err, errX)
    err = multierror.Append(err, errY)

    merr, ok := err.(*multierror.Error)
    if !ok {
        fmt.Println("failed to assert multierror")
        return
    }
    for _, err := range merr.Errors {
        fmt.Println(err.Error())
    }
    fmt.Println(merr.Error())

    fmt.Println(errors.As(err, &errX))
    fmt.Println(errors.Is(err, errY))
}

multierror-append: err1
multierror-append: err2
2 errors occurred:
        * multierror-append: err1
        * multierror-append: err2


true
true

Conclusion

• If you do not need to handle all errors, or if you want to suspend goroutine on error, errgroup is suitable.
• go-multierror is suitable if you want to handle all errors that occur.