How to write a microservice in Go with Go kit

I've searched a lot on the Internet (I consider my google-fu quite good) for some well written tutorial about writing simple "RESTful" microservices in Go with Go kit.

I was not able to find any...

Examples from the Go kit repository are good, but documentation is dull, IMHO.

I then decided to buy this book, called Go Programming Blueprints, 2nd Edition which is quite good, but has only two chapters dedicated to Go kit (one for actual development of a microservice and one for the actual deploy) and I don't really care for now about gRPC, which examples on chapter 10 of the book are also implementing. To much scaffolding code if you ask me :P

Sooo, I decided to give back something to the community and write a tutorial, in order to "learn by doing". This tutorial will be heavily inspired by the book mentioned above, and probably could be improved in many ways.

Feel free to give a feedback!

You will find the link to the complete source code for the microservice on my blog, the original source of this article. coding.napolux.com

What is Go kit?

From the Go kit README.md:

Go kit is a programming toolkit for building microservices (or elegant monoliths) in Go. We solve common problems in distributed systems and application architecture so you can focus on delivering business value

[...]

Go is a great general-purpose language, but microservices require a certain amount of specialized support. RPC safety, system observability, infrastructure integration, even program design — Go kit fills in the gaps left by the standard library, and makes Go a first-class language for writing microservices in any organization.

I don't want to discuss much about it: I'm too new to Go to have a strong opinion. The community is of course divided in the ones that like it and the ones that don't. You can also find a good article about differences in frameworks for Go microservices here.

What are we going to do?

We are going to create a very basic microservice that will return and validate a date... The goal is understand how Go kit works, nothing more than that. You can easily replicate all the logic without Go kit, but I'm here to learn, and so...

I hope you'll have a glimpse and a good starting point for your next projects!

Our microservice will have some endpoints.

One GET endpoint at /status that will return a simple answer that will confirm that the microservice is up and running
One GET endpoint at /get that will return today's date
One POST endpoint at /validate that will receive a date string in the dd/mm/yyyy (the only existing date format if you ask me, take this USA!) format and validate it according to a simple regex...

Again, you will find the link to the complete source code for the microservice on my blog, the original source of this article. coding.napolux.com

Let's start!!!

Prerequisites

You should have Golang installed & working on your machine. I've found the official download package working better (I had some problems with the env. vars) than the Homebrew installation on my Macbook.

Plus, you should know the Go language, I'm not going to explain what a struct is, for example.

The napodate microservice

Ok, let's start by creating a new folder in our $GOPATH folder called napodate. This will also be the name of our package.

Put a service.go file there. Let's add our service interface on top of the file.

package napodate

import "context"

// Service provides some "date capabilities" to your application
type Service interface {
    Status(ctx context.Context) (string, error)
    Get(ctx context.Context) (string, error)
    Validate(ctx context.Context, date string) (bool, error)
}

Here we're defining the "blueprint" for our service: in Go kit, you have to model a service as an interface. As described above, we will need three endpoints that will be mapped, going forward, to this interface.

Why are we using the context package? Read https://blog.golang.org/context:

At Google, we developed a context package that makes it easy to pass request-scoped values, cancelation signals, and deadlines across API boundaries to all the goroutines involved in handling a request

Basically, this is needed because our microservice should be made from the beginning to handle concurrent requests and a context for every request is mandatory.

You don't want to mix up stuff. More on this later in the tutorial. We are not using that much now, but get used to it! :P We now have our microservice interface.

Implementing our service

As you probably know, an interface is nothing without an implementation, so let's implement our service. Let's add some more code to service.go.

type dateService struct{}

// NewService makes a new Service.
func NewService() Service {
    return dateService{}
}

// Status only tell us that our service is ok!
func (dateService) Status(ctx context.Context) (string, error) {
    return "ok", nil
}

// Get will return today's date
func (dateService) Get(ctx context.Context) (string, error) {
    now := time.Now()
    return now.Format("02/01/2006"), nil
}

// Validate will check if the date today's date
func (dateService) Validate(ctx context.Context, date string) (bool, error) {
    _, err := time.Parse("02/01/2006", date)
    if err != nil {
        return false, err
    }
    return true, nil
}

The newly defined type dateService (an empty struct) is how we are going to group together the methods of our service, while "hiding" somehow the implementation to the rest of the world.

See NewService() as a constructor for our "object". This is what we'll call to get an instance of our service, all while masking the internal logic as good programmers should do.

Let's write a test

A good example of how NewService() is used can be seen in the test for our service. Go on and create a service_test.go file.

package napodate

import (
    "context"
    "testing"
    "time"
)

func TestStatus(t *testing.T) {
    srv, ctx := setup()

    s, err := srv.Status(ctx)
    if err != nil {
        t.Errorf("Error: %s", err)
    }

    // testing status
    ok := s == "ok"
    if !ok {
        t.Errorf("expected service to be ok")
    }
}

func TestGet(t *testing.T) {
    srv, ctx := setup()
    d, err := srv.Get(ctx)
    if err != nil {
        t.Errorf("Error: %s", err)
    }

    time := time.Now()
    today := time.Format("02/01/2006")

    // testing today's date
    ok := today == d
    if !ok {
        t.Errorf("expected dates to be equal")
    }
}

func TestValidate(t *testing.T) {
    srv, ctx := setup()
    b, err := srv.Validate(ctx, "31/12/2019")
    if err != nil {
        t.Errorf("Error: %s", err)
    }

    // testing that the date is valid
    if !b {
        t.Errorf("date should be valid")
    }

    // testing an invalid date
    b, err = srv.Validate(ctx, "31/31/2019")
    if b {
        t.Errorf("date should be invalid")
    }

    // testing a USA date date
    b, err = srv.Validate(ctx, "12/31/2019")
    if b {
        t.Errorf("USA date should be invalid")
    }
}

func setup() (srv Service, ctx context.Context) {
    return NewService(), context.Background()
}

I made the tests more readable, but you should really write them with Subtests, for a more up-to-date syntax.

Tests are green (!) but focus for a second on the setup() method. For every test we return an instance of our service using NewService() and the context.

Transports

Our service will be exposed using HTTP. We are now going to model the accepted HTTP requests and responses. Go and create a file called transport.go in the same folder of service.go.

package napodate

import (
    "context"
    "encoding/json"
    "net/http"
)

// In the first part of the file we are mapping requests and responses to their JSON payload.
type getRequest struct{}

type getResponse struct {
    Date string `json:"date"`
    Err  string `json:"err,omitempty"`
}

type validateRequest struct {
    Date string `json:"date"`
}

type validateResponse struct {
    Valid bool   `json:"valid"`
    Err   string `json:"err,omitempty"`
}

type statusRequest struct{}

type statusResponse struct {
    Status string `json:"status"`
}

// In the second part we will write "decoders" for our incoming requests
func decodeGetRequest(ctx context.Context, r *http.Request) (interface{}, error) {
    var req getRequest
    return req, nil
}

func decodeValidateRequest(ctx context.Context, r *http.Request) (interface{}, error) {
    var req validateRequest
    err := json.NewDecoder(r.Body).Decode(&req)
    if err != nil {
        return nil, err
    }
    return req, nil
}

func decodeStatusRequest(ctx context.Context, r *http.Request) (interface{}, error) {
    var req statusRequest
    return req, nil
}

// Last but not least, we have the encoder for the response output
func encodeResponse(ctx context.Context, w http.ResponseWriter, response interface{}) error {
    return json.NewEncoder(w).Encode(response)
}

You will find the link to the complete source code for the microservice on my blog, the original source of this article. coding.napolux.com

A bit of code if you ask me, but you'll find comments in the transport.go in the repo file that will help you navigate it.

In the first part of the file we are mapping requests and responses to their JSON payload. For statusRequest and getRequest we don't need much, since no payload is sent to the server. For validateRequest we are going to pass a date to be validated, so here is the date field.

Responses are pretty straightforward too.

In the second part we will write "decoders" for our incoming requests, telling the service how he should translate requests and map them to the correct request struct. get and status are empty, I know, but they're there for the sake of completeness. Remember, I'm learning by doing...

Last but not least, we have the encoder for the response output, which is a simple JSON encoder: given an object, we'll return a JSON object from it.

That's it for transports, let's create our endpoints!

Endpoints

Let's create a new file endpoint.go. This file will contain our endpoints that will map the request coming from the client to our internal service

package napodate

import (
    "context"
    "errors"

    "github.com/go-kit/kit/endpoint"
)

// Endpoints are exposed
type Endpoints struct {
    GetEndpoint      endpoint.Endpoint
    StatusEndpoint   endpoint.Endpoint
    ValidateEndpoint endpoint.Endpoint
}

// MakeGetEndpoint returns the response from our service "get"
func MakeGetEndpoint(srv Service) endpoint.Endpoint {
    return func(ctx context.Context, request interface{}) (interface{}, error) {
        _ = request.(getRequest) // we really just need the request, we don't use any value from it
        d, err := srv.Get(ctx)
        if err != nil {
            return getResponse{d, err.Error()}, nil
        }
        return getResponse{d, ""}, nil
    }
}

// MakeStatusEndpoint returns the response from our service "status"
func MakeStatusEndpoint(srv Service) endpoint.Endpoint {
    return func(ctx context.Context, request interface{}) (interface{}, error) {
        _ = request.(statusRequest) // we really just need the request, we don't use any value from it
        s, err := srv.Status(ctx)
        if err != nil {
            return statusResponse{s}, err
        }
        return statusResponse{s}, nil
    }
}

// MakeValidateEndpoint returns the response from our service "validate"
func MakeValidateEndpoint(srv Service) endpoint.Endpoint {
    return func(ctx context.Context, request interface{}) (interface{}, error) {
        req := request.(validateRequest)
        b, err := srv.Validate(ctx, req.Date)
        if err != nil {
            return validateResponse{b, err.Error()}, nil
        }
        return validateResponse{b, ""}, nil
    }
}

// Get endpoint mapping
func (e Endpoints) Get(ctx context.Context) (string, error) {
    req := getRequest{}
    resp, err := e.GetEndpoint(ctx, req)
    if err != nil {
        return "", err
    }
    getResp := resp.(getResponse)
    if getResp.Err != "" {
        return "", errors.New(getResp.Err)
    }
    return getResp.Date, nil
}

// Status endpoint mapping
func (e Endpoints) Status(ctx context.Context) (string, error) {
    req := statusRequest{}
    resp, err := e.StatusEndpoint(ctx, req)
    if err != nil {
        return "", err
    }
    statusResp := resp.(statusResponse)
    return statusResp.Status, nil
}

// Validate endpoint mapping
func (e Endpoints) Validate(ctx context.Context, date string) (bool, error) {
    req := validateRequest{Date: date}
    resp, err := e.ValidateEndpoint(ctx, req)
    if err != nil {
        return false, err
    }
    validateResp := resp.(validateResponse)
    if validateResp.Err != "" {
        return false, errors.New(validateResp.Err)
    }
    return validateResp.Valid, nil
}

Let's dig a bit into this... In order to expose all our service methods Get(), Status() and Validate() as endpoints we are going to write functions that will handle the incoming requests, call the corresponding service method and depending on the response will build and return a proper object.

These methods are the Make... ones. They will receive the servuce as argument, whe then use a type assertion to "force" the request type to a specific one and use it to call the service method for it.

After these Make... methods, that will be used in the main.go file, we will write the endpoints to comply with the service interface

type Endpoints struct {
    GetEndpoint      endpoint.Endpoint
    StatusEndpoint   endpoint.Endpoint
    ValidateEndpoint endpoint.Endpoint
}

Let's make an example:

// Status endpoint mapping
func (e Endpoints) Status(ctx context.Context) (string, error) {
    req := statusRequest{}
    resp, err := e.StatusEndpoint(ctx, req)
    if err != nil {
        return "", err
    }
    statusResp := resp.(statusResponse)
    return statusResp.Status, nil
}

This method will allow us to use the endpoints as Go methods.

The HTTP server

For our microservice we need an HTTP server. Go is pretty helpful with this, but I chose https://github.com/gorilla/mux for our routes since its syntax looks so clean and concise, so let's create a little cute HTTP server with mappings to our endpoints.

Create a new file called server.go in your project.

package napodate

import (
    "context"
    "net/http"

    httptransport "github.com/go-kit/kit/transport/http"
    "github.com/gorilla/mux"
)

// NewHTTPServer is a good little server
func NewHTTPServer(ctx context.Context, endpoints Endpoints) http.Handler {
    r := mux.NewRouter()
    r.Use(commonMiddleware) // @see https://stackoverflow.com/a/51456342

    r.Methods("GET").Path("/status").Handler(httptransport.NewServer(
        endpoints.StatusEndpoint,
        decodeStatusRequest,
        encodeResponse,
    ))

    r.Methods("GET").Path("/get").Handler(httptransport.NewServer(
        endpoints.GetEndpoint,
        decodeGetRequest,
        encodeResponse,
    ))

    r.Methods("POST").Path("/validate").Handler(httptransport.NewServer(
        endpoints.ValidateEndpoint,
        decodeValidateRequest,
        encodeResponse,
    ))

    return r
}

func commonMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        w.Header().Add("Content-Type", "application/json")
        next.ServeHTTP(w, r)
    })
}

The endpoints will be passed to the server from the main.go file, and commonMiddleware(), not exposed will take care to add specific headers to every response.

And finally, our main.go file

Let's wrap up! We have a service with endpoints. We have an HTTP server, we just need a place where we can wrap up everything and of course it's our main.go file. Put it into a new folder, let's call it cmd.

package main

import (
    "context"
    "flag"
    "fmt"
    "log"
    "net/http"
    "os"
    "os/signal"
    "syscall"

    "napodate"
)

func main() {
    var (
        httpAddr = flag.String("http", ":8080", "http listen address")
    )
    flag.Parse()
    ctx := context.Background()
    // our napodate service
    srv := napodate.NewService()
    errChan := make(chan error)

    go func() {
        c := make(chan os.Signal, 1)
        signal.Notify(c, syscall.SIGINT, syscall.SIGTERM)
        errChan <- fmt.Errorf("%s", <-c)
    }()

    // mapping endpoints
    endpoints := napodate.Endpoints{
        GetEndpoint:      napodate.MakeGetEndpoint(srv),
        StatusEndpoint:   napodate.MakeStatusEndpoint(srv),
        ValidateEndpoint: napodate.MakeValidateEndpoint(srv),
    }

    // HTTP transport
    go func() {
        log.Println("napodate is listening on port:", *httpAddr)
        handler := napodate.NewHTTPServer(ctx, endpoints)
        errChan <- http.ListenAndServe(*httpAddr, handler)
    }()

    log.Fatalln(<-errChan)
}

Let's analyze this file together. We declare the main package and import what we need.

We use a flag to make the listening port configurable, the default port for our service will be the classic 8080 but we can lunch it with whatever port we wont thanks to the flag.

What follows is the setup of our server: we create a context (see above for an explanation of what a context is) and we get our service. An error channel is also setup.

Channels are the pipes that connect concurrent goroutines. You can send values into channels from one goroutine and receive those values into another goroutine.

We then create two goroutines. One to stop the server when we press CTRL+C and one that will actually listen for incoming requests.

Look at handler := napodate.NewHTTPServer(ctx, endpoints) this handler will map our service endpoints (do you remember the Make... methods from above?) and return the correct answer.

Where did you see NewHTTPServer() before?

As soon as the channel receives an error message, the server will stop and die.

Let's the service!

If you did everything correctly, by running

go run cmd/main.go

from your project folder, you should be able to curl your microservice!

curl http://localhost:8080/get
{"date":"14/04/2019"}

curl http://localhost:8080/status
{"status":"ok"}

curl -XPOST -d '{"date":"32/12/2020"}' http://localhost:8080/validate
{"valid":false,"err":"parsing time \"32/12/2020\": day out of range"}

curl -XPOST -d '{"date":"12/12/2021"}' http://localhost:8080/validate
{"valid":true}

Wrapping up

We created a new microservice from scratch, even if it's super simple, it's a good way to start using Go kit with the Go programming language.

Hope you enjoyed this tutorial as much as I did!

You will find the link to the complete source code for the microservice on my blog, the original source of this article. coding.napolux.com

Blog

How to write a microservice in Go with Go kit

Francesco Napoletano

Feel free to give a feedback!

What is Go kit?

What are we going to do?

Prerequisites

The napodate microservice

Implementing our service

Let's write a test

Transports

Endpoints

The HTTP server

And finally, our main.go file

Let's the service!

Wrapping up

Join Our Newsletter. No Spam, Only the good stuff.

Related