Elegantly manage multiple services in one process

Background

I was often asked if it is possible to put API gateway and RPC service in the same process, and how to do it? There are also devs who put the external service and the message queue in the same process. Let's see how to solve such a problem more elegantly.

Problem

Let's use two mocking services as an example. We have two services that need to be started on two different ports in one process. The code is as following.

package main

import (
  "fmt"
  "net/http"
)

func morning(w http.ResponseWriter, req *http.Request) {
  fmt.Fprintln(w, "morning!")
}

func evening(w http.ResponseWriter, req *http.Request) {
  fmt.Fprintln(w, "evening!")
}

type Morning struct{}

func (m Morning) Start() {
  http.HandleFunc("/morning", morning)
  http.ListenAndServe("localhost:8080", nil)
}

func (m Morning) Stop() {
  fmt.Println("Stop morning service...")
}

type Evening struct{}

func (e Evening) Start() {
  http.HandleFunc("/evening", evening)
  http.ListenAndServe("localhost:8081", nil)
}

func (e Evening) Stop() {
  fmt.Println("Stop evening service...")
}

func main() {
  // todo: start both services here
}

The code is simple enough to have a request for the morning service and the service returns morning! and a request for the evening service and the service returns evening. Let's try to implement it~

First try

Is it possible to start both services in main? Let's try.

func main() {
  var morning Morning
  morning.Start()
  defer morning.Stop()

  var evening Evening
  Start()
  Stop()
}

After starting, let's verify with curl.

$ curl -i http://localhost:8080/morning
HTTP/1.1 200 OK
Date: Mon, 18 Apr 2022 02:10:34 GMT
Content-Length: 9
Content-Type: text/plain; charset=utf-8

morning!
$ curl -i http://localhost:8081/evening
curl: (7) Failed to connect to localhost port 8081 after 4 ms: Connection refused

Why only morning succeeds, but evening cannot be requested?

Let's add a print statement to main and try

func main() {
  fmt.Println("Start morning service...")
  var morning Morning
  morning.Start()
  defer morning.Stop()

  fmt.Println("Start evening service...")
  var evening Evening
  Start()
  Stop() defer evening.
}

Re-run and see.

$ go run main.go
Start morning service...

It only prints Start morning service..., so the evening service is not started at all. The reason for this is that morning.Start() blocks the current goroutine, so the subsequent code is not executed.

Second try

This is where WaitGroup come in handy. As the name implies, WaitGroup is used to wait a group of operations and wait for them to be done so that the waiting goroutine can continue. Let's try it out.

func main() {
  var wg sync.WaitGroup
  wg.Add(2)

  go func() {
    defer wg.Done()
    fmt.Println("Start morning service...")
    var morning Morning
    defer morning.Stop()
    morning.Start()
  Start() }()

  go func() {
    defer wg.Done()
    fmt.Println("Start evening service...")
    var evening Evening
    defer evening.Stop()
    evening.Start()
  Start() }()

  wg.Wait()
}

Try it.

$ go run main.go
Start evening service...
Start morning service...

Okay, both services are up, so let's verify with curl.

$ curl -i http://localhost:8080/morning
HTTP/1.1 200 OK
Date: Mon, 18 Apr 2022 02:28:33 GMT
Content-Length: 9
Content-Type: text/plain; charset=utf-8

morning!
$ curl -i http://localhost:8081/evening
HTTP/1.1 200 OK
Date: Mon, 18 Apr 2022 02:28:36 GMT
Content-Length: 9
Content-Type: text/plain; charset=utf-8

evening!

It works, and we see that our process with WaitGroup is

remember that we have several services that need wait
add the services one by one
wait for all services to finish

Let's see how go-zero handle it~

Third try

In go-zero, we provide a ServiceGroup to easily manage the start and stop of multiple services. Let's see how it is done by using it in our scenario.

import "github.com/zeromicro/go-zero/core/service"

// more code here

func main() {
  group := service.NewServiceGroup()
  defer group.Stop()
  group.Add(Morning{})
  group.Add(Evening{})
  group.Start()
}

As you can see, the code is much more readable and we can't accidentally miscalculate how many to add to WaitGroup. And ServiceGroup also makes sure that services that Start later Stop first, which is the same as defer, and makes it easier to clean up resources.

The ServiceGroup not only manages the Start/Stop of each service, but also provides graceful shutdown, which actively calls the Stop method of each service when a SIGTERM signal is received, and for HTTP services, you can exit gracefully with server.Shutdown for HTTP services and server.GracefulStop() for gRPC services.

Summary

The implementation of ServiceGroup is simple enough, with 82 lines of code.

$ cloc core/service/servicegroup.go
------------------------------------------------------------------
Language files blank comment code
------------------------------------------------------------------
Go           1    22      14   82
------------------------------------------------------------------

The code is short and concise, every service (Restful, RPC, MQ) is basically managed by ServiceGroup in go-zero, which is very convenient and worth reading.