TheLazy.Dev
Portfolio GitHub

Blog

go |temporal |docker |microservices

Building more reliable applications with Temporal

ifedayo

Ifedayo Adesiyan

Posted on February 23, 2023

Building more reliable applications with Temporal

The Temporal microservice orchestrator is a key component of the Temporal platform that provides coordination and management of microservices as they interact with one another to complete complex workflows. The orchestrator provides features such as task management, error handling, retries, and compensation to ensure that workflows are executed accurately and consistently, even in the face of failures or unexpected events.
Using Temporal in application development can help simplify the development of complex, time-sensitive workflows and improve the reliability and scalability of the applications. By using the Temporal orchestrator, developers can focus on the implementation of their business logic, and leave the coordination and management of the underlying microservices to the platform.

Illustration

Microservices architecture, for example, is used in company XYZ. The microservices approach to software development helps their teams deploy faster, but it comes with some issues, one of which is data consistency. How can data changes in microservice A be propagated to microservices B and C? send it through an event?
Yes, that works, but what if B updates itself and C had hiccups and just could not make the update?
Then that means we need to have a mechanism that allows us to handle such failures, make retries, and what else? How many situations like the one described above require us to write failure and retry logic?
Hence, the use of Temporal as a microservice orchestrator helps us solve the issues stated above.

Prerequisites

The prerequisite for this tutorial is having Go and Docker installed and configured.
Verify that you've installed Go by opening a command prompt and typing the following command



go version
Enter fullscreen mode Exit fullscreen mode

It should return something similar to this



go version go1.19.3 darwin/amd64
Enter fullscreen mode Exit fullscreen mode

Also, verify docker is installed with



docker --version
#Docker version 20.10.22, build 3a2c30b
Enter fullscreen mode Exit fullscreen mode

Choice

Temporal can be used inside and outside your project directory, as long as your application can access it. Your individual needs and the project's infrastructure will determine your choice.
It may be simpler to maintain your project's dependencies and setup if you decide to execute Temporal inside of your project directory because everything is contained there. However, running Temporal outside of your project directory might improve the degree of concern separation and make it simpler to manage numerous projects that share a single Temporal instance.
However, in this guide, we will run the temporal server inside our project directory.

Hello-workflow

We will be using the simple hello-workflow to orchestrate tasks within a distributed system. The starter and workers are the two main components that enable the distributed system to run, and in this guide, we will be implementing the starter and worker.
Here's what our project directory will look like.



├── dynamicconfig
│ ├── development-cass.yaml
│ └── development-sql.yaml
| └── docker.yaml
|
├── src
│ ├── helloworkflow 
| |        └── workflow.go
│ ├── starter
| |        └── main.go
| └── worker
|          └── main.go
├── .env
├── docker-compose.yml
Enter fullscreen mode Exit fullscreen mode

You create a new Go project, say tutorial-guide and cd into it, then open the directory on your text editor. You create dynamicconfig folder in the root directory of your project and add the 3 yaml files. These files are needed to have the temporal-server up.



# development-cass.yaml
system.forceSearchAttributesCacheRefreshOnRead:
  - value: true # Dev setup only. Please don't turn this on in production.
    constraints: {}
Enter fullscreen mode Exit fullscreen mode 


# development-sql.yaml
limit.maxIDLength:
  - value: 255
    constraints: {}
system.forceSearchAttributesCacheRefreshOnRead:
  - value: true # Dev setup only. Please don't turn this on in production.
    constraints: {}
Enter fullscreen mode Exit fullscreen mode 


# docker.yaml
apiVersion: v1
clusters:
  - cluster:
      certificate-authority: path-to-your-ca.rt #e.g/System/Volumes/Data/Users/<name>/.minikube/ca.crt
      server: https://192.168.99.100:8443
    name: minikube
contexts:
  - context:
      cluster: minikube
      user: minikube
    name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
  - name: minikube
    user:
      client-certificate: path-to-proxy-client-ca.key #e.g/System/Volumes/Data/Users/<name>/.minikube/proxy-client-ca.key
      client-key: path-to-proxy-client-ca.key #e.g/System/Volumes/Data/Users/<name>/.minikube/proxy-client-ca.key
Enter fullscreen mode Exit fullscreen mode

Use docker.yaml file to override the default dynamic config value (they are specified when creating the service config). More information about the docker.yaml file and how to use it [https://github.com/temporalio/docker-compose/tree/main/dynamicconfig]



# docker-compose.yaml
version: '3.5'

services:
  elasticsearch:
    container_name: temporal-elasticsearch
    environment:
      - cluster.routing.allocation.disk.threshold_enabled=true
      - cluster.routing.allocation.disk.watermark.low=512mb
      - cluster.routing.allocation.disk.watermark.high=256mb
      - cluster.routing.allocation.disk.watermark.flood_stage=128mb
      - discovery.type=single-node
      - ES_JAVA_OPTS=-Xms256m -Xmx256m
      - xpack.security.enabled=false
    image: elasticsearch:${ELASTICSEARCH_VERSION}
    networks:
      - temporal-network
    expose:
      - 9200
    volumes:
      - /var/lib/elasticsearch/data
  postgresql:
    container_name: temporal-postgresql
    environment:
      POSTGRES_PASSWORD: temporal
      POSTGRES_USER: temporal
    image: postgres:${POSTGRESQL_VERSION}
    networks:
      - temporal-network
    expose:
      - 5432
    volumes:
      - /var/lib/postgresql/data
  temporal:
    container_name: temporal
    depends_on:
      - postgresql
      - elasticsearch
    environment:
      - DB=postgresql
      - DB_PORT=5432
      - POSTGRES_USER=temporal
      - POSTGRES_PWD=temporal
      - POSTGRES_SEEDS=postgresql
      - DYNAMIC_CONFIG_FILE_PATH=config/dynamicconfig/development-sql.yaml
      - ENABLE_ES=true
      - ES_SEEDS=elasticsearch
      - ES_VERSION=v7
    image: temporalio/auto-setup:${TEMPORAL_VERSION}
    networks:
      - temporal-network
    ports:
      - 7233:7233
    labels:
      kompose.volume.type: configMap
    volumes:
      - ./dynamicconfig:/etc/temporal/config/dynamicconfig
  temporal-admin-tools:
    container_name: temporal-admin-tools
    depends_on:
      - temporal
    environment:
      - TEMPORAL_CLI_ADDRESS=temporal:7233
    image: temporalio/admin-tools:${TEMPORAL_VERSION}
    networks:
      - temporal-network
    stdin_open: true
    tty: true
  temporal-ui:
    container_name: temporal-ui
    depends_on:
      - temporal
    environment:
      - TEMPORAL_ADDRESS=temporal:7233
      - TEMPORAL_CORS_ORIGINS=http://localhost:3000
    image: temporalio/ui:${TEMPORAL_UI_VERSION}
    networks:
      - temporal-network
    ports:
      - 8080:8080

networks:
  temporal-network:
    driver: bridge
    name: temporal-network
Enter fullscreen mode Exit fullscreen mode

Add the environment variables needed for the docker-compose.yaml file.



# .env
COMPOSE_PROJECT_NAME=temporal
CASSANDRA_VERSION=3.11.9
ELASTICSEARCH_VERSION=7.16.2
MYSQL_VERSION=8
POSTGRESQL_VERSION=13
TEMPORAL_VERSION=1.19.1
TEMPORAL_UI_VERSION=2.9.1
Enter fullscreen mode Exit fullscreen mode

The source code for the workflow that the Temporal Server runs is located in the workflow.go file. It is in charge of generating activities, monitoring their completion, and controlling the workflow. It specifies every action the workflow must do and is started by an event, such as a message from a queue or the addition of a new data item to the system.



// workflow.go
package helloworkflow

import (
 "context"
 "time"

 "go.temporal.io/sdk/workflow"
)

func Workflow(ctx workflow.Context, name string) (string, error) {
 ao := workflow.ActivityOptions{
  ScheduleToStartTimeout: time.Minute,
  StartToCloseTimeout:    time.Minute,
 }

 ctx = workflow.WithActivityOptions(ctx, ao)

 logger := workflow.GetLogger(ctx)

 var result string
 err := workflow.ExecuteActivity(ctx, Activity, name).Get(ctx, &result)
 if err != nil {
  logger.Error("Activity failed", "Error", err)
 }

 return result, nil
}

func Activity(ctx context.Context, name string) (string, error) {
 return "Hello " + name, nil
}
Enter fullscreen mode Exit fullscreen mode

The starter is responsible for coordinating the workflow executions within the system. It is responsible for scheduling the tasks to be executed and keeping the task statuses up to date. The starter also acts as a gateway for users to interact with the system via its API.
Now, we add these to our main.go file under the starter directory.



// starter/main.go
package main

import (
 "context"
 "log"

 "github.com/theifedayo/hello-workflow/src/helloworkflow"
 "go.temporal.io/sdk/client"
)

func main() {
 c, err := client.NewClient(client.Options{

 })
 if err != nil {
  log.Fatalln("Unable to make client", err)
 }

 defer c.Close()

 workflowOptions := client.StartWorkflowOptions{
  ID:        "hello_world_workflowID",
  TaskQueue: "hello-world",
 }

 we, err := c.ExecuteWorkflow(context.Background(), workflowOptions, helloworkflow.Workflow, "ifedayo")
 if err != nil {
  log.Fatalln("Unable to execute workflow", err)
 }

 var result string
 // store the result of the run
 err = we.Get(context.Background(), &result)
 if err != nil {
  log.Fatalln("Unable to get workflow result", err)
 }
 log.Println("workflow result:", result)
}
Enter fullscreen mode Exit fullscreen mode

Workers are responsible for executing the tasks. These workers are deployed across different nodes in the system, and their job is to receive the tasks from the starter and execute them. The workers are also responsible for sending the results back to the starter. The Workers can be scaled up or down depending on the workload.



// worker/main.go
package main

import (
 "log"

 "github.com/theifedayo/hello-workflow/src/helloworkflow"
 "go.temporal.io/sdk/client"
 "go.temporal.io/sdk/worker"
)

func main() {
 c, err := client.NewClient(client.Options{})
 if err != nil {
  log.Fatalln("Unable to make client", err)
 }

 defer c.Close()

 w := worker.New(c, "hello-world", worker.Options{})
 w.RegisterWorkflow(helloworkflow.Workflow)
 w.RegisterActivity(helloworkflow.Activity)

 err = w.Run(worker.InterruptCh())
 if err != nil {
  log.Fatalln("Unable to start workflow", err)
 }
}
Enter fullscreen mode Exit fullscreen mode

Starting Everything Up

To start our workflow, we need our temporal server up. Go to your terminal and cd to your project directory and run



docker-compose up
Enter fullscreen mode Exit fullscreen mode

screenshot of docker services up
This will start up all the services in our docker-compose.yaml.
Next up, we start our worker.
In general, you should start the workers before the starter, as the starter typically depends on the workers being available to perform their tasks. This means that the workers should be started and initialized before the starter begins to coordinate their activities.



go run src/worker/main.go
Enter fullscreen mode Exit fullscreen mode 


2023/02/16 14:03:42 INFO  No logger configured for temporal client. Created default one.
2023/02/16 14:03:43 INFO  Started Worker Namespace default TaskQueue hello-world WorkerID 4465@Ifedayos-MacBook-Pro.local@
Enter fullscreen mode Exit fullscreen mode

And finally, we start the workflow




go run src/starter/main.go
Enter fullscreen mode Exit fullscreen mode 



2023/02/16 14:05:52 INFO  No logger configured for temporal client. Created default one.

2023/02/16 14:05:52 workflow result: Hello ifedayo
Enter fullscreen mode Exit fullscreen mode




Viewing Workflows

Navigating to the browser on localhost:8080, we have a UI that gives more information about the workflow

screenshot of list of workflows

screenshot of detail of a selected workflow
Temporal also provides a CLI tool for interacting with the Temporal server, tctl, which also performs various administrative tasks, such as starting and stopping workflows, querying workflow information, and managing workflow executions.




tctl workflow list
Enter fullscreen mode Exit fullscreen mode 



WORKFLOW TYPE |      WORKFLOW ID       |                RUN ID                | TASK QUEUE  | START TIME | EXECUTION TIME | END TIME


Workflow      | hello_world_workflowID | 0454098f-cdd9-4f64-af8b-ab77a6f86c35 | hello-world | 13:05:52   | 13:05:52       | 13:05:52
Enter fullscreen mode Exit fullscreen mode




Conclusion

In this writing, we've learned how what temporal is, why use it, an illustration of using it in a company, choices of setting up a temporal server, how to start the server inside your go project directory, running workflow and worker, and finally viewing your workflows in a UI or CLI.

Cheers 🥂! You're one step closer to building more reliable applications with Temporal.

💖 💪 🙅 🚩
ifedayo
Ifedayo Adesiyan

Posted on February 23, 2023

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Building more reliable applications with Temporal
go Building more reliable applications with Temporal

February 23, 2023