Go-lang Gorilla API + MongoDB in 30 minutes
myk_okoth_ogodo
Posted on July 24, 2022
Now hold your horses, before you start the thirty minute timer there is some house keeping we need to do first.One, we need to talk about No-SQL database family that MongoDB belongs to,two, we also need to touch on how to install MongoDB in our system (i am assuming you are using one of the Linux distros), then we will integrate this MongoDB database with a simple Gorilla-Mux API.
The code to this article can be found here.
No-SQL(Non-Relational Databases)
These are document-oriented databases that have a different data storing format to SQL database such as MySQL or PostgreSQL.There are various flavors of non-relational databases based on their data model. Main types are document databases like MongoDB, key-value databases like Redis, wide-column databases like Apache and Cassandra and finally graph databases like Orient-DB, Dgraph.
No-SQL databases give developers the freedom to store huge amounts of unstructured data. This is due to the fact that No-SQL databases are less structured or confined in format thus allowing for greater flexibility and adaptability in the type of data stored.
A popular example of an application that uses a No-SQL database would be Facebook's Messenger app that uses an Apache HBase to stream data to Hadoop clusters. Apache HBase is a wide-column database modeled on by Google's Big Table.
Characteristics of No-SQL(non-Relational) Databases:
- Simple to integrate and use with apps for developers
- Allows for Horizontal scaling or Scaling out.
- Allows for Fast Querying
- Have Flexible Schema.
Dive deeper into this topic at this article here.
Installing MongoDB
We first update our apt and install the MongoDB as shown below:
mykmyk@skynet:~$ sudo apt update
then;
mykmyk@skynet:~$ sudo apt-get install mongodb
Now, to confirm if your MongoBD database has installed
correctly, check that by doing the following:
mykmyk@skynet:~$ mongo
MongoDB shell version v3.6.3
connecting to: mongodb://127.0.0.1:27017
MongoDB server version: 3.6.3
Server has startup warnings:
2022-07-23T22:58:19.824+0300 I STORAGE [initandlisten]
2022-07-23T22:58:19.824+0300 I STORAGE [initandlisten] ** WARNING: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine
2022-07-23T22:58:19.824+0300 I STORAGE [initandlisten] ** See http://dochub.mongodb.org/core/prodnotes-filesystem
2022-07-23T22:58:30.835+0300 I CONTROL [initandlisten]
2022-07-23T22:58:30.835+0300 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2022-07-23T22:58:30.835+0300 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2022-07-23T22:58:30.835+0300 I CONTROL [initandlisten]
>
If you ran into errors while you were installing MongoDB, i refer you to this well written article, it should guide sufficiently enough to get the database safely installed on your system.
To check the databases that are avaible in our MongoDB DMS:
> show databases;
admin 0.000GB
appdb 0.000GB
config 0.000GB
local 0.000GB
test 0.000GB
The above command "show databases" lists all available databases , by default,admin,test, and local are the three that should be available before you create any new database;
We are going to create a database called, "booksdb", so while still in your MongoDB shell:
> use booksdb;
switched to db booksdb
The command "use booksdb" above switches from the current database to the booksdb database that we just created. Its interesting to note that the "booksdb" wont show up if we query all the databases available using the "show databases" command. Go ahead and try it.
> use booksdb;
switched to db booksdb
> show databases;
admin 0.000GB
appdb 0.000GB
config 0.000GB
local 0.000GB
test 0.000GB
The above phenomenon results from the internal implementation of MongoDB,a database is only created in MongoDB when actual initial data is inserted into it , so lets go ahead and do just that.
still on your MongoDB shell type in:
> db.books.insertOne({_id:5, title: 'A Song Of Ice and Fire',authors:['George R.R Martin','Phyllis Eisenstein'],publishdate:'August 1,1996',characters:['Jon Snow','Daenerys Targaryen','Eddard Star','Sansa Stark','Jammie Lannister','Rob Stark','Cerser Lannister'],publisher:{name:'Bantam Books',country:'United States',website:'www.randomhousebooks.com'}})
{ "acknowledged" : true, "insertedId" : 5 }
The json we just inserted has an ID called _id, we could either provide that id while inserting our books object or we could leave it to MongoDB to insert it for us. Lets try and insert a book without specifying an id below and see what MongoDB will do:
> db.books.insertOne({title: 'A Feast Of Crows',authors:['George R.R Martin'],publishdate:'August 1,1996',characters:['Jon Snow','Daenerys Targaryen','Eddard Star','Sansa Stark','Jammie Lannister','Rob Stark','Cerser Lannister'],publisher:{name:'Bantam Books',country:'United States',website:'www.randomhousebooks.com'}})
{
"acknowledged" : true,
"insertedId" : ObjectId("62dda3f1457193537fd4d245")
}
>
As you can see above in the insert aknowledgment JSON response, insertId value has now changed to a long hash value that is generated internally by MongoDB, in our case it is "62dda3f1457193537fd4d245"
To search for books we use the find() keyword as shown below:
> db.books.find()
{ "_id" : 5, "title" : "A Song Of Ice and Fire", "authors" : [ "George R.R Martin", "Phyllis Eisenstein" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }
{ "_id" : ObjectId("62dda3f1457193537fd4d245"), "title" : "A Feast Of Crows", "authors" : [ "George R.R Martin" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }
>
The find() without any supplied arguments returns all the
documents in our "booksdb" collection. To return a single document we use the findOne(), if empty it returns latest document i.e:
> db.books.findOne()
{
"_id" : 5,
"title" : "A Song Of Ice and Fire",
"authors" : [
"George R.R Martin",
"Phyllis Eisenstein"
],
"publishdate" : "August 1,1996",
"characters" : [
"Jon Snow",
"Daenerys Targaryen",
"Eddard Star",
"Sansa Stark",
"Jammie Lannister",
"Rob Stark",
"Cerser Lannister"
],
"publisher" : {
"name" : "Bantam Books",
"country" : "United States",
"website" : "www.randomhousebooks.com"
}
}
>
We can supply filtering parameters to our find() collection queries as shown below:
> db.books.find({title:{$eq: "A Feast Of Crows"}})
{ "_id" : ObjectId("62dda3f1457193537fd4d245"), "title" : "A Feast Of Crows", "authors" : [ "George R.R Martin" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }
We can delete a document from a given collection using the deleteOne and deleteMany functions:
> db.books.deleteOne({"_id": ObjectId("62dda3f1457193537fd4d245")})
{ "acknowledged" : true, "deletedCount" : 1 }
Next stage we concentrate on building up our very simple GorillaMux API:
Gorilla-Mux API
First things first,to allow our API to talk to the MongoDB database we will be using a database driver called mgo.
mgo- A Rich MongoDB driver for Go
This is MongoDB driver for go-lang that enables developers to build applications like in our case and have that API interface directly with the MongoDB database without having to go through the Mongo interactive shell. This diver acts as a wrapper around the MongoDB API.
Check out the documentation on this driver here, for a deeper understanding on its implementation and application.
mgo installation:
First lets create a folder for our application, create a folder called MongoGorilla:
mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/GoMongo$ mkdir MongoGorilla
then run go mod init, that create go.mod file to track the dependencies that we will use to build our API application like mgo driver above:
mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ go mod init
go: creating new go.mod: module github.com/myk4040okothogodo/MongoGorilla
mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ ll
total 12
drwxrwxr-x 2 mykmyk mykmyk 4096 Jul 23 23:24 ./
drwxrwxr-x 20 mykmyk mykmyk 4096 Jul 23 23:09 ../
-rw-rw-r-- 1 mykmyk mykmyk 58 Jul 23 23:24 go.mod
Now, let install our driver "mgo" as shown below:
mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ go get gopkg.in/mgo.v2
go: downloading gopkg.in/mgo.v2 v2.0.0-20190816093944-a6b53ec6cb22
go: added gopkg.in/mgo.v2 v2.0.0-20190816093944-a6b53ec6cb22
Gorilla-Mux
Next we should add the gorilla/mux package , that will help use implement a very simple router in our API, to help match incoming request to respective handlers.
Dive deeper into the package and how it implements its dispatching features here.
To install the package and have it accessible in our project, while in our project root folder that houses the go.mod file, type in the following:
mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ go get -u github.com/gorilla/mux
go: downloading github.com/gorilla/mux v1.8.0
go: added github.com/gorilla/mux v1.8.0
mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$
API Building
Now lets write a simple program that talks to the MongoDB database using our "mgo" driver and inserts a book " A song of Ice and Fire" book record.
Create a file called "main.go" open it and add the following code shown below:
1 package main
2
3 import (
4 "os"
5 "os/signal"
6 "context"
7 "time"
8 "net/http"
9 "io/ioutil"
10 "encoding/json"
11 "github.com/hashicorp/go-hclog"
12 "log"
13 "github.com/gorilla/mux"
14 mgo "gopkg.in/mgo.v2"
15 "gopkg.in/mgo.v2/bson"
16
17 )
18
19
20 // The struct below holds our database session information
21 type DBSession struct {
22 session *mgo.Session
23 collection *mgo.Collection
24 }
25
26
27 //Book struct holds book data
28 type Book struct {
29 ID bson.ObjectId `json: "id" bson:"_id,omitempty"`
30 Title string `json: "title" bson:"title"`
31 Authors []string `json: "authors" bson: "authors"`
32 Genre []string `json: "genre" bson: "genre"`
33 PublishDate string `json: "publishdate" bson: "publishdate"`
34 Characters []string `json: "characters" bson: "characters"`
35 Publisher Publisher `json: "publisher" bson:"publisher"`
36 }
37
38
39 // Publisher is nested in Movie
40 type Publisher struct {
41 Name string `json: "budget" bson:"name"`
42 Country string `json: "country" bson:"country"`
43 website string `json: "website" bson:"website"`
44 }
45
46
47 // GetBook fetches a book with a given ID
48 func (db *DBSession) GetBook (w http.ResponseWriter, r *http.Request){
49 vars := mux.Vars(r)
50
51 w.WriteHeader(http.StatusOK)
52 var book Book
53 err := db.collection.Find(bson.M{"_id": bson.ObjectIdHex(vars["id"])}).One(&book)
54 if err != nil {
55 w.Write([]byte(err.Error()))
56 } else {
57 w.Header().Set("Content-Type", "application/json")
58 response, _ := json.Marshal(book)
59 w.Write(response)
60 }
61 }
62
63
64 //PostBook adds a new book to our MongoDB collection
65 func (db *DBSession) PostBook (w http.ResponseWriter, r *http.Request){
66 var book Book
67 postBody, _ := ioutil.ReadAll(r.Body)
68 json.Unmarshal(postBody, &book)
69
70 //Create a Hash ID to insert
71 book.ID = bson.NewObjectId()
72 err := db.collection.Insert(book)
73 if err != nil {
74 w.Write([]byte(err.Error()))
75 } else {
76 w.Header().Set("Content-Type","application/json")
77 response, _ := json.Marshal(book)
78 w.Write(response)
79 }
80 }
81
82
83
84
85 //UpdateBook modifies the data of an existing book resource
86 func (db *DBSession) UpdateBook(w http.ResponseWriter, r *http.Request){
87 vars := mux.Vars(r)
88 var book Book
89
90 putBody, _ := ioutil.ReadAll(r.Body)
91 json.Unmarshal(putBody, &book)
92 err := db.collection.Update(bson.M{"_id":
93 bson.ObjectIdHex(vars["id"])}, bson.M{"$set": &book})
94
95 if err != nil {
96 w.WriteHeader(http.StatusOK)
97 w.Write([]byte(err.Error()))
98 } else {
99 w.Header().Set("Content-Type","text")
100 w.Write([]byte("Update succesfully"))
101 }
102 }
103
104
105 //DeleteBook removes the data from the db
106 func (db *DBSession) DeleteBook (w http.ResponseWriter, r *http.Request){
107 vars := mux.Vars(r)
108 err := db.collection.Remove(bson.M{"_id":
109 bson.ObjectIdHex(vars["id"])})
110 if err != nil {
111 w.WriteHeader(http.StatusOK)
112 w.Write([]byte(err.Error()))
113 } else {
114 w.Header().Set("Content-Type", "text")
115 w.Write([]byte("Delete Succesfully"))
116 }
117 }
118
119 func main() {
120 l := hclog.Default()
121 session, err := mgo.Dial("127.0.0.1")
122 c := session.DB("booksdb").C("books")
123 db := &DBSession{session: session, collection:c}
124 addr := "127.0.0.1:8000"
125 if err != nil {
126 panic(err)
127 }
128 defer session.Close()
129
130 //logger := log.New(os.Stdout, "", log.Ldate | log.Ltime)
131 // Create a new router
132 r := mux.NewRouter()
133
134 //Attach an elegant path with handler
135 r.HandleFunc("/api/books/{id:[a-zA-Z0-9]*}", db.GetBook).Methods("GET")
136 r.HandleFunc("/api/books", db.PostBook).Methods("POST")
137 r.HandleFunc("/api/books/{id:[a-zA-Z0-9]*}", db.UpdateBook).Methods("PUT")
138 r.HandleFunc("/api/books/{id:[a-zA-Z0-9]*}", db.DeleteBook).Methods("DELETE")
139
140 srv := &http.Server{
141 Handler: r,
142 Addr: addr,
143 ErrorLog: l.StandardLogger(&hclog.StandardLoggerOptions{}),
144 IdleTimeout: time.Minute,
145 WriteTimeout: 15 * time.Second,
146 ReadTimeout: 15 * time.Second,
147 }
148
149 //start the server
150 go func() {
151 l.Info("Starting server on port 8000 ")
152 err := srv.ListenAndServe()
153 if err != nil {
154 l.Error("Error starting the server :", "error", err)
155 os.Exit(1)
156 }
157 }()
158
159
160 //Trap sigterm or interupt and gracefully shutdown the server
161 ch := make(chan os.Signal, 1)
162 signal.Notify(ch, os.Interrupt)
163 signal.Notify(ch, os.Kill)
164
165 //Block untill a signal is received
166 sig := <- ch
167 log.Println("Got signal :", sig)
168
169
170 //gracefully shutdown the server, waiting max 30 for current operations to complete
171 ctx, _ := context.WithTimeout(context.Background(), 30 * time.Second)
172 srv.Shutdown(ctx)
173
174 }
Remember to run the command "go mod tidy", to have all the direct and indirect dependecies brought in.
In the above code we are first creating two structs , DBSession and Book struct;
DBSession structure will be used to hold our database session and collection properties of the mgo driver. Book structure will hold our book data, also note that the Book struct has a nested Publisher struct.
Next we define a receiver Pointer Method for our Book struct that we intend to have implement Handler interface.In this case we first define the method GetBook, that fetches a book from the database using an id that has been supplied as part of the request.It then returns this method as a response. We then define the second method, called PostBook that does just what its title suggests, it takes a book sent in as part of the Request body and then attempts to insert this book into our database. We the created another handler method called UpdateBook, that takes the updated values of a book that already exists in our database and tries to alter one of its values. Our Final Handler is the DeleteBook Method that is used the data pertaining to a book that we have stored in our database. We identify this particular book using an id that is sent in as part of the request.
We then create our main function, in this we instantiate a session with our database that we will use for all our subsequent queries and updates, we then create a new router using the NewRouter() method of gorilla/mux, we then attach our handlers to this router to enable dispatching of the various request to their subsequent handlers. We then launch our server as a go-routine , we then create a channel to listen and respond to server terminating signals.
To add a new book to our database and to test our POST endpoint lets make a post request to our API, open your terminal from any location and make the following curl request:
mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ curl -X POST \
> http://localhost:8000/api/books \
> -H 'cache-control: no-cache' \
> -H 'content-type: application/json' \
> -H 'postman-token: 6ef9507e-65b3-c3dd-4748-3a2a3e055c9c' \
> -d '{"title" :"A Dance with Dragons","authors":[ "George R.R Martin", "Phyllis Eisenstein"],"publishdate":"August 6,1998","characters":[ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ],"publisher": { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" }}'
You will receive a reply from the API server that will look like:
{"ID":"62ddc5e89c99ea1313ae2568","Title":"A Dance with Dragons","Authors":["George R.R Martin","Phyllis Eisenstein"],"Genre":null,"PublishDate":"August 6,1998","Characters":["Jon Snow","Daenerys Targaryen","Eddard Star","Sansa Stark","Jammie Lannister","Rob Stark","Cerser Lannister"],"publisher": { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" }}
Go to your database and confirm that there us a new entry, whose id has been auto-generated by MongoDB for us, it should look like:
{ "_id" : ObjectId("62ddc5e87071de3d88044140"), "id" : ObjectId("62ddc5e89c99ea1313ae2568"), "title" : "A Dance with Dragons", "authors" : [ "George R.R Martin", "Phyllis Eisenstein" ], "genre" : [ ], "publishdate" : "August 6,1998", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" }}
Lets test our GetBook endpoint, we are go supply the id that was returned when we added our book using the PostBook endpoint above, i.e in my case "62ddc5e89c99ea1313ae2568":
mykmyk@skynet:~/go/src/github.com/myk4040okothogodo/MongoGorilla$ curl -X GET \
> http://localhost:8000/api/books/62ddc5e89c99ea1313ae2568 \
> -H 'cache-control: no-cache' \
> -H 'postman-token: 00282916-e7f8-5977-ea34-d8f89aeb43e2'
It should return:
{"ID":"62ddc5e89c99ea1313ae2568","Title":"A Dance with Dragons","Authors":["George R.R Martin","Phyllis Eisenstein"],"Genre":[],"PublishDate":"August 6,1998","Characters":["Jon Snow","Daenerys Targaryen","Eddard Star","Sansa Stark","Jammie Lannister","Rob Stark","Cerser Lannister"],"publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" }}
To test the delete endpoint:
curl -X DELETE http://localhost:8000/api/books/62ddc5e87071de3d88044140
it should return:
Delete Successfully
Now if we check our database using the find(), command we find that, the specific book "A Dance with Dragons" has been removed i.e:
> db.books.find()
{ "_id" : 5, "title" : "A Song Of Ice and Fire", "authors" : [ "George R.R Martin", "Phyllis Eisenstein" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }
{ "_id" : ObjectId("62dda3f1457193537fd4d245"), "title" : "A Feast Of Crows", "authors" : [ "George R.R Martin" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }
To test the Put endpoint, that we will use to alter specific values of of book data:
I want to change the publisher to the book "A Feast of Crows" from "Bantam Books" to "Voyager" and i want to remove one of the authors "Phyllis Eisenstein"
the books currently looks like:
{ "_id" : 62dda3f1457193537fd4d245, "title" : "A Feast Of Crows", "authors" : [ "George R.R Martin", "Phyllis Eisenstein" ], "publishdate" : "August 1,1996", "characters" : [ "Jon Snow", "Daenerys Targaryen", "Eddard Star", "Sansa Stark", "Jammie Lannister", "Rob Stark", "Cerser Lannister" ], "publisher" : { "name" : "Bantam Books", "country" : "United States", "website" : "www.randomhousebooks.com" } }
Can you figure out how to alter the data, hint:
curl -X PUT http://localhost:8000/api/books/.................
If you get stuck put it down in the comments and i will be sure to help.
Conclusion
In this article we have mainly looked at MongoDB, how to install and add data to it using its interactive shell, then we talked about building a simple gorilla-mux api and having it interface with our database to store, delete and alter the details of books send using HTTP request methods like PUT, POST, GET and DELETE.
Goodbye,see you soon.
Posted on July 24, 2022
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
