Attempting to Learn Go
Here you can find the code I've been writing for my Attempting to Learn Go posts that I've been writing and posting over on Dev.to.
Attempting to Learn Go - Building Dev Log Part 03
Steve Layton
Posted on January 26, 2019
Building Out A Dev Blog
Here we go again! If you've been following these posts, welcome back, and if not welcome aboard! This time we'll be continuing our work to extend the basic code we put together to create a static site generator. We actually jump quite a bit forward this week. We've now are at a point where we can convert an entire directory of well-formatted markdown files into HTML ready to host. Anyway, let's keep on with our attempts to learn Go...
Diving In
I'm still toying around with a different way to show our updated code but, for now, I think the code base is small enough I can just start from the top and work my way down. I'm going to gloss over the bits that haven't changed since last week so, if you haven't read that I suggest you do!
Now then, let's jump down to our first real changes.
package main
import (
"bufio"
"bytes"
"fmt"
"html/template"
"io/ioutil"
"os"
"regexp"
"strings"
"github.com/microcosm-cc/bluemonday"
"github.com/russross/blackfriday"
yaml "gopkg.in/yaml.v2"
)
const delim = "---"
type post struct {
Title string
Published bool
Description string
Tags []string
CoverImage string
Series string
PostBody template.HTML
}
var templ = `<!DOCTYPE html>
<html lang="en">
<head>
<title>{{.Title}}</title>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="HandheldFriendly" content="True">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="referrer" content="no-referrer-when-downgrade" />
<meta name="description" content="{{.Description}}" />
</head>
<body>
<div class="post">
<h1>{{.Title}}</h1>
{{.PostBody}}
</div>
</body>
</html>
`
Naming Stuff Is Hard
Here we get to the first (small) change, the function is the same as last week but I've renamed it from loadFile() to getContents(). It seems to more accurately capture what it does.
func getContents(f *string) ([]byte, error) {
b, err := ioutil.ReadFile(*f)
if err != nil {
return nil, err
}
return b, nil
}
One thing you should notice throughout this section is I've tried to make sure we're are breaking out code into standalone functions. This should keep our main() nice and make it so we can write tests. Below we have parseFM() which wraps up our un-marshaling and returning the first stage of our front matter. It may be more correctly called parseYAML() but, we'll leave it as is for now. isNil() has not changed and still is not really used, I will see to that before the next post, possibly.
func parseFM(b *[]byte) (map[string]interface{}, error) {
m := make(map[string]interface{})
err := yaml.Unmarshal(*b, &m)
if err != nil {
msg := fmt.Sprintf("error: %v\ninput:\n%s", err, b)
return nil, fmt.Errorf(msg)
}
return m, nil
}
func isNil(i interface{}) bool {
if i != nil {
return false
}
return true
}
As before we need to split up the data we loaded from the markdown file. A call to splitData() will take care of it. I've fed in various bits of malformed data and the if statement seemed to work as expected each time. I also ran my current set of markdown dev.to posts through and each was converted without a problem. They can all be found over at dev.shindakun.net.
func splitData(f *[]byte) ([][]byte, error) {
b := bytes.Split(*f, []byte(delim))
if len(b) < 3 || len(b[0]) != 0 {
return nil, fmt.Errorf("Front matter is damaged")
}
return b, nil
}
Once we've split the data and parsed out the front matter we call makePost() to build the post struct and the template. I'm passing back the *post because we're currently executing the template just before we write the file to disk, not in this function itself. It will also be handy when I build out something to create an index.html. Again, we're not really using isNil() as intended mostly because I don't like the if/else jumble that it will create. We'll get that taken care of one of these days.
// makePost creates the post struct, returns that and the template HTML
func makePost(fm map[string]interface{}, contents []byte,
s [][]byte) (*template.Template, *post) {
p := &post{}
if isNil(fm["title"]) {
panic("isNil tripped at title")
} else {
p.Title = fm["title"].(string)
}
p.Published = fm["published"].(bool)
p.Description = fm["description"].(string)
// TODO: Strip space after comma prior to parse?
tmp := fm["tags"].(string)
p.Tags = strings.Split(tmp, ", ")
p.CoverImage = fm["cover_image"].(string)
p.Series = fm["series"].(string)
pBody := contents[len(s[1])+(len(delim)*2):]
out := blackfriday.Run(pBody)
bm := bluemonday.UGCPolicy()
bm.AllowAttrs("class").Matching(regexp.MustCompile("^language-[a-zA-Z0-9]+$")).OnElements("code")
p.PostBody = template.HTML(bm.SanitizeBytes(out))
tm := template.Must(template.New("msg").Parse(templ))
return tm, p
}
There you go, we're down to our main() function! Right of the bat, you'll see that we're now using ioutil.ReadDir() to get a list of the contents of the current directory.
func main() {
d, err := ioutil.ReadDir(".")
if err != nil {
panic(err)
}
We then range through the list of the current directory and immediately check to see if the current file ends with .md. If it does the address of our string gets passed into getContents() and if all goes well we'll get the contents back.
for _, f := range d {
if t := f.Name(); strings.HasSuffix(t, ".md") {
contents, err := getContents(&t)
if err != nil {
panic(err)
}
We'll pass a pointer to our contents into splitData() and then pass the second chunk of s, that is s[1], into parseFM(). This gives us the base building blocks for our post. Both splitData() and parseFM() could probably be moved into makePost(), then we could pass the contents directly into it and do all the work there. That's something to consider for when refactoring.
s, err := splitData(&contents)
if err != nil {
panic(err)
}
fm, err := parseFM(&s[1])
if err != nil {
msg := fmt.Sprintf("error: %v\ninput:\n%s", err, s[1])
panic(msg)
}
tm, p := makePost(fm, contents, s)
In this current version of the code, we're going to take the current filename and lop off the extension. We'll then create a new file with the same name as the original and append .html. Again, this is all currently happening in the same directory and we're being really sloppy and not checking for existing files or anything. os.Create() uses 0666 (or equivalent permissions in Windows) for the file permissions so in theory we should be able to write and overwrite with no issues. This is another case where we might not be doing what may be considered "correct" - instead, we are just going through an iterative process. This lets everyone see how the code has changed, warts and all.
File created, o, we then use bufio.NewWriter() to create a "Writer" with a buffer. We can then execute our template and write directly into the file which is closed after we flush the buffer.
fin := strings.TrimSuffix(t, ".md")
o, err := os.Create(fin + ".html")
if err != nil {
panic(err)
}
defer o.Close()
buf := bufio.NewWriter(o)
err = tm.Execute(buf, p)
if err != nil {
panic(err)
}
buf.Flush()
}
}
}
If we have any files left we jump back up to the top of our loop and do the whole things again!
Next Time
We have just about everything in place! But first, we need to create and write out index.html. I think I'm going implement that over the next day or so and put up a quick follow up post. After that, I'll need to decide if we want to do one more post in the "dev site" series and extend the code to use external templates. It shouldn't be that much more work, I suppose I'll wait and see how busy the next week looks on Monday. See you next time!
You can find the code for this and most of the other Attempting to Learn Go posts in the repo on GitHub.
shindakun
/
atlg
Source repo for the "Attempting to Learn Go" posts I've been putting up over on dev.to
| Enjoy this post? |
|---|
| How about buying me a coffee? |
💖 💪 🙅 🚩
Steve Layton
Posted on January 26, 2019
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
