Flamegraphs part 1

sherpaurgen

urgensherpa

Posted on October 6, 2022

Flamegraphs part 1

As a developer/sysadmin you can create flamegraphs to to create visualizations of system performance data recorded with the perf tool. This perf output shows a stack trace followed by a count, for a total of #N number of samples

git clone the flamegraph scripts:

cd /home/ubuntu
git clone https://github.com/brendangregg/FlameGraph

Sampling a go programme which downloads an linuxmint iso

package main
import (
    "fmt"
    "net/http"
    "io"
    "os"
)

func check(e error) {
    if e != nil {
        panic(e)
    }
}

func main() {
    d1 := []byte("helo world\n")
    for i := 0; i < 10000; i++ {
        resp, err := http.Get("https://mirrors.layeronline.com/linuxmint/stable/21/linuxmint-21-cinnamon-64bit.iso")
        //body, err := io.ReadAll(resp.Body)
        fmt.Println(resp.StatusCode)
        check(err)
        defer resp.Body.Close()
        file, err := os.Create("/tmp/hello.iso")
        size, err := io.Copy(file, resp.Body)
        defer file.Close()
        fmt.Printf("downloaded %s with size %d", file, size)
        err = os.WriteFile("/tmp/check.txt", d1, 0644)
        check(err)
    }
}
Enter fullscreen mode Exit fullscreen mode

compile/build the main.go and run it ./main
Get the process id of main e.g ps aux | grep main

perf record -a -F 99 -g -p 1464 -- sleep 20
Running above command creates a perf.data file

perf script > perf.script //it will by default read perf.data from current working directory and redirects stdout to a file perf.script(ascii file)
This command reads the input file and displays the trace recorded.

Creating flame graph:-

./FlameGraph/stackcollapse-perf.pl perf.script  | ./FlameGraph/flamegraph.pl > flame1006.svg
Enter fullscreen mode Exit fullscreen mode

Download and view flame1006.svg in browser

flamegraph for pid 1464

Here we can observe that io.copybuffer function is using most cpu time (io.copy source)
looking further net.(*netFD).Read is the function call using most cpu time. This net.(*netFD).Read implements func (*IPConn) Read
Conn is a generic stream-oriented network connection.
this function reads data from the connection. This func https://go.dev/src/net/http/transfer.go ΒΆ
we also see ksys_read() is called. This function is responsible for retrieving the struct fd that corresponds with the file descriptor passed in by the user. The struct fd structure contains the struct file_operations structure within it.

Image description
sock_read_iter is fired when receiving a message on a socket
By looking at the graph we can conclude that the cpu usage by main programme is spent mostly on reading data from the connection.

πŸ’– πŸ’ͺ πŸ™… 🚩
sherpaurgen
urgensherpa

Posted on October 6, 2022

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related