Open-source contributions are a great way to help new community members learn new skills. One of the most critical aspects of these contributions is that they are always done in a collaborative effort. However, extracting insights from raw contributions data can be hard without graphical representations involved.

The good news is that visualization is a powerful technique for communicating information. It can be used in many different ways, such as analyzing data, making inferences, and creating compelling presentations.

Visualizing data has become more critical in today's world because of the increase in technological advancements. Visualizations allow people to see patterns and trends that might have been difficult to notice otherwise.

Visualizing contributions is not always necessary for all open source projects. However, it can benefit certain types of projects or specific audiences unfamiliar with the project.

It is quite important to mention that contributing to open source projects is not just about writing code; it also includes bug reports, documentation, and other contributions.

In this article, we will explore how FFmpeg and Gource, in particular, can help us visualize the contributions of a given project and understand how it has evolved.

Psst: if you want to follow along with this tutorial, consider the following steps, assuming you are a Linux user:

1- Clone the web3.py repo and cd into it:

$ git clone https://github.com/ethereum/web3.py && cd web3.py

2- Create a new directory called img, download and save my Github avatar inside that directory:

$ mkdir img && cd img

👉 Table Of Contents (TOC).

• What Is Gource?
• Gource Installation
• Gource Usage
• What Is FFmpeg?
• FFmpeg Installation
• FFmpeg Usage
• Putting It All Together
• Wrapping Up
• References

What is Gource?

🔝 Go To TOC.

Gource is a free and open-source software visualization tool used to create 3D animated visualizations of the source code development of a given software [0]. The visualization is generated by connecting to a version control repository, such as a git repository, and using the commit logs, or manually generated logs, to produce an animated video. This means that it is not designed to show off any particular aspect of design or function; instead, it shows how much work went into creating something and how it evolved.

Gource was created by Andrew Caudwell [1]. It supports Git, Bazaar, Mercurial and SVN as version control systems. Apparently, the name "Gource" was inspired by the word "Graph" and the term "Source", according to rumors. What makes Gource so popular is the fact that it is featured on Debian [2] and ubuntu [3] manpages.

Gource can be used for many purposes, such as showing the contributions of individual programmers in an open-source project, or to provide an overview of the different parts of a complex program and how they are connected.

In the following section, we will go over the process of installing Gource on your machine.

Gource Installation

🔝 Go To TOC.

Gource is a cross-platform tool which means it is available for Linux / Windows / Mac. Being a humbled Linux user, you can install it with apt:

$ apt install gource

However, as stated in the readme file [4], Gource requires OpenGL to run on your machine.

If you are a Mac user, I got you covered. But I can't predict what can go wrong for you. you can install this tool using homebrew [5]:

$ brew install gource

That's all you need for the installation part. Now, let's go over some of the common usages of this tool.

Gource Usage

🔝 Go To TOC.

Schematically, the Gource command-line interface expects the following arguments:

$ gource [options] [path]

• options: You can run gource -H to take a look at the available options.
• path: the path of the .git directory. By default, the value for this argument is .git/. Therefore, you don't have to explicitly provide it if you are inside a project folder that has a .git directory.

With that noted, you can run Gource without any argument like the following:

$ gource

Gource immediately opens up an animated video window after running the command, but it doesn't save it on your machine. Notice that it will start streaming from the first commit to the project, which is not convenient for large projects. To speed things up, you can provide additional arguments.

But, before we dive into the details of the available options, let's run the following command and observe what is going to happen.

$ gource -1280x720 --title "web3" --seconds-per-day 0.1 --auto-skip-seconds 0.1 --bloom-intensity 0.05 --max-user-speed 500 --highlight-users --highlight-dirs --hide filenames --multi-sampling

As you can notice, it will generate an animated video at a relatively normal speed. Now let's discuss a bit about the options used in the above command:

• -1280x720: the resolution of the video.
• --title "web3": The video's title which will be displayed in the bottom left of the video.
• --seconds-per-day 0.1: Speed in seconds per day (default: 10). This argument represents a fraction of time. 0.1 means 1 second per 10 days. If you are an active contributor to the project, try to increase this value. 0.1 can be considered a modest developer.
• --auto-skip-seconds 0.1: This parameter tells the program to jump 0.1 seconds if there is an idle time, when there is no commits, to jump to the next entry (default: 3 seconds).
• --bloom-intensity 0.05: Adjust the intensity of the bloom (default: 0.75).
• --max-user-speed 500: The velocity at which users can move around(default: 500).
• --highlight-users: Highlight all users contributed to the project.
• --highlight-dirs: Highlight the names of all directories.
• --hide filenames: Don't show filenames.
• --multi-sampling: Enable multi-sampling for additional smoothing.

Gource will take so long to exit for a relatively sizeable open-source project because of when the project started and how much work went into it over the years. However, you can specify a start and stop date to visualize a particular date duration.

$ gource -1280x720 --title "web3" --seconds-per-day 0.1 --auto-skip-seconds 0.1 --bloom-intensity 0.05 --max-user-speed 500 --highlight-users --highlight-dirs --hide filenames --multi-sampling  --start-date "2022-02-04" --stop-date "2022-03-22"

If you want to understand what is going on in that animation, we can refer to the readme file of Gource, and I quote:

The repository is visualized as a tree where the root of the repository is the centre, directories are branches and files are leaves. Contributors to the source code appear and disappear as they contribute to specific files and directories.

To highlight a specific user, you can provide a --highlight-user argument and a string representing the username. You can also provide an image for a particular user using the --user-image-dir option(Directory containing .jpg or .png images of users. e.g., "Full Name.png" to use as avatars.). Additionally, you can change the highlighter's color using the --highlight-colour argument(by default, it is the FFFFFF hex value representing the white color). Our command will result in the following:

$ gource -1280x720 --title "web3" --seconds-per-day 0.1 --auto-skip-seconds 0.1 --bloom-intensity 0.05 --max-user-speed 500 --highlight-dirs --hide filenames,dirnames --multi-sampling  --start-date "2022-02-04" --stop-date "2022-03-22" --highlight-user "Mahmoud Harmouch" --user-image-dir "./img/" --highlight-colour 00FF00

You can also track movements of a particular user using the --follow-user and --camera-mode options

$ gource -1280x720 --title "web3" --seconds-per-day 0.1 --auto-skip-seconds 0.1 --bloom-intensity 0.05 --max-user-speed 500 --highlight-dirs --hide filenames,dirnames --multi-sampling  --start-date "2022-02-04" --stop-date "2022-03-22" --highlight-user "Mahmoud Harmouch" --user-image-dir "./img/" --highlight-colour 00FF00 --follow-user "Mahmoud Harmouch" --camera-mode track

Now, if we want to write this animated video frame by frame onto your disk(using the -o gource.ppm argument), we will end up with a huge file. Let's take the following scenario:

Let's take our Gource command. We want to record a video with a resolution of 1280x720 pixels per frame. We need to spend 3 bytes per pixel for color-coding. Additionally, the video runs at 30 frames per second for a total duration of 1 minute.

resolution = 1280 * 720 (pixels / frame)
cost_per_pixel = 3 (bytes / pixel)
duration_in_seconds = 1 * 60 (seconds)
frame_rate = 30 (frames / second)

storage_required = duration_in_seconds * frame_rate * resolution * cost_per_pixel (bytes)

If you do the math, this video will require approximately 4.6 GB of storage for just one minute of the recording! That's why you have to use some sort of compression with the help of the FFmpeg command. FFmpeg uses a codec to compress the video stream generated by Gource. It converts raw digital video to a compressed format.

That's all you need to know about Gource. You can refer to the Readme file [6] if you are looking for additional options or looking them up within your terminal using the gource -H command. Now, let's move on to the FFmpeg section in order to learn how we can store that recording locally on a drive.

What is FFmpeg?

🔝 Go To TOC.

FFmpeg is an open-source project for manipulating, converting, and streaming video and audio data [8]. It is most often used to convert video from one format to another, but it can also be used for live streaming and recording. It is the best known of several such projects.

The project was started in 2000 by Fabrice Bellard [9] in response to a need for a tool to convert video from one format to another, with particular attention to the then-new MPEG-4 standard. The name "FFmpeg" was originally an acronym for "Fast Forward MPEG Player" [10].

The project has since expanded its scope to include encoding, transcoding, streaming, recording, playback (conversion), and multimedia content manipulation.

For complete and detailed information on how this tool works, you can refer to the documentation, which does a great job in terms of explaining things [11].

FFmpeg Installation

🔝 Go To TOC.

To install FFmpeg on a Linux machine, open your favorite terminal and type the following command:

$ apt install ffmpeg

or, if you are using a Mac OS, you can use homebrew to install it on your machine:

$ brew install ffmpeg

Having FFmpeg installed on your machine, let's take a look at some useful commands that can be helpful in your daily work with multimedia content.

FFmpeg Usage

🔝 Go To TOC.

Schematically, the FFmpeg command line program expects the following number of arguments to accomplish its job:

 ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

The primary usage of this command is the following:

$ ffmpeg -i input.mp4 -r 24 output.gif

It will convert an mp4 video file into a gif with a frame rate equal to 24. However, for the purpose of this article, we need to read a stream of data from gource and save it on a local drive. To do so, we need to change the input filename to - to read from STDIN. Recall from the above Gource Usage section, we need to compress the data generated by Gource.

In the next section, we are going to put things together to generate a nicely compressed video file.

Putting It All Together

🔝 Go To TOC.

To use FFmpeg along with Gource, I found a handy tutorial on the wiki page [12] of the Gource repo.

With that noted, our final command will result in the following:

$ gource -1280x720 --title "web3" --seconds-per-day 0.1 --auto-skip-seconds 0.1 --bloom-intensity 0.05 --max-user-speed 500 --highlight-dirs --hide filenames,dirnames --multi-sampling  --start-date "2022-02-04" --stop-date "2022-03-22" --highlight-user "Mahmoud Harmouch" --user-image-dir "./img/" --highlight-colour 00FF00 --follow-user "Mahmoud Harmouch" --camera-mode track -o - | ffmpeg -y -r 60 -f image2pipe -vcodec ppm -i - -vcodec libx264 -preset ultrafast -pix_fmt yuv420p -crf 1 -threads 2 -bf 0 gource.mp4

This command will generate the following recording:

The FFmpeg command will convert the ppm file generated by Gource to a regular mp4 format. This is done using Gource -o( abbreviation for --output-ppm-stream) and converting this stream of images into the mp4 format using the FFmpeg utility.

Wrapping up

🔝 Go To TOC.

I will conclude by stating that open-source visualization tools are potent and flexible enough to create complex visualizations helpful in extracting meaningful insights from contributions data.

This constitutes our guide for today on using Gource to visualize the development of your project beautifully. Using this tool, you can quite impress your colleagues or coworkers with some cool visualizations to show off your contributions ;-).

Not a popular argument, but some may argue that using this type of tool can harm your privacy to some extent by tracking your contributions to a given project. Some people don't like this idea, so they keep their contributions private. For others, it is some sort of cringe to show off your work, and I can't agree more. Just stay calm, keep coding, and deliver something useful for the whole damn world. But it can motivate you by creating a feedback loop that is essential to your professional development and maintain your happiness in the long run.

As always, this article is a gift to you, and you can share it with whomever you like or use it in any way that would be beneficial to your personal and professional development. By supporting this blog, you keep me motivated to publish high-quality content. Thank you in advance for your ultimate support!

I am looking forward to your comments about using this tool in the comments section below. And I hope you enjoyed reading this article and learned something valuable from it.

That's all for today's article. Thank you for reading! Stay hydrated, folks; see you in the next one.

References

🔝 Go To TOC.

[0] Andrew Caudwell. Software version control visualization. Github Repository. Retrieved 2022-04-26.

[1] Github. Andrew Caudwell. Github Repository. Retrieved 2022-04-26.

[2] Debian. Gource - a software version control visualization. Debian manpages. Retrieved 2022-04-26.

[3] Ubuntu. Gource - a software version control visualization. Ubuntu manpages. Retrieved 2022-04-26.

[4] Andrew Caudwell. Requirements. Github Repository. Retrieved 2022-04-26.

[5] Homebrew Formulae. gource install command. Homebrew Formulae website. Retrieved 2022-04-26.

[6] Andrew Caudwell. Using Gource. Github Repository. Retrieved 2022-04-26.

[7] Bellard, Fabrice. The transcoding process in ffmpeg for each output. FFmpeg website. Retrieved 2022-04-26.

[8] Bellard, Fabrice. Description. FFmpeg website. Retrieved 2022-04-26.

[9] Bellard, Fabrice. FFmpeg License and Legal Considerations. FFmpeg website. Retrieved 2022-04-26.

[10] Bellard, Fabrice. FFmpeg naming and logo. FFmpeg developer mailing list. FFmpeg website. Retrieved 2022-04-26.

[11] Bellard, Fabrice. Detailed description. FFmpeg website. Retrieved 2022-04-26.

[12] Andrew Caudwell. Gource wiki pages. Github Repository. Retrieved 2022-04-26.