🗃️ Syncing my notes - My Plain Text Journey Part III

This is an ongoing series:

After you start using one folder with your plain text notes and lists, that’s all great until you want to work on them from multiple devices. That’s when you start looking at different ways to sync your notes.

My syncing requirements were simple:

Access from all desktop operating systems, iOS, and Android
Version control
Headless command line access

In the past I would have recommended Dropbox but their free level no longer works, and the paid tiers are way too much. Plus, I had issues with conflicted files… A lot. None of the other sync services support command line only access. I thought about installing Nextcloud or Seafile, but there is a simpler option, Git.

Why Git?

Git has several advantages. File versioning is built in, it works well offline, and I can use it from any machine. The biggest disadvantage is that it doesn’t deal well with large binary files. There are ways to deal with those, but I’m not storing anything of the sort in my notes repository.

Don’t be afraid of Git. I barely know how to use it effectively, but it boils down to knowing the following commands:

git add
git commit -am “Commit Message”
git pull
git push

On the desktop/laptop, I use SparkleShare, which means I don’t even need to know those commands!

Setting up the repo

The main repo is a private repository on Github. I thought about storing it myself on a server, but I like the offsiteness of Github. It will also inevitably be more secure than anything I run.

Create/sign in to your Github account and create a new private repo. I call mine, unsurprisingly, notes. You may already have your notes folder somewhere on your computer. Don’t worry, we’ll move everything from that folder to your Git notes folder soon enough.

Syncing on the desktop

To sync on the desktop, under macOS,Windows, or Linux, I use the open source application SparkleShare. If you already have git set up or are using it on your desktop you can use that, but SparkleShare takes care of the syncing for you automatically. You don’t have to remember to do the pull/commit/push dance with git. SparkleShare does that all for you.

After you install SparkleShare, you add the SSH key from SparkleShare to your GitHub account, and then you can add your notes repo from GitHub to SparkleShare. Inside the SparkleShare folder, your notes folder will magically appear!

From then on any change you do to your notes folder will automatically be synced and sent up to GitHub.

Setting up at the command line

Majority of the time I interact with my notes folder from the command line. I have it on a virtual private server (VPS) that is accessible from almost any device I use. If I know I’m not going to have Internet access, I will clone my repo to whatever machine I am on at the time, but most of the time I use mosh to connect to my VPS and work on my notes from there.

To automate the add/pull/commit/push cycle with git, I wrote a very simple bass script called notesync.sh. It takes care of keeping my notes repo in sync.

#!/bin/bash

cd ${HOME}/notes
gstatus=`git status --porcelain`

if [${#gstatus} -ne 0]
then
    git add --all
    git commit -am "Git Sync `date` on `hostname` $gstatus"
    git pull
    git push
else 
    git pull
fi

To run this automatically, I set up a service that will run the sync script every two minutes. You could throw it in the cron also.

An important git setting

When I first moved my journal to get, I would randomly have merge conflicts. Git doesn’t like it when you add lines to the end of a file from various machines and then try to commit those changes. It doesn’t know what order those line should be in. After searching all over the Internet, I found a setting that will take care of this for me. In the root of the repo, I have a .gitattributes file with the following contents:

*.markdown merge=union
*.taskpaper merge=union
*.md merge=union
*.txt merge=union

No more merge conflicts, although sometimes I have to go into the journal and re-order a few things. It doesn’t happen enough to investigate further.

Telling SparkleShare you pushed something

Finally, on the machines your run git, you will want to add a post hook that will tell SparkleShare you pushed changes to the repo. Without this post hook, SparkleShare will still sync, but it might take 15 minutes because it has to poll for changes instead of getting a notification that something changed.

For the post took to work, you’ll need to have socat installed. Once socat is installed, it’s a very simple post hook from SparkleShare that uses socat to send out a notification to sparkle share.

#!/bin/sh

#
# This Git hook sends update messages to a notification server, so
# that manual git pushes to a repository also will be noticed by 
# SparkleShare clients.
#
# For information on running your own notification service:
# https://github.com/travisghansen/fanout
#
# For use with Gitolite:
#
# Copy this file to .gitolite/hooks/common/post-update.
# Run "gl-setup" again.
#
# For use with standard Git repositories:
#
# Copy this file to .git/hooks/post-update in 
# the remote repository
#
# Make sure to "chmod -x" this hook after the file has been copied
#

# Change these if you run your own service somewhere
SERVER="notifications.sparkleshare.org"
PORT="443"

# Don't edit below this line
exec > /dev/null 2>&1

CHANNEL=$(git show HEAD:.sparkleshare)
MESSAGE=$(git rev-list --max-count=1 HEAD)

DATA="announce ${CHANNEL} ${MESSAGE}"
echo "${DATA}\n" | socat - TCP-CONNECT:${SERVER}:${PORT} &

exit 0

iOS

To sync on my iPhone, I bought the app Working Copy. It’s pretty amazing, and well worth the price. No subscriptions here. Working Copy syncs the repo to my phone, and then I can use any app on my phone that supports the new Files app to edit documents in my notes repo. I usually use iA Writer. There are a ton of markdown apps you can check out though.

Android

With android, I use the app Termux. It gives you a full Linux environment on your phone or android tablet. I use this to sync the repo and also sometimes will use them in Termux to write in my notes. Most of the time, however, I use the app Markor. It supports markdown and the todo.txt task managing format.

To automate the syncing on android, I did pay for the widget that allows you to add shortcuts to scripts in the Termux environment to your home screen.

All synced up

Using git has a ton of nice features. I get free version control for all of my text files, and each machine has a back up of everything I’ve done with my notes folder. And since I use it on a few different machines, I have a nice distributed back up along with the copy that is being held at GitHub.