(the lack of) Keybase Filesystem Events

For one of my clients (small company) it seemed like a useful feature to notify employees when someone added, removed or renamed a file on their shared Keybase team folder so that they don't have to check every time.

The idea originated when I saw that in the compact window of the Keybase GUI, you see a list of recent file changes. I asked Keybase support if I can read the events somehow from somewhere. After all, it is an Electron application.
The keybase filesystem does not implement any real events, so I created a hacky little script that simulates them.

I wrote a hacky little script that globs the k:/team/name folder and emits changes. I simply run this script periodically (every 30 seconds) as a sort of cronjob.

To break it down:

glob a personal or team folder
compare result with previous run
if a new file is added, emit a create event
if a file is no longer there, emit a delete event
if the mtime (last modified time) of the file changed, emit change event

To check whether a file has been renamed is a little trickier because the old file will be detected as missing/deleted and the renamed file will be seen as a new file. In order to make this work, when a file is supposedly created and another one is deleted, compare the mtime of the new file to the old file. If they are the same it is veeeeeeeery likely that it has been renamed. (can't think of much exceptions)

I know ... its hacky but hey; but the following is currently tested and working on all operating systems which is better than nothing at all.

/* filesystem I/O and child process */
const fs = require('fs')
const childProcess = require('child_process')

/* events */
var Emitter = require('./emitter')
var emitter = new Emitter() 

/* subscribe to all events and use console.log as temporary callback */
emitter.on('*', console.log) // probably pipe to some GUI / notification

/* buffer file for file list */
const oldFiles = 'teamname.json'
/* the directory to watch */
let directory = 'k:/team/teamname/**/*'

/* buffer file otherwise initialize empty array */
let old = fs.existsSync(oldFiles) ? JSON.parse(fs.readFileSync(oldFiles, 'utf8')) : []

/* normalize object with lstat data to JSON compatible format */
const normalize = files => JSON.parse(JSON.stringify(files))

/* start time */
let start = Date.now()

/* match all files and directories */
let files = normalize(JSON.parse(childProcess.execSync(`glob --json "${directory}"`).toString()).map(file => [file, fs.lstatSync(file).mtime]))

/* time it took = end time - start time */
let end = Date.now()
let difference = end - start

emitter.emit('done', { took: difference/1000 }) // seconds

Array.prototype.has = function(file) {
    for(let item of this) {
        if(item[0]==file[0])
        return true
    }
    return false
}

Array.prototype.hasmtime = function(mtime) {
    for(let item of this) {
        if(item[1]==mtime)
        return item
    }
    return false
}

let newfiles = []

/* iterate old files */
old.forEach((file, index) => {
    let [f, mtime] = file 
    let fl 
    /* find new file with filename = this old filename */
    fl = files.filter(file => file[0] == f)
    /* because map returns array */
    if(fl.length > 0) { 
        fl = fl[0]
        /* if mtime has changed, emit change event */
        if(mtime!=fl[1])
            emitter.emit('changed', fl)
    }
    /* if new files does not have this current old file */
    if(!files.has(file)) {
        /* check to see if a file with exactly the same mtime exists */
        let changed = files.hasmtime(file[1])
        if(changed) {
            /* veeery likely that file is renamed */
            emitter.emit('rename', { from: file[0], to: changed[0]})
        } else {
            /* file no longer exists and no same mtime file found */
            emitter.emit('deleted', file)
        }
    }
})

/* if a new file does not exist anywhere in old files */
files.forEach(file => !old.has(file) ? (emitter.emit('new', file) && newfiles.push(file)) : null)

/* finally, write the new files to file for next iteration */
fs.writeFileSync(oldFiles, JSON.stringify(files))

Wrap this up in a setInterval and voila!

Note: depending on your internet speed and the amount of files/directories it takes about 5 to 15 seconds to glob the whole folder. Take that into account when deciding the delay between each run. Also, it is not always (almost never) that important that the notifications are immediate. (30 seconds to a minute is acceptable)

Note: you can find the the Emitter class I used in this article.

Note: I am using this for keybase but this will also work on local folders, FTP and shared network folders.

Todo: you can improve the certainty that a file is renamed by not only checking the last modified time but also check if the filesize is the same. You can read the filesize from lstat just like the last modified time.

Blog

(the lack of) Keybase Filesystem Events

Jochem Stoel

Join Our Newsletter. No Spam, Only the good stuff.

Related