Processing data in a shell... with JavaScript!

spyke

spyke

Posted on January 9, 2020

Processing data in a shell... with JavaScript!

Processing data in a shell may be very convenient, especially for experienced terminal users. You pipe text data through various commands, redirect streams, read and write files.

However, shell commands are diverse. Identical commands on some operating systems may have different options or those options may have different meaning. Some commands exist in different versions (grep, rgrep, sgrep, etc.), all with small tweaks here and there. Usually, it's not that difficult and just requires some time to remember and find your favorites.

But if you're a JavaScript developer, you have an additional tool to choose. With 0.5.0 release of the undercut library there's a new command line utility under the same name allowing you to build pipelines with JavaScript and achieve quantum supremacy in a shell.

To try the undercut command you will need Node.js 10.13 or later. Global installation is recommended for ease of use:

$ npm i -g @undercut/cli
Enter fullscreen mode Exit fullscreen mode

What it can do? Everything that undercut push lines can + any suitable JavaScript expression.

Specify your operations in quotes separated by spaces:

$ undercut 'op1' 'op2' 'opN'
Enter fullscreen mode Exit fullscreen mode

Incoming data from stdin will be processed sequentially line-by-line by these operations and passed further into stdout.

Let's read a text file, trim its lines, remove lines shorter than 10 symbols, and print it on the screen:

#   ↙[read a file]  ↙[pipe it to undercut] 
$ cat strings.txt | undercut 'map(s => s.trim())' 'filter(s => s.length > 10)'
#                               ↖[operation 1]       ↖[operation 2]
Endless world!
A very long string...
Enter fullscreen mode Exit fullscreen mode

It should be very familiar if you've ever used array methods like Array.prototype.map() or Array.prototype.filter().

If we want to save it to a file instead of printing on the screen, then we just need to add stdout redirection like so:

$ cat strings.txt | undercut 'map(s => s.trim())' > processed.txt
#                           [redirect to a file]↗
Enter fullscreen mode Exit fullscreen mode

This is a standard shell mechanism. But you can do more.

Use an Iterable as a source instead of stdin:

$ undercut -s 'range(0, 5)' 'map(Math.sqrt)' 'sum()'
6.146264369941973
Enter fullscreen mode Exit fullscreen mode

Import an installed npm package and use it:

$ undercut -i 'pad::left-pad' -s 'range(0, 3)' 'map(x => pad(x, 3))'
000
001
002
Enter fullscreen mode Exit fullscreen mode

Or even enter text data from keyboard by skipping the source. Results will be printed to stdout after you signal the end of input with Ctrl + D:

$ undercut 'map(s => s.toUpperCase(s))'
Tom
Sam
# Ctrl + D to finish the input.
TOM
SAM
Enter fullscreen mode Exit fullscreen mode

If you want to view the list of operations or just learn more about the undercut, feel free to visit undercut.js.org for the documentation or GutHub repo for the code.

💖 💪 🙅 🚩
spyke
spyke

Posted on January 9, 2020

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related