Tweaking less(1) for JSON parsing

lbonanomi

lbonanomi

Posted on September 15, 2019

Tweaking less(1) for JSON parsing

I use less pretty frequently (and I'll bet you do too) because less can read your mind:

  • Run it against a text file and it displays it neatly a-page-at-a-time.
  • Run it against a gzipped tarball and it displays an index of the files inside.
  • Run it against an RPM and it displays the file index and changelog.
  • Run it against a flat JSON file and it displays:
{"data":{"__schema":{"queryType":{"name":"Query"},"mutationType":{"name":"Mutation"},"subscriptionType":null,"types":
[{"kind":"SCALAR","name":"Boolean","description":"Represents `true` or `false` values.","fields":null,"inputFields":null,
"interfaces":null,"enumValues":null,"possibleTypes":null},{"kind":"SCALAR","name":"String","description":"Represents 
textual data as UTF-8 character sequences. This type is most often used by GraphQL to represent free-form human-readable 
text.","fields":null,"inputFields":null,"interfaces":null,"enumValues":null,"possibleTypes":null},{"kind":"OBJECT",
"name":"Query","description":"The query root of GitHub's GraphQL interface.","fields":[{"name":"codeOfConduct",
"description":"Look up a code of conduct by its key","args":[{"name":"key","description":"The code of conduct's key",

...a wall of un-parsed JSON data.

Let's teach this faithful dog some new tricks with a shell script edit and a simple function.

less and flat JSON files

A really cool feature of the less utility is the shell script lesspipe.sh that substitutes input file contents with output from a command run on the input file, with rules selected based on that input file's extension. We can easily add another extension handler to the rules for ".json" files to replace raw data with the output of a jq run like this:

*.json) jq . < "$1"; handle_exit_status $? ;;

Now we can update our own $LESSOPEN by running export LESSOPEN="||$HOME/lesspipe.sh %s"

less and streaming JSON

The trouble with tweaking lesspipe.sh is that it doesn't do anything for paging STDOUT, which is my usual workflow for exploring a REST or GraphQL endpoint. Rather than buffering potentially huge chunks of JSON and using a "big" tool to determine the file type, we can make a best-effort check by seeing if the first 2 characters are legal starts to a JSON document ('"', '[' or '{'). If STDOUT starts as legal JSON we will try feeding it to jq | less and just pushing it to less if jq chokes. This approach will cost us a few extra forks, but I think the simplicity is worth it.

function jqless() {
    if [[ -e "$1" ]]
    then
        /bin/less "$@"
    else
        read -u 0 -n2 X
        [[ $(echo "$X" | tr -d [:space:] | cut -c1,2 | egrep "\[|\"|{"{2}) ]] &&\
        (((echo -n "$X" && cat) | jq . | /bin/less ) || (echo -n "$X" && cat) | /bin/less ) ||\
        (echo -n "$X" && cat) | /bin/less
    fi
}
💖 💪 🙅 🚩
lbonanomi
lbonanomi

Posted on September 15, 2019

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Tweaking less(1) for JSON parsing
bash Tweaking less(1) for JSON parsing

September 15, 2019