jq, xmlstarlet and friends.

Json

Jq

Commandline json mapping and filtering tool.

# get json keys
jq 'keys' <file.json # or e.g. '.item[0] | keys'

# filter, map, reduce (partially stolen)
cat file.json | jq '.[] | select(.age == 36)'
cat file.json | jq '.[] | select((.users| length)  == 5)'
cat file.json | jq 'map({ _id, email })'
cat file.json | jq 'reduce .[] as $item (0; . + $item.age)'


# wrap entry stream in array
echo '[ { "success": true },{"success":false} ]' | jq '[.[].success]'
#[  true,  false ]


# Misc - csv, date formatting

cat file.json | jq -r 'map([.type, (.time| ./1000 | strflocaltime("%Y-%m-%d %I:%M%p"))]) | .[] | @csv'

Jq alternatives

  • jid - auto completing & interactive jq. Doesn’t seem to support pipes or anything more complex from jq syntax.
  • jq-node version (better functions)
  • pxi is faster alternative (3 times faster than jq) probably. Supports more than just json.

JWT decoding

From https://gist.github.com/angelo-v/e0208a18d455e2e6ea3c40ad637aac53 jq -R 'gsub("-";"+") | gsub("_";"/") | split(".") | .[1] | @base64d | fromjson'

Gron

gron - flatten json into grepable key - value form. gron -u to undo it.

Xml

TODO xmlstarlet

Todo 🤷‍♂️