How to use jq on linux Environment

jq is a command-line tool for parsing JSON. It can be used to retrieve and output specific information from your system, e.g. how many nodes or dataflows ran for more than 10 minutes, or who logged into the system

Note: JQ is a third party tool. If you cannot install the binaries on your server, you can download the logs from the server and then run JQ on a different machine that has access to the logs. 

How to Set up JQ on Linux

1. Download jq from here: https://stedolan.github.io/jq/download/
2. Go into Path : /usr/local/bin
3. Place the downloaded file into the newly created Bin folder.
4. Rename the file to “jq” (so the full name should be jq.exe)  ( It should be something like in below highlighted screenshot)

5. Run ` jq ` in the command line like below in highlighted screenshot and text should appear, indicating that jq is ready to be used

How to run the commands along with  sample commands for the tool

The command can be run directly from the path where the lae-audit.log  is located .

This reads the audit log file

jq<lae-audit.log

This will show you the graphs that ran last night and their status and how long they took:

jq 'select(.auditCode=="dataflowProcessed") | { graph: .arguments.graph, status: .arguments.status, timestamp: .timestamp, duration: .arguments.duration }' < lae-audit.log

This will show you how many graphs ran

jq 'select(.auditCode=="dataflowProcessed") | { graph: .arguments.graph, status: .arguments.status, timestamp: .timestamp, duration: .arguments.duration }' < lae-audit.log | egrep graph | wc -l

This will show how many graphs ran for longer than one hour

jq 'select(.auditCode=="dataflowProcessed" and .arguments.duration > "01:00:00:000") | { graph: .arguments.graph, status: .arguments.status, timestamp: .timestamp, duration: .arguments.duration }' < lae-audit.log| egrep graph | wc -l

This will show how many graphs ran for longer than two hours

jq 'select(.auditCode=="dataflowProcessed" and .arguments.duration > "02:00:00:000") | { graph: .arguments.graph, status: .arguments.status, timestamp: .timestamp, duration: .arguments.duration }' < lae-audit.log| egrep graph | wc -l

This will show the nodes that ran for more than 500s

jq 'select(.auditCode=="nodeProcessed" and .arguments.elapsedTimeMS>500000) | {node: .arguments.node, timestamp: .timestamp, duration: .arguments.elapsedTimeMS, outputRecordCounts: .arguments.outputRecordCounts }' < lae-audit.log

This will show you how many nodes ran last night

jq 'select(.auditCode=="nodeProcessed") ' < lae-audit.log | egrep elapsedTimeMS | wc -l

This counts the sleep nodes

jq 'select(.auditCode=="nodeProcessed")' < lae-audit.log | egrep 'Sleep"' | wc -l