1
0
mirror of https://github.com/kellyjonbrazil/jc.git synced 2026-04-03 17:44:07 +02:00

Compare commits

...

19 Commits

Author SHA1 Message Date
Kelly Brazil
976fd7d9bd readme update 2019-10-22 17:24:56 -07:00
Kelly Brazil
d8337870ca update documentation 2019-10-22 17:21:00 -07:00
Kelly Brazil
39a8aec77f v0.8.1 build 2019-10-22 17:15:02 -07:00
Kelly Brazil
306d539b6b readme update 2019-10-22 16:50:01 -07:00
Kelly Brazil
f3087b8a8e update readme and formatting 2019-10-22 16:40:27 -07:00
Kelly Brazil
414c2ecef8 fix iptables parser 2019-10-22 16:32:55 -07:00
Kelly Brazil
776ef2d1be add iptables parser 2019-10-22 15:42:29 -07:00
Kelly Brazil
9ac5746996 add uname parser 2019-10-22 13:28:15 -07:00
Kelly Brazil
a3e55d97c0 add mount parser 2019-10-22 12:54:41 -07:00
Kelly Brazil
b15227e7ba add lsblk parser 2019-10-22 11:55:11 -07:00
Kelly Brazil
ec3d1f84ce fix free parser 2019-10-22 11:26:58 -07:00
Kelly Brazil
753d5fd9fe readme update 2019-10-22 11:17:21 -07:00
Kelly Brazil
73a0d70c92 readme update 2019-10-22 11:15:44 -07:00
Kelly Brazil
c2c189f3e6 readme update 2019-10-22 11:14:19 -07:00
Kelly Brazil
36bc55a310 fix df 2019-10-22 11:11:41 -07:00
Kelly Brazil
a023001cd3 add df, env, and free parsers 2019-10-22 11:10:11 -07:00
Kelly Brazil
e3750b4962 documentation enhancements 2019-10-22 07:40:42 -07:00
Kelly Brazil
b5ea08e55b fix transport protocol 2019-10-21 18:22:51 -07:00
Kelly Brazil
8e71b8e352 fix jq example 2019-10-21 18:11:51 -07:00
14 changed files with 1362 additions and 79 deletions

555
README.md
View File

@@ -1,12 +1,12 @@
# JC
JSON CLI output utility
`jc` is used to JSONify the output of many standard linux cli tools for easier parsing in scripts. Parsers for `ls`, `ifconfig`, `ps`, `route`, and `netstat` are currently included and more can be added via modules.
`jc` is used to JSONify the output of many standard linux cli tools for easier parsing in scripts. See the **Parsers** section for supported commands.
This allows further command line processing of output with tools like `jq` simply by piping commands:
```
$ ls -l /usr/bin | jc --ls | jq .[] | jq 'select(.bytes > 50000000)'
$ ls -l /usr/bin | jc --ls | jq '.[] | select(.bytes > 50000000)'
{
"filename": "emacs",
"flags": "-r-xr-xr-x",
@@ -18,12 +18,32 @@ $ ls -l /usr/bin | jc --ls | jq .[] | jq 'select(.bytes > 50000000)'
}
```
The `jc` parsers can also be used as python modules by referencing them via:
The `jc` parsers can also be used as python modules:
```
import jc.parsers.[parser]
data = 'data to parse'
jc.parsers.[parser].parse(data)
>>> import jc.parsers.ls
>>>
>>> data='''-rwxr-xr-x 1 root wheel 23648 May 3 22:26 cat
... -rwxr-xr-x 1 root wheel 30016 May 3 22:26 chmod
... -rwxr-xr-x 1 root wheel 29024 May 3 22:26 cp
... -rwxr-xr-x 1 root wheel 375824 May 3 22:26 csh
... -rwxr-xr-x 1 root wheel 28608 May 3 22:26 date
... -rwxr-xr-x 1 root wheel 32000 May 3 22:26 dd
... -rwxr-xr-x 1 root wheel 23392 May 3 22:26 df
... -rwxr-xr-x 1 root wheel 18128 May 3 22:26 echo'''
>>>
>>> jc.parsers.ls.parse(data)
[{'filename': 'cat', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'bytes': 23648, 'date': 'May 3 22:26'}, {'filename': 'chmod', 'flags': '-rwxr-xr-x', 'links': 1,
'owner': 'root', 'group': 'wheel', 'bytes': 30016, 'date': 'May 3 22:26'}, {'filename': 'cp',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 29024,
'date': 'May 3 22:26'}, {'filename': 'csh', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'bytes': 375824, 'date': 'May 3 22:26'}, {'filename': 'date',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 28608,
'date': 'May 3 22:26'}, {'filename': 'dd', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'bytes': 32000, 'date': 'May 3 22:26'}, {'filename': 'df', 'flags': '-rwxr-xr-x',
'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 23392, 'date': 'May 3 22:26'},
{'filename': 'echo', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'bytes': 18128, 'date': 'May 3 22:26'}]
```
In this case the output will be a python dictionary instead of JSON.
@@ -39,17 +59,93 @@ jc [parser] [options]
`jc` accepts piped input from `STDIN` and outputs a JSON representation of the previous command's output to `STDOUT`. The JSON output can be compact or pretty formatted.
Parsers:
### Parsers
- `--df` enables the `df` parser
- `--env` enables the `env` parser
- `--free` enables the `free` parser
- `--ifconfig` enables the `ifconfig` parser
- `--iptables` enables the `iptables` parser
- `--ls` enables the `ls` parser
- `--lsblk` enables the `lsblk` parser
- `--mount` enables the `mount` parser
- `--netstat` enables the `netstat` parser
- `--ps` enables the `ps` parser
- `--route` enables the `route` parser
- `--uname` enables the `uname -a` parser
Options:
### Options
- `-p` specifies whether to pretty format the JSON output
## Examples
### df
```
$ df | jc --df -p
[
{
"Filesystem": "udev",
"1K-blocks": "977500",
"Used": "0",
"Available": "977500",
"Use%": "0%",
"Mounted": "/dev"
},
{
"Filesystem": "tmpfs",
"1K-blocks": "201732",
"Used": "1180",
"Available": "200552",
"Use%": "1%",
"Mounted": "/run"
},
{
"Filesystem": "/dev/sda2",
"1K-blocks": "20508240",
"Used": "5747284",
"Available": "13696152",
"Use%": "30%",
"Mounted": "/"
},
{
"Filesystem": "tmpfs",
"1K-blocks": "1008648",
"Used": "0",
"Available": "1008648",
"Use%": "0%",
"Mounted": "/dev/shm"
},
...
]
```
### env
```
$ env | jc --env -p
[
{
"TERM": "xterm-256color"
},
{
"SHELL": "/bin/bash"
},
{
"USER": "root"
},
{
"PATH": "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
},
{
"PWD": "/bin"
},
{
"LANG": "en_US.UTF-8"
},
{
"HOME": "/root"
},
{
"_": "/usr/bin/env"
}
]
```
### ifconfig
```
$ ifconfig | jc --ifconfig -p
@@ -134,6 +230,325 @@ $ ifconfig | jc --ifconfig -p
}
]
```
### iptables
```
$ sudo iptables -L -t nat | jc --iptables -p
[
{
"chain": "PREROUTING",
"rules": [
{
"target": "PREROUTING_direct",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "PREROUTING_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "PREROUTING_ZONES",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "DOCKER",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere",
"options": "ADDRTYPE match dst-type LOCAL"
}
]
},
{
"chain": "INPUT",
"rules": []
},
{
"chain": "OUTPUT",
"rules": [
{
"target": "OUTPUT_direct",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "DOCKER",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "!loopback/8",
"options": "ADDRTYPE match dst-type LOCAL"
}
]
},
...
]
```
```
$ sudo iptables -vnL -t filter | jc --iptables -p
[
{
"chain": "INPUT",
"rules": [
{
"pkts": "1571",
"bytes": "3394K",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "lo",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_direct",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DROP",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate INVALID"
},
{
"pkts": "710",
"bytes": "60078",
"target": "REJECT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "reject-with icmp-host-prohibited"
}
]
},
{
"chain": "FORWARD",
"rules": [
{
"pkts": "0",
"bytes": "0",
"target": "DOCKER-ISOLATION",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DOCKER",
"prot": "all",
"opt": "--",
"in": "*",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "docker0",
"out": "!docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "docker0",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "lo",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_direct",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_IN_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_IN_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_OUT_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_OUT_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DROP",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate INVALID"
},
{
"pkts": "0",
"bytes": "0",
"target": "REJECT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "reject-with icmp-host-prohibited"
}
]
},
...
]
```
### ls
```
$ ls -l /bin | jc --ls -p
@@ -168,12 +583,104 @@ $ ls -l /bin | jc --ls -p
...
]
```
### lsblk
```
$ lsblk | jc --lsblk -p
[
{
"NAME": "loop0",
"MAJ:MIN": "7:0",
"RM": "0",
"SIZE": "54.5M",
"RO": "1",
"TYPE": "loop",
"MOUNTPOINT": "/snap/core18/1223"
},
{
"NAME": "sda",
"MAJ:MIN": "8:0",
"RM": "0",
"SIZE": "20G",
"RO": "0",
"TYPE": "disk"
},
{
"NAME": "sda1",
"MAJ:MIN": "8:1",
"RM": "0",
"SIZE": "1M",
"RO": "0",
"TYPE": "part"
},
{
"NAME": "sda2",
"MAJ:MIN": "8:2",
"RM": "0",
"SIZE": "20G",
"RO": "0",
"TYPE": "part",
"MOUNTPOINT": "/"
},
{
"NAME": "sr0",
"MAJ:MIN": "11:0",
"RM": "1",
"SIZE": "64.8M",
"RO": "0",
"TYPE": "rom"
}
]
```
### mount
```
$ mount | jc --mount -p
[
{
"filesystem": "sysfs",
"mount_point": "/sys",
"type": "sysfs",
"access": [
"rw",
"nosuid",
"nodev",
"noexec",
"relatime"
]
},
{
"filesystem": "proc",
"mount_point": "/proc",
"type": "proc",
"access": [
"rw",
"nosuid",
"nodev",
"noexec",
"relatime"
]
},
{
"filesystem": "udev",
"mount_point": "/dev",
"type": "devtmpfs",
"access": [
"rw",
"nosuid",
"relatime",
"size=977500k",
"nr_inodes=244375",
"mode=755"
]
},
...
]
```
### netstat
```
$ netstat -p | jc --netstat -p
[
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "localhost.localdo",
"local_port": "34480",
@@ -186,7 +693,7 @@ $ netstat -p | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "localhost.localdo",
"local_port": "34478",
@@ -204,7 +711,7 @@ $ netstat -p | jc --netstat -p
$ netstat -lpn | jc --netstat -p
[
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "127.0.0.1",
"local_port": "42351",
@@ -217,7 +724,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "127.0.0.53",
"local_port": "53",
@@ -230,7 +737,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "0.0.0.0",
"local_port": "22",
@@ -243,7 +750,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv6",
"local_address": "::",
"local_port": "22",
@@ -256,7 +763,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "udp",
"transport_protocol": "udp",
"network_protocol": "ipv4",
"local_address": "127.0.0.53",
"local_port": "53",
@@ -268,7 +775,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "udp",
"transport_protocol": "udp",
"network_protocol": "ipv4",
"local_address": "192.168.71.131",
"local_port": "68",
@@ -374,6 +881,20 @@ $ route -n | jc --route -p
}
]
```
### uname -a
```
$ uname -a | jc --uname -p
{
"kernel_name": "Linux",
"node_name": "user-ubuntu",
"kernel_release": "4.15.0-65-generic",
"operating_system": "GNU/Linux",
"hardware_platform": "x86_64",
"processor": "x86_64",
"machine": "x86_64",
"kernel_version": "#74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019"
}
```
## Contributions
Feel free to add/improve code or parsers!

View File

@@ -1,6 +1,15 @@
jc changelog
20191021 v0.6.3
20191022 v0.8.1
- Add env parser
- Add df parser
- Add free parser
- Add lsblk parser
- Add mount parser
- Add uname parser
- Add iptables parser
20191021 v0.6.4
- Flatten netstat parser output
- Clean up argument parsing
- Fix command help

View File

@@ -5,55 +5,67 @@
This module serializes standard unix command line output to structured JSON
output.
Example:
CLI Example:
$ ls -al | jc | jq .
$ ls -l /bin | jc --ls -p
[
{
"filename": ".",
"suffix": Null,
"bytes": 224,
"date_updated": "Oct 1 12:09",
"owner_user": "joeuser",
"owner_group": "staff",
"flags": "drwxr-xr-x+",
"link_to": Null,
"links": 47
},
{
"filename": "..",
"suffix": Null,
"bytes": 224,
"date_updated": "Oct 1 12:09",
"owner_user": "admin",
"owner_group": "root",
"flags": "drwxr-xr-x",
"link_to": Null,
"links": 7
},
{
"filename": "testfile.txt",
"suffix": "txt",
"bytes": 14686,
"date_updated": "Oct 1 12:09",
"owner_user": "joeuser",
"owner_group": "staff",
"flags": "-rwxr-xr-x@",
"link_to": Null,
"links": 1
},
{
"filename": "ncat",
"suffix": Null,
"bytes": 14686,
"date_updated": "Oct 1 12:09",
"owner_user": "joeuser",
"owner_group": "staff",
"flags": "lrwxr-xr-x",
"link_to": "../Cellar/nmap/7.70/bin/ncat",
"links": 1
}
{
"filename": "bash",
"flags": "-r-xr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 618416,
"date": "May 3 22:26"
},
{
"filename": "cat",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 23648,
"date": "May 3 22:26"
},
{
"filename": "chmod",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 30016,
"date": "May 3 22:26"
},
...
]
Module Example:
>>> import jc.parsers.ls
>>>
>>> data='''-rwxr-xr-x 1 root wheel 23648 May 3 22:26 cat
... -rwxr-xr-x 1 root wheel 30016 May 3 22:26 chmod
... -rwxr-xr-x 1 root wheel 29024 May 3 22:26 cp
... -rwxr-xr-x 1 root wheel 375824 May 3 22:26 csh
... -rwxr-xr-x 1 root wheel 28608 May 3 22:26 date
... -rwxr-xr-x 1 root wheel 32000 May 3 22:26 dd
... -rwxr-xr-x 1 root wheel 23392 May 3 22:26 df
... -rwxr-xr-x 1 root wheel 18128 May 3 22:26 echo'''
>>>
>>> jc.parsers.ls.parse(data)
[{'filename': 'cat', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'bytes': 23648, 'date': 'May 3 22:26'}, {'filename': 'chmod', 'flags': '-rwxr-xr-x', 'links': 1,
'owner': 'root', 'group': 'wheel', 'bytes': 30016, 'date': 'May 3 22:26'}, {'filename': 'cp',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 29024,
'date': 'May 3 22:26'}, {'filename': 'csh', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'bytes': 375824, 'date': 'May 3 22:26'}, {'filename': 'date',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 28608,
'date': 'May 3 22:26'}, {'filename': 'dd', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'bytes': 32000, 'date': 'May 3 22:26'}, {'filename': 'df', 'flags': '-rwxr-xr-x',
'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 23392, 'date': 'May 3 22:26'},
{'filename': 'echo', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'bytes': 18128, 'date': 'May 3 22:26'}]
"""
name = 'jc'

View File

@@ -6,11 +6,18 @@ Main input module
import sys
import json
import jc.parsers.df
import jc.parsers.env
import jc.parsers.free
import jc.parsers.ifconfig
import jc.parsers.iptables
import jc.parsers.ls
import jc.parsers.lsblk
import jc.parsers.mount
import jc.parsers.netstat
import jc.parsers.ps
import jc.parsers.route
import jc.parsers.uname
def main():
@@ -20,12 +27,30 @@ def main():
if '-p' in sys.argv:
pretty = True
if '--ifconfig' in sys.argv:
if '--df' in sys.argv:
result = jc.parsers.df.parse(data)
elif '--env' in sys.argv:
result = jc.parsers.env.parse(data)
elif '--free' in sys.argv:
result = jc.parsers.free.parse(data)
elif '--ifconfig' in sys.argv:
result = jc.parsers.ifconfig.parse(data)
elif '--iptables' in sys.argv:
result = jc.parsers.iptables.parse(data)
elif '--ls' in sys.argv:
result = jc.parsers.ls.parse(data)
elif '--lsblk' in sys.argv:
result = jc.parsers.lsblk.parse(data)
elif '--mount' in sys.argv:
result = jc.parsers.mount.parse(data)
elif '--netstat' in sys.argv:
result = jc.parsers.netstat.parse(data)
@@ -35,15 +60,25 @@ def main():
elif '--route' in sys.argv:
result = jc.parsers.route.parse(data)
elif '--uname' in sys.argv:
result = jc.parsers.uname.parse(data)
else:
print('jc: missing arguments', file=sys.stderr)
print('jc: missing arguments\n', file=sys.stderr)
print('Usage: jc [parser] [options]\n', file=sys.stderr)
print('Parsers:', file=sys.stderr)
print(' --df df parser', file=sys.stderr)
print(' --env env parser', file=sys.stderr)
print(' --free free parser', file=sys.stderr)
print(' --ifconfig iconfig parser', file=sys.stderr)
print(' --iptables iptables parser', file=sys.stderr)
print(' --ls ls parser', file=sys.stderr)
print(' --lsblk lsblk parser', file=sys.stderr)
print(' --mount mount parser', file=sys.stderr)
print(' --netstat netstat parser', file=sys.stderr)
print(' --ps ps parser', file=sys.stderr)
print(' --route route parser\n', file=sys.stderr)
print(' --route route parser', file=sys.stderr)
print(' --uname uname parser\n', file=sys.stderr)
print('Options:', file=sys.stderr)
print(' -p pretty print output\n', file=sys.stderr)
print('Example:', file=sys.stderr)

55
jc/parsers/df.py Normal file
View File

@@ -0,0 +1,55 @@
"""jc - JSON CLI output utility df Parser
Usage:
specify --df as the first argument if the piped input is coming from df
Example:
$ df | jc --df -p
[
{
"Filesystem": "udev",
"1K-blocks": "977500",
"Used": "0",
"Available": "977500",
"Use%": "0%",
"Mounted": "/dev"
},
{
"Filesystem": "tmpfs",
"1K-blocks": "201732",
"Used": "1180",
"Available": "200552",
"Use%": "1%",
"Mounted": "/run"
},
{
"Filesystem": "/dev/sda2",
"1K-blocks": "20508240",
"Used": "5747284",
"Available": "13696152",
"Use%": "30%",
"Mounted": "/"
},
{
"Filesystem": "tmpfs",
"1K-blocks": "1008648",
"Used": "0",
"Available": "1008648",
"Use%": "0%",
"Mounted": "/dev/shm"
},
...
]
"""
def parse(data):
# code adapted from Conor Heine at:
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
return [dict(zip(headers, r)) for r in raw_data]

53
jc/parsers/env.py Normal file
View File

@@ -0,0 +1,53 @@
"""jc - JSON CLI output utility env Parser
Usage:
specify --env as the first argument if the piped input is coming from env
Example:
$ env | jc --env -p
[
{
"TERM": "xterm-256color"
},
{
"SHELL": "/bin/bash"
},
{
"USER": "root"
},
{
"PATH": "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
},
{
"PWD": "/bin"
},
{
"LANG": "en_US.UTF-8"
},
{
"HOME": "/root"
},
{
"_": "/usr/bin/env"
}
]
"""
def parse(data):
output = []
linedata = data.splitlines()
# Clear any blank lines
cleandata = list(filter(None, linedata))
if cleandata:
for entry in cleandata:
output_line = {}
parsed_line = entry.split('=', maxsplit=1)
output_line[parsed_line[0]] = parsed_line[1]
output.append(output_line)
return output

45
jc/parsers/free.py Normal file
View File

@@ -0,0 +1,45 @@
"""jc - JSON CLI output utility free Parser
Usage:
specify --free as the first argument if the piped input is coming from free
Example:
$ free | jc --free -p
[
{
"type": "Mem",
"total": "2017300",
"used": "213104",
"free": "1148452",
"shared": "1176",
"buff/cache": "655744",
"available": "1622204"
},
{
"type": "Swap",
"total": "2097148",
"used": "0",
"free": "2097148"
}
]
"""
def parse(data):
# code adapted from Conor Heine at:
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h]
headers.insert(0, "type")
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
output = [dict(zip(headers, r)) for r in raw_data]
for entry in output:
entry['type'] = entry['type'].rstrip(':')
return output

366
jc/parsers/iptables.py Normal file
View File

@@ -0,0 +1,366 @@
"""jc - JSON CLI output utility ipables Parser
Usage:
Specify --iptables as the first argument if the piped input is coming from iptables
Supports -vLn for all tables
Examples:
$ sudo iptables -L -t nat | jc --iptables -p
[
{
"chain": "PREROUTING",
"rules": [
{
"target": "PREROUTING_direct",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "PREROUTING_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "PREROUTING_ZONES",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "DOCKER",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere",
"options": "ADDRTYPE match dst-type LOCAL"
}
]
},
{
"chain": "INPUT",
"rules": []
},
{
"chain": "OUTPUT",
"rules": [
{
"target": "OUTPUT_direct",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "DOCKER",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "!loopback/8",
"options": "ADDRTYPE match dst-type LOCAL"
}
]
},
...
]
$ sudo iptables -vnL -t filter | jc --iptables -p
[
{
"chain": "INPUT",
"rules": [
{
"pkts": "1571",
"bytes": "3394K",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "lo",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_direct",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DROP",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate INVALID"
},
{
"pkts": "710",
"bytes": "60078",
"target": "REJECT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "reject-with icmp-host-prohibited"
}
]
},
{
"chain": "FORWARD",
"rules": [
{
"pkts": "0",
"bytes": "0",
"target": "DOCKER-ISOLATION",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DOCKER",
"prot": "all",
"opt": "--",
"in": "*",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "docker0",
"out": "!docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "docker0",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "lo",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_direct",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_IN_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_IN_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_OUT_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_OUT_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DROP",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate INVALID"
},
{
"pkts": "0",
"bytes": "0",
"target": "REJECT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "reject-with icmp-host-prohibited"
}
]
},
...
]
"""
class state():
output = []
chain = {}
headers = []
def parse(data):
cleandata = data.splitlines()
for line in cleandata:
if line.find('Chain') == 0:
state.output.append(state.chain)
state.chain = {}
state.headers = []
parsed_line = line.split()
state.chain['chain'] = parsed_line[1]
state.chain['rules'] = []
continue
if line.find('target') == 0 or line.find('pkts') == 1:
state.headers = []
state.headers = [h for h in ' '.join(line.strip().split()).split() if h]
state.headers.append("options")
continue
else:
rule = line.split(maxsplit=len(state.headers) - 1)
temp_rule = dict(zip(state.headers, rule))
if temp_rule:
state.chain['rules'].append(temp_rule)
state.output = list(filter(None, state.output))
return state.output

View File

@@ -72,7 +72,7 @@ $ ls -al /usr/bin | jc --ls -p
...
]
$ $ ls -l /usr/bin | jc --ls | jq .[] | jq 'select(.bytes > 50000000)'
$ $ ls -l /usr/bin | jc --ls | jq '.[] | 'select(.bytes > 50000000)'
{
"filename": "emacs",
"flags": "-r-xr-xr-x",

70
jc/parsers/lsblk.py Normal file
View File

@@ -0,0 +1,70 @@
"""jc - JSON CLI output utility lsblk Parser
Usage:
specify --lsblk as the first argument if the piped input is coming from lsblk
Example:
$ lsblk | jc --lsblk -p
[
{
"NAME": "loop0",
"MAJ:MIN": "7:0",
"RM": "0",
"SIZE": "54.5M",
"RO": "1",
"TYPE": "loop",
"MOUNTPOINT": "/snap/core18/1223"
},
{
"NAME": "sda",
"MAJ:MIN": "8:0",
"RM": "0",
"SIZE": "20G",
"RO": "0",
"TYPE": "disk"
},
{
"NAME": "sda1",
"MAJ:MIN": "8:1",
"RM": "0",
"SIZE": "1M",
"RO": "0",
"TYPE": "part"
},
{
"NAME": "sda2",
"MAJ:MIN": "8:2",
"RM": "0",
"SIZE": "20G",
"RO": "0",
"TYPE": "part",
"MOUNTPOINT": "/"
},
{
"NAME": "sr0",
"MAJ:MIN": "11:0",
"RM": "1",
"SIZE": "64.8M",
"RO": "0",
"TYPE": "rom"
}
]
"""
def parse(data):
# code adapted from Conor Heine at:
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
output = [dict(zip(headers, r)) for r in raw_data]
for entry in output:
entry['NAME'] = entry['NAME'].encode('ascii', errors='ignore').decode()
return output

75
jc/parsers/mount.py Normal file
View File

@@ -0,0 +1,75 @@
"""jc - JSON CLI output utility mount Parser
Usage:
specify --mount as the first argument if the piped input is coming from mount
Example:
$ mount | jc --mount -p
[
{
"filesystem": "sysfs",
"mount_point": "/sys",
"type": "sysfs",
"access": [
"rw",
"nosuid",
"nodev",
"noexec",
"relatime"
]
},
{
"filesystem": "proc",
"mount_point": "/proc",
"type": "proc",
"access": [
"rw",
"nosuid",
"nodev",
"noexec",
"relatime"
]
},
{
"filesystem": "udev",
"mount_point": "/dev",
"type": "devtmpfs",
"access": [
"rw",
"nosuid",
"relatime",
"size=977500k",
"nr_inodes=244375",
"mode=755"
]
},
...
]
"""
def parse(data):
output = []
linedata = data.splitlines()
# Clear any blank lines
cleandata = list(filter(None, linedata))
if cleandata:
for entry in cleandata:
output_line = {}
parsed_line = entry.split()
output_line['filesystem'] = parsed_line[0]
output_line['mount_point'] = parsed_line[2]
output_line['type'] = parsed_line[4]
access = parsed_line[5].lstrip('(').rstrip(')').split(',')
output_line['access'] = access
output.append(output_line)
return output

View File

@@ -13,7 +13,7 @@ Examples:
$ netstat -p | jc --netstat -p
[
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "localhost.localdo",
"local_port": "34480",
@@ -26,7 +26,7 @@ $ netstat -p | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "localhost.localdo",
"local_port": "34478",
@@ -43,7 +43,7 @@ $ netstat -p | jc --netstat -p
$ netstat -lpn | jc --netstat -p
[
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "127.0.0.1",
"local_port": "42351",
@@ -56,7 +56,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "127.0.0.53",
"local_port": "53",
@@ -69,7 +69,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "0.0.0.0",
"local_port": "22",
@@ -82,7 +82,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv6",
"local_address": "::",
"local_port": "22",
@@ -95,7 +95,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "udp",
"transport_protocol": "udp",
"network_protocol": "ipv4",
"local_address": "127.0.0.53",
"local_port": "53",
@@ -107,7 +107,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "udp",
"transport_protocol": "udp",
"network_protocol": "ipv4",
"local_address": "192.168.71.131",
"local_port": "68",
@@ -129,7 +129,7 @@ def parse_line(entry):
output_line = {}
if entry.find('tcp') == 0:
output_line['session_protocol'] = 'tcp'
output_line['transport_protocol'] = 'tcp'
if entry.find('p6') == 2:
output_line['network_protocol'] = 'ipv6'
@@ -138,7 +138,7 @@ def parse_line(entry):
output_line['network_protocol'] = 'ipv4'
elif entry.find('udp') == 0:
output_line['session_protocol'] = 'udp'
output_line['transport_protocol'] = 'udp'
if entry.find('p6') == 2:
output_line['network_protocol'] = 'ipv6'

42
jc/parsers/uname.py Normal file
View File

@@ -0,0 +1,42 @@
"""jc - JSON CLI output utility uname Parser
Usage:
specify --uname as the first argument if the piped input is coming from uname
Limitations:
must use 'uname -a'
Example:
$ uname -a | jc --uname -p
{
"kernel_name": "Linux",
"node_name": "user-ubuntu",
"kernel_release": "4.15.0-65-generic",
"operating_system": "GNU/Linux",
"hardware_platform": "x86_64",
"processor": "x86_64",
"machine": "x86_64",
"kernel_version": "#74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019"
}
"""
def parse(data):
output = {}
parsed_line = data.split(maxsplit=3)
output['kernel_name'] = parsed_line.pop(0)
output['node_name'] = parsed_line.pop(0)
output['kernel_release'] = parsed_line.pop(0)
parsed_line = parsed_line[-1].rsplit(maxsplit=4)
output['operating_system'] = parsed_line.pop(-1)
output['hardware_platform'] = parsed_line.pop(-1)
output['processor'] = parsed_line.pop(-1)
output['machine'] = parsed_line.pop(-1)
output['kernel_version'] = parsed_line.pop(0)
return output

View File

@@ -5,7 +5,7 @@ with open('README.md', 'r') as f:
setuptools.setup(
name='jc',
version='0.6.3',
version='0.8.1',
author='Kelly Brazil',
author_email='kellyjonbrazil@gmail.com',
description='This tool serializes the output of popular command line tools to structured JSON output.',