1
0
mirror of https://github.com/kellyjonbrazil/jc.git synced 2026-04-03 17:44:07 +02:00

Compare commits

..

37 Commits

Author SHA1 Message Date
Kelly Brazil
e7921b65f5 Merge pull request #4 from kellyjonbrazil/dev
Dev v0.9.1
2019-10-23 18:41:55 -07:00
Kelly Brazil
2cc1b1bd54 version bump 2019-10-23 18:39:24 -07:00
Kelly Brazil
58ae976db0 documentation update 2019-10-23 18:30:55 -07:00
Kelly Brazil
66772392ae add lsmod parser 2019-10-23 18:04:54 -07:00
Kelly Brazil
29c47c03a6 documentation update 2019-10-23 17:37:25 -07:00
Kelly Brazil
91eb9a4d13 use None instead of -- 2019-10-23 17:27:23 -07:00
Kelly Brazil
a1a3de32ec add lsof parser 2019-10-23 17:22:25 -07:00
Kelly Brazil
9c47fd05bf doco fix 2019-10-23 14:11:13 -07:00
Kelly Brazil
649c0aa7c1 add documentation 2019-10-23 14:10:10 -07:00
Kelly Brazil
3db758764e add jobs parser 2019-10-23 14:05:47 -07:00
Kelly Brazil
802f1510eb tighten if statements 2019-10-23 10:27:05 -07:00
Kelly Brazil
56901788de stop blocking when no pipe and enhance help text 2019-10-23 09:51:29 -07:00
Kelly Brazil
679ae6d5dc version bump 2019-10-23 09:20:11 -07:00
Kelly Brazil
b15c8c352a simplify state variables 2019-10-23 08:46:54 -07:00
Kelly Brazil
393e8bc560 Merge pull request #3 from kellyjonbrazil/dev
Dev v0.8.1
2019-10-22 17:26:40 -07:00
Kelly Brazil
976fd7d9bd readme update 2019-10-22 17:24:56 -07:00
Kelly Brazil
d8337870ca update documentation 2019-10-22 17:21:00 -07:00
Kelly Brazil
39a8aec77f v0.8.1 build 2019-10-22 17:15:02 -07:00
Kelly Brazil
306d539b6b readme update 2019-10-22 16:50:01 -07:00
Kelly Brazil
f3087b8a8e update readme and formatting 2019-10-22 16:40:27 -07:00
Kelly Brazil
414c2ecef8 fix iptables parser 2019-10-22 16:32:55 -07:00
Kelly Brazil
776ef2d1be add iptables parser 2019-10-22 15:42:29 -07:00
Kelly Brazil
9ac5746996 add uname parser 2019-10-22 13:28:15 -07:00
Kelly Brazil
a3e55d97c0 add mount parser 2019-10-22 12:54:41 -07:00
Kelly Brazil
b15227e7ba add lsblk parser 2019-10-22 11:55:11 -07:00
Kelly Brazil
ec3d1f84ce fix free parser 2019-10-22 11:26:58 -07:00
Kelly Brazil
753d5fd9fe readme update 2019-10-22 11:17:21 -07:00
Kelly Brazil
73a0d70c92 readme update 2019-10-22 11:15:44 -07:00
Kelly Brazil
c2c189f3e6 readme update 2019-10-22 11:14:19 -07:00
Kelly Brazil
36bc55a310 fix df 2019-10-22 11:11:41 -07:00
Kelly Brazil
a023001cd3 add df, env, and free parsers 2019-10-22 11:10:11 -07:00
Kelly Brazil
e3750b4962 documentation enhancements 2019-10-22 07:40:42 -07:00
Kelly Brazil
b5ea08e55b fix transport protocol 2019-10-21 18:22:51 -07:00
Kelly Brazil
8e71b8e352 fix jq example 2019-10-21 18:11:51 -07:00
Kelly Brazil
4c8610c54f fixed build 2019-10-21 17:59:32 -07:00
Kelly Brazil
c8f886dc8f fix example 2019-10-21 17:56:53 -07:00
Kelly Brazil
4cfc2d22b3 update changelog 2019-10-21 17:38:40 -07:00
17 changed files with 1872 additions and 89 deletions

717
README.md
View File

@@ -1,12 +1,12 @@
# JC
JSON CLI output utility
`jc` is used to JSONify the output of many standard linux cli tools for easier parsing in scripts. Parsers for `ls`, `ifconfig`, `ps`, `route`, and `netstat` are currently included and more can be added via modules.
`jc` is used to JSONify the output of many standard linux cli tools for easier parsing in scripts. See the **Parsers** section for supported commands.
This allows further command line processing of output with tools like `jq` simply by piping commands:
```
$ ls -l /usr/bin | jc --ls | jq .[] | jq 'select(.bytes > 50000000)'
$ ls -l /usr/bin | jc --ls | jq '.[] | select(.bytes > 50000000)'
{
"filename": "emacs",
"flags": "-r-xr-xr-x",
@@ -18,12 +18,32 @@ $ ls -l /usr/bin | jc --ls | jq .[] | jq 'select(.bytes > 50000000)'
}
```
The `jc` parsers can also be used as python modules by referencing them via:
The `jc` parsers can also be used as python modules:
```
import jc.parsers.[parser]
data = 'data to parse'
jc.parsers.[parser].parse(data)
>>> import jc.parsers.ls
>>>
>>> data='''-rwxr-xr-x 1 root wheel 23648 May 3 22:26 cat
... -rwxr-xr-x 1 root wheel 30016 May 3 22:26 chmod
... -rwxr-xr-x 1 root wheel 29024 May 3 22:26 cp
... -rwxr-xr-x 1 root wheel 375824 May 3 22:26 csh
... -rwxr-xr-x 1 root wheel 28608 May 3 22:26 date
... -rwxr-xr-x 1 root wheel 32000 May 3 22:26 dd
... -rwxr-xr-x 1 root wheel 23392 May 3 22:26 df
... -rwxr-xr-x 1 root wheel 18128 May 3 22:26 echo'''
>>>
>>> jc.parsers.ls.parse(data)
[{'filename': 'cat', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'bytes': 23648, 'date': 'May 3 22:26'}, {'filename': 'chmod', 'flags': '-rwxr-xr-x', 'links': 1,
'owner': 'root', 'group': 'wheel', 'bytes': 30016, 'date': 'May 3 22:26'}, {'filename': 'cp',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 29024,
'date': 'May 3 22:26'}, {'filename': 'csh', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'bytes': 375824, 'date': 'May 3 22:26'}, {'filename': 'date',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 28608,
'date': 'May 3 22:26'}, {'filename': 'dd', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'bytes': 32000, 'date': 'May 3 22:26'}, {'filename': 'df', 'flags': '-rwxr-xr-x',
'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 23392, 'date': 'May 3 22:26'},
{'filename': 'echo', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'bytes': 18128, 'date': 'May 3 22:26'}]
```
In this case the output will be a python dictionary instead of JSON.
@@ -39,17 +59,96 @@ jc [parser] [options]
`jc` accepts piped input from `STDIN` and outputs a JSON representation of the previous command's output to `STDOUT`. The JSON output can be compact or pretty formatted.
Parsers:
### Parsers
- `--df` enables the `df` parser
- `--env` enables the `env` parser
- `--free` enables the `free` parser
- `--ifconfig` enables the `ifconfig` parser
- `--iptables` enables the `iptables` parser
- `--jobs` enables the `jobs` parser
- `--ls` enables the `ls` parser
- `--lsblk` enables the `lsblk` parser
- `--lsmod` enables the `lsmod` parser
- `--lsof` enables the `lsof` parser
- `--mount` enables the `mount` parser
- `--netstat` enables the `netstat` parser
- `--ps` enables the `ps` parser
- `--route` enables the `route` parser
- `--uname` enables the `uname -a` parser
Options:
### Options
- `-p` specifies whether to pretty format the JSON output
## Examples
### df
```
$ df | jc --df -p
[
{
"Filesystem": "udev",
"1K-blocks": "977500",
"Used": "0",
"Available": "977500",
"Use%": "0%",
"Mounted": "/dev"
},
{
"Filesystem": "tmpfs",
"1K-blocks": "201732",
"Used": "1180",
"Available": "200552",
"Use%": "1%",
"Mounted": "/run"
},
{
"Filesystem": "/dev/sda2",
"1K-blocks": "20508240",
"Used": "5747284",
"Available": "13696152",
"Use%": "30%",
"Mounted": "/"
},
{
"Filesystem": "tmpfs",
"1K-blocks": "1008648",
"Used": "0",
"Available": "1008648",
"Use%": "0%",
"Mounted": "/dev/shm"
},
...
]
```
### env
```
$ env | jc --env -p
[
{
"TERM": "xterm-256color"
},
{
"SHELL": "/bin/bash"
},
{
"USER": "root"
},
{
"PATH": "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
},
{
"PWD": "/bin"
},
{
"LANG": "en_US.UTF-8"
},
{
"HOME": "/root"
},
{
"_": "/usr/bin/env"
}
]
```
### ifconfig
```
$ ifconfig | jc --ifconfig -p
@@ -134,6 +233,363 @@ $ ifconfig | jc --ifconfig -p
}
]
```
### iptables
```
$ sudo iptables -L -t nat | jc --iptables -p
[
{
"chain": "PREROUTING",
"rules": [
{
"target": "PREROUTING_direct",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "PREROUTING_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "PREROUTING_ZONES",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "DOCKER",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere",
"options": "ADDRTYPE match dst-type LOCAL"
}
]
},
{
"chain": "INPUT",
"rules": []
},
{
"chain": "OUTPUT",
"rules": [
{
"target": "OUTPUT_direct",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "DOCKER",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "!loopback/8",
"options": "ADDRTYPE match dst-type LOCAL"
}
]
},
...
]
```
```
$ sudo iptables -vnL -t filter | jc --iptables -p
[
{
"chain": "INPUT",
"rules": [
{
"pkts": "1571",
"bytes": "3394K",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "lo",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_direct",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DROP",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate INVALID"
},
{
"pkts": "710",
"bytes": "60078",
"target": "REJECT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "reject-with icmp-host-prohibited"
}
]
},
{
"chain": "FORWARD",
"rules": [
{
"pkts": "0",
"bytes": "0",
"target": "DOCKER-ISOLATION",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DOCKER",
"prot": "all",
"opt": "--",
"in": "*",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "docker0",
"out": "!docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "docker0",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "lo",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_direct",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_IN_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_IN_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_OUT_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_OUT_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DROP",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate INVALID"
},
{
"pkts": "0",
"bytes": "0",
"target": "REJECT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "reject-with icmp-host-prohibited"
}
]
},
...
]
```
### jobs
```
$ jobs -l | jc --jobs -p
[
{
"job_number": 1,
"pid": 14798,
"status": "Running",
"command": "sleep 10000 &"
},
{
"job_number": 2,
"pid": 14799,
"status": "Running",
"command": "sleep 10001 &"
},
{
"job_number": 3,
"pid": 14800,
"status": "Running",
"command": "sleep 10002 &"
},
{
"job_number": 4,
"pid": 14814,
"history": "previous",
"status": "Running",
"command": "sleep 10003 &"
},
{
"job_number": 5,
"pid": 14815,
"history": "current",
"status": "Running",
"command": "sleep 10004 &"
}
]
```
### ls
```
$ ls -l /bin | jc --ls -p
@@ -168,12 +624,225 @@ $ ls -l /bin | jc --ls -p
...
]
```
### lsblk
```
$ lsblk | jc --lsblk -p
[
{
"NAME": "loop0",
"MAJ:MIN": "7:0",
"RM": "0",
"SIZE": "54.5M",
"RO": "1",
"TYPE": "loop",
"MOUNTPOINT": "/snap/core18/1223"
},
{
"NAME": "sda",
"MAJ:MIN": "8:0",
"RM": "0",
"SIZE": "20G",
"RO": "0",
"TYPE": "disk"
},
{
"NAME": "sda1",
"MAJ:MIN": "8:1",
"RM": "0",
"SIZE": "1M",
"RO": "0",
"TYPE": "part"
},
{
"NAME": "sda2",
"MAJ:MIN": "8:2",
"RM": "0",
"SIZE": "20G",
"RO": "0",
"TYPE": "part",
"MOUNTPOINT": "/"
},
{
"NAME": "sr0",
"MAJ:MIN": "11:0",
"RM": "1",
"SIZE": "64.8M",
"RO": "0",
"TYPE": "rom"
}
]
```
### lsmod
```
$ lsmod | jc --lsmod -p
[
{
"Module": "nf_nat_ipv4",
"Size": "14115",
"Used": "1",
"By": [
"iptable_nat"
]
},
{
"Module": "nf_nat",
"Size": "26583",
"Used": "3",
"By": [
"nf_nat_ipv4",
"nf_nat_ipv6",
"nf_nat_masquerade_ipv4"
]
},
{
"Module": "iptable_mangle",
"Size": "12695",
"Used": "1"
},
{
"Module": "iptable_security",
"Size": "12705",
"Used": "1"
},
{
"Module": "iptable_raw",
"Size": "12678",
"Used": "1"
},
{
"Module": "nf_conntrack",
"Size": "139224",
"Used": "7",
"By": [
"nf_nat",
"nf_nat_ipv4",
"nf_nat_ipv6",
"xt_conntrack",
"nf_nat_masquerade_ipv4",
"nf_conntrack_ipv4",
"nf_conntrack_ipv6"
]
},
...
]
```
### lsof
```
$ sudo lsof | jc --lsof -p
[
{
"COMMAND": "systemd",
"PID": "1",
"TID": null,
"USER": "root",
"FD": "cwd",
"TYPE": "DIR",
"DEVICE": "253,0",
"SIZE/OFF": "224",
"NODE": "64",
"NAME": "/"
},
{
"COMMAND": "systemd",
"PID": "1",
"TID": null,
"USER": "root",
"FD": "rtd",
"TYPE": "DIR",
"DEVICE": "253,0",
"SIZE/OFF": "224",
"NODE": "64",
"NAME": "/"
},
{
"COMMAND": "systemd",
"PID": "1",
"TID": null,
"USER": "root",
"FD": "txt",
"TYPE": "REG",
"DEVICE": "253,0",
"SIZE/OFF": "1624520",
"NODE": "50360451",
"NAME": "/usr/lib/systemd/systemd"
},
{
"COMMAND": "systemd",
"PID": "1",
"TID": null,
"USER": "root",
"FD": "mem",
"TYPE": "REG",
"DEVICE": "253,0",
"SIZE/OFF": "20064",
"NODE": "8146",
"NAME": "/usr/lib64/libuuid.so.1.3.0"
},
{
"COMMAND": "systemd",
"PID": "1",
"TID": null,
"USER": "root",
"FD": "mem",
"TYPE": "REG",
"DEVICE": "253,0",
"SIZE/OFF": "265600",
"NODE": "8147",
"NAME": "/usr/lib64/libblkid.so.1.1.0"
},
...
]
```
### mount
```
$ mount | jc --mount -p
[
{
"filesystem": "sysfs",
"mount_point": "/sys",
"type": "sysfs",
"access": [
"rw",
"nosuid",
"nodev",
"noexec",
"relatime"
]
},
{
"filesystem": "proc",
"mount_point": "/proc",
"type": "proc",
"access": [
"rw",
"nosuid",
"nodev",
"noexec",
"relatime"
]
},
{
"filesystem": "udev",
"mount_point": "/dev",
"type": "devtmpfs",
"access": [
"rw",
"nosuid",
"relatime",
"size=977500k",
"nr_inodes=244375",
"mode=755"
]
},
...
]
```
### netstat
```
$ netstat -p | jc --netstat -p
[
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "localhost.localdo",
"local_port": "34480",
@@ -186,7 +855,7 @@ $ netstat -p | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "localhost.localdo",
"local_port": "34478",
@@ -204,7 +873,7 @@ $ netstat -p | jc --netstat -p
$ netstat -lpn | jc --netstat -p
[
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "127.0.0.1",
"local_port": "42351",
@@ -217,7 +886,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "127.0.0.53",
"local_port": "53",
@@ -230,7 +899,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "0.0.0.0",
"local_port": "22",
@@ -243,7 +912,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv6",
"local_address": "::",
"local_port": "22",
@@ -256,7 +925,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "udp",
"transport_protocol": "udp",
"network_protocol": "ipv4",
"local_address": "127.0.0.53",
"local_port": "53",
@@ -268,7 +937,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "udp",
"transport_protocol": "udp",
"network_protocol": "ipv4",
"local_address": "192.168.71.131",
"local_port": "68",
@@ -374,6 +1043,20 @@ $ route -n | jc --route -p
}
]
```
### uname -a
```
$ uname -a | jc --uname -p
{
"kernel_name": "Linux",
"node_name": "user-ubuntu",
"kernel_release": "4.15.0-65-generic",
"operating_system": "GNU/Linux",
"hardware_platform": "x86_64",
"processor": "x86_64",
"machine": "x86_64",
"kernel_version": "#74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019"
}
```
## Contributions
Feel free to add/improve code or parsers!

View File

@@ -1,8 +1,26 @@
jc changelog
20191021 v0.6.1
20191023 v0.9.1
- Add jobs parser
- Add lsof parser
- Add lsmod parser
- No blocking if no piped data
- Better help text
- Clean up iptables parser code
20191022 v0.8.1
- Add env parser
- Add df parser
- Add free parser
- Add lsblk parser
- Add mount parser
- Add uname parser
- Add iptables parser
20191021 v0.6.4
- Flatten netstat parser output
- Clean up argument parsing
- Fix command help
20191018 v0.5.5
- Fix netstat -p parsing for Ubuntu

View File

@@ -5,55 +5,67 @@
This module serializes standard unix command line output to structured JSON
output.
Example:
CLI Example:
$ ls -al | jc | jq .
$ ls -l /bin | jc --ls -p
[
{
"filename": ".",
"suffix": Null,
"bytes": 224,
"date_updated": "Oct 1 12:09",
"owner_user": "joeuser",
"owner_group": "staff",
"flags": "drwxr-xr-x+",
"link_to": Null,
"links": 47
},
{
"filename": "..",
"suffix": Null,
"bytes": 224,
"date_updated": "Oct 1 12:09",
"owner_user": "admin",
"owner_group": "root",
"flags": "drwxr-xr-x",
"link_to": Null,
"links": 7
},
{
"filename": "testfile.txt",
"suffix": "txt",
"bytes": 14686,
"date_updated": "Oct 1 12:09",
"owner_user": "joeuser",
"owner_group": "staff",
"flags": "-rwxr-xr-x@",
"link_to": Null,
"links": 1
},
{
"filename": "ncat",
"suffix": Null,
"bytes": 14686,
"date_updated": "Oct 1 12:09",
"owner_user": "joeuser",
"owner_group": "staff",
"flags": "lrwxr-xr-x",
"link_to": "../Cellar/nmap/7.70/bin/ncat",
"links": 1
}
{
"filename": "bash",
"flags": "-r-xr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 618416,
"date": "May 3 22:26"
},
{
"filename": "cat",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 23648,
"date": "May 3 22:26"
},
{
"filename": "chmod",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 30016,
"date": "May 3 22:26"
},
...
]
Module Example:
>>> import jc.parsers.ls
>>>
>>> data='''-rwxr-xr-x 1 root wheel 23648 May 3 22:26 cat
... -rwxr-xr-x 1 root wheel 30016 May 3 22:26 chmod
... -rwxr-xr-x 1 root wheel 29024 May 3 22:26 cp
... -rwxr-xr-x 1 root wheel 375824 May 3 22:26 csh
... -rwxr-xr-x 1 root wheel 28608 May 3 22:26 date
... -rwxr-xr-x 1 root wheel 32000 May 3 22:26 dd
... -rwxr-xr-x 1 root wheel 23392 May 3 22:26 df
... -rwxr-xr-x 1 root wheel 18128 May 3 22:26 echo'''
>>>
>>> jc.parsers.ls.parse(data)
[{'filename': 'cat', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'bytes': 23648, 'date': 'May 3 22:26'}, {'filename': 'chmod', 'flags': '-rwxr-xr-x', 'links': 1,
'owner': 'root', 'group': 'wheel', 'bytes': 30016, 'date': 'May 3 22:26'}, {'filename': 'cp',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 29024,
'date': 'May 3 22:26'}, {'filename': 'csh', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'bytes': 375824, 'date': 'May 3 22:26'}, {'filename': 'date',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 28608,
'date': 'May 3 22:26'}, {'filename': 'dd', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'bytes': 32000, 'date': 'May 3 22:26'}, {'filename': 'df', 'flags': '-rwxr-xr-x',
'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 23392, 'date': 'May 3 22:26'},
{'filename': 'echo', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'bytes': 18128, 'date': 'May 3 22:26'}]
"""
name = 'jc'

View File

@@ -6,26 +6,94 @@ Main input module
import sys
import json
import jc.parsers.df
import jc.parsers.env
import jc.parsers.free
import jc.parsers.ifconfig
import jc.parsers.iptables
import jc.parsers.jobs
import jc.parsers.ls
import jc.parsers.lsblk
import jc.parsers.lsmod
import jc.parsers.lsof
import jc.parsers.mount
import jc.parsers.netstat
import jc.parsers.ps
import jc.parsers.route
import jc.parsers.uname
def helptext():
print('Usage: jc [parser] [options]\n', file=sys.stderr)
print('Parsers:', file=sys.stderr)
print(' --df df parser', file=sys.stderr)
print(' --env env parser', file=sys.stderr)
print(' --free free parser', file=sys.stderr)
print(' --ifconfig iconfig parser', file=sys.stderr)
print(' --iptables iptables parser', file=sys.stderr)
print(' --jobs jobs parser', file=sys.stderr)
print(' --ls ls parser', file=sys.stderr)
print(' --lsblk lsblk parser', file=sys.stderr)
print(' --lsmod lsmod parser', file=sys.stderr)
print(' --lsof lsof parser', file=sys.stderr)
print(' --mount mount parser', file=sys.stderr)
print(' --netstat netstat parser', file=sys.stderr)
print(' --ps ps parser', file=sys.stderr)
print(' --route route parser', file=sys.stderr)
print(' --uname uname parser\n', file=sys.stderr)
print('Options:', file=sys.stderr)
print(' -p pretty print output\n', file=sys.stderr)
print('Example:', file=sys.stderr)
print(' ls -al | jc --ls -p\n', file=sys.stderr)
def main():
if sys.stdin.isatty():
print('jc: missing piped data\n', file=sys.stderr)
helptext()
exit()
data = sys.stdin.read()
pretty = False
# options
if '-p' in sys.argv:
pretty = True
if '--ifconfig' in sys.argv:
# parsers
if '--df' in sys.argv:
result = jc.parsers.df.parse(data)
elif '--env' in sys.argv:
result = jc.parsers.env.parse(data)
elif '--free' in sys.argv:
result = jc.parsers.free.parse(data)
elif '--ifconfig' in sys.argv:
result = jc.parsers.ifconfig.parse(data)
elif '--iptables' in sys.argv:
result = jc.parsers.iptables.parse(data)
elif '--jobs' in sys.argv:
result = jc.parsers.jobs.parse(data)
elif '--ls' in sys.argv:
result = jc.parsers.ls.parse(data)
elif '--lsblk' in sys.argv:
result = jc.parsers.lsblk.parse(data)
elif '--lsmod' in sys.argv:
result = jc.parsers.lsmod.parse(data)
elif '--lsof' in sys.argv:
result = jc.parsers.lsof.parse(data)
elif '--mount' in sys.argv:
result = jc.parsers.mount.parse(data)
elif '--netstat' in sys.argv:
result = jc.parsers.netstat.parse(data)
@@ -35,19 +103,12 @@ def main():
elif '--route' in sys.argv:
result = jc.parsers.route.parse(data)
elif '--uname' in sys.argv:
result = jc.parsers.uname.parse(data)
else:
print('jc: missing arguments', file=sys.stderr)
print('Usage: jc [parser] [options]\n', file=sys.stderr)
print('Parsers:', file=sys.stderr)
print(' --ifconfig iconfig parser', file=sys.stderr)
print(' --ls ls parser', file=sys.stderr)
print(' --netstat netstat parser', file=sys.stderr)
print(' --ps ps parser', file=sys.stderr)
print(' --route route parser\n', file=sys.stderr)
print('Options:', file=sys.stderr)
print(' -p pretty print output\n', file=sys.stderr)
print('Example:', file=sys.stderr)
print(' ls -al | jc -p --ls\n', file=sys.stderr)
print('jc: missing or incorrect arguments\n', file=sys.stderr)
helptext()
exit()
# output resulting dictionary as json

55
jc/parsers/df.py Normal file
View File

@@ -0,0 +1,55 @@
"""jc - JSON CLI output utility df Parser
Usage:
specify --df as the first argument if the piped input is coming from df
Example:
$ df | jc --df -p
[
{
"Filesystem": "udev",
"1K-blocks": "977500",
"Used": "0",
"Available": "977500",
"Use%": "0%",
"Mounted": "/dev"
},
{
"Filesystem": "tmpfs",
"1K-blocks": "201732",
"Used": "1180",
"Available": "200552",
"Use%": "1%",
"Mounted": "/run"
},
{
"Filesystem": "/dev/sda2",
"1K-blocks": "20508240",
"Used": "5747284",
"Available": "13696152",
"Use%": "30%",
"Mounted": "/"
},
{
"Filesystem": "tmpfs",
"1K-blocks": "1008648",
"Used": "0",
"Available": "1008648",
"Use%": "0%",
"Mounted": "/dev/shm"
},
...
]
"""
def parse(data):
# code adapted from Conor Heine at:
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
return [dict(zip(headers, r)) for r in raw_data]

53
jc/parsers/env.py Normal file
View File

@@ -0,0 +1,53 @@
"""jc - JSON CLI output utility env Parser
Usage:
specify --env as the first argument if the piped input is coming from env
Example:
$ env | jc --env -p
[
{
"TERM": "xterm-256color"
},
{
"SHELL": "/bin/bash"
},
{
"USER": "root"
},
{
"PATH": "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
},
{
"PWD": "/bin"
},
{
"LANG": "en_US.UTF-8"
},
{
"HOME": "/root"
},
{
"_": "/usr/bin/env"
}
]
"""
def parse(data):
output = []
linedata = data.splitlines()
# Clear any blank lines
cleandata = list(filter(None, linedata))
if cleandata:
for entry in cleandata:
output_line = {}
parsed_line = entry.split('=', maxsplit=1)
output_line[parsed_line[0]] = parsed_line[1]
output.append(output_line)
return output

45
jc/parsers/free.py Normal file
View File

@@ -0,0 +1,45 @@
"""jc - JSON CLI output utility free Parser
Usage:
specify --free as the first argument if the piped input is coming from free
Example:
$ free | jc --free -p
[
{
"type": "Mem",
"total": "2017300",
"used": "213104",
"free": "1148452",
"shared": "1176",
"buff/cache": "655744",
"available": "1622204"
},
{
"type": "Swap",
"total": "2097148",
"used": "0",
"free": "2097148"
}
]
"""
def parse(data):
# code adapted from Conor Heine at:
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h]
headers.insert(0, "type")
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
output = [dict(zip(headers, r)) for r in raw_data]
for entry in output:
entry['type'] = entry['type'].rstrip(':')
return output

364
jc/parsers/iptables.py Normal file
View File

@@ -0,0 +1,364 @@
"""jc - JSON CLI output utility ipables Parser
Usage:
Specify --iptables as the first argument if the piped input is coming from iptables
Supports -vLn for all tables
Examples:
$ sudo iptables -L -t nat | jc --iptables -p
[
{
"chain": "PREROUTING",
"rules": [
{
"target": "PREROUTING_direct",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "PREROUTING_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "PREROUTING_ZONES",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "DOCKER",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere",
"options": "ADDRTYPE match dst-type LOCAL"
}
]
},
{
"chain": "INPUT",
"rules": []
},
{
"chain": "OUTPUT",
"rules": [
{
"target": "OUTPUT_direct",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "DOCKER",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "!loopback/8",
"options": "ADDRTYPE match dst-type LOCAL"
}
]
},
...
]
$ sudo iptables -vnL -t filter | jc --iptables -p
[
{
"chain": "INPUT",
"rules": [
{
"pkts": "1571",
"bytes": "3394K",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "lo",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_direct",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DROP",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate INVALID"
},
{
"pkts": "710",
"bytes": "60078",
"target": "REJECT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "reject-with icmp-host-prohibited"
}
]
},
{
"chain": "FORWARD",
"rules": [
{
"pkts": "0",
"bytes": "0",
"target": "DOCKER-ISOLATION",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DOCKER",
"prot": "all",
"opt": "--",
"in": "*",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "docker0",
"out": "!docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "docker0",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "lo",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_direct",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_IN_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_IN_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_OUT_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_OUT_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DROP",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate INVALID"
},
{
"pkts": "0",
"bytes": "0",
"target": "REJECT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "reject-with icmp-host-prohibited"
}
]
},
...
]
"""
def parse(data):
output = []
chain = {}
headers = []
cleandata = data.splitlines()
for line in cleandata:
if line.find('Chain') == 0:
output.append(chain)
chain = {}
headers = []
parsed_line = line.split()
chain['chain'] = parsed_line[1]
chain['rules'] = []
continue
elif line.find('target') == 0 or line.find('pkts') == 1:
headers = []
headers = [h for h in ' '.join(line.strip().split()).split() if h]
headers.append("options")
continue
else:
rule = line.split(maxsplit=len(headers) - 1)
temp_rule = dict(zip(headers, rule))
if temp_rule:
chain['rules'].append(temp_rule)
output = list(filter(None, output))
return output

108
jc/parsers/jobs.py Normal file
View File

@@ -0,0 +1,108 @@
"""jc - JSON CLI output utility jobs Parser
Usage:
specify --jobs as the first argument if the piped input is coming from jobs
Also supports the -l option
Example:
$ jobs -l | jc --jobs -p
[
{
"job_number": 1,
"pid": 14798,
"status": "Running",
"command": "sleep 10000 &"
},
{
"job_number": 2,
"pid": 14799,
"status": "Running",
"command": "sleep 10001 &"
},
{
"job_number": 3,
"pid": 14800,
"status": "Running",
"command": "sleep 10002 &"
},
{
"job_number": 4,
"pid": 14814,
"history": "previous",
"status": "Running",
"command": "sleep 10003 &"
},
{
"job_number": 5,
"pid": 14815,
"history": "current",
"status": "Running",
"command": "sleep 10004 &"
}
]
"""
import string
def parse(data):
output = []
linedata = data.splitlines()
# Clear any blank lines
cleandata = list(filter(None, linedata))
if cleandata:
for entry in cleandata:
output_line = {}
remainder = []
job_number = ''
pid = ''
job_history = ''
parsed_line = entry.split(maxsplit=2)
# check if -l was used
if parsed_line[1][0] in string.digits:
pid = parsed_line.pop(1)
remainder = parsed_line.pop(1)
job_number = parsed_line.pop(0)
remainder = remainder.split(maxsplit=1)
# rebuild parsed_line
parsed_line = []
for r in remainder:
parsed_line.append(r)
parsed_line.insert(0, job_number)
# check for + or - in first field
if parsed_line[0].find('+') != -1:
job_history = 'current'
parsed_line[0] = parsed_line[0].rstrip('+')
if parsed_line[0].find('-') != -1:
job_history = 'previous'
parsed_line[0] = parsed_line[0].rstrip('-')
# clean up first field
parsed_line[0] = parsed_line[0].lstrip('[').rstrip(']')
# create list of dictionaries
output_line['job_number'] = int(parsed_line[0])
if pid:
output_line['pid'] = int(pid)
if job_history:
output_line['history'] = job_history
output_line['status'] = parsed_line[1]
output_line['command'] = parsed_line[2]
output.append(output_line)
return output

View File

@@ -72,7 +72,7 @@ $ ls -al /usr/bin | jc --ls -p
...
]
$ $ ls -l /usr/bin | jc --ls | jq .[] | jq 'select(.bytes > 50000000)'
$ $ ls -l /usr/bin | jc --ls | jq '.[] | 'select(.bytes > 50000000)'
{
"filename": "emacs",
"flags": "-r-xr-xr-x",

70
jc/parsers/lsblk.py Normal file
View File

@@ -0,0 +1,70 @@
"""jc - JSON CLI output utility lsblk Parser
Usage:
specify --lsblk as the first argument if the piped input is coming from lsblk
Example:
$ lsblk | jc --lsblk -p
[
{
"NAME": "loop0",
"MAJ:MIN": "7:0",
"RM": "0",
"SIZE": "54.5M",
"RO": "1",
"TYPE": "loop",
"MOUNTPOINT": "/snap/core18/1223"
},
{
"NAME": "sda",
"MAJ:MIN": "8:0",
"RM": "0",
"SIZE": "20G",
"RO": "0",
"TYPE": "disk"
},
{
"NAME": "sda1",
"MAJ:MIN": "8:1",
"RM": "0",
"SIZE": "1M",
"RO": "0",
"TYPE": "part"
},
{
"NAME": "sda2",
"MAJ:MIN": "8:2",
"RM": "0",
"SIZE": "20G",
"RO": "0",
"TYPE": "part",
"MOUNTPOINT": "/"
},
{
"NAME": "sr0",
"MAJ:MIN": "11:0",
"RM": "1",
"SIZE": "64.8M",
"RO": "0",
"TYPE": "rom"
}
]
"""
def parse(data):
# code adapted from Conor Heine at:
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
output = [dict(zip(headers, r)) for r in raw_data]
for entry in output:
entry['NAME'] = entry['NAME'].encode('ascii', errors='ignore').decode()
return output

80
jc/parsers/lsmod.py Normal file
View File

@@ -0,0 +1,80 @@
"""jc - JSON CLI output utility lsmod Parser
Usage:
specify --lsmod as the first argument if the piped input is coming from lsmod
Example:
$ lsmod | jc --lsmod -p
[
{
"Module": "nf_nat_ipv4",
"Size": "14115",
"Used": "1",
"By": [
"iptable_nat"
]
},
{
"Module": "nf_nat",
"Size": "26583",
"Used": "3",
"By": [
"nf_nat_ipv4",
"nf_nat_ipv6",
"nf_nat_masquerade_ipv4"
]
},
{
"Module": "iptable_mangle",
"Size": "12695",
"Used": "1"
},
{
"Module": "iptable_security",
"Size": "12705",
"Used": "1"
},
{
"Module": "iptable_raw",
"Size": "12678",
"Used": "1"
},
{
"Module": "nf_conntrack",
"Size": "139224",
"Used": "7",
"By": [
"nf_nat",
"nf_nat_ipv4",
"nf_nat_ipv6",
"xt_conntrack",
"nf_nat_masquerade_ipv4",
"nf_conntrack_ipv4",
"nf_conntrack_ipv6"
]
},
...
]
"""
def parse(data):
# code adapted from Conor Heine at:
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h]
headers.pop(-1)
headers.append('By')
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
output = [dict(zip(headers, r)) for r in raw_data]
for mod in output:
if 'By' in mod:
mod['By'] = mod['By'].split(',')
return output

117
jc/parsers/lsof.py Normal file
View File

@@ -0,0 +1,117 @@
"""jc - JSON CLI output utility lsof Parser
Usage:
specify --lsof as the first argument if the piped input is coming from lsof
Limitations:
No additional columns are supported
Example:
$ sudo lsof | jc --lsof -p
[
{
"COMMAND": "systemd",
"PID": "1",
"TID": null,
"USER": "root",
"FD": "cwd",
"TYPE": "DIR",
"DEVICE": "253,0",
"SIZE/OFF": "224",
"NODE": "64",
"NAME": "/"
},
{
"COMMAND": "systemd",
"PID": "1",
"TID": null,
"USER": "root",
"FD": "rtd",
"TYPE": "DIR",
"DEVICE": "253,0",
"SIZE/OFF": "224",
"NODE": "64",
"NAME": "/"
},
{
"COMMAND": "systemd",
"PID": "1",
"TID": null,
"USER": "root",
"FD": "txt",
"TYPE": "REG",
"DEVICE": "253,0",
"SIZE/OFF": "1624520",
"NODE": "50360451",
"NAME": "/usr/lib/systemd/systemd"
},
{
"COMMAND": "systemd",
"PID": "1",
"TID": null,
"USER": "root",
"FD": "mem",
"TYPE": "REG",
"DEVICE": "253,0",
"SIZE/OFF": "20064",
"NODE": "8146",
"NAME": "/usr/lib64/libuuid.so.1.3.0"
},
{
"COMMAND": "systemd",
"PID": "1",
"TID": null,
"USER": "root",
"FD": "mem",
"TYPE": "REG",
"DEVICE": "253,0",
"SIZE/OFF": "265600",
"NODE": "8147",
"NAME": "/usr/lib64/libblkid.so.1.1.0"
},
...
]
"""
def parse(data):
output = []
linedata = data.splitlines()
# Clear any blank lines
cleandata = list(filter(None, linedata))
if cleandata:
# find column value of last character of each header
header_row = cleandata.pop(0)
headers = header_row.split()
header_spec = []
for i, h in enumerate(headers):
# header tuple is (index, header_name, col)
header_spec.append((i, h, header_row.find(h) + len(h)))
# parse lines
for entry in cleandata:
output_line = {}
# normalize data by inserting Null for missing data
temp_line = entry.split(maxsplit=len(headers) - 1)
for spec in header_spec:
if spec[1] == 'COMMAND' or spec[1] == 'NAME':
continue
if entry[spec[2] - 1] == ' ':
temp_line.insert(spec[0], None)
name = ' '.join(temp_line[9:])
fixed_line = temp_line[0:9]
fixed_line.append(name)
output_line = dict(zip(headers, fixed_line))
output.append(output_line)
return output

75
jc/parsers/mount.py Normal file
View File

@@ -0,0 +1,75 @@
"""jc - JSON CLI output utility mount Parser
Usage:
specify --mount as the first argument if the piped input is coming from mount
Example:
$ mount | jc --mount -p
[
{
"filesystem": "sysfs",
"mount_point": "/sys",
"type": "sysfs",
"access": [
"rw",
"nosuid",
"nodev",
"noexec",
"relatime"
]
},
{
"filesystem": "proc",
"mount_point": "/proc",
"type": "proc",
"access": [
"rw",
"nosuid",
"nodev",
"noexec",
"relatime"
]
},
{
"filesystem": "udev",
"mount_point": "/dev",
"type": "devtmpfs",
"access": [
"rw",
"nosuid",
"relatime",
"size=977500k",
"nr_inodes=244375",
"mode=755"
]
},
...
]
"""
def parse(data):
output = []
linedata = data.splitlines()
# Clear any blank lines
cleandata = list(filter(None, linedata))
if cleandata:
for entry in cleandata:
output_line = {}
parsed_line = entry.split()
output_line['filesystem'] = parsed_line[0]
output_line['mount_point'] = parsed_line[2]
output_line['type'] = parsed_line[4]
access = parsed_line[5].lstrip('(').rstrip(')').split(',')
output_line['access'] = access
output.append(output_line)
return output

View File

@@ -13,7 +13,7 @@ Examples:
$ netstat -p | jc --netstat -p
[
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "localhost.localdo",
"local_port": "34480",
@@ -26,7 +26,7 @@ $ netstat -p | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "localhost.localdo",
"local_port": "34478",
@@ -43,7 +43,7 @@ $ netstat -p | jc --netstat -p
$ netstat -lpn | jc --netstat -p
[
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "127.0.0.1",
"local_port": "42351",
@@ -56,7 +56,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "127.0.0.53",
"local_port": "53",
@@ -69,7 +69,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "0.0.0.0",
"local_port": "22",
@@ -82,7 +82,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "tcp",
"transport_protocol": "tcp",
"network_protocol": "ipv6",
"local_address": "::",
"local_port": "22",
@@ -95,7 +95,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "udp",
"transport_protocol": "udp",
"network_protocol": "ipv4",
"local_address": "127.0.0.53",
"local_port": "53",
@@ -107,7 +107,7 @@ $ netstat -lpn | jc --netstat -p
"send_q": 0
},
{
"session_protocol": "udp",
"transport_protocol": "udp",
"network_protocol": "ipv4",
"local_address": "192.168.71.131",
"local_port": "68",
@@ -129,7 +129,7 @@ def parse_line(entry):
output_line = {}
if entry.find('tcp') == 0:
output_line['session_protocol'] = 'tcp'
output_line['transport_protocol'] = 'tcp'
if entry.find('p6') == 2:
output_line['network_protocol'] = 'ipv6'
@@ -138,7 +138,7 @@ def parse_line(entry):
output_line['network_protocol'] = 'ipv4'
elif entry.find('udp') == 0:
output_line['session_protocol'] = 'udp'
output_line['transport_protocol'] = 'udp'
if entry.find('p6') == 2:
output_line['network_protocol'] = 'ipv6'

42
jc/parsers/uname.py Normal file
View File

@@ -0,0 +1,42 @@
"""jc - JSON CLI output utility uname Parser
Usage:
specify --uname as the first argument if the piped input is coming from uname
Limitations:
must use 'uname -a'
Example:
$ uname -a | jc --uname -p
{
"kernel_name": "Linux",
"node_name": "user-ubuntu",
"kernel_release": "4.15.0-65-generic",
"operating_system": "GNU/Linux",
"hardware_platform": "x86_64",
"processor": "x86_64",
"machine": "x86_64",
"kernel_version": "#74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019"
}
"""
def parse(data):
output = {}
parsed_line = data.split(maxsplit=3)
output['kernel_name'] = parsed_line.pop(0)
output['node_name'] = parsed_line.pop(0)
output['kernel_release'] = parsed_line.pop(0)
parsed_line = parsed_line[-1].rsplit(maxsplit=4)
output['operating_system'] = parsed_line.pop(-1)
output['hardware_platform'] = parsed_line.pop(-1)
output['processor'] = parsed_line.pop(-1)
output['machine'] = parsed_line.pop(-1)
output['kernel_version'] = parsed_line.pop(0)
return output

View File

@@ -5,7 +5,7 @@ with open('README.md', 'r') as f:
setuptools.setup(
name='jc',
version='0.6.2',
version='0.9.1',
author='Kelly Brazil',
author_email='kellyjonbrazil@gmail.com',
description='This tool serializes the output of popular command line tools to structured JSON output.',