python/jc
1
0
Fork 0
mirror of https://github.com/kellyjonbrazil/jc.git synced 2025-07-13 01:20:24 +02:00
Code Issues Releases Activity

Merge pull request #5 from kellyjonbrazil/dev

Browse Source 
Dev v1.0.1
This commit is contained in:
Kelly Brazil
2019-10-25 19:43:24 -07:00
committed by GitHub
parent ea0cf0acf2 c9849ce0db
commit c3eaf59836
21 changed files with 1040 additions and 737 deletions
Download Patch File Download Diff File Expand all files Collapse all files

749
README.md
View File

@ -6,19 +6,19 @@ JSON CLI output utility
This allows further command line processing of output with tools like `jq` simply by piping commands: This allows further command line processing of output with tools like `jq` simply by piping commands:
``` ```
$ ls -l /usr/bin | jc --ls | jq '.[] | select(.bytes > 50000000)' $ ls -l /usr/bin | jc --ls | jq '.[] | select(.size|tonumber > 50000000)'
{ {
"filename": "emacs", "filename": "emacs",
"flags": "-r-xr-xr-x", "flags": "-r-xr-xr-x",
"links": 1, "links": 1,
"owner": "root", "owner": "root",
"group": "wheel", "group": "wheel",
"bytes": 117164432, "size": "117164432",
"date": "May 3 22:26" "date": "May 3 22:26"
} }
``` ```
The `jc` parsers can also be used as python modules: The `jc` parsers can also be used as python modules. In this case the output will be a python dictionary instead of JSON:
``` ```
>>> import jc.parsers.ls >>> import jc.parsers.ls
>>> >>>
@ -32,29 +32,28 @@ The `jc` parsers can also be used as python modules:
... -rwxr-xr-x 1 root wheel 18128 May 3 22:26 echo''' ... -rwxr-xr-x 1 root wheel 18128 May 3 22:26 echo'''
>>> >>>
>>> jc.parsers.ls.parse(data) >>> jc.parsers.ls.parse(data)
[{'filename': 'cat', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', [{'filename': 'cat', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'size': '23648',
'bytes': 23648, 'date': 'May 3 22:26'}, {'filename': 'chmod', 'flags': '-rwxr-xr-x', 'links': 1, 'date': 'May 3 22:26'}, {'filename': 'chmod', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'owner': 'root', 'group': 'wheel', 'bytes': 30016, 'date': 'May 3 22:26'}, {'filename': 'cp', 'group': 'wheel', 'size': '30016', 'date': 'May 3 22:26'}, {'filename': 'cp', 'flags': '-rwxr-xr-x',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 29024, 'links': 1, 'owner': 'root', 'group': 'wheel', 'size': '29024', 'date': 'May 3 22:26'}, {'filename': 'csh',
'date': 'May 3 22:26'}, {'filename': 'csh', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'size': '375824', 'date': 'May 3 22:26'},
'group': 'wheel', 'bytes': 375824, 'date': 'May 3 22:26'}, {'filename': 'date', {'filename': 'date', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'size': '28608',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 28608, 'date': 'May 3 22:26'}, {'filename': 'dd', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'date': 'May 3 22:26'}, {'filename': 'dd', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'size': '32000', 'date': 'May 3 22:26'}, {'filename': 'df', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'bytes': 32000, 'date': 'May 3 22:26'}, {'filename': 'df', 'flags': '-rwxr-xr-x', 'group': 'wheel', 'size': '23392', 'date': 'May 3 22:26'}, {'filename': 'echo', 'flags': '-rwxr-xr-x',
'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 23392, 'date': 'May 3 22:26'}, 'links': 1, 'owner': 'root', 'group': 'wheel', 'size': '18128', 'date': 'May 3 22:26'}]
{'filename': 'echo', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'bytes': 18128, 'date': 'May 3 22:26'}]
``` ```
In this case the output will be a python dictionary instead of JSON.
The goal is to keep the resulting JSON as flat and simple as possible. Also, keys have been converted to lowercase and special characters are replaced whenever possible. Numbers are kept as strings because, depending on context or the output options, numbers can sometimes turn into strings. (e.g 'human readable' options)
## Installation ## Installation
``` ```
$ pip3 install jc $ pip3 install --upgrade jc
``` ```
## Usage ## Usage
``` ```
jc [parser] [options] jc PARSER [OPTIONS]
``` ```
`jc` accepts piped input from `STDIN` and outputs a JSON representation of the previous command's output to `STDOUT`. The JSON output can be compact or pretty formatted. `jc` accepts piped input from `STDIN` and outputs a JSON representation of the previous command's output to `STDOUT`. The JSON output can be compact or pretty formatted.
@ -63,6 +62,7 @@ jc [parser] [options]
- `--df` enables the `df` parser - `--df` enables the `df` parser
- `--env` enables the `env` parser - `--env` enables the `env` parser
- `--free` enables the `free` parser - `--free` enables the `free` parser
- `--history` enables the `history` parser
- `--ifconfig` enables the `ifconfig` parser - `--ifconfig` enables the `ifconfig` parser
- `--iptables` enables the `iptables` parser - `--iptables` enables the `iptables` parser
- `--jobs` enables the `jobs` parser - `--jobs` enables the `jobs` parser
@ -75,6 +75,8 @@ jc [parser] [options]
- `--ps` enables the `ps` parser - `--ps` enables the `ps` parser
- `--route` enables the `route` parser - `--route` enables the `route` parser
- `--uname` enables the `uname -a` parser - `--uname` enables the `uname -a` parser
- `--uptime` enables the `uptime` parser
- `--w` enables the `w` parser
### Options ### Options
- `-p` specifies whether to pretty format the JSON output - `-p` specifies whether to pretty format the JSON output
@ -85,70 +87,87 @@ jc [parser] [options]
$ df | jc --df -p $ df | jc --df -p
[ [
{ {
"Filesystem": "udev", "filesystem": "udev",
"1K-blocks": "977500", "1k-blocks": "977500",
"Used": "0", "used": "0",
"Available": "977500", "available": "977500",
"Use%": "0%", "use_percent": "0%",
"Mounted": "/dev" "mounted": "/dev"
}, },
{ {
"Filesystem": "tmpfs", "filesystem": "tmpfs",
"1K-blocks": "201732", "1k-blocks": "201732",
"Used": "1180", "used": "1204",
"Available": "200552", "available": "200528",
"Use%": "1%", "use_percent": "1%",
"Mounted": "/run" "mounted": "/run"
}, },
{ {
"Filesystem": "/dev/sda2", "filesystem": "/dev/sda2",
"1K-blocks": "20508240", "1k-blocks": "20508240",
"Used": "5747284", "used": "5748312",
"Available": "13696152", "available": "13695124",
"Use%": "30%", "use_percent": "30%",
"Mounted": "/" "mounted": "/"
}, },
{ {
"Filesystem": "tmpfs", "filesystem": "tmpfs",
"1K-blocks": "1008648", "1k-blocks": "1008648",
"Used": "0", "used": "0",
"Available": "1008648", "available": "1008648",
"Use%": "0%", "use_percent": "0%",
"Mounted": "/dev/shm" "mounted": "/dev/shm"
}, }
... ...
] ]
``` ```
### env ### env
``` ```
$ env | jc --env -p $ env | jc --env -p
[
{
"TERM": "xterm-256color"
},
{
"SHELL": "/bin/bash"
},
{
"USER": "root"
},
{
"PATH": "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
},
{
"PWD": "/bin"
},
{
"LANG": "en_US.UTF-8"
},
{
"HOME": "/root"
},
{ {
"TERM": "xterm-256color",
"SHELL": "/bin/bash",
"USER": "root",
"PATH": "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin",
"PWD": "/root",
"LANG": "en_US.UTF-8",
"HOME": "/root",
"LOGNAME": "root",
"_": "/usr/bin/env" "_": "/usr/bin/env"
} }
```
### free
```
$ free | jc --free -p
[
{
"type": "Mem",
"total": "2017300",
"used": "213104",
"free": "1148452",
"shared": "1176",
"buff_cache": "655744",
"available": "1622204"
},
{
"type": "Swap",
"total": "2097148",
"used": "0",
"free": "2097148"
}
] ]
``` ```
### history
```
$ history | jc --history -p
{
"n118": "sleep 100",
"n119": "ls /bin",
"n120": "echo \"hello\"",
"n121": "docker images",
...
}
```
### ifconfig ### ifconfig
``` ```
$ ifconfig | jc --ifconfig -p $ ifconfig | jc --ifconfig -p
@ -557,69 +576,92 @@ $ sudo iptables -vnL -t filter | jc --iptables -p
$ jobs -l | jc --jobs -p $ jobs -l | jc --jobs -p
[ [
{ {
"job_number": 1, "job_number": "1",
"pid": 14798, "pid": "19510",
"status": "Running", "status": "Running",
"command": "sleep 10000 &" "command": "sleep 1000 &"
}, },
{ {
"job_number": 2, "job_number": "2",
"pid": 14799, "pid": "19511",
"status": "Running", "status": "Running",
"command": "sleep 10001 &" "command": "sleep 1001 &"
}, },
{ {
"job_number": 3, "job_number": "3",
"pid": 14800, "pid": "19512",
"status": "Running",
"command": "sleep 10002 &"
},
{
"job_number": 4,
"pid": 14814,
"history": "previous", "history": "previous",
"status": "Running", "status": "Running",
"command": "sleep 10003 &" "command": "sleep 1002 &"
}, },
{ {
"job_number": 5, "job_number": "4",
"pid": 14815, "pid": "19513",
"history": "current", "history": "current",
"status": "Running", "status": "Running",
"command": "sleep 10004 &" "command": "sleep 1003 &"
} }
] ]
``` ```
### ls ### ls
``` ```
$ ls -l /bin | jc --ls -p $ ls -l /usr/bin | jc --ls -p
[ [
{
"filename": "apropos",
"link_to": "whatis",
"flags": "lrwxrwxrwx.",
"links": "1",
"owner": "root",
"group": "root",
"size": "6",
"date": "Aug 15 10:53"
},
{
"filename": "arch",
"flags": "-rwxr-xr-x.",
"links": "1",
"owner": "root",
"group": "root",
"size": "33080",
"date": "Aug 19 23:25"
},
{
"filename": "awk",
"link_to": "gawk",
"flags": "lrwxrwxrwx.",
"links": "1",
"owner": "root",
"group": "root",
"size": "4",
"date": "Aug 15 10:53"
},
{
"filename": "base64",
"flags": "-rwxr-xr-x.",
"links": "1",
"owner": "root",
"group": "root",
"size": "37360",
"date": "Aug 19 23:25"
},
{
"filename": "basename",
"flags": "-rwxr-xr-x.",
"links": "1",
"owner": "root",
"group": "root",
"size": "29032",
"date": "Aug 19 23:25"
},
{ {
"filename": "bash", "filename": "bash",
"flags": "-r-xr-xr-x", "flags": "-rwxr-xr-x.",
"links": 1, "links": "1",
"owner": "root", "owner": "root",
"group": "wheel", "group": "root",
"bytes": 618416, "size": "964600",
"date": "May 3 22:26" "date": "Aug 8 05:06"
},
{
"filename": "cat",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 23648,
"date": "May 3 22:26"
},
{
"filename": "chmod",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 30016,
"date": "May 3 22:26"
}, },
... ...
] ]
@ -629,46 +671,55 @@ $ ls -l /bin | jc --ls -p
$ lsblk | jc --lsblk -p $ lsblk | jc --lsblk -p
[ [
{ {
"NAME": "loop0", "name": "sda",
"MAJ:MIN": "7:0", "maj_min": "8:0",
"RM": "0", "rm": "0",
"SIZE": "54.5M", "size": "20G",
"RO": "1", "ro": "0",
"TYPE": "loop", "type": "disk"
"MOUNTPOINT": "/snap/core18/1223"
}, },
{ {
"NAME": "sda", "name": "sda1",
"MAJ:MIN": "8:0", "maj_min": "8:1",
"RM": "0", "rm": "0",
"SIZE": "20G", "size": "1G",
"RO": "0", "ro": "0",
"TYPE": "disk" "type": "part",
"mountpoint": "/boot"
}, },
{ {
"NAME": "sda1", "name": "sda2",
"MAJ:MIN": "8:1", "maj_min": "8:2",
"RM": "0", "rm": "0",
"SIZE": "1M", "size": "19G",
"RO": "0", "ro": "0",
"TYPE": "part" "type": "part"
}, },
{ {
"NAME": "sda2", "name": "centos-root",
"MAJ:MIN": "8:2", "maj_min": "253:0",
"RM": "0", "rm": "0",
"SIZE": "20G", "size": "17G",
"RO": "0", "ro": "0",
"TYPE": "part", "type": "lvm",
"MOUNTPOINT": "/" "mountpoint": "/"
}, },
{ {
"NAME": "sr0", "name": "centos-swap",
"MAJ:MIN": "11:0", "maj_min": "253:1",
"RM": "1", "rm": "0",
"SIZE": "64.8M", "size": "2G",
"RO": "0", "ro": "0",
"TYPE": "rom" "type": "lvm",
"mountpoint": "[SWAP]"
},
{
"name": "sr0",
"maj_min": "11:0",
"rm": "1",
"size": "1024M",
"ro": "0",
"type": "rom"
} }
] ]
``` ```
@ -676,44 +727,12 @@ $ lsblk | jc --lsblk -p
``` ```
$ lsmod | jc --lsmod -p $ lsmod | jc --lsmod -p
[ [
...
{ {
"Module": "nf_nat_ipv4", "module": "nf_conntrack",
"Size": "14115", "size": "139224",
"Used": "1", "used": "7",
"By": [ "by": [
"iptable_nat"
]
},
{
"Module": "nf_nat",
"Size": "26583",
"Used": "3",
"By": [
"nf_nat_ipv4",
"nf_nat_ipv6",
"nf_nat_masquerade_ipv4"
]
},
{
"Module": "iptable_mangle",
"Size": "12695",
"Used": "1"
},
{
"Module": "iptable_security",
"Size": "12705",
"Used": "1"
},
{
"Module": "iptable_raw",
"Size": "12678",
"Used": "1"
},
{
"Module": "nf_conntrack",
"Size": "139224",
"Used": "7",
"By": [
"nf_nat", "nf_nat",
"nf_nat_ipv4", "nf_nat_ipv4",
"nf_nat_ipv6", "nf_nat_ipv6",
@ -723,72 +742,99 @@ $ lsmod | jc --lsmod -p
"nf_conntrack_ipv6" "nf_conntrack_ipv6"
] ]
}, },
{
"module": "ip_set",
"size": "45799",
"used": "0"
},
{
"module": "nfnetlink",
"size": "14519",
"used": "1",
"by": [
"ip_set"
]
},
{
"module": "ebtable_filter",
"size": "12827",
"used": "1"
},
{
"module": "ebtables",
"size": "35009",
"used": "2",
"by": [
"ebtable_nat",
"ebtable_filter"
]
},
... ...
] ]
``` ```
### lsof ### lsof
``` ```
$ sudo lsof | jc --lsof -p $ sudo lsof | jc --lsof -p | more
[ [
{ {
"COMMAND": "systemd", "command": "systemd",
"PID": "1", "pid": "1",
"TID": null, "tid": null,
"USER": "root", "user": "root",
"FD": "cwd", "fd": "cwd",
"TYPE": "DIR", "type": "DIR",
"DEVICE": "253,0", "device": "8,2",
"SIZE/OFF": "224", "size_off": "4096",
"NODE": "64", "node": "2",
"NAME": "/" "name": "/"
}, },
{ {
"COMMAND": "systemd", "command": "systemd",
"PID": "1", "pid": "1",
"TID": null, "tid": null,
"USER": "root", "user": "root",
"FD": "rtd", "fd": "rtd",
"TYPE": "DIR", "type": "DIR",
"DEVICE": "253,0", "device": "8,2",
"SIZE/OFF": "224", "size_off": "4096",
"NODE": "64", "node": "2",
"NAME": "/" "name": "/"
}, },
{ {
"COMMAND": "systemd", "command": "systemd",
"PID": "1", "pid": "1",
"TID": null, "tid": null,
"USER": "root", "user": "root",
"FD": "txt", "fd": "txt",
"TYPE": "REG", "type": "REG",
"DEVICE": "253,0", "device": "8,2",
"SIZE/OFF": "1624520", "size_off": "1595792",
"NODE": "50360451", "node": "668802",
"NAME": "/usr/lib/systemd/systemd" "name": "/lib/systemd/systemd"
}, },
{ {
"COMMAND": "systemd", "command": "systemd",
"PID": "1", "pid": "1",
"TID": null, "tid": null,
"USER": "root", "user": "root",
"FD": "mem", "fd": "mem",
"TYPE": "REG", "type": "REG",
"DEVICE": "253,0", "device": "8,2",
"SIZE/OFF": "20064", "size_off": "1700792",
"NODE": "8146", "node": "656167",
"NAME": "/usr/lib64/libuuid.so.1.3.0" "name": "/lib/x86_64-linux-gnu/libm-2.27.so"
}, },
{ {
"COMMAND": "systemd", "command": "systemd",
"PID": "1", "pid": "1",
"TID": null, "tid": null,
"USER": "root", "user": "root",
"FD": "mem", "fd": "mem",
"TYPE": "REG", "type": "REG",
"DEVICE": "253,0", "device": "8,2",
"SIZE/OFF": "265600", "size_off": "121016",
"NODE": "8147", "node": "655394",
"NAME": "/usr/lib64/libblkid.so.1.1.0" "name": "/lib/x86_64-linux-gnu/libudev.so.1.6.9"
}, },
... ...
] ]
@ -851,8 +897,8 @@ $ netstat -p | jc --netstat -p
"state": "ESTABLISHED", "state": "ESTABLISHED",
"pid": 53550, "pid": 53550,
"program_name": "git-remote-ht", "program_name": "git-remote-ht",
"receive_q": 0, "receive_q": "0",
"send_q": 0 "send_q": "0"
}, },
{ {
"transport_protocol": "tcp", "transport_protocol": "tcp",
@ -864,39 +910,26 @@ $ netstat -p | jc --netstat -p
"state": "ESTABLISHED", "state": "ESTABLISHED",
"pid": 53550, "pid": 53550,
"program_name": "git-remote-ht", "program_name": "git-remote-ht",
"receive_q": 0, "receive_q": "0",
"send_q": 0 "send_q": "0"
} }
] ]
``` ```
``` ```
$ netstat -lpn | jc --netstat -p $ sudo netstat -lpn | jc --netstat -p
[ [
{ {
"transport_protocol": "tcp", "transport_protocol": "tcp",
"network_protocol": "ipv4", "network_protocol": "ipv4",
"local_address": "127.0.0.1", "local_address": "127.0.0.1",
"local_port": "42351", "local_port": "25",
"foreign_address": "0.0.0.0", "foreign_address": "0.0.0.0",
"foreign_port": "*", "foreign_port": "*",
"state": "LISTEN", "state": "LISTEN",
"pid": 1112, "pid": "1584",
"program_name": "containerd", "program_name": "master",
"receive_q": 0, "receive_q": "0",
"send_q": 0 "send_q": "0"
},
{
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "127.0.0.53",
"local_port": "53",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"state": "LISTEN",
"pid": 885,
"program_name": "systemd-resolve",
"receive_q": 0,
"send_q": 0
}, },
{ {
"transport_protocol": "tcp", "transport_protocol": "tcp",
@ -906,140 +939,130 @@ $ netstat -lpn | jc --netstat -p
"foreign_address": "0.0.0.0", "foreign_address": "0.0.0.0",
"foreign_port": "*", "foreign_port": "*",
"state": "LISTEN", "state": "LISTEN",
"pid": 1127, "pid": "1213",
"program_name": "sshd", "program_name": "sshd",
"receive_q": 0, "receive_q": "0",
"send_q": 0 "send_q": "0"
}, },
{ {
"transport_protocol": "tcp", "transport_protocol": "tcp",
"network_protocol": "ipv6", "network_protocol": "ipv6",
"local_address": "::", "local_address": "::1",
"local_port": "22", "local_port": "25",
"foreign_address": "::", "foreign_address": "::",
"foreign_port": "*", "foreign_port": "*",
"state": "LISTEN", "state": "LISTEN",
"pid": 1127, "pid": "1584",
"program_name": "sshd", "program_name": "master",
"receive_q": 0, "receive_q": "0",
"send_q": 0 "send_q": "0"
}, },
{ {
"transport_protocol": "udp", "transport_protocol": "udp",
"network_protocol": "ipv4", "network_protocol": "ipv4",
"local_address": "127.0.0.53", "local_address": "0.0.0.0",
"local_port": "53",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"pid": 885,
"program_name": "systemd-resolve",
"receive_q": 0,
"send_q": 0
},
{
"transport_protocol": "udp",
"network_protocol": "ipv4",
"local_address": "192.168.71.131",
"local_port": "68", "local_port": "68",
"foreign_address": "0.0.0.0", "foreign_address": "0.0.0.0",
"foreign_port": "*", "foreign_port": "*",
"pid": 867, "pid": "19177",
"program_name": "systemd-network", "program_name": "dhclient",
"receive_q": 0, "receive_q": "0",
"send_q": 0 "send_q": "0"
} },
...
] ]
``` ```
### ps ### ps
``` ```
$ ps -ef | jc --ps -p $ ps -ef | jc --ps -p
[ [
...
{ {
"UID": "root", "uid": "root",
"PID": "1", "pid": "545",
"PPID": "0", "ppid": "1",
"C": "0", "c": "0",
"STIME": "13:58", "stime": "Oct21",
"TTY": "?", "tty": "?",
"TIME": "00:00:05", "time": "00:00:03",
"CMD": "/lib/systemd/systemd --system --deserialize 35" "cmd": "/usr/lib/systemd/systemd-journald"
}, },
{ {
"UID": "root", "uid": "root",
"PID": "2", "pid": "566",
"PPID": "0", "ppid": "1",
"C": "0", "c": "0",
"STIME": "13:58", "stime": "Oct21",
"TTY": "?", "tty": "?",
"TIME": "00:00:00", "time": "00:00:00",
"CMD": "[kthreadd]" "cmd": "/usr/sbin/lvmetad -f"
}, },
{ {
"UID": "root", "uid": "root",
"PID": "4", "pid": "580",
"PPID": "2", "ppid": "1",
"C": "0", "c": "0",
"STIME": "13:58", "stime": "Oct21",
"TTY": "?", "tty": "?",
"TIME": "00:00:00", "time": "00:00:00",
"CMD": "[kworker/0:0H]" "cmd": "/usr/lib/systemd/systemd-udevd"
}, },
{ {
"UID": "root", "uid": "root",
"PID": "6", "pid": "659",
"PPID": "2", "ppid": "2",
"C": "0", "c": "0",
"STIME": "13:58", "stime": "Oct21",
"TTY": "?", "tty": "?",
"TIME": "00:00:00", "time": "00:00:00",
"CMD": "[mm_percpu_wq]" "cmd": "[kworker/u257:0]"
},
{
"uid": "root",
"pid": "666",
"ppid": "2",
"c": "0",
"stime": "Oct21",
"tty": "?",
"time": "00:00:00",
"cmd": "[hci0]"
}, },
... ...
] ]
``` ```
### route ### route
``` ```
$ route -n | jc --route -p $ route | jc --route -p
[ [
{ {
"Destination": "0.0.0.0", "destination": "default",
"Gateway": "192.168.71.2", "gateway": "gateway",
"Genmask": "0.0.0.0", "genmask": "0.0.0.0",
"Flags": "UG", "flags": "UG",
"Metric": "100", "metric": "100",
"Ref": "0", "ref": "0",
"Use": "0", "use": "0",
"Iface": "ens33" "iface": "ens33"
}, },
{ {
"Destination": "172.17.0.0", "destination": "172.17.0.0",
"Gateway": "0.0.0.0", "gateway": "0.0.0.0",
"Genmask": "255.255.0.0", "genmask": "255.255.0.0",
"Flags": "U", "flags": "U",
"Metric": "0", "metric": "0",
"Ref": "0", "ref": "0",
"Use": "0", "use": "0",
"Iface": "docker0" "iface": "docker0"
}, },
{ {
"Destination": "192.168.71.0", "destination": "192.168.71.0",
"Gateway": "0.0.0.0", "gateway": "0.0.0.0",
"Genmask": "255.255.255.0", "genmask": "255.255.255.0",
"Flags": "U", "flags": "U",
"Metric": "0", "metric": "100",
"Ref": "0", "ref": "0",
"Use": "0", "use": "0",
"Iface": "ens33" "iface": "ens33"
},
{
"Destination": "192.168.71.2",
"Gateway": "0.0.0.0",
"Genmask": "255.255.255.255",
"Flags": "UH",
"Metric": "100",
"Ref": "0",
"Use": "0",
"Iface": "ens33"
} }
] ]
``` ```
@ -1057,6 +1080,44 @@ $ uname -a | jc --uname -p
"kernel_version": "#74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019" "kernel_version": "#74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019"
} }
``` ```
### uptime
```
$ uptime | jc --uptime -p
{
"time": "16:52",
"uptime": "3 days, 4:49",
"users": "5",
"load_1m": "1.85",
"load_5m": "1.90",
"load_15m": "1.91"
}
```
### w
```
$ w | jc --w -p
[
{
"user": "root",
"tty": "ttyS0",
"from": "-",
"login_at": "Mon20",
"idle": "0.00s",
"jcpu": "14.70s",
"pcpu": "0.00s",
"what": "bash"
},
{
"user": "root",
"tty": "pts/0",
"from": "192.168.71.1",
"login_at": "Thu22",
"idle": "22:46m",
"jcpu": "0.05s",
"pcpu": "0.05s",
"what": "-bash"
}
]
```
## Contributions ## Contributions
Feel free to add/improve code or parsers! Feel free to add/improve code or parsers!

11
changelog.txt
View File

@ -1,5 +1,16 @@
jc changelog jc changelog
20191025 v1.0.1
- Add w parser
- Add uptime parser
- Add history parser
- Fix uptime parser
- Flatten env parser output
- Remove problematic characters from key names in: df, free, history, lsblk, lsof, and w
- Where possible, lowercase all keys (except cases like env where the key is the variable name)
- Remove integer values
- Handle CTRL-C gracefully
20191023 v0.9.1 20191023 v0.9.1
- Add jobs parser - Add jobs parser
- Add lsof parser - Add lsof parser

26
jc/jc.py
View File

@ -5,10 +5,12 @@ Main input module
""" """
import sys import sys
import signal
import json import json
import jc.parsers.df import jc.parsers.df
import jc.parsers.env import jc.parsers.env
import jc.parsers.free import jc.parsers.free
import jc.parsers.history
import jc.parsers.ifconfig import jc.parsers.ifconfig
import jc.parsers.iptables import jc.parsers.iptables
import jc.parsers.jobs import jc.parsers.jobs
@ -21,14 +23,17 @@ import jc.parsers.netstat
import jc.parsers.ps import jc.parsers.ps
import jc.parsers.route import jc.parsers.route
import jc.parsers.uname import jc.parsers.uname
import jc.parsers.uptime
import jc.parsers.w
def helptext(): def helptext():
print('Usage: jc [parser] [options]\n', file=sys.stderr) print('Usage: jc PARSER [OPTIONS]\n', file=sys.stderr)
print('Parsers:', file=sys.stderr) print('Parsers:', file=sys.stderr)
print(' --df df parser', file=sys.stderr) print(' --df df parser', file=sys.stderr)
print(' --env env parser', file=sys.stderr) print(' --env env parser', file=sys.stderr)
print(' --free free parser', file=sys.stderr) print(' --free free parser', file=sys.stderr)
print(' --history history parser', file=sys.stderr)
print(' --ifconfig iconfig parser', file=sys.stderr) print(' --ifconfig iconfig parser', file=sys.stderr)
print(' --iptables iptables parser', file=sys.stderr) print(' --iptables iptables parser', file=sys.stderr)
print(' --jobs jobs parser', file=sys.stderr) print(' --jobs jobs parser', file=sys.stderr)
@ -40,14 +45,22 @@ def helptext():
print(' --netstat netstat parser', file=sys.stderr) print(' --netstat netstat parser', file=sys.stderr)
print(' --ps ps parser', file=sys.stderr) print(' --ps ps parser', file=sys.stderr)
print(' --route route parser', file=sys.stderr) print(' --route route parser', file=sys.stderr)
print(' --uname uname parser\n', file=sys.stderr) print(' --uname uname parser', file=sys.stderr)
print(' --uptime uptime parser', file=sys.stderr)
print(' --w w parser\n', file=sys.stderr)
print('Options:', file=sys.stderr) print('Options:', file=sys.stderr)
print(' -p pretty print output\n', file=sys.stderr) print(' -p pretty print output\n', file=sys.stderr)
print('Example:', file=sys.stderr) print('Example:', file=sys.stderr)
print(' ls -al | jc --ls -p\n', file=sys.stderr) print(' ls -al | jc --ls -p\n', file=sys.stderr)
def ctrlc(signum, frame):
exit()
def main(): def main():
signal.signal(signal.SIGINT, ctrlc)
if sys.stdin.isatty(): if sys.stdin.isatty():
print('jc: missing piped data\n', file=sys.stderr) print('jc: missing piped data\n', file=sys.stderr)
helptext() helptext()
@ -70,6 +83,9 @@ def main():
elif '--free' in sys.argv: elif '--free' in sys.argv:
result = jc.parsers.free.parse(data) result = jc.parsers.free.parse(data)
elif '--history' in sys.argv:
result = jc.parsers.history.parse(data)
elif '--ifconfig' in sys.argv: elif '--ifconfig' in sys.argv:
result = jc.parsers.ifconfig.parse(data) result = jc.parsers.ifconfig.parse(data)
@ -106,6 +122,12 @@ def main():
elif '--uname' in sys.argv: elif '--uname' in sys.argv:
result = jc.parsers.uname.parse(data) result = jc.parsers.uname.parse(data)
elif '--uptime' in sys.argv:
result = jc.parsers.uptime.parse(data)
elif '--w' in sys.argv:
result = jc.parsers.w.parse(data)
else: else:
print('jc: missing or incorrect arguments\n', file=sys.stderr) print('jc: missing or incorrect arguments\n', file=sys.stderr)
helptext() helptext()

57
jc/parsers/df.py
View File

@ -8,37 +8,37 @@ Example:
$ df | jc --df -p $ df | jc --df -p
[ [
{ {
"Filesystem": "udev", "filesystem": "udev",
"1K-blocks": "977500", "1k-blocks": "977500",
"Used": "0", "used": "0",
"Available": "977500", "available": "977500",
"Use%": "0%", "use_percent": "0%",
"Mounted": "/dev" "mounted": "/dev"
}, },
{ {
"Filesystem": "tmpfs", "filesystem": "tmpfs",
"1K-blocks": "201732", "1k-blocks": "201732",
"Used": "1180", "used": "1204",
"Available": "200552", "available": "200528",
"Use%": "1%", "use_percent": "1%",
"Mounted": "/run" "mounted": "/run"
}, },
{ {
"Filesystem": "/dev/sda2", "filesystem": "/dev/sda2",
"1K-blocks": "20508240", "1k-blocks": "20508240",
"Used": "5747284", "used": "5748312",
"Available": "13696152", "available": "13695124",
"Use%": "30%", "use_percent": "30%",
"Mounted": "/" "mounted": "/"
}, },
{ {
"Filesystem": "tmpfs", "filesystem": "tmpfs",
"1K-blocks": "1008648", "1k-blocks": "1008648",
"Used": "0", "used": "0",
"Available": "1008648", "available": "1008648",
"Use%": "0%", "use_percent": "0%",
"Mounted": "/dev/shm" "mounted": "/dev/shm"
}, }
... ...
] ]
""" """
@ -50,6 +50,11 @@ def parse(data):
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501 # https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines() cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h] headers = [h for h in ' '.join(cleandata[0].lower().strip().split()).split() if h]
# clean up 'use%' header
# even though % in a key is valid json, it can make things difficult
headers = ['use_percent' if x == 'use%' else x for x in headers]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:]) raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
return [dict(zip(headers, r)) for r in raw_data] return [dict(zip(headers, r)) for r in raw_data]

38
jc/parsers/env.py
View File

@ -4,38 +4,24 @@ Usage:
specify --env as the first argument if the piped input is coming from env specify --env as the first argument if the piped input is coming from env
Example: Example:
$ env | jc --env -p $ env | jc --env -p
[
{
"TERM": "xterm-256color"
},
{
"SHELL": "/bin/bash"
},
{
"USER": "root"
},
{
"PATH": "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
},
{
"PWD": "/bin"
},
{
"LANG": "en_US.UTF-8"
},
{
"HOME": "/root"
},
{ {
"TERM": "xterm-256color",
"SHELL": "/bin/bash",
"USER": "root",
"PATH": "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin",
"PWD": "/root",
"LANG": "en_US.UTF-8",
"HOME": "/root",
"LOGNAME": "root",
"_": "/usr/bin/env" "_": "/usr/bin/env"
} }
]
""" """
def parse(data): def parse(data):
output = [] output = {}
linedata = data.splitlines() linedata = data.splitlines()
@ -45,9 +31,7 @@ def parse(data):
if cleandata: if cleandata:
for entry in cleandata: for entry in cleandata:
output_line = {}
parsed_line = entry.split('=', maxsplit=1) parsed_line = entry.split('=', maxsplit=1)
output_line[parsed_line[0]] = parsed_line[1] output[parsed_line[0]] = parsed_line[1]
output.append(output_line)
return output return output

9
jc/parsers/free.py
View File

@ -13,7 +13,7 @@ $ free | jc --free -p
"used": "213104", "used": "213104",
"free": "1148452", "free": "1148452",
"shared": "1176", "shared": "1176",
"buff/cache": "655744", "buff_cache": "655744",
"available": "1622204" "available": "1622204"
}, },
{ {
@ -32,10 +32,13 @@ def parse(data):
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501 # https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines() cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h] headers = [h for h in ' '.join(cleandata[0].lower().strip().split()).split() if h]
headers.insert(0, "type") headers.insert(0, "type")
# clean up 'buff/cache' header
# even though forward slash in a key is valid json, it can make things difficult
headers = ['buff_cache' if x == 'buff/cache' else x for x in headers]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:]) raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
output = [dict(zip(headers, r)) for r in raw_data] output = [dict(zip(headers, r)) for r in raw_data]

38
jc/parsers/history.py Normal file
View File

@ -0,0 +1,38 @@
"""jc - JSON CLI output utility history Parser
Usage:
specify --history as the first argument if the piped input is coming from history
Example:
$ history | jc --history -p
{
"n118": "sleep 100",
"n119": "ls /bin",
"n120": "echo \"hello\"",
"n121": "docker images",
...
}
"""
def parse(data):
output = {}
# split lines and clear out any non-ascii chars
linedata = data.encode('ascii', errors='ignore').decode().splitlines()
# Clear any blank lines
cleandata = list(filter(None, linedata))
if cleandata:
for entry in cleandata:
try:
parsed_line = entry.split(maxsplit=1)
# prepend alpha character n to key so the resulting JSON is easier to work with
output['n' + parsed_line[0]] = parsed_line[1]
except IndexError:
# need to catch indexerror in case there is weird input from prior commands
pass
return output

81
jc/parsers/ifconfig.py
View File

@ -8,7 +8,86 @@ Usage:
Example: Example:
$ ifconfig | jc --ifconfig -p $ ifconfig | jc --ifconfig -p
[
{
"name": "docker0",
"flags": "4099",
"state": "UP,BROADCAST,MULTICAST",
"mtu": "1500",
"ipv4_addr": "172.17.0.1",
"ipv4_mask": "255.255.0.0",
"ipv4_bcast": "0.0.0.0",
"mac_addr": "02:42:53:18:31:cc",
"type": "Ethernet",
"rx_packets": "0",
"rx_errors": "0",
"rx_dropped": "0",
"rx_overruns": "0",
"rx_frame": "0",
"tx_packets": "0",
"tx_errors": "0",
"tx_dropped": "0",
"tx_overruns": "0",
"tx_carrier": "0",
"tx_collisions": "0",
"ipv6_addr": null,
"ipv6_mask": null,
"ipv6_scope": null,
"metric": null
},
{
"name": "ens33",
"flags": "4163",
"state": "UP,BROADCAST,RUNNING,MULTICAST",
"mtu": "1500",
"ipv4_addr": "192.168.71.135",
"ipv4_mask": "255.255.255.0",
"ipv4_bcast": "192.168.71.255",
"ipv6_addr": "fe80::c1cb:715d:bc3e:b8a0",
"ipv6_mask": "64",
"ipv6_scope": "link",
"mac_addr": "00:0c:29:3b:58:0e",
"type": "Ethernet",
"rx_packets": "26348",
"rx_errors": "0",
"rx_dropped": "0",
"rx_overruns": "0",
"rx_frame": "0",
"tx_packets": "5308",
"tx_errors": "0",
"tx_dropped": "0",
"tx_overruns": "0",
"tx_carrier": "0",
"tx_collisions": "0",
"metric": null
},
{
"name": "lo",
"flags": "73",
"state": "UP,LOOPBACK,RUNNING",
"mtu": "65536",
"ipv4_addr": "127.0.0.1",
"ipv4_mask": "255.0.0.0",
"ipv4_bcast": null,
"ipv6_addr": "::1",
"ipv6_mask": "128",
"ipv6_scope": "host",
"mac_addr": null,
"type": "Local Loopback",
"rx_packets": "64",
"rx_errors": "0",
"rx_dropped": "0",
"rx_overruns": "0",
"rx_frame": "0",
"tx_packets": "64",
"tx_errors": "0",
"tx_dropped": "0",
"tx_overruns": "0",
"tx_carrier": "0",
"tx_collisions": "0",
"metric": null
}
]
""" """
from collections import namedtuple from collections import namedtuple
from ifconfigparser import IfconfigParser from ifconfigparser import IfconfigParser

2
jc/parsers/iptables.py
View File

@ -348,7 +348,7 @@ def parse(data):
elif line.find('target') == 0 or line.find('pkts') == 1: elif line.find('target') == 0 or line.find('pkts') == 1:
headers = [] headers = []
headers = [h for h in ' '.join(line.strip().split()).split() if h] headers = [h for h in ' '.join(line.lower().strip().split()).split() if h]
headers.append("options") headers.append("options")
continue continue

36
jc/parsers/jobs.py
View File

@ -10,41 +10,33 @@ Example:
$ jobs -l | jc --jobs -p $ jobs -l | jc --jobs -p
[ [
{ {
"job_number": 1, "job_number": "1",
"pid": 14798, "pid": "19510",
"status": "Running", "status": "Running",
"command": "sleep 10000 &" "command": "sleep 1000 &"
}, },
{ {
"job_number": 2, "job_number": "2",
"pid": 14799, "pid": "19511",
"status": "Running", "status": "Running",
"command": "sleep 10001 &" "command": "sleep 1001 &"
}, },
{ {
"job_number": 3, "job_number": "3",
"pid": 14800, "pid": "19512",
"status": "Running",
"command": "sleep 10002 &"
},
{
"job_number": 4,
"pid": 14814,
"history": "previous", "history": "previous",
"status": "Running", "status": "Running",
"command": "sleep 10003 &" "command": "sleep 1002 &"
}, },
{ {
"job_number": 5, "job_number": "4",
"pid": 14815, "pid": "19513",
"history": "current", "history": "current",
"status": "Running", "status": "Running",
"command": "sleep 10004 &" "command": "sleep 1003 &"
} }
] ]
""" """
import string import string
@ -95,9 +87,9 @@ def parse(data):
parsed_line[0] = parsed_line[0].lstrip('[').rstrip(']') parsed_line[0] = parsed_line[0].lstrip('[').rstrip(']')
# create list of dictionaries # create list of dictionaries
output_line['job_number'] = int(parsed_line[0]) output_line['job_number'] = parsed_line[0]
if pid: if pid:
output_line['pid'] = int(pid) output_line['pid'] = pid
if job_history: if job_history:
output_line['history'] = job_history output_line['history'] = job_history
output_line['status'] = parsed_line[1] output_line['status'] = parsed_line[1]

95
jc/parsers/ls.py
View File

@ -5,81 +5,96 @@ Usage:
ls options supported: ls options supported:
- None - None
- l - lah
- a
Examples: Examples:
$ ls -a /usr/bin | jc --ls -p $ ls /usr/bin | jc --ls -p
[ [
{ {
"filename": "." "filename": "apropos"
}, },
{ {
"filename": ".." "filename": "arch"
}, },
{ {
"filename": "2to3-" "filename": "awk"
}, },
{ {
"filename": "2to3-2.7" "filename": "base64"
},
{
"filename": "AssetCacheLocatorUtil"
}, },
... ...
] ]
$ ls -al /usr/bin | jc --ls -p $ ls -l /usr/bin | jc --ls -p
[ [
{ {
"filename": ".", "filename": "apropos",
"flags": "drwxr-xr-x", "link_to": "whatis",
"links": 970, "flags": "lrwxrwxrwx.",
"links": "1",
"owner": "root", "owner": "root",
"group": "wheel", "group": "root",
"bytes": 31040, "size": "6",
"date": "Aug 27 21:20" "date": "Aug 15 10:53"
}, },
{ {
"filename": "..", "filename": "arch",
"flags": "drwxr-xr-x@", "flags": "-rwxr-xr-x.",
"links": 9, "links": "1",
"owner": "root", "owner": "root",
"group": "wheel", "group": "root",
"bytes": 288, "size": "33080",
"date": "May 3 22:14" "date": "Aug 19 23:25"
}, },
{ {
"filename": "2to3-", "filename": "awk",
"flags": "-rwxr-xr-x", "link_to": "gawk",
"links": 4, "flags": "lrwxrwxrwx.",
"links": "1",
"owner": "root", "owner": "root",
"group": "wheel", "group": "root",
"bytes": 925, "size": "4",
"date": "Feb 22 2019" "date": "Aug 15 10:53"
}, },
{ {
"filename": "2to3-2.7", "filename": "base64",
"link_to": "../../System/Library/Frameworks/Python.framework/Versions/2.7/bin/2to3-2.7", "flags": "-rwxr-xr-x.",
"flags": "lrwxr-xr-x", "links": "1",
"links": 1,
"owner": "root", "owner": "root",
"group": "wheel", "group": "root",
"bytes": 74, "size": "37360",
"date": "May 4 02:12" "date": "Aug 19 23:25"
},
{
"filename": "basename",
"flags": "-rwxr-xr-x.",
"links": "1",
"owner": "root",
"group": "root",
"size": "29032",
"date": "Aug 19 23:25"
},
{
"filename": "bash",
"flags": "-rwxr-xr-x.",
"links": "1",
"owner": "root",
"group": "root",
"size": "964600",
"date": "Aug 8 05:06"
}, },
... ...
] ]
$ $ ls -l /usr/bin | jc --ls | jq '.[] | 'select(.bytes > 50000000)' $ ls -l /usr/bin | jc --ls | jq '.[] | select(.size|tonumber > 50000000)'
{ {
"filename": "emacs", "filename": "emacs",
"flags": "-r-xr-xr-x", "flags": "-r-xr-xr-x",
"links": 1, "links": 1,
"owner": "root", "owner": "root",
"group": "wheel", "group": "wheel",
"bytes": 117164432, "size": "117164432",
"date": "May 3 22:26" "date": "May 3 22:26"
} }
""" """
@ -117,10 +132,10 @@ def parse(data):
output_line['link_to'] = filename_field[1] output_line['link_to'] = filename_field[1]
output_line['flags'] = parsed_line[0] output_line['flags'] = parsed_line[0]
output_line['links'] = int(parsed_line[1]) output_line['links'] = parsed_line[1]
output_line['owner'] = parsed_line[2] output_line['owner'] = parsed_line[2]
output_line['group'] = parsed_line[3] output_line['group'] = parsed_line[3]
output_line['bytes'] = int(parsed_line[4]) output_line['size'] = parsed_line[4]
output_line['date'] = ' '.join(parsed_line[5:8]) output_line['date'] = ' '.join(parsed_line[5:8])
output.append(output_line) output.append(output_line)
else: else:

81
jc/parsers/lsblk.py
View File

@ -8,46 +8,55 @@ Example:
$ lsblk | jc --lsblk -p $ lsblk | jc --lsblk -p
[ [
{ {
"NAME": "loop0", "name": "sda",
"MAJ:MIN": "7:0", "maj_min": "8:0",
"RM": "0", "rm": "0",
"SIZE": "54.5M", "size": "20G",
"RO": "1", "ro": "0",
"TYPE": "loop", "type": "disk"
"MOUNTPOINT": "/snap/core18/1223"
}, },
{ {
"NAME": "sda", "name": "sda1",
"MAJ:MIN": "8:0", "maj_min": "8:1",
"RM": "0", "rm": "0",
"SIZE": "20G", "size": "1G",
"RO": "0", "ro": "0",
"TYPE": "disk" "type": "part",
"mountpoint": "/boot"
}, },
{ {
"NAME": "sda1", "name": "sda2",
"MAJ:MIN": "8:1", "maj_min": "8:2",
"RM": "0", "rm": "0",
"SIZE": "1M", "size": "19G",
"RO": "0", "ro": "0",
"TYPE": "part" "type": "part"
}, },
{ {
"NAME": "sda2", "name": "centos-root",
"MAJ:MIN": "8:2", "maj_min": "253:0",
"RM": "0", "rm": "0",
"SIZE": "20G", "size": "17G",
"RO": "0", "ro": "0",
"TYPE": "part", "type": "lvm",
"MOUNTPOINT": "/" "mountpoint": "/"
}, },
{ {
"NAME": "sr0", "name": "centos-swap",
"MAJ:MIN": "11:0", "maj_min": "253:1",
"RM": "1", "rm": "0",
"SIZE": "64.8M", "size": "2G",
"RO": "0", "ro": "0",
"TYPE": "rom" "type": "lvm",
"mountpoint": "[SWAP]"
},
{
"name": "sr0",
"maj_min": "11:0",
"rm": "1",
"size": "1024M",
"ro": "0",
"type": "rom"
} }
] ]
""" """
@ -59,12 +68,16 @@ def parse(data):
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501 # https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines() cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h] headers = [h for h in ' '.join(cleandata[0].lower().strip().split()).split() if h]
# clean up 'maj:min' header
# even though colon in a key is valid json, it can make things difficult
headers = ['maj_min' if x == 'maj:min' else x for x in headers]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:]) raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
output = [dict(zip(headers, r)) for r in raw_data] output = [dict(zip(headers, r)) for r in raw_data]
for entry in output: for entry in output:
entry['NAME'] = entry['NAME'].encode('ascii', errors='ignore').decode() entry['name'] = entry['name'].encode('ascii', errors='ignore').decode()
return output return output

78
jc/parsers/lsmod.py
View File

@ -7,44 +7,12 @@ Example:
$ lsmod | jc --lsmod -p $ lsmod | jc --lsmod -p
[ [
...
{ {
"Module": "nf_nat_ipv4", "module": "nf_conntrack",
"Size": "14115", "size": "139224",
"Used": "1", "used": "7",
"By": [ "by": [
"iptable_nat"
]
},
{
"Module": "nf_nat",
"Size": "26583",
"Used": "3",
"By": [
"nf_nat_ipv4",
"nf_nat_ipv6",
"nf_nat_masquerade_ipv4"
]
},
{
"Module": "iptable_mangle",
"Size": "12695",
"Used": "1"
},
{
"Module": "iptable_security",
"Size": "12705",
"Used": "1"
},
{
"Module": "iptable_raw",
"Size": "12678",
"Used": "1"
},
{
"Module": "nf_conntrack",
"Size": "139224",
"Used": "7",
"By": [
"nf_nat", "nf_nat",
"nf_nat_ipv4", "nf_nat_ipv4",
"nf_nat_ipv6", "nf_nat_ipv6",
@ -54,6 +22,33 @@ $ lsmod | jc --lsmod -p
"nf_conntrack_ipv6" "nf_conntrack_ipv6"
] ]
}, },
{
"module": "ip_set",
"size": "45799",
"used": "0"
},
{
"module": "nfnetlink",
"size": "14519",
"used": "1",
"by": [
"ip_set"
]
},
{
"module": "ebtable_filter",
"size": "12827",
"used": "1"
},
{
"module": "ebtables",
"size": "35009",
"used": "2",
"by": [
"ebtable_nat",
"ebtable_filter"
]
},
... ...
] ]
""" """
@ -65,16 +60,13 @@ def parse(data):
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501 # https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines() cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h] headers = [h for h in ' '.join(cleandata[0].lower().strip().split()).split() if h]
headers.pop(-1)
headers.append('By')
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:]) raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
output = [dict(zip(headers, r)) for r in raw_data] output = [dict(zip(headers, r)) for r in raw_data]
for mod in output: for mod in output:
if 'By' in mod: if 'by' in mod:
mod['By'] = mod['By'].split(',') mod['by'] = mod['by'].split(',')
return output return output

125
jc/parsers/lsof.py
View File

@ -5,71 +5,72 @@ Usage:
Example: Example:
$ sudo lsof | jc --lsof -p $ sudo lsof | jc --lsof -p | more
[ [
{ {
"COMMAND": "systemd", "command": "systemd",
"PID": "1", "pid": "1",
"TID": null, "tid": null,
"USER": "root", "user": "root",
"FD": "cwd", "fd": "cwd",
"TYPE": "DIR", "type": "DIR",
"DEVICE": "253,0", "device": "8,2",
"SIZE/OFF": "224", "size_off": "4096",
"NODE": "64", "node": "2",
"NAME": "/" "name": "/"
}, },
{ {
"COMMAND": "systemd", "command": "systemd",
"PID": "1", "pid": "1",
"TID": null, "tid": null,
"USER": "root", "user": "root",
"FD": "rtd", "fd": "rtd",
"TYPE": "DIR", "type": "DIR",
"DEVICE": "253,0", "device": "8,2",
"SIZE/OFF": "224", "size_off": "4096",
"NODE": "64", "node": "2",
"NAME": "/" "name": "/"
}, },
{ {
"COMMAND": "systemd", "command": "systemd",
"PID": "1", "pid": "1",
"TID": null, "tid": null,
"USER": "root", "user": "root",
"FD": "txt", "fd": "txt",
"TYPE": "REG", "type": "REG",
"DEVICE": "253,0", "device": "8,2",
"SIZE/OFF": "1624520", "size_off": "1595792",
"NODE": "50360451", "node": "668802",
"NAME": "/usr/lib/systemd/systemd" "name": "/lib/systemd/systemd"
}, },
{ {
"COMMAND": "systemd", "command": "systemd",
"PID": "1", "pid": "1",
"TID": null, "tid": null,
"USER": "root", "user": "root",
"FD": "mem", "fd": "mem",
"TYPE": "REG", "type": "REG",
"DEVICE": "253,0", "device": "8,2",
"SIZE/OFF": "20064", "size_off": "1700792",
"NODE": "8146", "node": "656167",
"NAME": "/usr/lib64/libuuid.so.1.3.0" "name": "/lib/x86_64-linux-gnu/libm-2.27.so"
}, },
{ {
"COMMAND": "systemd", "command": "systemd",
"PID": "1", "pid": "1",
"TID": null, "tid": null,
"USER": "root", "user": "root",
"FD": "mem", "fd": "mem",
"TYPE": "REG", "type": "REG",
"DEVICE": "253,0", "device": "8,2",
"SIZE/OFF": "265600", "size_off": "121016",
"NODE": "8147", "node": "655394",
"NAME": "/usr/lib64/libblkid.so.1.1.0" "name": "/lib/x86_64-linux-gnu/libudev.so.1.6.9"
}, },
... ...
] ]
""" """
import string
def parse(data): def parse(data):
@ -83,10 +84,15 @@ def parse(data):
if cleandata: if cleandata:
# find column value of last character of each header # find column value of last character of each header
header_row = cleandata.pop(0) header_text = cleandata.pop(0).lower()
headers = header_row.split()
header_spec = []
# clean up 'size/off' header
# even though forward slash in a key is valid json, it can make things difficult
header_row = header_text.replace('size/off', 'size_off')
headers = header_row.split()
header_spec = []
for i, h in enumerate(headers): for i, h in enumerate(headers):
# header tuple is (index, header_name, col) # header tuple is (index, header_name, col)
header_spec.append((i, h, header_row.find(h) + len(h))) header_spec.append((i, h, header_row.find(h) + len(h)))
@ -99,10 +105,15 @@ def parse(data):
temp_line = entry.split(maxsplit=len(headers) - 1) temp_line = entry.split(maxsplit=len(headers) - 1)
for spec in header_spec: for spec in header_spec:
if spec[1] == 'COMMAND' or spec[1] == 'NAME':
index = spec[0]
header_name = spec[1]
col = spec[2] - 1 # subtract one since column starts at 0 instead of 1
if header_name == 'command' or header_name == 'name':
continue continue
if entry[spec[2] - 1] == ' ': if entry[col] in string.whitespace:
temp_line.insert(spec[0], None) temp_line.insert(index, None)
name = ' '.join(temp_line[9:]) name = ' '.join(temp_line[9:])
fixed_line = temp_line[0:9] fixed_line = temp_line[0:9]

84
jc/parsers/netstat.py
View File

@ -22,8 +22,8 @@ $ netstat -p | jc --netstat -p
"state": "ESTABLISHED", "state": "ESTABLISHED",
"pid": 53550, "pid": 53550,
"program_name": "git-remote-ht", "program_name": "git-remote-ht",
"receive_q": 0, "receive_q": "0",
"send_q": 0 "send_q": "0"
}, },
{ {
"transport_protocol": "tcp", "transport_protocol": "tcp",
@ -35,38 +35,25 @@ $ netstat -p | jc --netstat -p
"state": "ESTABLISHED", "state": "ESTABLISHED",
"pid": 53550, "pid": 53550,
"program_name": "git-remote-ht", "program_name": "git-remote-ht",
"receive_q": 0, "receive_q": "0",
"send_q": 0 "send_q": "0"
} }
] ]
$ netstat -lpn | jc --netstat -p $ sudo netstat -lpn | jc --netstat -p
[ [
{ {
"transport_protocol": "tcp", "transport_protocol": "tcp",
"network_protocol": "ipv4", "network_protocol": "ipv4",
"local_address": "127.0.0.1", "local_address": "127.0.0.1",
"local_port": "42351", "local_port": "25",
"foreign_address": "0.0.0.0", "foreign_address": "0.0.0.0",
"foreign_port": "*", "foreign_port": "*",
"state": "LISTEN", "state": "LISTEN",
"pid": 1112, "pid": "1584",
"program_name": "containerd", "program_name": "master",
"receive_q": 0, "receive_q": "0",
"send_q": 0 "send_q": "0"
},
{
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "127.0.0.53",
"local_port": "53",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"state": "LISTEN",
"pid": 885,
"program_name": "systemd-resolve",
"receive_q": 0,
"send_q": 0
}, },
{ {
"transport_protocol": "tcp", "transport_protocol": "tcp",
@ -76,48 +63,37 @@ $ netstat -lpn | jc --netstat -p
"foreign_address": "0.0.0.0", "foreign_address": "0.0.0.0",
"foreign_port": "*", "foreign_port": "*",
"state": "LISTEN", "state": "LISTEN",
"pid": 1127, "pid": "1213",
"program_name": "sshd", "program_name": "sshd",
"receive_q": 0, "receive_q": "0",
"send_q": 0 "send_q": "0"
}, },
{ {
"transport_protocol": "tcp", "transport_protocol": "tcp",
"network_protocol": "ipv6", "network_protocol": "ipv6",
"local_address": "::", "local_address": "::1",
"local_port": "22", "local_port": "25",
"foreign_address": "::", "foreign_address": "::",
"foreign_port": "*", "foreign_port": "*",
"state": "LISTEN", "state": "LISTEN",
"pid": 1127, "pid": "1584",
"program_name": "sshd", "program_name": "master",
"receive_q": 0, "receive_q": "0",
"send_q": 0 "send_q": "0"
}, },
{ {
"transport_protocol": "udp", "transport_protocol": "udp",
"network_protocol": "ipv4", "network_protocol": "ipv4",
"local_address": "127.0.0.53", "local_address": "0.0.0.0",
"local_port": "53",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"pid": 885,
"program_name": "systemd-resolve",
"receive_q": 0,
"send_q": 0
},
{
"transport_protocol": "udp",
"network_protocol": "ipv4",
"local_address": "192.168.71.131",
"local_port": "68", "local_port": "68",
"foreign_address": "0.0.0.0", "foreign_address": "0.0.0.0",
"foreign_port": "*", "foreign_port": "*",
"pid": 867, "pid": "19177",
"program_name": "systemd-network", "program_name": "dhclient",
"receive_q": 0, "receive_q": "0",
"send_q": 0 "send_q": "0"
} },
...
] ]
""" """
import string import string
@ -161,15 +137,15 @@ def parse_line(entry):
output_line['state'] = parsed_line[5] output_line['state'] = parsed_line[5]
if len(parsed_line) > 6 and parsed_line[6][0] in string.digits: if len(parsed_line) > 6 and parsed_line[6][0] in string.digits:
output_line['pid'] = int(parsed_line[6].split('/')[0]) output_line['pid'] = parsed_line[6].split('/')[0]
output_line['program_name'] = parsed_line[6].split('/')[1] output_line['program_name'] = parsed_line[6].split('/')[1]
else: else:
if parsed_line[5][0] in string.digits: if parsed_line[5][0] in string.digits:
output_line['pid'] = int(parsed_line[5].split('/')[0]) output_line['pid'] = parsed_line[5].split('/')[0]
output_line['program_name'] = parsed_line[5].split('/')[1] output_line['program_name'] = parsed_line[5].split('/')[1]
output_line['receive_q'] = int(parsed_line[1]) output_line['receive_q'] = parsed_line[1]
output_line['send_q'] = int(parsed_line[2]) output_line['send_q'] = parsed_line[2]
return output_line return output_line

83
jc/parsers/ps.py
View File

@ -11,45 +11,56 @@ Example:
$ ps -ef | jc --ps -p $ ps -ef | jc --ps -p
[ [
...
{ {
"UID": "root", "uid": "root",
"PID": "1", "pid": "545",
"PPID": "0", "ppid": "1",
"C": "0", "c": "0",
"STIME": "13:58", "stime": "Oct21",
"TTY": "?", "tty": "?",
"TIME": "00:00:05", "time": "00:00:03",
"CMD": "/lib/systemd/systemd --system --deserialize 35" "cmd": "/usr/lib/systemd/systemd-journald"
}, },
{ {
"UID": "root", "uid": "root",
"PID": "2", "pid": "566",
"PPID": "0", "ppid": "1",
"C": "0", "c": "0",
"STIME": "13:58", "stime": "Oct21",
"TTY": "?", "tty": "?",
"TIME": "00:00:00", "time": "00:00:00",
"CMD": "[kthreadd]" "cmd": "/usr/sbin/lvmetad -f"
}, },
{ {
"UID": "root", "uid": "root",
"PID": "4", "pid": "580",
"PPID": "2", "ppid": "1",
"C": "0", "c": "0",
"STIME": "13:58", "stime": "Oct21",
"TTY": "?", "tty": "?",
"TIME": "00:00:00", "time": "00:00:00",
"CMD": "[kworker/0:0H]" "cmd": "/usr/lib/systemd/systemd-udevd"
}, },
{ {
"UID": "root", "uid": "root",
"PID": "6", "pid": "659",
"PPID": "2", "ppid": "2",
"C": "0", "c": "0",
"STIME": "13:58", "stime": "Oct21",
"TTY": "?", "tty": "?",
"TIME": "00:00:00", "time": "00:00:00",
"CMD": "[mm_percpu_wq]" "cmd": "[kworker/u257:0]"
},
{
"uid": "root",
"pid": "666",
"ppid": "2",
"c": "0",
"stime": "Oct21",
"tty": "?",
"time": "00:00:00",
"cmd": "[hci0]"
}, },
... ...
] ]
@ -62,6 +73,12 @@ def parse(data):
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501 # https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines() cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h] headers = [h for h in ' '.join(cleandata[0].lower().strip().split()).split() if h]
# clean up '%cpu' and '%mem' headers
# even though % in a key is valid json, it can make things difficult
headers = ['cpu_percent' if x == '%cpu' else x for x in headers]
headers = ['mem_percent' if x == '%mem' else x for x in headers]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:]) raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
return [dict(zip(headers, r)) for r in raw_data] return [dict(zip(headers, r)) for r in raw_data]

62
jc/parsers/route.py
View File

@ -6,47 +6,37 @@ Usage:
Example: Example:
$ route -n | jc --route -p $ route | jc --route -p
[ [
{ {
"Destination": "0.0.0.0", "destination": "default",
"Gateway": "192.168.71.2", "gateway": "gateway",
"Genmask": "0.0.0.0", "genmask": "0.0.0.0",
"Flags": "UG", "flags": "UG",
"Metric": "100", "metric": "100",
"Ref": "0", "ref": "0",
"Use": "0", "use": "0",
"Iface": "ens33" "iface": "ens33"
}, },
{ {
"Destination": "172.17.0.0", "destination": "172.17.0.0",
"Gateway": "0.0.0.0", "gateway": "0.0.0.0",
"Genmask": "255.255.0.0", "genmask": "255.255.0.0",
"Flags": "U", "flags": "U",
"Metric": "0", "metric": "0",
"Ref": "0", "ref": "0",
"Use": "0", "use": "0",
"Iface": "docker0" "iface": "docker0"
}, },
{ {
"Destination": "192.168.71.0", "destination": "192.168.71.0",
"Gateway": "0.0.0.0", "gateway": "0.0.0.0",
"Genmask": "255.255.255.0", "genmask": "255.255.255.0",
"Flags": "U", "flags": "U",
"Metric": "0", "metric": "100",
"Ref": "0", "ref": "0",
"Use": "0", "use": "0",
"Iface": "ens33" "iface": "ens33"
},
{
"Destination": "192.168.71.2",
"Gateway": "0.0.0.0",
"Genmask": "255.255.255.255",
"Flags": "UH",
"Metric": "100",
"Ref": "0",
"Use": "0",
"Iface": "ens33"
} }
] ]
""" """
@ -58,6 +48,6 @@ def parse(data):
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501 # https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines()[1:] cleandata = data.splitlines()[1:]
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h] headers = [h for h in ' '.join(cleandata[0].lower().strip().split()).split() if h]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:]) raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
return [dict(zip(headers, r)) for r in raw_data] return [dict(zip(headers, r)) for r in raw_data]

2
jc/parsers/uname.py
View File

@ -26,6 +26,8 @@ def parse(data):
output = {} output = {}
parsed_line = data.split(maxsplit=3) parsed_line = data.split(maxsplit=3)
if len(parsed_line) > 1:
output['kernel_name'] = parsed_line.pop(0) output['kernel_name'] = parsed_line.pop(0)
output['node_name'] = parsed_line.pop(0) output['node_name'] = parsed_line.pop(0)
output['kernel_release'] = parsed_line.pop(0) output['kernel_release'] = parsed_line.pop(0)

45
jc/parsers/uptime.py Normal file
View File

@ -0,0 +1,45 @@
"""jc - JSON CLI output utility uptime Parser
Usage:
specify --uptime as the first argument if the piped input is coming from uptime
Example:
$ uptime | jc --uptime -p
{
"time": "16:52",
"uptime": "3 days, 4:49",
"users": "5",
"load_1m": "1.85",
"load_5m": "1.90",
"load_15m": "1.91"
}
"""
def parse(data):
output = {}
cleandata = data.splitlines()
if cleandata:
parsed_line = cleandata[0].split()
# allow space for odd times
while len(parsed_line) < 20:
parsed_line.insert(2, ' ')
# find first part of time
for i, word in enumerate(parsed_line[2:]):
if word != ' ':
marker = i + 2
break
output['time'] = parsed_line[0]
output['uptime'] = ' '.join(parsed_line[marker:13]).lstrip().rstrip(',')
output['users'] = parsed_line[13]
output['load_1m'] = parsed_line[17].rstrip(',')
output['load_5m'] = parsed_line[18].rstrip(',')
output['load_15m'] = parsed_line[19]
return output

47
jc/parsers/w.py Normal file
View File

@ -0,0 +1,47 @@
"""jc - JSON CLI output utility w Parser
Usage:
specify --w as the first argument if the piped input is coming from w
Example:
$ w | jc --w -p
[
{
"user": "root",
"tty": "ttyS0",
"from": "-",
"login_at": "Mon20",
"idle": "0.00s",
"jcpu": "14.70s",
"pcpu": "0.00s",
"what": "bash"
},
{
"user": "root",
"tty": "pts/0",
"from": "192.168.71.1",
"login_at": "Thu22",
"idle": "22:46m",
"jcpu": "0.05s",
"pcpu": "0.05s",
"what": "-bash"
}
]
"""
def parse(data):
# code adapted from Conor Heine at:
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines()[1:]
headers = [h for h in ' '.join(cleandata[0].lower().strip().split()).split() if h]
# clean up 'login@' header
# even though @ in a key is valid json, it can make things difficult
headers = ['login_at' if x == 'login@' else x for x in headers]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
return [dict(zip(headers, r)) for r in raw_data]

2
setup.py
View File

@ -5,7 +5,7 @@ with open('README.md', 'r') as f:
setuptools.setup( setuptools.setup(
name='jc', name='jc',
version='0.9.1', version='1.0.1',
author='Kelly Brazil', author='Kelly Brazil',
author_email='kellyjonbrazil@gmail.com', author_email='kellyjonbrazil@gmail.com',
description='This tool serializes the output of popular command line tools to structured JSON output.', description='This tool serializes the output of popular command line tools to structured JSON output.',