1
0
mirror of https://github.com/kellyjonbrazil/jc.git synced 2026-04-03 17:44:07 +02:00

Compare commits

...

56 Commits

Author SHA1 Message Date
Kelly Brazil
976fd7d9bd readme update 2019-10-22 17:24:56 -07:00
Kelly Brazil
d8337870ca update documentation 2019-10-22 17:21:00 -07:00
Kelly Brazil
39a8aec77f v0.8.1 build 2019-10-22 17:15:02 -07:00
Kelly Brazil
306d539b6b readme update 2019-10-22 16:50:01 -07:00
Kelly Brazil
f3087b8a8e update readme and formatting 2019-10-22 16:40:27 -07:00
Kelly Brazil
414c2ecef8 fix iptables parser 2019-10-22 16:32:55 -07:00
Kelly Brazil
776ef2d1be add iptables parser 2019-10-22 15:42:29 -07:00
Kelly Brazil
9ac5746996 add uname parser 2019-10-22 13:28:15 -07:00
Kelly Brazil
a3e55d97c0 add mount parser 2019-10-22 12:54:41 -07:00
Kelly Brazil
b15227e7ba add lsblk parser 2019-10-22 11:55:11 -07:00
Kelly Brazil
ec3d1f84ce fix free parser 2019-10-22 11:26:58 -07:00
Kelly Brazil
753d5fd9fe readme update 2019-10-22 11:17:21 -07:00
Kelly Brazil
73a0d70c92 readme update 2019-10-22 11:15:44 -07:00
Kelly Brazil
c2c189f3e6 readme update 2019-10-22 11:14:19 -07:00
Kelly Brazil
36bc55a310 fix df 2019-10-22 11:11:41 -07:00
Kelly Brazil
a023001cd3 add df, env, and free parsers 2019-10-22 11:10:11 -07:00
Kelly Brazil
e3750b4962 documentation enhancements 2019-10-22 07:40:42 -07:00
Kelly Brazil
b5ea08e55b fix transport protocol 2019-10-21 18:22:51 -07:00
Kelly Brazil
8e71b8e352 fix jq example 2019-10-21 18:11:51 -07:00
Kelly Brazil
4c8610c54f fixed build 2019-10-21 17:59:32 -07:00
Kelly Brazil
c8f886dc8f fix example 2019-10-21 17:56:53 -07:00
Kelly Brazil
4cfc2d22b3 update changelog 2019-10-21 17:38:40 -07:00
Kelly Brazil
59238c8540 Merge pull request #2 from kellyjonbrazil/dev
Dev v0.6.2
2019-10-21 17:36:33 -07:00
Kelly Brazil
30080c0165 reorder parsers 2019-10-21 17:26:00 -07:00
Kelly Brazil
fab80bb3b4 readme update 2019-10-21 17:20:12 -07:00
Kelly Brazil
a9f2df8054 move parsed_line var lower 2019-10-21 14:27:26 -07:00
Kelly Brazil
1d110be6cb update doco 2019-10-21 14:13:31 -07:00
Kelly Brazil
be81b5e1ed readme update 2019-10-21 13:47:22 -07:00
Kelly Brazil
5f88f7d8a0 netstat cleanup 2019-10-21 13:41:53 -07:00
Kelly Brazil
e57c7cc8ef change output from dict to list 2019-10-21 13:23:29 -07:00
Kelly Brazil
b216627c10 flatten netstat output 2019-10-21 13:19:00 -07:00
Kelly Brazil
6e925eab13 clean up arg parsing 2019-10-21 13:07:30 -07:00
Kelly Brazil
d54d906c57 update readme 2019-10-18 19:03:11 -07:00
Kelly Brazil
0040febbf0 Merge pull request #1 from kellyjonbrazil/dev
Dev v0.5.5
2019-10-18 18:57:32 -07:00
Kelly Brazil
e416c77bed version bump 2019-10-18 18:56:26 -07:00
Kelly Brazil
9181d242aa bump version 2019-10-18 18:47:20 -07:00
Kelly Brazil
d6f94c53a4 fix blank output case in ls 2019-10-18 18:46:25 -07:00
Kelly Brazil
a9294f32a0 ls fixes 2019-10-18 18:40:56 -07:00
Kelly Brazil
4d93b38fe4 add route documentation 2019-10-18 14:18:34 -07:00
Kelly Brazil
77b74c5507 add route parser 2019-10-18 13:59:30 -07:00
Kelly Brazil
51a1d3e6f4 version bump 2019-10-18 13:51:36 -07:00
Kelly Brazil
2eba30422b skip first line 2019-10-18 13:43:49 -07:00
Kelly Brazil
43ed09ce5b add route parser 2019-10-18 13:38:11 -07:00
Kelly Brazil
367ab54f94 formatting 2019-10-18 13:34:28 -07:00
Kelly Brazil
4f552e370e acknowledgment 2019-10-18 13:26:24 -07:00
Kelly Brazil
7571139f79 bump version 2019-10-18 13:20:38 -07:00
Kelly Brazil
8ec1bec317 add ps parser 2019-10-18 13:19:39 -07:00
Kelly Brazil
c04895407f ubuntu fixes 2019-10-18 12:57:02 -07:00
Kelly Brazil
cec73d6131 linting 2019-10-18 09:57:22 -07:00
Kelly Brazil
756c2bc9ac add acknowledgments 2019-10-18 09:57:10 -07:00
Kelly Brazil
084048987c changelog update 2019-10-17 17:59:46 -07:00
Kelly Brazil
02d97394dd changelog update 2019-10-17 17:28:14 -07:00
Kelly Brazil
177f948f97 version update 2019-10-17 16:11:55 -07:00
Kelly Brazil
c2b013150e add changelog 2019-10-17 15:51:39 -07:00
Kelly Brazil
0bec67c29c double quotes to single quotes 2019-10-17 15:39:12 -07:00
Kelly Brazil
8073d15fe1 readme update 2019-10-17 15:03:32 -07:00
17 changed files with 1955 additions and 502 deletions

944
README.md
View File

@@ -1,52 +1,153 @@
# JC
JSON CLI output utility
`jc` is used to JSONify the output of many standard linux cli tools for easier parsing in scripts. Parsers for `ls`, `ifconfig`, and `netstat` are currently included and more can be added via modules.
`jc` is used to JSONify the output of many standard linux cli tools for easier parsing in scripts. See the **Parsers** section for supported commands.
This allows further command line processing of output with tools like `jq` simply by piping commands:
```
$ ls -l /usr/bin | jc --ls | jq '.[] | select(.bytes > 50000000)'
{
"filename": "emacs",
"flags": "-r-xr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 117164432,
"date": "May 3 22:26"
}
```
The `jc` parsers can also be used as python modules:
```
>>> import jc.parsers.ls
>>>
>>> data='''-rwxr-xr-x 1 root wheel 23648 May 3 22:26 cat
... -rwxr-xr-x 1 root wheel 30016 May 3 22:26 chmod
... -rwxr-xr-x 1 root wheel 29024 May 3 22:26 cp
... -rwxr-xr-x 1 root wheel 375824 May 3 22:26 csh
... -rwxr-xr-x 1 root wheel 28608 May 3 22:26 date
... -rwxr-xr-x 1 root wheel 32000 May 3 22:26 dd
... -rwxr-xr-x 1 root wheel 23392 May 3 22:26 df
... -rwxr-xr-x 1 root wheel 18128 May 3 22:26 echo'''
>>>
>>> jc.parsers.ls.parse(data)
[{'filename': 'cat', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'bytes': 23648, 'date': 'May 3 22:26'}, {'filename': 'chmod', 'flags': '-rwxr-xr-x', 'links': 1,
'owner': 'root', 'group': 'wheel', 'bytes': 30016, 'date': 'May 3 22:26'}, {'filename': 'cp',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 29024,
'date': 'May 3 22:26'}, {'filename': 'csh', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'bytes': 375824, 'date': 'May 3 22:26'}, {'filename': 'date',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 28608,
'date': 'May 3 22:26'}, {'filename': 'dd', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'bytes': 32000, 'date': 'May 3 22:26'}, {'filename': 'df', 'flags': '-rwxr-xr-x',
'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 23392, 'date': 'May 3 22:26'},
{'filename': 'echo', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'bytes': 18128, 'date': 'May 3 22:26'}]
```
In this case the output will be a python dictionary instead of JSON.
## Installation
```
$ pip3 install jc
```
## Usage
`jc` accepts piped input from `STDIN` and outputs a JSON representation of the previous command to `STDOUT`. The JSON output can be compact or pretty formatted.
```
jc [parser] [options]
```
The first argument is required and identifies the command that is piping output to `jc` input. For example:
- `--ls` enables the `ls` parser
`jc` accepts piped input from `STDIN` and outputs a JSON representation of the previous command's output to `STDOUT`. The JSON output can be compact or pretty formatted.
### Parsers
- `--df` enables the `df` parser
- `--env` enables the `env` parser
- `--free` enables the `free` parser
- `--ifconfig` enables the `ifconfig` parser
- `--iptables` enables the `iptables` parser
- `--ls` enables the `ls` parser
- `--lsblk` enables the `lsblk` parser
- `--mount` enables the `mount` parser
- `--netstat` enables the `netstat` parser
- `--ps` enables the `ps` parser
- `--route` enables the `route` parser
- `--uname` enables the `uname -a` parser
The second `-p` argument is optional and specifies whether to pretty format the JSON output.
### Options
- `-p` specifies whether to pretty format the JSON output
## Examples
### df
```
$ ls -l /bin | jc --ls -p
$ df | jc --df -p
[
{
"filename": "bash",
"flags": "-r-xr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 618416,
"date": "May 3 22:26"
"Filesystem": "udev",
"1K-blocks": "977500",
"Used": "0",
"Available": "977500",
"Use%": "0%",
"Mounted": "/dev"
},
{
"filename": "cat",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 23648,
"date": "May 3 22:26"
"Filesystem": "tmpfs",
"1K-blocks": "201732",
"Used": "1180",
"Available": "200552",
"Use%": "1%",
"Mounted": "/run"
},
{
"filename": "chmod",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 30016,
"date": "May 3 22:26"
"Filesystem": "/dev/sda2",
"1K-blocks": "20508240",
"Used": "5747284",
"Available": "13696152",
"Use%": "30%",
"Mounted": "/"
},
{
"Filesystem": "tmpfs",
"1K-blocks": "1008648",
"Used": "0",
"Available": "1008648",
"Use%": "0%",
"Mounted": "/dev/shm"
},
...
]
```
### env
```
$ env | jc --env -p
[
{
"TERM": "xterm-256color"
},
{
"SHELL": "/bin/bash"
},
{
"USER": "root"
},
{
"PATH": "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
},
{
"PWD": "/bin"
},
{
"LANG": "en_US.UTF-8"
},
{
"HOME": "/root"
},
{
"_": "/usr/bin/env"
}
]
```
### ifconfig
```
$ ifconfig | jc --ifconfig -p
[
{
@@ -128,131 +229,676 @@ $ ifconfig | jc --ifconfig -p
"metric": null
}
]
```
### iptables
```
$ sudo iptables -L -t nat | jc --iptables -p
[
{
"chain": "PREROUTING",
"rules": [
{
"target": "PREROUTING_direct",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "PREROUTING_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "PREROUTING_ZONES",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "DOCKER",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere",
"options": "ADDRTYPE match dst-type LOCAL"
}
]
},
{
"chain": "INPUT",
"rules": []
},
{
"chain": "OUTPUT",
"rules": [
{
"target": "OUTPUT_direct",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "DOCKER",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "!loopback/8",
"options": "ADDRTYPE match dst-type LOCAL"
}
]
},
...
]
```
```
$ sudo iptables -vnL -t filter | jc --iptables -p
[
{
"chain": "INPUT",
"rules": [
{
"pkts": "1571",
"bytes": "3394K",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "lo",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_direct",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DROP",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate INVALID"
},
{
"pkts": "710",
"bytes": "60078",
"target": "REJECT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "reject-with icmp-host-prohibited"
}
]
},
{
"chain": "FORWARD",
"rules": [
{
"pkts": "0",
"bytes": "0",
"target": "DOCKER-ISOLATION",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DOCKER",
"prot": "all",
"opt": "--",
"in": "*",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "docker0",
"out": "!docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "docker0",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "lo",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_direct",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_IN_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_IN_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_OUT_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_OUT_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DROP",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate INVALID"
},
{
"pkts": "0",
"bytes": "0",
"target": "REJECT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "reject-with icmp-host-prohibited"
}
]
},
...
]
```
### ls
```
$ ls -l /bin | jc --ls -p
[
{
"filename": "bash",
"flags": "-r-xr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 618416,
"date": "May 3 22:26"
},
{
"filename": "cat",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 23648,
"date": "May 3 22:26"
},
{
"filename": "chmod",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 30016,
"date": "May 3 22:26"
},
...
]
```
### lsblk
```
$ lsblk | jc --lsblk -p
[
{
"NAME": "loop0",
"MAJ:MIN": "7:0",
"RM": "0",
"SIZE": "54.5M",
"RO": "1",
"TYPE": "loop",
"MOUNTPOINT": "/snap/core18/1223"
},
{
"NAME": "sda",
"MAJ:MIN": "8:0",
"RM": "0",
"SIZE": "20G",
"RO": "0",
"TYPE": "disk"
},
{
"NAME": "sda1",
"MAJ:MIN": "8:1",
"RM": "0",
"SIZE": "1M",
"RO": "0",
"TYPE": "part"
},
{
"NAME": "sda2",
"MAJ:MIN": "8:2",
"RM": "0",
"SIZE": "20G",
"RO": "0",
"TYPE": "part",
"MOUNTPOINT": "/"
},
{
"NAME": "sr0",
"MAJ:MIN": "11:0",
"RM": "1",
"SIZE": "64.8M",
"RO": "0",
"TYPE": "rom"
}
]
```
### mount
```
$ mount | jc --mount -p
[
{
"filesystem": "sysfs",
"mount_point": "/sys",
"type": "sysfs",
"access": [
"rw",
"nosuid",
"nodev",
"noexec",
"relatime"
]
},
{
"filesystem": "proc",
"mount_point": "/proc",
"type": "proc",
"access": [
"rw",
"nosuid",
"nodev",
"noexec",
"relatime"
]
},
{
"filesystem": "udev",
"mount_point": "/dev",
"type": "devtmpfs",
"access": [
"rw",
"nosuid",
"relatime",
"size=977500k",
"nr_inodes=244375",
"mode=755"
]
},
...
]
```
### netstat
```
$ netstat -p | jc --netstat -p
{
"client": {
"tcp": {
"ipv4": [
{
"local_address": "localhost.localdo",
"local_port": "34480",
"foreign_address": "lb-192-30-255-113",
"foreign_port": "https",
"state": "ESTABLISHED",
"pid": 53550,
"program_name": "git-remote-ht",
"receive_q": 0,
"send_q": 0
},
{
"local_address": "localhost.localdo",
"local_port": "34478",
"foreign_address": "lb-192-30-255-113",
"foreign_port": "https",
"state": "ESTABLISHED",
"pid": 53550,
"program_name": "git-remote-ht",
"receive_q": 0,
"send_q": 0
}
]
}
[
{
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "localhost.localdo",
"local_port": "34480",
"foreign_address": "lb-192-30-255-113",
"foreign_port": "https",
"state": "ESTABLISHED",
"pid": 53550,
"program_name": "git-remote-ht",
"receive_q": 0,
"send_q": 0
},
{
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "localhost.localdo",
"local_port": "34478",
"foreign_address": "lb-192-30-255-113",
"foreign_port": "https",
"state": "ESTABLISHED",
"pid": 53550,
"program_name": "git-remote-ht",
"receive_q": 0,
"send_q": 0
}
}
$ netstat -lp | jc --netstat -p
{
"server": {
"tcp": {
"ipv4": [
{
"local_address": "localhost",
"local_port": "smtp",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"state": "LISTEN",
"pid": 1594,
"program_name": "master",
"receive_q": 0,
"send_q": 0
},
{
"local_address": "0.0.0.0",
"local_port": "ssh",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"state": "LISTEN",
"pid": 21918,
"program_name": "sshd",
"receive_q": 0,
"send_q": 0
}
],
"ipv6": [
{
"local_address": "localhost",
"local_port": "smtp",
"foreign_address": "[::]",
"foreign_port": "*",
"state": "LISTEN",
"pid": 1594,
"program_name": "master",
"receive_q": 0,
"send_q": 0
},
{
"local_address": "[::]",
"local_port": "ssh",
"foreign_address": "[::]",
"foreign_port": "*",
"state": "LISTEN",
"pid": 21918,
"program_name": "sshd",
"receive_q": 0,
"send_q": 0
}
]
},
"udp": {
"ipv4": [
{
"local_address": "0.0.0.0",
"local_port": "bootpc",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"pid": 13903,
"program_name": "dhclient",
"receive_q": 0,
"send_q": 0
},
{
"local_address": "localhost",
"local_port": "323",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"pid": 30926,
"program_name": "chronyd",
"receive_q": 0,
"send_q": 0
}
],
"ipv6": [
{
"local_address": "localhost",
"local_port": "323",
"foreign_address": "[::]",
"foreign_port": "*",
"pid": 30926,
"program_name": "chronyd",
"receive_q": 0,
"send_q": 0
}
]
}
]
```
```
$ netstat -lpn | jc --netstat -p
[
{
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "127.0.0.1",
"local_port": "42351",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"state": "LISTEN",
"pid": 1112,
"program_name": "containerd",
"receive_q": 0,
"send_q": 0
},
{
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "127.0.0.53",
"local_port": "53",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"state": "LISTEN",
"pid": 885,
"program_name": "systemd-resolve",
"receive_q": 0,
"send_q": 0
},
{
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "0.0.0.0",
"local_port": "22",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"state": "LISTEN",
"pid": 1127,
"program_name": "sshd",
"receive_q": 0,
"send_q": 0
},
{
"transport_protocol": "tcp",
"network_protocol": "ipv6",
"local_address": "::",
"local_port": "22",
"foreign_address": "::",
"foreign_port": "*",
"state": "LISTEN",
"pid": 1127,
"program_name": "sshd",
"receive_q": 0,
"send_q": 0
},
{
"transport_protocol": "udp",
"network_protocol": "ipv4",
"local_address": "127.0.0.53",
"local_port": "53",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"pid": 885,
"program_name": "systemd-resolve",
"receive_q": 0,
"send_q": 0
},
{
"transport_protocol": "udp",
"network_protocol": "ipv4",
"local_address": "192.168.71.131",
"local_port": "68",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"pid": 867,
"program_name": "systemd-network",
"receive_q": 0,
"send_q": 0
}
]
```
### ps
```
$ ps -ef | jc --ps -p
[
{
"UID": "root",
"PID": "1",
"PPID": "0",
"C": "0",
"STIME": "13:58",
"TTY": "?",
"TIME": "00:00:05",
"CMD": "/lib/systemd/systemd --system --deserialize 35"
},
{
"UID": "root",
"PID": "2",
"PPID": "0",
"C": "0",
"STIME": "13:58",
"TTY": "?",
"TIME": "00:00:00",
"CMD": "[kthreadd]"
},
{
"UID": "root",
"PID": "4",
"PPID": "2",
"C": "0",
"STIME": "13:58",
"TTY": "?",
"TIME": "00:00:00",
"CMD": "[kworker/0:0H]"
},
{
"UID": "root",
"PID": "6",
"PPID": "2",
"C": "0",
"STIME": "13:58",
"TTY": "?",
"TIME": "00:00:00",
"CMD": "[mm_percpu_wq]"
},
...
]
```
### route
```
$ route -n | jc --route -p
[
{
"Destination": "0.0.0.0",
"Gateway": "192.168.71.2",
"Genmask": "0.0.0.0",
"Flags": "UG",
"Metric": "100",
"Ref": "0",
"Use": "0",
"Iface": "ens33"
},
{
"Destination": "172.17.0.0",
"Gateway": "0.0.0.0",
"Genmask": "255.255.0.0",
"Flags": "U",
"Metric": "0",
"Ref": "0",
"Use": "0",
"Iface": "docker0"
},
{
"Destination": "192.168.71.0",
"Gateway": "0.0.0.0",
"Genmask": "255.255.255.0",
"Flags": "U",
"Metric": "0",
"Ref": "0",
"Use": "0",
"Iface": "ens33"
},
{
"Destination": "192.168.71.2",
"Gateway": "0.0.0.0",
"Genmask": "255.255.255.255",
"Flags": "UH",
"Metric": "100",
"Ref": "0",
"Use": "0",
"Iface": "ens33"
}
]
```
### uname -a
```
$ uname -a | jc --uname -p
{
"kernel_name": "Linux",
"node_name": "user-ubuntu",
"kernel_release": "4.15.0-65-generic",
"operating_system": "GNU/Linux",
"hardware_platform": "x86_64",
"processor": "x86_64",
"machine": "x86_64",
"kernel_version": "#74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019"
}
```
## Contributions
Feel free to add/improve code or parsers!
## Acknowledgments
- `ifconfig-parser` module from https://github.com/KnightWhoSayNi/ifconfig-parser
- Parsing code from Conor Heine at https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501

24
changelog.txt Normal file
View File

@@ -0,0 +1,24 @@
jc changelog
20191022 v0.8.1
- Add env parser
- Add df parser
- Add free parser
- Add lsblk parser
- Add mount parser
- Add uname parser
- Add iptables parser
20191021 v0.6.4
- Flatten netstat parser output
- Clean up argument parsing
- Fix command help
20191018 v0.5.5
- Fix netstat -p parsing for Ubuntu
- Add ps parser
- Add route parser
- ls parser fixes
20191017 v0.2.0
- ifconfig, ls, and netstat support

View File

@@ -1,61 +1,71 @@
"""JC - JSON CLI output utility
v0.1
* kellyjonbrazil@gmail.com
This module serializes standard unix command line output to structured JSON
output.
Example:
CLI Example:
$ ls -al | jc | jq .
$ ls -l /bin | jc --ls -p
[
{
"filename": ".",
"suffix": Null,
"bytes": 224,
"date_updated": "Oct 1 12:09",
"owner_user": "joeuser",
"owner_group": "staff",
"flags": "drwxr-xr-x+",
"link_to": Null,
"links": 47
},
{
"filename": "..",
"suffix": Null,
"bytes": 224,
"date_updated": "Oct 1 12:09",
"owner_user": "admin",
"owner_group": "root",
"flags": "drwxr-xr-x",
"link_to": Null,
"links": 7
},
{
"filename": "testfile.txt",
"suffix": "txt",
"bytes": 14686,
"date_updated": "Oct 1 12:09",
"owner_user": "joeuser",
"owner_group": "staff",
"flags": "-rwxr-xr-x@",
"link_to": Null,
"links": 1
},
{
"filename": "ncat",
"suffix": Null,
"bytes": 14686,
"date_updated": "Oct 1 12:09",
"owner_user": "joeuser",
"owner_group": "staff",
"flags": "lrwxr-xr-x",
"link_to": "../Cellar/nmap/7.70/bin/ncat",
"links": 1
}
{
"filename": "bash",
"flags": "-r-xr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 618416,
"date": "May 3 22:26"
},
{
"filename": "cat",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 23648,
"date": "May 3 22:26"
},
{
"filename": "chmod",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "wheel",
"bytes": 30016,
"date": "May 3 22:26"
},
...
]
Module Example:
>>> import jc.parsers.ls
>>>
>>> data='''-rwxr-xr-x 1 root wheel 23648 May 3 22:26 cat
... -rwxr-xr-x 1 root wheel 30016 May 3 22:26 chmod
... -rwxr-xr-x 1 root wheel 29024 May 3 22:26 cp
... -rwxr-xr-x 1 root wheel 375824 May 3 22:26 csh
... -rwxr-xr-x 1 root wheel 28608 May 3 22:26 date
... -rwxr-xr-x 1 root wheel 32000 May 3 22:26 dd
... -rwxr-xr-x 1 root wheel 23392 May 3 22:26 df
... -rwxr-xr-x 1 root wheel 18128 May 3 22:26 echo'''
>>>
>>> jc.parsers.ls.parse(data)
[{'filename': 'cat', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'bytes': 23648, 'date': 'May 3 22:26'}, {'filename': 'chmod', 'flags': '-rwxr-xr-x', 'links': 1,
'owner': 'root', 'group': 'wheel', 'bytes': 30016, 'date': 'May 3 22:26'}, {'filename': 'cp',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 29024,
'date': 'May 3 22:26'}, {'filename': 'csh', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'bytes': 375824, 'date': 'May 3 22:26'}, {'filename': 'date',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 28608,
'date': 'May 3 22:26'}, {'filename': 'dd', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'bytes': 32000, 'date': 'May 3 22:26'}, {'filename': 'df', 'flags': '-rwxr-xr-x',
'links': 1, 'owner': 'root', 'group': 'wheel', 'bytes': 23392, 'date': 'May 3 22:26'},
{'filename': 'echo', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'bytes': 18128, 'date': 'May 3 22:26'}]
"""
name = 'jc'

View File

@@ -6,38 +6,91 @@ Main input module
import sys
import json
import jc.parsers.df
import jc.parsers.env
import jc.parsers.free
import jc.parsers.ifconfig
import jc.parsers.iptables
import jc.parsers.ls
import jc.parsers.lsblk
import jc.parsers.mount
import jc.parsers.netstat
import jc.parsers.ps
import jc.parsers.route
import jc.parsers.uname
def main():
pretty = False
data = sys.stdin.read()
pretty = False
if len(sys.argv) < 2:
print(f'\nError: jc\n Must specify parser. (e.g. --ls, --netstat, --ifconfig, etc.)')
print(' Use -p to pretty print')
print(f'\nExample: ls -al | jc --ls -p\n')
exit()
if '-p' in sys.argv:
pretty = True
arg = sys.argv[1]
if '--df' in sys.argv:
result = jc.parsers.df.parse(data)
if len(sys.argv) > 2:
if sys.argv[2] == '-p':
pretty = True
elif '--env' in sys.argv:
result = jc.parsers.env.parse(data)
if arg == '--ifconfig':
elif '--free' in sys.argv:
result = jc.parsers.free.parse(data)
elif '--ifconfig' in sys.argv:
result = jc.parsers.ifconfig.parse(data)
elif arg == '--ls':
elif '--iptables' in sys.argv:
result = jc.parsers.iptables.parse(data)
elif '--ls' in sys.argv:
result = jc.parsers.ls.parse(data)
elif arg == '--netstat':
elif '--lsblk' in sys.argv:
result = jc.parsers.lsblk.parse(data)
elif '--mount' in sys.argv:
result = jc.parsers.mount.parse(data)
elif '--netstat' in sys.argv:
result = jc.parsers.netstat.parse(data)
elif '--ps' in sys.argv:
result = jc.parsers.ps.parse(data)
elif '--route' in sys.argv:
result = jc.parsers.route.parse(data)
elif '--uname' in sys.argv:
result = jc.parsers.uname.parse(data)
else:
print('jc: missing arguments\n', file=sys.stderr)
print('Usage: jc [parser] [options]\n', file=sys.stderr)
print('Parsers:', file=sys.stderr)
print(' --df df parser', file=sys.stderr)
print(' --env env parser', file=sys.stderr)
print(' --free free parser', file=sys.stderr)
print(' --ifconfig iconfig parser', file=sys.stderr)
print(' --iptables iptables parser', file=sys.stderr)
print(' --ls ls parser', file=sys.stderr)
print(' --lsblk lsblk parser', file=sys.stderr)
print(' --mount mount parser', file=sys.stderr)
print(' --netstat netstat parser', file=sys.stderr)
print(' --ps ps parser', file=sys.stderr)
print(' --route route parser', file=sys.stderr)
print(' --uname uname parser\n', file=sys.stderr)
print('Options:', file=sys.stderr)
print(' -p pretty print output\n', file=sys.stderr)
print('Example:', file=sys.stderr)
print(' ls -al | jc --ls -p\n', file=sys.stderr)
exit()
# output resulting dictionary as json
if pretty:
print(json.dumps(result, indent=2))
else:
print(json.dumps(result))
if __name__ == '__main__':
main()

55
jc/parsers/df.py Normal file
View File

@@ -0,0 +1,55 @@
"""jc - JSON CLI output utility df Parser
Usage:
specify --df as the first argument if the piped input is coming from df
Example:
$ df | jc --df -p
[
{
"Filesystem": "udev",
"1K-blocks": "977500",
"Used": "0",
"Available": "977500",
"Use%": "0%",
"Mounted": "/dev"
},
{
"Filesystem": "tmpfs",
"1K-blocks": "201732",
"Used": "1180",
"Available": "200552",
"Use%": "1%",
"Mounted": "/run"
},
{
"Filesystem": "/dev/sda2",
"1K-blocks": "20508240",
"Used": "5747284",
"Available": "13696152",
"Use%": "30%",
"Mounted": "/"
},
{
"Filesystem": "tmpfs",
"1K-blocks": "1008648",
"Used": "0",
"Available": "1008648",
"Use%": "0%",
"Mounted": "/dev/shm"
},
...
]
"""
def parse(data):
# code adapted from Conor Heine at:
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
return [dict(zip(headers, r)) for r in raw_data]

53
jc/parsers/env.py Normal file
View File

@@ -0,0 +1,53 @@
"""jc - JSON CLI output utility env Parser
Usage:
specify --env as the first argument if the piped input is coming from env
Example:
$ env | jc --env -p
[
{
"TERM": "xterm-256color"
},
{
"SHELL": "/bin/bash"
},
{
"USER": "root"
},
{
"PATH": "/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin"
},
{
"PWD": "/bin"
},
{
"LANG": "en_US.UTF-8"
},
{
"HOME": "/root"
},
{
"_": "/usr/bin/env"
}
]
"""
def parse(data):
output = []
linedata = data.splitlines()
# Clear any blank lines
cleandata = list(filter(None, linedata))
if cleandata:
for entry in cleandata:
output_line = {}
parsed_line = entry.split('=', maxsplit=1)
output_line[parsed_line[0]] = parsed_line[1]
output.append(output_line)
return output

45
jc/parsers/free.py Normal file
View File

@@ -0,0 +1,45 @@
"""jc - JSON CLI output utility free Parser
Usage:
specify --free as the first argument if the piped input is coming from free
Example:
$ free | jc --free -p
[
{
"type": "Mem",
"total": "2017300",
"used": "213104",
"free": "1148452",
"shared": "1176",
"buff/cache": "655744",
"available": "1622204"
},
{
"type": "Swap",
"total": "2097148",
"used": "0",
"free": "2097148"
}
]
"""
def parse(data):
# code adapted from Conor Heine at:
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h]
headers.insert(0, "type")
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
output = [dict(zip(headers, r)) for r in raw_data]
for entry in output:
entry['type'] = entry['type'].rstrip(':')
return output

View File

@@ -13,9 +13,10 @@ $ ifconfig | jc --ifconfig -p
from collections import namedtuple
from ifconfigparser import IfconfigParser
def parse(data):
output = []
parsed = IfconfigParser(console_output=data)
interfaces = parsed.get_interfaces()
@@ -24,7 +25,5 @@ def parse(data):
d = interfaces[iface]._asdict()
dct = dict(d)
output.append(dct)
return output

366
jc/parsers/iptables.py Normal file
View File

@@ -0,0 +1,366 @@
"""jc - JSON CLI output utility ipables Parser
Usage:
Specify --iptables as the first argument if the piped input is coming from iptables
Supports -vLn for all tables
Examples:
$ sudo iptables -L -t nat | jc --iptables -p
[
{
"chain": "PREROUTING",
"rules": [
{
"target": "PREROUTING_direct",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "PREROUTING_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "PREROUTING_ZONES",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "DOCKER",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere",
"options": "ADDRTYPE match dst-type LOCAL"
}
]
},
{
"chain": "INPUT",
"rules": []
},
{
"chain": "OUTPUT",
"rules": [
{
"target": "OUTPUT_direct",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "anywhere"
},
{
"target": "DOCKER",
"prot": "all",
"opt": "--",
"source": "anywhere",
"destination": "!loopback/8",
"options": "ADDRTYPE match dst-type LOCAL"
}
]
},
...
]
$ sudo iptables -vnL -t filter | jc --iptables -p
[
{
"chain": "INPUT",
"rules": [
{
"pkts": "1571",
"bytes": "3394K",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "lo",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_direct",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "711",
"bytes": "60126",
"target": "INPUT_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DROP",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate INVALID"
},
{
"pkts": "710",
"bytes": "60078",
"target": "REJECT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "reject-with icmp-host-prohibited"
}
]
},
{
"chain": "FORWARD",
"rules": [
{
"pkts": "0",
"bytes": "0",
"target": "DOCKER-ISOLATION",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DOCKER",
"prot": "all",
"opt": "--",
"in": "*",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "docker0",
"out": "!docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "docker0",
"out": "docker0",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate RELATED,ESTABLISHED"
},
{
"pkts": "0",
"bytes": "0",
"target": "ACCEPT",
"prot": "all",
"opt": "--",
"in": "lo",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_direct",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_IN_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_IN_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_OUT_ZONES_SOURCE",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "FORWARD_OUT_ZONES",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0"
},
{
"pkts": "0",
"bytes": "0",
"target": "DROP",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "ctstate INVALID"
},
{
"pkts": "0",
"bytes": "0",
"target": "REJECT",
"prot": "all",
"opt": "--",
"in": "*",
"out": "*",
"source": "0.0.0.0/0",
"destination": "0.0.0.0/0",
"options": "reject-with icmp-host-prohibited"
}
]
},
...
]
"""
class state():
output = []
chain = {}
headers = []
def parse(data):
cleandata = data.splitlines()
for line in cleandata:
if line.find('Chain') == 0:
state.output.append(state.chain)
state.chain = {}
state.headers = []
parsed_line = line.split()
state.chain['chain'] = parsed_line[1]
state.chain['rules'] = []
continue
if line.find('target') == 0 or line.find('pkts') == 1:
state.headers = []
state.headers = [h for h in ' '.join(line.strip().split()).split() if h]
state.headers.append("options")
continue
else:
rule = line.split(maxsplit=len(state.headers) - 1)
temp_rule = dict(zip(state.headers, rule))
if temp_rule:
state.chain['rules'].append(temp_rule)
state.output = list(filter(None, state.output))
return state.output

View File

@@ -72,7 +72,7 @@ $ ls -al /usr/bin | jc --ls -p
...
]
$ $ ls -l /usr/bin | jc --ls | jq .[] | jq 'select(.bytes > 50000000)'
$ $ ls -l /usr/bin | jc --ls | jq '.[] | 'select(.bytes > 50000000)'
{
"filename": "emacs",
"flags": "-r-xr-xr-x",
@@ -85,46 +85,48 @@ $ $ ls -l /usr/bin | jc --ls | jq .[] | jq 'select(.bytes > 50000000)'
"""
import re
def parse(data):
output = []
cleandata = data.splitlines()
linedata = data.splitlines()
# Delete first line if it starts with 'total'
if cleandata[0].find('total') == 0:
cleandata.pop(0)
if linedata:
if linedata[0].find('total') == 0:
linedata.pop(0)
# Delete last line if it is blank
if cleandata[-1] == '':
cleandata.pop(-1)
# Clear any blank lines
cleandata = list(filter(None, linedata))
# Check if -l was used to parse extra data
if re.match('^[-dclpsbDCMnP?]([-r][-w][-xsS]){2}([-r][-w][-xtT])[+]?', cleandata[0]):
for entry in cleandata:
output_line = {}
if cleandata:
# Check if -l was used to parse extra data
if re.match('^[-dclpsbDCMnP?]([-r][-w][-xsS]){2}([-r][-w][-xtT])[+]?', cleandata[0]):
for entry in cleandata:
output_line = {}
parsed_line = entry.split()
parsed_line = entry.split(maxsplit=8)
# split filenames and links
filename_field = ' '.join(parsed_line[8:]).split(' -> ')
# split filenames and links
filename_field = parsed_line[8].split(' -> ')
# create list of dictionaries
output_line['filename'] = filename_field[0]
# create list of dictionaries
output_line['filename'] = filename_field[0]
if len(filename_field) > 1:
output_line['link_to'] = filename_field[1]
if len(filename_field) > 1:
output_line['link_to'] = filename_field[1]
output_line['flags'] = parsed_line[0]
output_line['links'] = int(parsed_line[1])
output_line['owner'] = parsed_line[2]
output_line['group'] = parsed_line[3]
output_line['bytes'] = int(parsed_line[4])
output_line['date'] = ' '.join(parsed_line[5:8])
output.append(output_line)
else:
for entry in cleandata:
output_line = {}
output_line['filename'] = entry
output.append(output_line)
output_line['flags'] = parsed_line[0]
output_line['links'] = int(parsed_line[1])
output_line['owner'] = parsed_line[2]
output_line['group'] = parsed_line[3]
output_line['bytes'] = int(parsed_line[4])
output_line['date'] = ' '.join(parsed_line[5:8])
output.append(output_line)
else:
for entry in cleandata:
output_line = {}
output_line['filename'] = entry
output.append(output_line)
return output

70
jc/parsers/lsblk.py Normal file
View File

@@ -0,0 +1,70 @@
"""jc - JSON CLI output utility lsblk Parser
Usage:
specify --lsblk as the first argument if the piped input is coming from lsblk
Example:
$ lsblk | jc --lsblk -p
[
{
"NAME": "loop0",
"MAJ:MIN": "7:0",
"RM": "0",
"SIZE": "54.5M",
"RO": "1",
"TYPE": "loop",
"MOUNTPOINT": "/snap/core18/1223"
},
{
"NAME": "sda",
"MAJ:MIN": "8:0",
"RM": "0",
"SIZE": "20G",
"RO": "0",
"TYPE": "disk"
},
{
"NAME": "sda1",
"MAJ:MIN": "8:1",
"RM": "0",
"SIZE": "1M",
"RO": "0",
"TYPE": "part"
},
{
"NAME": "sda2",
"MAJ:MIN": "8:2",
"RM": "0",
"SIZE": "20G",
"RO": "0",
"TYPE": "part",
"MOUNTPOINT": "/"
},
{
"NAME": "sr0",
"MAJ:MIN": "11:0",
"RM": "1",
"SIZE": "64.8M",
"RO": "0",
"TYPE": "rom"
}
]
"""
def parse(data):
# code adapted from Conor Heine at:
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
output = [dict(zip(headers, r)) for r in raw_data]
for entry in output:
entry['NAME'] = entry['NAME'].encode('ascii', errors='ignore').decode()
return output

75
jc/parsers/mount.py Normal file
View File

@@ -0,0 +1,75 @@
"""jc - JSON CLI output utility mount Parser
Usage:
specify --mount as the first argument if the piped input is coming from mount
Example:
$ mount | jc --mount -p
[
{
"filesystem": "sysfs",
"mount_point": "/sys",
"type": "sysfs",
"access": [
"rw",
"nosuid",
"nodev",
"noexec",
"relatime"
]
},
{
"filesystem": "proc",
"mount_point": "/proc",
"type": "proc",
"access": [
"rw",
"nosuid",
"nodev",
"noexec",
"relatime"
]
},
{
"filesystem": "udev",
"mount_point": "/dev",
"type": "devtmpfs",
"access": [
"rw",
"nosuid",
"relatime",
"size=977500k",
"nr_inodes=244375",
"mode=755"
]
},
...
]
"""
def parse(data):
output = []
linedata = data.splitlines()
# Clear any blank lines
cleandata = list(filter(None, linedata))
if cleandata:
for entry in cleandata:
output_line = {}
parsed_line = entry.split()
output_line['filesystem'] = parsed_line[0]
output_line['mount_point'] = parsed_line[2]
output_line['type'] = parsed_line[4]
access = parsed_line[5].lstrip('(').rstrip(')').split(',')
output_line['access'] = access
output.append(output_line)
return output

View File

@@ -11,152 +11,145 @@ Limitations:
Examples:
$ netstat -p | jc --netstat -p
{
"client": {
"tcp": {
"ipv4": [
{
"local_address": "localhost.localdo",
"local_port": "34480",
"foreign_address": "lb-192-30-255-113",
"foreign_port": "https",
"state": "ESTABLISHED",
"pid": 53550,
"program_name": "git-remote-ht",
"receive_q": 0,
"send_q": 0
},
{
"local_address": "localhost.localdo",
"local_port": "34478",
"foreign_address": "lb-192-30-255-113",
"foreign_port": "https",
"state": "ESTABLISHED",
"pid": 53550,
"program_name": "git-remote-ht",
"receive_q": 0,
"send_q": 0
}
]
}
[
{
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "localhost.localdo",
"local_port": "34480",
"foreign_address": "lb-192-30-255-113",
"foreign_port": "https",
"state": "ESTABLISHED",
"pid": 53550,
"program_name": "git-remote-ht",
"receive_q": 0,
"send_q": 0
},
{
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "localhost.localdo",
"local_port": "34478",
"foreign_address": "lb-192-30-255-113",
"foreign_port": "https",
"state": "ESTABLISHED",
"pid": 53550,
"program_name": "git-remote-ht",
"receive_q": 0,
"send_q": 0
}
}
]
$ netstat -lp | jc --netstat -p
{
"server": {
"tcp": {
"ipv4": [
{
"local_address": "localhost",
"local_port": "smtp",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"state": "LISTEN",
"pid": 1594,
"program_name": "master",
"receive_q": 0,
"send_q": 0
},
{
"local_address": "0.0.0.0",
"local_port": "ssh",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"state": "LISTEN",
"pid": 21918,
"program_name": "sshd",
"receive_q": 0,
"send_q": 0
}
],
"ipv6": [
{
"local_address": "localhost",
"local_port": "smtp",
"foreign_address": "[::]",
"foreign_port": "*",
"state": "LISTEN",
"pid": 1594,
"program_name": "master",
"receive_q": 0,
"send_q": 0
},
{
"local_address": "[::]",
"local_port": "ssh",
"foreign_address": "[::]",
"foreign_port": "*",
"state": "LISTEN",
"pid": 21918,
"program_name": "sshd",
"receive_q": 0,
"send_q": 0
}
]
},
"udp": {
"ipv4": [
{
"local_address": "0.0.0.0",
"local_port": "bootpc",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"pid": 13903,
"program_name": "dhclient",
"receive_q": 0,
"send_q": 0
},
{
"local_address": "localhost",
"local_port": "323",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"pid": 30926,
"program_name": "chronyd",
"receive_q": 0,
"send_q": 0
}
],
"ipv6": [
{
"local_address": "localhost",
"local_port": "323",
"foreign_address": "[::]",
"foreign_port": "*",
"pid": 30926,
"program_name": "chronyd",
"receive_q": 0,
"send_q": 0
}
]
}
$ netstat -lpn | jc --netstat -p
[
{
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "127.0.0.1",
"local_port": "42351",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"state": "LISTEN",
"pid": 1112,
"program_name": "containerd",
"receive_q": 0,
"send_q": 0
},
{
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "127.0.0.53",
"local_port": "53",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"state": "LISTEN",
"pid": 885,
"program_name": "systemd-resolve",
"receive_q": 0,
"send_q": 0
},
{
"transport_protocol": "tcp",
"network_protocol": "ipv4",
"local_address": "0.0.0.0",
"local_port": "22",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"state": "LISTEN",
"pid": 1127,
"program_name": "sshd",
"receive_q": 0,
"send_q": 0
},
{
"transport_protocol": "tcp",
"network_protocol": "ipv6",
"local_address": "::",
"local_port": "22",
"foreign_address": "::",
"foreign_port": "*",
"state": "LISTEN",
"pid": 1127,
"program_name": "sshd",
"receive_q": 0,
"send_q": 0
},
{
"transport_protocol": "udp",
"network_protocol": "ipv4",
"local_address": "127.0.0.53",
"local_port": "53",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"pid": 885,
"program_name": "systemd-resolve",
"receive_q": 0,
"send_q": 0
},
{
"transport_protocol": "udp",
"network_protocol": "ipv4",
"local_address": "192.168.71.131",
"local_port": "68",
"foreign_address": "0.0.0.0",
"foreign_port": "*",
"pid": 867,
"program_name": "systemd-network",
"receive_q": 0,
"send_q": 0
}
}
]
"""
import string
output = {}
output = []
class state():
section = ''
session = ''
network = ''
client_tcp_ip4 = []
client_tcp_ip6 = []
client_udp_ip4 = []
client_udp_ip6 = []
server_tcp_ip4 = []
server_tcp_ip6 = []
server_udp_ip4 = []
server_udp_ip6 = []
def parse_line(entry):
parsed_line = entry.split()
output_line = {}
if entry.find('tcp') == 0:
output_line['transport_protocol'] = 'tcp'
if entry.find('p6') == 2:
output_line['network_protocol'] = 'ipv6'
else:
output_line['network_protocol'] = 'ipv4'
elif entry.find('udp') == 0:
output_line['transport_protocol'] = 'udp'
if entry.find('p6') == 2:
output_line['network_protocol'] = 'ipv6'
else:
output_line['network_protocol'] = 'ipv4'
else:
return
parsed_line = entry.split()
output_line['local_address'] = parsed_line[3].rsplit(':', 1)[0]
output_line['local_port'] = parsed_line[3].rsplit(':', 1)[-1]
output_line['foreign_address'] = parsed_line[4].rsplit(':', 1)[0]
@@ -164,151 +157,41 @@ def parse_line(entry):
if len(parsed_line) > 5:
if parsed_line[5][0] not in string.digits:
if parsed_line[5][0] not in string.digits and parsed_line[5][0] != '-':
output_line['state'] = parsed_line[5]
if len(parsed_line) > 6:
if len(parsed_line) > 6 and parsed_line[6][0] in string.digits:
output_line['pid'] = int(parsed_line[6].split('/')[0])
output_line['program_name'] = parsed_line[6].split('/')[1]
else:
output_line['pid'] = int(parsed_line[5].split('/')[0])
output_line['program_name'] = parsed_line[5].split('/')[1]
if parsed_line[5][0] in string.digits:
output_line['pid'] = int(parsed_line[5].split('/')[0])
output_line['program_name'] = parsed_line[5].split('/')[1]
output_line['receive_q'] = int(parsed_line[1])
output_line['send_q'] = int(parsed_line[2])
return output_line
def parse(data):
cleandata = data.splitlines()
for line in cleandata:
if line.find('Active Internet connections (w/o servers)') == 0:
state.section = "client"
continue
if line.find('Active Internet connections (only servers)') == 0:
state.section = "server"
continue
if line.find('Proto') == 0:
continue
if line.find('Active UNIX') == 0:
break
if state.section == 'client':
if line.find('tcp') == 0:
state.session = 'tcp'
if line.find('p6') == 2:
state.network = 'ipv6'
else:
state.network = 'ipv4'
elif line.find('udp') == 0:
state.session = 'udp'
if line.find('p6') == 2:
state.network = 'ipv6'
else:
state.network = 'ipv4'
elif state.section == 'server':
if line.find('tcp') == 0:
state.session = 'tcp'
if line.find('p6') == 2:
state.network = 'ipv6'
else:
state.network = 'ipv4'
elif line.find('udp') == 0:
state.session = 'udp'
if line.find('p6') == 2:
state.network = 'ipv6'
else:
state.network = 'ipv4'
if state.section == 'client' and state.session == 'tcp' and state.network == 'ipv4':
state.client_tcp_ip4.append(parse_line(line))
output.append(parse_line(line))
if state.section == 'client' and state.session == 'tcp' and state.network == 'ipv6':
state.client_tcp_ip6.append(parse_line(line))
if state.section == 'client' and state.session == 'udp' and state.network == 'ipv4':
state.client_udp_ip4.append(parse_line(line))
if state.section == 'client' and state.session == 'udp' and state.network == 'ipv6':
state.client_udp_ip6.append(parse_line(line))
if state.section == 'server' and state.session == 'tcp' and state.network == 'ipv4':
state.server_tcp_ip4.append(parse_line(line))
if state.section == 'server' and state.session == 'tcp' and state.network == 'ipv6':
state.server_tcp_ip6.append(parse_line(line))
if state.section == 'server' and state.session == 'udp' and state.network == 'ipv4':
state.server_udp_ip4.append(parse_line(line))
if state.section == 'server' and state.session == 'udp' and state.network == 'ipv6':
state.server_udp_ip6.append(parse_line(line))
state.session = ''
state.network = ''
# build dictionary
if state.client_tcp_ip4:
if 'client' not in output:
output['client'] = {}
if 'tcp' not in output['client']:
output['client']['tcp'] = {}
output['client']['tcp']['ipv4'] = state.client_tcp_ip4
if state.client_tcp_ip6:
if 'client' not in output:
output['client'] = {}
if 'tcp' not in output['client']:
output['client']['tcp'] = {}
output['client']['tcp']['ipv6'] = state.client_tcp_ip6
if state.client_udp_ip4:
if 'client' not in output:
output['client'] = {}
if 'udp' not in output['client']:
output['client']['udp'] = {}
output['client']['udp']['ipv4'] = state.client_udp_ip4
if state.client_udp_ip6:
if 'client' not in output:
output['client'] = {}
if 'udp' not in output['client']:
output['client']['udp'] = {}
output['client']['udp']['ipv6'] = state.client_udp_ip6
if state.server_tcp_ip4:
if 'server' not in output:
output['server'] = {}
if 'tcp' not in output['server']:
output['server']['tcp'] = {}
output['server']['tcp']['ipv4'] = state.server_tcp_ip4
if state.server_tcp_ip6:
if 'server' not in output:
output['server'] = {}
if 'tcp' not in output['server']:
output['server']['tcp'] = {}
output['server']['tcp']['ipv6'] = state.server_tcp_ip6
if state.server_udp_ip4:
if 'server' not in output:
output['server'] = {}
if 'udp' not in output['server']:
output['server']['udp'] = {}
output['server']['udp']['ipv4'] = state.server_udp_ip4
if state.server_udp_ip6:
if 'server' not in output:
output['server'] = {}
if 'udp' not in output['server']:
output['server']['udp'] = {}
output['server']['udp']['ipv6'] = state.server_udp_ip6
return output
clean_output = list(filter(None, output))
return clean_output

67
jc/parsers/ps.py Normal file
View File

@@ -0,0 +1,67 @@
"""jc - JSON CLI output utility ps Parser
Usage:
specify --ps as the first argument if the piped input is coming from ps
ps options supported:
- ef
- axu
Example:
$ ps -ef | jc --ps -p
[
{
"UID": "root",
"PID": "1",
"PPID": "0",
"C": "0",
"STIME": "13:58",
"TTY": "?",
"TIME": "00:00:05",
"CMD": "/lib/systemd/systemd --system --deserialize 35"
},
{
"UID": "root",
"PID": "2",
"PPID": "0",
"C": "0",
"STIME": "13:58",
"TTY": "?",
"TIME": "00:00:00",
"CMD": "[kthreadd]"
},
{
"UID": "root",
"PID": "4",
"PPID": "2",
"C": "0",
"STIME": "13:58",
"TTY": "?",
"TIME": "00:00:00",
"CMD": "[kworker/0:0H]"
},
{
"UID": "root",
"PID": "6",
"PPID": "2",
"C": "0",
"STIME": "13:58",
"TTY": "?",
"TIME": "00:00:00",
"CMD": "[mm_percpu_wq]"
},
...
]
"""
def parse(data):
# code adapted from Conor Heine at:
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines()
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
return [dict(zip(headers, r)) for r in raw_data]

63
jc/parsers/route.py Normal file
View File

@@ -0,0 +1,63 @@
"""jc - JSON CLI output utility route Parser
Usage:
specify --route as the first argument if the piped input is coming from route
Example:
$ route -n | jc --route -p
[
{
"Destination": "0.0.0.0",
"Gateway": "192.168.71.2",
"Genmask": "0.0.0.0",
"Flags": "UG",
"Metric": "100",
"Ref": "0",
"Use": "0",
"Iface": "ens33"
},
{
"Destination": "172.17.0.0",
"Gateway": "0.0.0.0",
"Genmask": "255.255.0.0",
"Flags": "U",
"Metric": "0",
"Ref": "0",
"Use": "0",
"Iface": "docker0"
},
{
"Destination": "192.168.71.0",
"Gateway": "0.0.0.0",
"Genmask": "255.255.255.0",
"Flags": "U",
"Metric": "0",
"Ref": "0",
"Use": "0",
"Iface": "ens33"
},
{
"Destination": "192.168.71.2",
"Gateway": "0.0.0.0",
"Genmask": "255.255.255.255",
"Flags": "UH",
"Metric": "100",
"Ref": "0",
"Use": "0",
"Iface": "ens33"
}
]
"""
def parse(data):
# code adapted from Conor Heine at:
# https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501
cleandata = data.splitlines()[1:]
headers = [h for h in ' '.join(cleandata[0].strip().split()).split() if h]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), cleandata[1:])
return [dict(zip(headers, r)) for r in raw_data]

42
jc/parsers/uname.py Normal file
View File

@@ -0,0 +1,42 @@
"""jc - JSON CLI output utility uname Parser
Usage:
specify --uname as the first argument if the piped input is coming from uname
Limitations:
must use 'uname -a'
Example:
$ uname -a | jc --uname -p
{
"kernel_name": "Linux",
"node_name": "user-ubuntu",
"kernel_release": "4.15.0-65-generic",
"operating_system": "GNU/Linux",
"hardware_platform": "x86_64",
"processor": "x86_64",
"machine": "x86_64",
"kernel_version": "#74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019"
}
"""
def parse(data):
output = {}
parsed_line = data.split(maxsplit=3)
output['kernel_name'] = parsed_line.pop(0)
output['node_name'] = parsed_line.pop(0)
output['kernel_release'] = parsed_line.pop(0)
parsed_line = parsed_line[-1].rsplit(maxsplit=4)
output['operating_system'] = parsed_line.pop(-1)
output['hardware_platform'] = parsed_line.pop(-1)
output['processor'] = parsed_line.pop(-1)
output['machine'] = parsed_line.pop(-1)
output['kernel_version'] = parsed_line.pop(0)
return output

View File

@@ -5,7 +5,7 @@ with open('README.md', 'r') as f:
setuptools.setup(
name='jc',
version='0.2.0',
version='0.8.1',
author='Kelly Brazil',
author_email='kellyjonbrazil@gmail.com',
description='This tool serializes the output of popular command line tools to structured JSON output.',