python/jc
1
0
Fork 0
mirror of https://github.com/kellyjonbrazil/jc.git synced 2025-07-13 01:20:24 +02:00
Code Issues Releases Activity

Merge pull request #287 from kellyjonbrazil/procfile

Browse Source 
Procfile
This commit is contained in:
Kelly Brazil
2022-09-15 16:15:30 +00:00
committed by GitHub
parent d3c1a73ced c348fa89a9
commit fd28fea3b6
45 changed files with 5342 additions and 90 deletions
Download Patch File Download Diff File Expand all files Collapse all files

47
README.md
View File

@ -70,7 +70,7 @@ values are converted, and, in some cases, additional semantic context fields are
added.
To access the raw, pre-processed JSON, use the `-r` cli option or the `raw=True`
function parameter in `parse()`.
function parameter in `parse()` when using `jc` as a python library.
Schemas for each parser can be found at the documentation link beside each
[**Parser**](#parsers) below.
@ -133,14 +133,18 @@ on Github.
`jc` accepts piped input from `STDIN` and outputs a JSON representation of the
previous command's output to `STDOUT`.
```bash
COMMAND | jc PARSER [OPTIONS]
COMMAND | jc [OPTIONS] PARSER
cat FILE | jc [OPTIONS] PARSER
echo STRING | jc [OPTIONS] PARSER
```
Alternatively, the "magic" syntax can be used by prepending `jc` to the command
to be converted. Options can be passed to `jc` immediately before the command is
given. (Note: command aliases and shell builtins are not supported)
to be converted or in front of the absolute path for Proc files. Options can be
passed to `jc` immediately before the command or Proc file path is given.
(Note: command aliases and shell builtins are not supported)
```bash
jc [OPTIONS] COMMAND
jc [OPTIONS] /proc/<path-to-procfile>
```
The JSON output can be compact (default) or pretty formatted with the `-p`
@ -225,6 +229,7 @@ option.
| ` --pip-show` | `pip show` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_show) |
| ` --plist` | PLIST file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/plist) |
| ` --postconf` | `postconf -M` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/postconf) |
| ` --proc` | `/proc/` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/proc) |
| ` --ps` | `ps` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ps) |
| ` --route` | `route` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/route) |
| ` --rpm-qi` | `rpm -qi` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/rpm_qi) |
@ -278,7 +283,7 @@ option.
| `-a` | `--about` | About `jc`. Prints information about `jc` and the parsers (in JSON or YAML, of course!) |
| `-C` | `--force-color` | Force color output even when using pipes (overrides `-m` and the `NO_COLOR` env variable) |
| `-d` | `--debug` | Debug mode. Prints trace messages if parsing issues are encountered (use`-dd` for verbose debugging) |
| `-h` | `--help` | Help. Use `jc -h --parser_name` for parser documentation |
| `-h` | `--help` | Help. Use `jc -h --parser_name` for parser documentation. Use twice to show hidden parsers (e.g. `-hh`) |
| `-m` | `--monochrome` | Monochrome output |
| `-M` | `--meta-out` | Add metadata to output including timestamp, parser name, magic command, magic command exit code, etc. | |
| `-p` | `--pretty` | Pretty format the JSON output |
@ -537,12 +542,12 @@ that case you can suppress the warning message with the `-q` cli option or the
macOS:
```bash
cat lsof.out | jc --lsof -q
cat lsof.out | jc -q --lsof
```
or Windows:
```bash
type lsof.out | jc --lsof -q
type lsof.out | jc -q --lsof
```
Tested on:
@ -586,7 +591,7 @@ documentation.
### arp
```bash
arp | jc --arp -p # or: jc -p arp
arp | jc -p --arp # or: jc -p arp
```
```json
[
@ -625,7 +630,7 @@ cat homes.csv
...
```
```bash
cat homes.csv | jc --csv -p
cat homes.csv | jc -p --csv
```
```json
[
@ -666,7 +671,7 @@ cat homes.csv | jc --csv -p
```
### /etc/hosts file
```bash
cat /etc/hosts | jc --hosts -p
cat /etc/hosts | jc -p --hosts
```
```json
[
@ -693,7 +698,7 @@ cat /etc/hosts | jc --hosts -p
```
### ifconfig
```bash
ifconfig | jc --ifconfig -p # or: jc -p ifconfig
ifconfig | jc -p --ifconfig # or: jc -p ifconfig
```
```json
[
@ -751,7 +756,7 @@ Port = 50022
ForwardX11 = no
```
```bash
cat example.ini | jc --ini -p
cat example.ini | jc -p --ini
```
```json
{
@ -773,7 +778,7 @@ cat example.ini | jc --ini -p
```
### ls
```bash
$ ls -l /usr/bin | jc --ls -p # or: jc -p ls -l /usr/bin
$ ls -l /usr/bin | jc -p --ls # or: jc -p ls -l /usr/bin
```
```json
[
@ -809,7 +814,7 @@ $ ls -l /usr/bin | jc --ls -p # or: jc -p ls -l /usr/bin
```
### netstat
```bash
netstat -apee | jc --netstat -p # or: jc -p netstat -apee
netstat -apee | jc -p --netstat # or: jc -p netstat -apee
```
```json
[
@ -897,7 +902,7 @@ netstat -apee | jc --netstat -p # or: jc -p netstat -apee
```
### /etc/passwd file
```bash
cat /etc/passwd | jc --passwd -p
cat /etc/passwd | jc -p --passwd
```
```json
[
@ -923,7 +928,7 @@ cat /etc/passwd | jc --passwd -p
```
### ping
```bash
ping 8.8.8.8 -c 3 | jc --ping -p # or: jc -p ping 8.8.8.8 -c 3
ping 8.8.8.8 -c 3 | jc -p --ping # or: jc -p ping 8.8.8.8 -c 3
```
```json
{
@ -976,7 +981,7 @@ ping 8.8.8.8 -c 3 | jc --ping -p # or: jc -p ping 8.8.8.8 -c 3
```
### ps
```bash
ps axu | jc --ps -p # or: jc -p ps axu
ps axu | jc -p --ps # or: jc -p ps axu
```
```json
[
@ -1023,7 +1028,7 @@ ps axu | jc --ps -p # or: jc -p ps axu
```
### traceroute
```bash
traceroute -m 2 8.8.8.8 | jc --traceroute -p
traceroute -m 2 8.8.8.8 | jc -p --traceroute
# or: jc -p traceroute -m 2 8.8.8.8
```
```json
@ -1088,7 +1093,7 @@ traceroute -m 2 8.8.8.8 | jc --traceroute -p
```
### uptime
```bash
uptime | jc --uptime -p # or: jc -p uptime
uptime | jc -p --uptime # or: jc -p uptime
```
```json
{
@ -1133,7 +1138,7 @@ cat cd_catalog.xml
...
```
```bash
cat cd_catalog.xml | jc --xml -p
cat cd_catalog.xml | jc -p --xml
```
```json
{
@ -1185,7 +1190,7 @@ spec:
mode: ISTIO_MUTUAL
```
```bash
cat istio.yaml | jc --yaml -p
cat istio.yaml | jc -p --yaml
```
```json
[

2
completions/jc_bash_completion.sh
View File

@ -4,7 +4,7 @@ _jc()
jc_about_options jc_about_mod_options jc_help_options jc_special_options
jc_commands=(acpi airport arp blkid chage cksum crontab date df dig dmidecode dpkg du env file finger free git gpg hciconfig id ifconfig iostat iptables iw jobs last lastb ls lsblk lsmod lsof lsusb md5 md5sum mdadm mount mpstat netstat nmcli ntpq pidstat ping ping6 pip pip3 postconf printenv ps route rpm rsync sfdisk sha1sum sha224sum sha256sum sha384sum sha512sum shasum ss stat sum sysctl systemctl systeminfo timedatectl top tracepath tracepath6 traceroute traceroute6 ufw uname update-alternatives upower uptime vdir vmstat w wc who xrandr zipinfo)
jc_parsers=(--acpi --airport --airport-s --arp --asciitable --asciitable-m --blkid --cef --cef-s --chage --cksum --crontab --crontab-u --csv --csv-s --date --df --dig --dir --dmidecode --dpkg-l --du --email-address --env --file --finger --free --fstab --git-log --git-log-s --gpg --group --gshadow --hash --hashsum --hciconfig --history --hosts --id --ifconfig --ini --iostat --iostat-s --ip-address --iptables --iso-datetime --iw-scan --jar-manifest --jobs --jwt --kv --last --ls --ls-s --lsblk --lsmod --lsof --lsusb --m3u --mdadm --mount --mpstat --mpstat-s --netstat --nmcli --ntpq --passwd --pidstat --pidstat-s --ping --ping-s --pip-list --pip-show --plist --postconf --ps --route --rpm-qi --rsync --rsync-s --sfdisk --shadow --ss --stat --stat-s --sysctl --syslog --syslog-s --syslog-bsd --syslog-bsd-s --systemctl --systemctl-lj --systemctl-ls --systemctl-luf --systeminfo --time --timedatectl --timestamp --top --top-s --tracepath --traceroute --ufw --ufw-appinfo --uname --update-alt-gs --update-alt-q --upower --uptime --url --vmstat --vmstat-s --w --wc --who --x509-cert --xml --xrandr --yaml --zipinfo)
jc_parsers=(--acpi --airport --airport-s --arp --asciitable --asciitable-m --blkid --cef --cef-s --chage --cksum --crontab --crontab-u --csv --csv-s --date --df --dig --dir --dmidecode --dpkg-l --du --email-address --env --file --finger --free --fstab --git-log --git-log-s --gpg --group --gshadow --hash --hashsum --hciconfig --history --hosts --id --ifconfig --ini --iostat --iostat-s --ip-address --iptables --iso-datetime --iw-scan --jar-manifest --jobs --jwt --kv --last --ls --ls-s --lsblk --lsmod --lsof --lsusb --m3u --mdadm --mount --mpstat --mpstat-s --netstat --nmcli --ntpq --passwd --pidstat --pidstat-s --ping --ping-s --pip-list --pip-show --plist --postconf --proc --ps --route --rpm-qi --rsync --rsync-s --sfdisk --shadow --ss --stat --stat-s --sysctl --syslog --syslog-s --syslog-bsd --syslog-bsd-s --systemctl --systemctl-lj --systemctl-ls --systemctl-luf --systeminfo --time --timedatectl --timestamp --top --top-s --tracepath --traceroute --ufw --ufw-appinfo --uname --update-alt-gs --update-alt-q --upower --uptime --url --vmstat --vmstat-s --w --wc --who --x509-cert --xml --xrandr --yaml --zipinfo)
jc_options=(--force-color -C --debug -d --monochrome -m --meta-out -M --pretty -p --quiet -q --raw -r --unbuffer -u --yaml-out -y)
jc_about_options=(--about -a)
jc_about_mod_options=(--pretty -p --yaml-out -y --monochrome -m --force-color -C)

3
completions/jc_zsh_completion.sh
View File

@ -95,7 +95,7 @@ _jc() {
'xrandr:run "xrandr" command with magic syntax.'
'zipinfo:run "zipinfo" command with magic syntax.'
)
jc_parsers=(--acpi --airport --airport-s --arp --asciitable --asciitable-m --blkid --cef --cef-s --chage --cksum --crontab --crontab-u --csv --csv-s --date --df --dig --dir --dmidecode --dpkg-l --du --email-address --env --file --finger --free --fstab --git-log --git-log-s --gpg --group --gshadow --hash --hashsum --hciconfig --history --hosts --id --ifconfig --ini --iostat --iostat-s --ip-address --iptables --iso-datetime --iw-scan --jar-manifest --jobs --jwt --kv --last --ls --ls-s --lsblk --lsmod --lsof --lsusb --m3u --mdadm --mount --mpstat --mpstat-s --netstat --nmcli --ntpq --passwd --pidstat --pidstat-s --ping --ping-s --pip-list --pip-show --plist --postconf --ps --route --rpm-qi --rsync --rsync-s --sfdisk --shadow --ss --stat --stat-s --sysctl --syslog --syslog-s --syslog-bsd --syslog-bsd-s --systemctl --systemctl-lj --systemctl-ls --systemctl-luf --systeminfo --time --timedatectl --timestamp --top --top-s --tracepath --traceroute --ufw --ufw-appinfo --uname --update-alt-gs --update-alt-q --upower --uptime --url --vmstat --vmstat-s --w --wc --who --x509-cert --xml --xrandr --yaml --zipinfo)
jc_parsers=(--acpi --airport --airport-s --arp --asciitable --asciitable-m --blkid --cef --cef-s --chage --cksum --crontab --crontab-u --csv --csv-s --date --df --dig --dir --dmidecode --dpkg-l --du --email-address --env --file --finger --free --fstab --git-log --git-log-s --gpg --group --gshadow --hash --hashsum --hciconfig --history --hosts --id --ifconfig --ini --iostat --iostat-s --ip-address --iptables --iso-datetime --iw-scan --jar-manifest --jobs --jwt --kv --last --ls --ls-s --lsblk --lsmod --lsof --lsusb --m3u --mdadm --mount --mpstat --mpstat-s --netstat --nmcli --ntpq --passwd --pidstat --pidstat-s --ping --ping-s --pip-list --pip-show --plist --postconf --proc --ps --route --rpm-qi --rsync --rsync-s --sfdisk --shadow --ss --stat --stat-s --sysctl --syslog --syslog-s --syslog-bsd --syslog-bsd-s --systemctl --systemctl-lj --systemctl-ls --systemctl-luf --systeminfo --time --timedatectl --timestamp --top --top-s --tracepath --traceroute --ufw --ufw-appinfo --uname --update-alt-gs --update-alt-q --upower --uptime --url --vmstat --vmstat-s --w --wc --who --x509-cert --xml --xrandr --yaml --zipinfo)
jc_parsers_describe=(
'--acpi:`acpi` command parser'
'--airport:`airport -I` command parser'
@ -172,6 +172,7 @@ _jc() {
'--pip-show:`pip show` command parser'
'--plist:PLIST file parser'
'--postconf:`postconf -M` command parser'
'--proc:`/proc/` file parser'
'--ps:`ps` command parser'
'--route:`route` command parser'
'--rpm-qi:`rpm -qi` command parser'

5
docs/lib.md
View File

@ -162,7 +162,8 @@ Parameters:
### all\_parser\_info
```python
def all_parser_info(documentation: bool = False) -> List[Dict]
def all_parser_info(documentation: bool = False,
show_hidden: bool = False) -> List[Dict]
```
Returns a list of dictionaries that includes metadata for all parser
@ -171,6 +172,8 @@ modules.
Parameters:
documentation: (boolean) include parser docstrings if True
show_hidden: (boolean) also show parsers marked as hidden
in their info metadata.
<a id="jc.lib.get_help"></a>

2
docs/parsers/lsof.md
View File

@ -140,6 +140,6 @@ Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Compatibility: linux, darwin, aix, freebsd
Version 1.6 by Kelly Brazil (kellyjonbrazil@gmail.com)

142
docs/parsers/proc.md Normal file
View File

@ -0,0 +1,142 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc"></a>
# jc.parsers.proc
jc - JSON Convert Proc file output parser
This parser automatically identifies the Proc file and calls the
corresponding parser to peform the parsing.
Magic syntax for converting `/proc` files is also supported by running
`jc /proc/<path to file>`. Any `jc` options must be specified before the
`/proc` path.
specific Proc file parsers can also be called directly, if desired and have
a naming convention of `proc-<name>` (cli) or `proc_<name>` (module).
Usage (cli):
$ cat /proc/meminfo | jc --proc
or
$ jc /proc/meminfo
or
$ cat /proc/meminfo | jc --proc-memifno
Usage (module):
import jc
result = jc.parse('proc', proc_file)
Schema:
See the specific Proc parser for the schema:
$ jc --help --proc-<name>
For example:
$ jc --help --proc-meminfo
Specific Proc file parser names can be found with `jc -hh` or `jc -a`.
Schemas can also be found online at:
https://kellyjonbrazil.github.io/jc/docs/parsers/proc_<name>
For example:
https://kellyjonbrazil.github.io/jc/docs/parsers/proc_meminfo
Examples:
$ cat /proc/modules | jc --proc -p
[
{
"module": "binfmt_misc",
"size": 24576,
"used": 1,
"used_by": [],
"status": "Live",
"location": "0xffffffffc0ab4000"
},
{
"module": "vsock_loopback",
"size": 16384,
"used": 0,
"used_by": [],
"status": "Live",
"location": "0xffffffffc0a14000"
},
{
"module": "vmw_vsock_virtio_transport_common",
"size": 36864,
"used": 1,
"used_by": [
"vsock_loopback"
],
"status": "Live",
"location": "0xffffffffc0a03000"
},
...
]
$ proc_modules | jc --proc_modules -p -r
[
{
"module": "binfmt_misc",
"size": "24576",
"used": "1",
"used_by": [],
"status": "Live",
"location": "0xffffffffc0ab4000"
},
{
"module": "vsock_loopback",
"size": "16384",
"used": "0",
"used_by": [],
"status": "Live",
"location": "0xffffffffc0a14000"
},
{
"module": "vmw_vsock_virtio_transport_common",
"size": "36864",
"used": "1",
"used_by": [
"vsock_loopback"
],
"status": "Live",
"location": "0xffffffffc0a03000"
},
...
]
<a id="jc.parsers.proc.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

128
docs/parsers/proc_buddyinfo.md Normal file
View File

@ -0,0 +1,128 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_buddyinfo"></a>
# jc.parsers.proc\_buddyinfo
jc - JSON Convert `/proc/buddyinfo` file parser
Usage (cli):
$ cat /proc/buddyinfo | jc --proc
or
$ jc /proc/buddyinfo
or
$ cat /proc/buddyinfo | jc --proc-buddyinfo
Usage (module):
import jc
result = jc.parse('proc', proc_buddyinfo_file)
or
import jc
result = jc.parse('proc_buddyinfo', proc_buddyinfo_file)
Schema:
All values are integers.
[
{
"node": integer,
"zone": string,
"free_chunks": [
integer # [0]
]
}
]
[0] array index correlates to the Order number.
E.g. free_chunks[0] is the value for Order 0
Examples:
$ cat /proc/buddyinfo | jc --proc -p
[
{
"node": 0,
"zone": "DMA",
"free_chunks": [
0,
0,
0,
1,
1,
1,
1,
1,
0,
1,
3
]
},
{
"node": 0,
"zone": "DMA32",
"free_chunks": [
78,
114,
82,
52,
38,
25,
13,
9,
3,
4,
629
]
},
{
"node": 0,
"zone": "Normal",
"free_chunks": [
0,
22,
8,
10,
1,
1,
2,
11,
13,
0,
0
]
}
]
<a id="jc.parsers.proc_buddyinfo.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

111
docs/parsers/proc_consoles.md Normal file
View File

@ -0,0 +1,111 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_consoles"></a>
# jc.parsers.proc\_consoles
jc - JSON Convert `/proc/consoles` file parser
Usage (cli):
$ cat /proc/consoles | jc --proc
or
$ jc /proc/consoles
or
$ cat /proc/consoles | jc --proc-consoles
Usage (module):
import jc
result = jc.parse('proc', proc_consoles_file)
or
import jc
result = jc.parse('proc_consoles', proc_consoles_file)
Schema:
[
{
"device": string,
"operations": string,
"operations_list": [
string # [0]
],
"flags": string,
"flags_list": [
string # [1]
],
"major": integer,
"minor": integer
}
]
[0] Values: read, write, unblank
[1] Values: enabled, preferred, primary boot, prink buffer,
braille device, safe when CPU offline
Examples:
$ cat /proc/consoles | jc --proc -p
[
{
"device": "tty0",
"operations": "-WU",
"operations_list": [
"write",
"unblank"
],
"flags": "ECp",
"flags_list": [
"enabled",
"preferred",
"printk buffer"
],
"major": 4,
"minor": 7
},
{
"device": "ttyS0",
"operations": "-W-",
"operations_list": [
"write"
],
"flags": "Ep",
"flags_list": [
"enabled",
"printk buffer"
],
"major": 4,
"minor": 64
}
]
<a id="jc.parsers.proc_consoles.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

247
docs/parsers/proc_cpuinfo.md Normal file
View File

@ -0,0 +1,247 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_cpuinfo"></a>
# jc.parsers.proc\_cpuinfo
jc - JSON Convert `/proc/cpuinfo` file parser
Usage (cli):
$ cat /proc/cpuinfo | jc --proc
or
$ jc /proc/cpuinfo
or
$ cat /proc/cpuinfo | jc --proc-cpuinfo
Usage (module):
import jc
result = jc.parse('proc', proc_cpuinfo_file)
or
import jc
result = jc.parse('proc_cpuinfo', proc_cpuinfo_file)
Schema:
Integer, float, and boolean ("yes"/"no") conversions are attempted. Blank
strings are converted to `null`.
"Well-known" keys like `cache size`, `address types`, `bugs`, and `flags`
are processed into sensible data types. (see below)
If this is not desired, then use the `--raw` (CLI) or `raw=True` (Module)
option.
[
{
"processor": integer,
"address sizes": string,
"address_size_physical": integer, # in bits
"address_size_virtual": integer, # in bits
"cache size": string,
"cache_size_num": integer,
"cache_size_unit": string,
"flags": [
string
],
"bugs": [
string
],
"bogomips": float,
<key>: string/int/float/boolean/null
}
]
Examples:
$ cat /proc/cpuinfo | jc --proc -p
[
{
"processor": 0,
"vendor_id": "GenuineIntel",
"cpu family": 6,
"model": 142,
"model name": "Intel(R) Core(TM) i5-8279U CPU @ 2.40GHz",
"stepping": 10,
"cpu MHz": 2400.0,
"cache size": "6144 KB",
"physical id": 0,
"siblings": 1,
"core id": 0,
"cpu cores": 1,
"apicid": 0,
"initial apicid": 0,
"fpu": true,
"fpu_exception": true,
"cpuid level": 22,
"wp": true,
"bogomips": 4800.0,
"clflush size": 64,
"cache_alignment": 64,
"address sizes": "45 bits physical, 48 bits virtual",
"power management": null,
"address_size_physical": 45,
"address_size_virtual": 48,
"cache_size_num": 6144,
"cache_size_unit": "KB",
"flags": [
"fpu",
"vme",
"de",
"pse",
"tsc",
"msr",
"pae",
"mce",
"cx8",
"apic",
"sep",
"mtrr",
"pge",
"mca",
"cmov",
"pat",
"pse36",
"clflush",
"mmx",
"fxsr",
"sse",
"sse2",
"ss",
"syscall",
"nx",
"pdpe1gb",
"rdtscp",
"lm",
"constant_tsc",
"arch_perfmon",
"nopl",
"xtopology",
"tsc_reliable",
"nonstop_tsc",
"cpuid",
"pni",
"pclmulqdq",
"ssse3",
"fma",
"cx16",
"pcid",
"sse4_1",
"sse4_2",
"x2apic",
"movbe",
"popcnt",
"tsc_deadline_timer",
"aes",
"xsave",
"avx",
"f16c",
"rdrand",
"hypervisor",
"lahf_lm",
"abm",
"3dnowprefetch",
"cpuid_fault",
"invpcid_single",
"pti",
"ssbd",
"ibrs",
"ibpb",
"stibp",
"fsgsbase",
"tsc_adjust",
"bmi1",
"avx2",
"smep",
"bmi2",
"invpcid",
"rdseed",
"adx",
"smap",
"clflushopt",
"xsaveopt",
"xsavec",
"xgetbv1",
"xsaves",
"arat",
"md_clear",
"flush_l1d",
"arch_capabilities"
],
"bugs": [
"cpu_meltdown",
"spectre_v1",
"spectre_v2",
"spec_store_bypass",
"l1tf",
"mds",
"swapgs",
"itlb_multihit",
"srbds"
]
},
...
]
$ proc_cpuinfo | jc --proc_cpuinfo -p -r
[
{
"processor": "0",
"vendor_id": "GenuineIntel",
"cpu family": "6",
"model": "142",
"model name": "Intel(R) Core(TM) i5-8279U CPU @ 2.40GHz",
"stepping": "10",
"cpu MHz": "2400.000",
"cache size": "6144 KB",
"physical id": "0",
"siblings": "1",
"core id": "0",
"cpu cores": "1",
"apicid": "0",
"initial apicid": "0",
"fpu": "yes",
"fpu_exception": "yes",
"cpuid level": "22",
"wp": "yes",
"flags": "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge ...",
"bugs": "cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass ...",
"bogomips": "4800.00",
"clflush size": "64",
"cache_alignment": "64",
"address sizes": "45 bits physical, 48 bits virtual",
"power management": ""
},
...
]
<a id="jc.parsers.proc_cpuinfo.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

143
docs/parsers/proc_crypto.md Normal file
View File

@ -0,0 +1,143 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_crypto"></a>
# jc.parsers.proc\_crypto
jc - JSON Convert `/proc/crypto` file parser
Usage (cli):
$ cat /proc/crypto | jc --proc
or
$ jc /proc/crypto
or
$ cat /proc/crypto | jc --proc-crypto
Usage (module):
import jc
result = jc.parse('proc', proc_crypto_file)
or
import jc
result = jc.parse('proc_crypto', proc_crypto_file)
Schema:
"Well-known" keys like `priority` and `refcnt` are converted to integers.
Also, keynames ending in "size" are converted to integers.
If this is not desired, then use the `--raw` (CLI) or `raw=True` (Module)
option.
[
{
"name": string,
"driver": string,
"module": string,
"priority": integer,
"refcnt": integer,
"selftest": string,
"internal": string,
"type": string,
"*size": integer
}
]
Examples:
$ cat /proc/crypto | jc --proc -p
[
{
"name": "ecdh",
"driver": "ecdh-generic",
"module": "ecdh_generic",
"priority": 100,
"refcnt": 1,
"selftest": "passed",
"internal": "no",
"type": "kpp"
},
{
"name": "blake2b-512",
"driver": "blake2b-512-generic",
"module": "blake2b_generic",
"priority": 100,
"refcnt": 1,
"selftest": "passed",
"internal": "no",
"type": "shash",
"blocksize": 128,
"digestsize": 64
},
...
]
$ proc_crypto | jc --proc_crypto -p -r
[
{
"name": "ecdh",
"driver": "ecdh-generic",
"module": "ecdh_generic",
"priority": "100",
"refcnt": "1",
"selftest": "passed",
"internal": "no",
"type": "kpp"
},
{
"name": "blake2b-512",
"driver": "blake2b-512-generic",
"module": "blake2b_generic",
"priority": "100",
"refcnt": "1",
"selftest": "passed",
"internal": "no",
"type": "shash",
"blocksize": "128",
"digestsize": "64"
},
{
"name": "blake2b-384",
"driver": "blake2b-384-generic",
"module": "blake2b_generic",
"priority": "100",
"refcnt": "1",
"selftest": "passed",
"internal": "no",
"type": "shash",
"blocksize": "128",
"digestsize": "48"
},
...
]
<a id="jc.parsers.proc_crypto.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

103
docs/parsers/proc_devices.md Normal file
View File

@ -0,0 +1,103 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_devices"></a>
# jc.parsers.proc\_devices
jc - JSON Convert `/proc/devices` file parser
Usage (cli):
$ cat /proc/devices | jc --proc
or
$ jc /proc/devices
or
$ cat /proc/devices | jc --proc-devices
Usage (module):
import jc
result = jc.parse('proc', proc_devices_file)
or
import jc
result = jc.parse('proc_devices', proc_devices_file)
Schema:
Since devices can be members of multiple groups, the value for each device
is a list.
{
"character": {
"<device number>": [
string
]
},
"block": {
"<device number>": [
string
]
}
}
Examples:
$ cat /proc/devices | jc --proc -p
{
"character": {
"1": [
"mem"
],
"4": [
"/dev/vc/0",
"tty",
"ttyS"
],
"5": [
"/dev/tty",
"/dev/console",
"/dev/ptmx",
"ttyprintk"
],
"block": {
"7": [
"loop"
],
"8": [
"sd"
],
"9": [
"md"
]
}
}
<a id="jc.parsers.proc_devices.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> Dict
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

202
docs/parsers/proc_diskstats.md Normal file
View File

@ -0,0 +1,202 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_diskstats"></a>
# jc.parsers.proc\_diskstats
jc - JSON Convert `/proc/diskstats` file parser
Usage (cli):
$ cat /proc/diskstats | jc --proc
or
$ jc /proc/diskstats
or
$ cat /proc/diskstats | jc --proc-diskstats
Usage (module):
import jc
result = jc.parse('proc', proc_diskstats_file)
or
import jc
result = jc.parse('proc_diskstats', proc_diskstats_file)
Schema:
[
{
"maj": integer,
"min": integer,
"device": string,
"reads_completed": integer,
"reads_merged": integer,
"sectors_read": integer,
"read_time_ms": integer,
"writes_completed": integer,
"writes_merged": integer,
"sectors_written": integer,
"write_time_ms": integer,
"io_in_progress": integer,
"io_time_ms": integer,
"weighted_io_time_ms": integer,
"discards_completed_successfully": integer,
"discards_merged": integer,
"sectors_discarded": integer,
"discarding_time_ms": integer,
"flush_requests_completed_successfully": integer,
"flushing_time_ms": integer
}
]
Examples:
$ cat /proc/diskstats | jc --proc -p
[
{
"maj": 7,
"min": 0,
"device": "loop0",
"reads_completed": 48,
"reads_merged": 0,
"sectors_read": 718,
"read_time_ms": 19,
"writes_completed": 0,
"writes_merged": 0,
"sectors_written": 0,
"write_time_ms": 0,
"io_in_progress": 0,
"io_time_ms": 36,
"weighted_io_time_ms": 19,
"discards_completed_successfully": 0,
"discards_merged": 0,
"sectors_discarded": 0,
"discarding_time_ms": 0,
"flush_requests_completed_successfully": 0,
"flushing_time_ms": 0
},
{
"maj": 7,
"min": 1,
"device": "loop1",
"reads_completed": 41,
"reads_merged": 0,
"sectors_read": 688,
"read_time_ms": 17,
"writes_completed": 0,
"writes_merged": 0,
"sectors_written": 0,
"write_time_ms": 0,
"io_in_progress": 0,
"io_time_ms": 28,
"weighted_io_time_ms": 17,
"discards_completed_successfully": 0,
"discards_merged": 0,
"sectors_discarded": 0,
"discarding_time_ms": 0,
"flush_requests_completed_successfully": 0,
"flushing_time_ms": 0
},
...
]
$ proc_diskstats | jc --proc_diskstats -p -r
[
{
"maj": "7",
"min": "0",
"device": "loop0",
"reads_completed": "48",
"reads_merged": "0",
"sectors_read": "718",
"read_time_ms": "19",
"writes_completed": "0",
"writes_merged": "0",
"sectors_written": "0",
"write_time_ms": "0",
"io_in_progress": "0",
"io_time_ms": "36",
"weighted_io_time_ms": "19",
"discards_completed_successfully": "0",
"discards_merged": "0",
"sectors_discarded": "0",
"discarding_time_ms": "0",
"flush_requests_completed_successfully": "0",
"flushing_time_ms": "0"
},
{
"maj": "7",
"min": "1",
"device": "loop1",
"reads_completed": "41",
"reads_merged": "0",
"sectors_read": "688",
"read_time_ms": "17",
"writes_completed": "0",
"writes_merged": "0",
"sectors_written": "0",
"write_time_ms": "0",
"io_in_progress": "0",
"io_time_ms": "28",
"weighted_io_time_ms": "17",
"discards_completed_successfully": "0",
"discards_merged": "0",
"sectors_discarded": "0",
"discarding_time_ms": "0",
"flush_requests_completed_successfully": "0",
"flushing_time_ms": "0"
},
{
"maj": "7",
"min": "2",
"device": "loop2",
"reads_completed": "119",
"reads_merged": "0",
"sectors_read": "2956",
"read_time_ms": "18",
"writes_completed": "0",
"writes_merged": "0",
"sectors_written": "0",
"write_time_ms": "0",
"io_in_progress": "0",
"io_time_ms": "56",
"weighted_io_time_ms": "18",
"discards_completed_successfully": "0",
"discards_merged": "0",
"sectors_discarded": "0",
"discarding_time_ms": "0",
"flush_requests_completed_successfully": "0",
"flushing_time_ms": "0"
},
...
]
<a id="jc.parsers.proc_diskstats.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

126
docs/parsers/proc_driver_rtc.md Normal file
View File

@ -0,0 +1,126 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_driver_rtc"></a>
# jc.parsers.proc\_driver\_rtc
jc - JSON Convert `/proc/driver_rtc` file parser
Usage (cli):
$ cat /proc/driver_rtc | jc --proc
or
$ jc /proc/driver_rtc
or
$ cat /proc/driver_rtc | jc --proc-driver-rtc
Usage (module):
import jc
result = jc.parse('proc', proc_driver_rtc_file)
or
import jc
result = jc.parse('proc_driver_rtc', proc_driver_rtc_file)
Schema:
"yes" and "no" values are converted to `true`/`false`. Integer conversions
are attempted. If you do not want this behavior, then use `--raw` (cli) or
`raw=True` (module).
{
"rtc_time": string,
"rtc_date": string,
"alrm_time": string,
"alrm_date": string,
"alarm_IRQ": boolean,
"alrm_pending": boolean,
"update IRQ enabled": boolean,
"periodic IRQ enabled": boolean,
"periodic IRQ frequency": integer,
"max user IRQ frequency": integer,
"24hr": boolean,
"periodic_IRQ": boolean,
"update_IRQ": boolean,
"HPET_emulated": boolean,
"BCD": boolean,
"DST_enable": boolean,
"periodic_freq": integer,
"batt_status": string
}
Examples:
$ cat /proc/driver_rtc | jc --proc -p
{
"rtc_time": "16:09:21",
"rtc_date": "2022-09-03",
"alrm_time": "00:00:00",
"alrm_date": "2022-09-03",
"alarm_IRQ": false,
"alrm_pending": false,
"update IRQ enabled": false,
"periodic IRQ enabled": false,
"periodic IRQ frequency": 1024,
"max user IRQ frequency": 64,
"24hr": true,
"periodic_IRQ": false,
"update_IRQ": false,
"HPET_emulated": true,
"BCD": true,
"DST_enable": false,
"periodic_freq": 1024,
"batt_status": "okay"
}
$ cat /proc/driver_rtc | jc --proc -p -r
{
"rtc_time": "16:09:21",
"rtc_date": "2022-09-03",
"alrm_time": "00:00:00",
"alrm_date": "2022-09-03",
"alarm_IRQ": "no",
"alrm_pending": "no",
"update IRQ enabled": "no",
"periodic IRQ enabled": "no",
"periodic IRQ frequency": "1024",
"max user IRQ frequency": "64",
"24hr": "yes",
"periodic_IRQ": "no",
"update_IRQ": "no",
"HPET_emulated": "yes",
"BCD": "yes",
"DST_enable": "no",
"periodic_freq": "1024",
"batt_status": "okay"
}
<a id="jc.parsers.proc_driver_rtc.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> Dict
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

81
docs/parsers/proc_filesystems.md Normal file
View File

@ -0,0 +1,81 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_filesystems"></a>
# jc.parsers.proc\_filesystems
jc - JSON Convert `/proc/filesystems` file parser
Usage (cli):
$ cat /proc/filesystems | jc --proc
or
$ jc /proc/filesystems
or
$ cat /proc/filesystems | jc --proc-filesystems
Usage (module):
import jc
result = jc.parse('proc', proc_filesystems_file)
or
import jc
result = jc.parse('proc_filesystems', proc_filesystems_file)
Schema:
[
{
"filesystem": string,
"nodev": boolean
}
]
Examples:
$ cat /proc/filesystems | jc --proc -p
[
{
"filesystem": "sysfs",
"nodev": true
},
{
"filesystem": "tmpfs",
"nodev": true
},
{
"filesystem": "bdev",
"nodev": true
},
...
]
<a id="jc.parsers.proc_filesystems.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

133
docs/parsers/proc_interrupts.md Normal file
View File

@ -0,0 +1,133 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_interrupts"></a>
# jc.parsers.proc\_interrupts
jc - JSON Convert `/proc/interrupts` file parser
Usage (cli):
$ cat /proc/interrupts | jc --proc
or
$ jc /proc/interrupts
or
$ cat /proc/interrupts | jc --proc-interrupts
Usage (module):
import jc
result = jc.parse('proc', proc_interrupts_file)
or
import jc
result = jc.parse('proc_interrupts', proc_interrupts_file)
Schema:
[
{
"irq": string,
"cpu_num": integer,
"interrupts": [
integer
],
"type": string,
"device": [
string
]
}
]
Examples:
$ cat /proc/interrupts | jc --proc -p
[
{
"irq": "0",
"cpu_num": 2,
"interrupts": [
18,
0
],
"type": "IO-APIC",
"device": [
"2-edge",
"timer"
]
},
{
"irq": "1",
"cpu_num": 2,
"interrupts": [
0,
73
],
"type": "IO-APIC",
"device": [
"1-edge",
"i8042"
]
},
...
]
$ proc_interrupts | jc --proc_interrupts -p -r
[
{
"irq": "0",
"cpu_num": 2,
"interrupts": [
"18",
"0"
],
"type": "IO-APIC",
"device": [
"2-edge",
"timer"
]
},
{
"irq": "1",
"cpu_num": 2,
"interrupts": [
"0",
"73"
],
"type": "IO-APIC",
"device": [
"1-edge",
"i8042"
]
},
...
]
<a id="jc.parsers.proc_interrupts.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

85
docs/parsers/proc_iomem.md Normal file
View File

@ -0,0 +1,85 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_iomem"></a>
# jc.parsers.proc\_iomem
jc - JSON Convert `/proc/iomem` file parser
Usage (cli):
$ cat /proc/iomem | jc --proc
or
$ jc /proc/iomem
or
$ cat /proc/iomem | jc --proc-iomem
Usage (module):
import jc
result = jc.parse('proc', proc_iomem_file)
or
import jc
result = jc.parse('proc_iomem', proc_iomem_file)
Schema:
[
{
"start": string,
"end": string,
"device": string
}
]
Examples:
$ cat /proc/iomem | jc --proc -p
[
{
"start": "00000000",
"end": "00000fff",
"device": "Reserved"
},
{
"start": "00001000",
"end": "0009e7ff",
"device": "System RAM"
},
{
"start": "0009e800",
"end": "0009ffff",
"device": "Reserved"
},
...
]
<a id="jc.parsers.proc_iomem.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

85
docs/parsers/proc_ioports.md Normal file
View File

@ -0,0 +1,85 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_ioports"></a>
# jc.parsers.proc\_ioports
jc - JSON Convert `/proc/ioports` file parser
Usage (cli):
$ cat /proc/ioports | jc --proc
or
$ jc /proc/ioports
or
$ cat /proc/ioports | jc --proc-ioports
Usage (module):
import jc
result = jc.parse('proc', proc_ioports_file)
or
import jc
result = jc.parse('proc_ioports', proc_ioports_file)
Schema:
[
{
"start": string,
"end": string,
"device": string
}
]
Examples:
$ cat /proc/ioports | jc --proc -p
[
{
"start": "0000",
"end": "0cf7",
"device": "PCI Bus 0000:00"
},
{
"start": "0000",
"end": "001f",
"device": "dma1"
},
{
"start": "0020",
"end": "0021",
"device": "PNP0001:00"
},
...
]
<a id="jc.parsers.proc_ioports.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

88
docs/parsers/proc_loadavg.md Normal file
View File

@ -0,0 +1,88 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_loadavg"></a>
# jc.parsers.proc\_loadavg
jc - JSON Convert `/proc/loadavg` file parser
Usage (cli):
$ cat /proc/loadavg | jc --proc
or
$ jc /proc/loadavg
or
$ cat /proc/loadavg | jc --proc-loadavg
Usage (module):
import jc
result = jc.parse('proc', proc_loadavg_file)
or
import jc
result = jc.parse('proc_loadavg', proc_loadavg_file)
Schema:
All values are integers.
{
"load_1m": float,
"load_5m": float,
"load_15m": float,
"running": integer,
"available": integer,
"last_pid": integer
}
Examples:
$ cat /proc/loadavg | jc --proc -p
{
"load_1m": 0.0,
"load_5m": 0.01,
"load_15m": 0.03,
"running": 2,
"available": 111,
"last_pid": 2039
}
$ cat /proc/loadavg | jc --proc -p -r
{
"load_1m": "0.00",
"load_5m": "0.01",
"load_15m": "0.03",
"running": "2",
"available": "111",
"last_pid": "2039"
}
<a id="jc.parsers.proc_loadavg.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> Dict
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

130
docs/parsers/proc_locks.md Normal file
View File

@ -0,0 +1,130 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_locks"></a>
# jc.parsers.proc\_locks
jc - JSON Convert `/proc/locks` file parser
Usage (cli):
$ cat /proc/locks | jc --proc
or
$ jc /proc/locks
or
$ cat /proc/locks | jc --proc-locks
Usage (module):
import jc
result = jc.parse('proc', proc_locks_file)
or
import jc
result = jc.parse('proc_locks', proc_locks_file)
Schema:
[
{
"id": integer,
"class": string,
"type": string,
"access": string,
"pid": integer,
"maj": string,
"min": string,
"inode": integer,
"start": string,
"end": string
}
]
Examples:
$ cat /proc/locks | jc --proc -p
[
{
"id": 1,
"class": "POSIX",
"type": "ADVISORY",
"access": "WRITE",
"pid": 877,
"maj": "00",
"min": "19",
"inode": 812,
"start": "0",
"end": "EOF"
},
{
"id": 2,
"class": "FLOCK",
"type": "ADVISORY",
"access": "WRITE",
"pid": 854,
"maj": "00",
"min": "19",
"inode": 805,
"start": "0",
"end": "EOF"
},
...
]
$ proc_locks | jc --proc_locks -p -r
[
{
"id": "1",
"class": "POSIX",
"type": "ADVISORY",
"access": "WRITE",
"pid": "877",
"maj": "00",
"min": "19",
"inode": "812",
"start": "0",
"end": "EOF"
},
{
"id": "2",
"class": "FLOCK",
"type": "ADVISORY",
"access": "WRITE",
"pid": "854",
"maj": "00",
"min": "19",
"inode": "805",
"start": "0",
"end": "EOF"
},
...
]
<a id="jc.parsers.proc_locks.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

118
docs/parsers/proc_meminfo.md Normal file
View File

@ -0,0 +1,118 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_meminfo"></a>
# jc.parsers.proc\_meminfo
jc - JSON Convert `/proc/meminfo` file parser
Usage (cli):
$ cat /proc/meminfo | jc --proc
or
$ jc /proc/meminfo
or
$ cat /proc/meminfo | jc --proc-meminfo
Usage (module):
import jc
result = jc.parse('proc', proc_meminfo_file)
or
import jc
result = jc.parse('proc_meminfo', proc_meminfo_file)
Schema:
All values are integers.
{
<keyName> integer
}
Examples:
$ cat /proc/meminfo | jc --proc -p
{
"MemTotal": 3997272,
"MemFree": 2760316,
"MemAvailable": 3386876,
"Buffers": 40452,
"Cached": 684856,
"SwapCached": 0,
"Active": 475816,
"Inactive": 322064,
"Active(anon)": 70216,
"Inactive(anon)": 148,
"Active(file)": 405600,
"Inactive(file)": 321916,
"Unevictable": 19476,
"Mlocked": 19476,
"SwapTotal": 3996668,
"SwapFree": 3996668,
"Dirty": 152,
"Writeback": 0,
"AnonPages": 92064,
"Mapped": 79464,
"Shmem": 1568,
"KReclaimable": 188216,
"Slab": 288096,
"SReclaimable": 188216,
"SUnreclaim": 99880,
"KernelStack": 5872,
"PageTables": 1812,
"NFS_Unstable": 0,
"Bounce": 0,
"WritebackTmp": 0,
"CommitLimit": 5995304,
"Committed_AS": 445240,
"VmallocTotal": 34359738367,
"VmallocUsed": 21932,
"VmallocChunk": 0,
"Percpu": 107520,
"HardwareCorrupted": 0,
"AnonHugePages": 0,
"ShmemHugePages": 0,
"ShmemPmdMapped": 0,
"FileHugePages": 0,
"FilePmdMapped": 0,
"HugePages_Total": 0,
"HugePages_Free": 0,
"HugePages_Rsvd": 0,
"HugePages_Surp": 0,
"Hugepagesize": 2048,
"Hugetlb": 0,
"DirectMap4k": 192320,
"DirectMap2M": 4001792,
"DirectMap1G": 2097152
}
<a id="jc.parsers.proc_meminfo.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> Dict
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

132
docs/parsers/proc_modules.md Normal file
View File

@ -0,0 +1,132 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.proc_modules"></a>
# jc.parsers.proc\_modules
jc - JSON Convert `/proc/modules` file parser
Usage (cli):
$ cat /proc/modules | jc --proc
or
$ jc /proc/modules
or
$ cat /proc/modules | jc --proc-modules
Usage (module):
import jc
result = jc.parse('proc', proc_modules_file)
or
import jc
result = jc.parse('proc_modules', proc_modules_file)
Schema:
[
{
"module": string,
"size": integer,
"used": integer,
"used_by": [
string
],
"status": string,
"location": string
}
]
Examples:
$ cat /proc/modules | jc --proc -p
[
{
"module": "binfmt_misc",
"size": 24576,
"used": 1,
"used_by": [],
"status": "Live",
"location": "0xffffffffc0ab4000"
},
{
"module": "vsock_loopback",
"size": 16384,
"used": 0,
"used_by": [],
"status": "Live",
"location": "0xffffffffc0a14000"
},
{
"module": "vmw_vsock_virtio_transport_common",
"size": 36864,
"used": 1,
"used_by": [
"vsock_loopback"
],
"status": "Live",
"location": "0xffffffffc0a03000"
},
...
]
$ proc_modules | jc --proc_modules -p -r
[
{
"module": "binfmt_misc",
"size": "24576",
"used": "1",
"used_by": [],
"status": "Live",
"location": "0xffffffffc0ab4000"
},
{
"module": "vsock_loopback",
"size": "16384",
"used": "0",
"used_by": [],
"status": "Live",
"location": "0xffffffffc0a14000"
},
{
"module": "vmw_vsock_virtio_transport_common",
"size": "36864",
"used": "1",
"used_by": [
"vsock_loopback"
],
"status": "Live",
"location": "0xffffffffc0a03000"
},
...
]
<a id="jc.parsers.proc_modules.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

85
jc/cli.py
View File

@ -11,8 +11,9 @@ import signal
import shlex
import subprocess
from .lib import (__version__, parser_info, all_parser_info, parsers,
_get_parser, _parser_is_streaming, standard_parser_mod_list,
plugin_parser_mod_list, streaming_parser_mod_list)
_get_parser, _parser_is_streaming, parser_mod_list,
standard_parser_mod_list, plugin_parser_mod_list,
streaming_parser_mod_list)
from . import utils
from .cli_data import long_options_map, new_pygments_colors, old_pygments_colors
from .shell_completions import bash_completion, zsh_completion
@ -120,11 +121,11 @@ def parser_shortname(parser_arg):
return parser_arg[2:]
def parsers_text(indent=0, pad=0):
def parsers_text(indent=0, pad=0, show_hidden=False):
"""Return the argument and description information from each parser"""
ptext = ''
padding_char = ' '
for p in all_parser_info():
for p in all_parser_info(show_hidden=show_hidden):
parser_arg = p.get('argument', 'UNKNOWN')
padding = pad - len(parser_arg)
parser_desc = p.get('description', 'No description available.')
@ -164,48 +165,62 @@ def about_jc():
'license': info.license,
'python_version': '.'.join((str(sys.version_info.major), str(sys.version_info.minor), str(sys.version_info.micro))),
'python_path': sys.executable,
'parser_count': len(all_parser_info()),
'parser_count': len(parser_mod_list()),
'standard_parser_count': len(standard_parser_mod_list()),
'streaming_parser_count': len(streaming_parser_mod_list()),
'plugin_parser_count': len(plugin_parser_mod_list()),
'parsers': all_parser_info()
'parsers': all_parser_info(show_hidden=True)
}
def helptext():
def helptext(show_hidden=False):
"""Return the help text with the list of parsers"""
parsers_string = parsers_text(indent=4, pad=20)
parsers_string = parsers_text(indent=4, pad=20, show_hidden=show_hidden)
options_string = options_text(indent=4, pad=20)
helptext_string = f'''\
jc converts the output of many commands and file-types to JSON or YAML
jc converts the output of many commands, file-types, and strings to JSON or YAML
Usage:
COMMAND | jc PARSER [OPTIONS]
or magic syntax:
Standard syntax:
COMMAND | jc [OPTIONS] PARSER
cat FILE | jc [OPTIONS] PARSER
echo STRING | jc [OPTIONS] PARSER
Magic syntax:
jc [OPTIONS] COMMAND
jc [OPTIONS] /proc/<path-to-procfile>
Parsers:
{parsers_string}
Options:
{options_string}
Examples:
Standard Syntax:
$ dig www.google.com | jc --dig --pretty
$ dig www.google.com | jc --pretty --dig
$ cat /proc/meminfo | jc --pretty --proc
Magic Syntax:
$ jc --pretty dig www.google.com
$ jc --pretty /proc/meminfo
Parser Documentation:
$ jc --help --dig
Show Hidden Parsers:
$ jc -hh
'''
return helptext_string
def help_doc(options):
def help_doc(options, show_hidden=False):
"""
Returns the parser documentation if a parser is found in the arguments,
otherwise the general help text is returned.
@ -228,7 +243,7 @@ def help_doc(options):
utils._safe_pager(doc_text)
return
utils._safe_print(helptext())
utils._safe_print(helptext(show_hidden=show_hidden))
return
@ -419,6 +434,11 @@ def magic_parser(args):
)
def open_text_file(path_string):
with open(path_string, 'r') as f:
return f.read()
def run_user_command(command):
"""
Use subprocess to run the user's command. Returns the STDOUT, STDERR,
@ -525,6 +545,7 @@ def main():
force_color = 'C' in options
mono = ('m' in options or bool(os.getenv('NO_COLOR'))) and not force_color
help_me = 'h' in options
verbose_help = options.count('h') > 1
pretty = 'p' in options
quiet = 'q' in options
ignore_exceptions = options.count('q') > 1
@ -552,7 +573,7 @@ def main():
sys.exit(0)
if help_me:
help_doc(sys.argv)
help_doc(sys.argv, show_hidden=verbose_help)
sys.exit(0)
if version_info:
@ -569,13 +590,38 @@ def main():
# if magic syntax used, try to run the command and error if it's not found, etc.
magic_stdout, magic_stderr, magic_exit_code = None, None, 0
run_command_str = ''
if run_command:
try:
run_command_str = shlex.join(run_command) # python 3.8+
except AttributeError:
run_command_str = ' '.join(run_command) # older python versions
if valid_command:
if run_command_str.startswith('/proc'):
try:
magic_found_parser = 'proc'
magic_stdout = open_text_file(run_command_str)
except OSError as e:
if debug:
raise
error_msg = os.strerror(e.errno)
utils.error_message([
f'"{run_command_str}" file could not be opened: {error_msg}.'
])
sys.exit(combined_exit_code(magic_exit_code, JC_ERROR_EXIT))
except Exception:
if debug:
raise
utils.error_message([
f'"{run_command_str}" file could not be opened. For details use the -d or -dd option.'
])
sys.exit(combined_exit_code(magic_exit_code, JC_ERROR_EXIT))
elif valid_command:
try:
magic_stdout, magic_stderr, magic_exit_code = run_user_command(run_command)
if magic_stderr:
@ -623,12 +669,7 @@ def main():
utils.error_message(['Missing or incorrect arguments. Use "jc -h" for help.'])
sys.exit(combined_exit_code(magic_exit_code, JC_ERROR_EXIT))
# check for input errors (pipe vs magic)
if not sys.stdin.isatty() and magic_stdout:
utils.error_message(['Piped data and Magic syntax used simultaneously. Use "jc -h" for help.'])
sys.exit(combined_exit_code(magic_exit_code, JC_ERROR_EXIT))
elif sys.stdin.isatty() and magic_stdout is None:
if sys.stdin.isatty() and magic_stdout is None:
utils.error_message(['Missing piped data. Use "jc -h" for help.'])
sys.exit(combined_exit_code(magic_exit_code, JC_ERROR_EXIT))

36
jc/lib.py
View File

@ -84,6 +84,22 @@ parsers = [
'pip-show',
'plist',
'postconf',
'proc',
'proc-buddyinfo',
'proc-consoles',
'proc-cpuinfo',
'proc-crypto',
'proc-devices',
'proc-diskstats',
'proc-driver-rtc',
'proc-filesystems',
'proc-interrupts',
'proc-iomem',
'proc-ioports',
'proc-loadavg',
'proc-locks',
'proc-meminfo',
'proc-modules',
'ps',
'route',
'rpm-qi',
@ -339,7 +355,9 @@ def parser_info(parser_mod_name: str, documentation: bool = False) -> Dict:
return info_dict
def all_parser_info(documentation: bool = False) -> List[Dict]:
def all_parser_info(documentation: bool = False,
show_hidden: bool = False
) -> List[Dict]:
"""
Returns a list of dictionaries that includes metadata for all parser
modules.
@ -347,8 +365,22 @@ def all_parser_info(documentation: bool = False) -> List[Dict]:
Parameters:
documentation: (boolean) include parser docstrings if True
show_hidden: (boolean) also show parsers marked as hidden
in their info metadata.
"""
return [parser_info(p, documentation=documentation) for p in parsers]
temp_list = [parser_info(p, documentation=documentation) for p in parsers]
p_list = []
if show_hidden:
p_list = temp_list
else:
for item in temp_list:
if not item.get('hidden', None):
p_list.append(item)
return p_list
def get_help(parser_mod_name: str) -> None:
"""

2
jc/parsers/lsof.py
View File

@ -124,7 +124,7 @@ class info():
description = '`lsof` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
compatible = ['linux', 'darwin', 'aix', 'freebsd']
magic_commands = ['lsof']

277
jc/parsers/proc.py Normal file
View File

@ -0,0 +1,277 @@
"""jc - JSON Convert Proc file output parser
This parser automatically identifies the Proc file and calls the
corresponding parser to peform the parsing.
Magic syntax for converting `/proc` files is also supported by running
`jc /proc/<path to file>`. Any `jc` options must be specified before the
`/proc` path.
specific Proc file parsers can also be called directly, if desired and have
a naming convention of `proc-<name>` (cli) or `proc_<name>` (module).
Usage (cli):
$ cat /proc/meminfo | jc --proc
or
$ jc /proc/meminfo
or
$ cat /proc/meminfo | jc --proc-memifno
Usage (module):
import jc
result = jc.parse('proc', proc_file)
Schema:
See the specific Proc parser for the schema:
$ jc --help --proc-<name>
For example:
$ jc --help --proc-meminfo
Specific Proc file parser names can be found with `jc -hh` or `jc -a`.
Schemas can also be found online at:
https://kellyjonbrazil.github.io/jc/docs/parsers/proc_<name>
For example:
https://kellyjonbrazil.github.io/jc/docs/parsers/proc_meminfo
Examples:
$ cat /proc/modules | jc --proc -p
[
{
"module": "binfmt_misc",
"size": 24576,
"used": 1,
"used_by": [],
"status": "Live",
"location": "0xffffffffc0ab4000"
},
{
"module": "vsock_loopback",
"size": 16384,
"used": 0,
"used_by": [],
"status": "Live",
"location": "0xffffffffc0a14000"
},
{
"module": "vmw_vsock_virtio_transport_common",
"size": 36864,
"used": 1,
"used_by": [
"vsock_loopback"
],
"status": "Live",
"location": "0xffffffffc0a03000"
},
...
]
$ proc_modules | jc --proc_modules -p -r
[
{
"module": "binfmt_misc",
"size": "24576",
"used": "1",
"used_by": [],
"status": "Live",
"location": "0xffffffffc0ab4000"
},
{
"module": "vsock_loopback",
"size": "16384",
"used": "0",
"used_by": [],
"status": "Live",
"location": "0xffffffffc0a14000"
},
{
"module": "vmw_vsock_virtio_transport_common",
"size": "36864",
"used": "1",
"used_by": [
"vsock_loopback"
],
"status": "Live",
"location": "0xffffffffc0a03000"
},
...
]
"""
import re
import importlib
from typing import List, Dict
import jc.utils
from jc.exceptions import ParseError
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
__version__ = info.version
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.input_type_check(data)
if jc.utils.has_data(data):
# signatures
buddyinfo_p = re.compile(r'^Node \d+, zone\s+\w+\s+(?:\d+\s+){11}\n')
consoles_p = re.compile(r'^\w+\s+[\-WUR]{3} \([ECBpba ]+\)\s+\d+:\d+\n')
cpuinfo_p = re.compile(r'^processor\t+: \d+.*bogomips\t+: \d+.\d\d\n', re.DOTALL)
crypto_p = re.compile(r'^name\s+:.*\ndriver\s+:.*\nmodule\s+:.*\n')
devices_p = re.compile(r'^Character devices:\n\s+\d+ .*\n')
diskstats_p = re.compile(r'^\s*\d+\s+\d\s\w+\s(?:\d+\s){10,16}\d+\n')
filesystems_p = re.compile(r'^(?:(?:nodev\t|\t)\w+\n){3}')
interrupts_p = re.compile(r'^\s+(?:CPU\d+ +)+\n\s*\d+:\s+\d+')
iomem_p = re.compile(r'^00000000-[0-9a-f]{8} : .*\n[0-9a-f]{8}-[0-9a-f]{8} : ')
ioports_p = re.compile(r'^0000-[0-9a-f]{4} : .*\n\s*0000-[0-9a-f]{4} : ')
loadavg_p = re.compile(r'^\d+.\d\d \d+.\d\d \d+.\d\d \d+/\d+ \d+$')
locks_p = re.compile(r'^\d+: (?:POSIX|FLOCK|OFDLCK)\s+(?:ADVISORY|MANDATORY)\s+(?:READ|WRITE) ')
meminfo_p = re.compile(r'^MemTotal:.*\nMemFree:.*\nMemAvailable:.*\n')
modules_p = re.compile(r'^\w+ \d+ \d+ (?:-|\w+,).*0x[0-9a-f]{16}\n')
mtrr_p = re.compile(r'^reg\d+: base=0x[0-9a-f]+ \(')
pagetypeinfo_p = re.compile(r'^Page block order:\s+\d+\nPages per block:\s+\d+\n\n')
partitions_p = re.compile(r'^major minor #blocks name\n\n\s*\d+\s+\d+\s+\d+ \w+\n')
slabinfo_p = re.compile(r'^slabinfo - version: \d+.\d+\n')
softirqs_p = re.compile(r'^\s+(CPU\d+\s+)+\n\s+HI:\s+\d')
stat_p = re.compile(r'^cpu\s+(?: \d+){10}.*intr ', re.DOTALL)
swaps_p = re.compile(r'^Filename\t\t\t\tType\t\tSize\t\tUsed\t\tPriority\n')
uptime_p = re.compile(r'^\d+.\d\d \d+.\d\d$')
version_p = re.compile(r'^.+\sversion\s[^\n]+$')
vmallocinfo_p = re.compile(r'^0x[0-9a-f]{16}-0x[0-9a-f]{16}\s+\d+ \w+\+\w+/\w+ ')
vmstat_p = re.compile(r'nr_free_pages \d+\n.* \d$', re.DOTALL)
zoneinfo_p = re.compile(r'^Node \d+, zone\s+\w+\n')
driver_rtc_p = re.compile(r'^rtc_time\t: .*\nrtc_date\t: .*\nalrm_time\t: .*\n')
net_arp_p = re.compile(r'^IP address\s+HW type\s+Flags\s+HW address\s+Mask\s+Device\n')
net_dev_p = re.compile(r'^Inter-\|\s+Receive\s+\|\s+Transmit\n')
net_dev_mcast_p = re.compile(r'^\d+\s+\w+\s+\d+\s+\d+\s+[0-9a-f]{12}')
net_if_inet6_p = re.compile(r'^[0-9a-f]{32} \d\d \d\d \d\d \d\d\s+\w+')
net_igmp_p = re.compile(r'^Idx\tDevice\s+:\s+Count\s+Querier\tGroup\s+Users\s+Timer\tReporter\n')
net_igmp6_p = re.compile(r'^\d+\s+\w+\s+[0-9a-f]{32}\s+\d+\s+[0-9A-F]{8}\s+\d+')
net_ipv6_route_p = re.compile(r'^[0-9a-f]{32} \d\d [0-9a-f]{32} \d\d [0-9a-f]{32} (?:[0-9a-f]{8} ){4}\s+\w+')
net_netlink_p = re.compile(r'^sk\s+Eth Pid\s+Groups\s+Rmem\s+Wmem')
net_netstat_p = re.compile(r'^TcpExt: SyncookiesSent SyncookiesRecv SyncookiesFailed')
net_packet_p = re.compile(r'^sk RefCnt Type Proto Iface R Rmem User Inode\n')
net_protocols_p = re.compile(r'^protocol size sockets memory press maxhdr slab module cl co di ac io in de sh ss gs se re sp bi br ha uh gp em\n')
net_route_p = re.compile(r'^Iface\tDestination\tGateway \tFlags\tRefCnt\tUse\tMetric\tMask\t\tMTU\tWindow\tIRTT\s+\n')
net_unix_p = re.compile(r'^Num RefCount Protocol Flags Type St Inode Path\n')
pid_status_p = re.compile(r'^Name:\t.+\nUmask:\t\d+\nState:\t.+\nTgid:\t\d+\n')
pid_statm_p = re.compile(r'^\d+ \d+ \d+\s\d+\s\d+\s\d+\s\d+$')
pid_stat_p = re.compile(r'^\d+ \(.{1,16}\) \w \d+ \d+ \d+ \d+ -?\d+ (?:\d+ ){43}\d+$')
pid_smaps_p = re.compile(r'^[0-9a-f]{12}-[0-9a-f]{12} [rwxsp\-]{4} [0-9a-f]{8} [0-9a-f]{2}:[0-9a-f]{2} \d+ [^\n]+\nSize:\s+\d+ \S\S\n')
pid_maps_p = re.compile(r'^[0-9a-f]{12}-[0-9a-f]{12} [rwxsp\-]{4} [0-9a-f]{8} [0-9a-f]{2}:[0-9a-f]{2} \d+ ')
pid_numa_maps_p = re.compile(r'^[a-f0-9]{12} default [^\n]+\n')
pid_io_p = re.compile(r'^rchar: \d+\nwchar: \d+\nsyscr: \d+\n')
pid_mountinfo_p = re.compile(r'^\d+ \d+ \d+:\d+ /.+\n')
pid_fdinfo_p = re.compile(r'^pos:\t\d+\nflags:\t\d+\nmnt_id:\t\d+\n')
scsi_device_info = re.compile(r"^'\w+' '.+' 0x\d+")
scsi_scsi_p = re.compile(r'^Attached devices:\nHost: \w+ ')
procmap = {
buddyinfo_p: 'proc_buddyinfo',
consoles_p: 'proc_consoles',
cpuinfo_p: 'proc_cpuinfo',
crypto_p: 'proc_crypto',
devices_p: 'proc_devices',
diskstats_p: 'proc_diskstats',
filesystems_p: 'proc_filesystems',
interrupts_p: 'proc_interrupts',
iomem_p: 'proc_iomem',
ioports_p: 'proc_ioports',
loadavg_p: 'proc_loadavg',
locks_p: 'proc_locks',
meminfo_p: 'proc_meminfo',
modules_p: 'proc_modules',
mtrr_p: 'proc_mtrr',
pagetypeinfo_p: 'proc_pagetypeinfo',
partitions_p: 'proc_partitions',
slabinfo_p: 'proc_slabinfo',
softirqs_p: 'proc_softirqs',
stat_p: 'proc_stat',
swaps_p: 'proc_swaps',
uptime_p: 'proc_uptime',
version_p: 'proc_version',
vmallocinfo_p: 'proc_vmallocinfo',
vmstat_p: 'proc_vmstat',
zoneinfo_p: 'proc_zoneinfo',
driver_rtc_p: 'proc_driver_rtc',
net_arp_p: 'proc_net_arp',
net_dev_p: 'proc_net_dev',
net_if_inet6_p: 'proc_net_if_inet6',
net_igmp_p: 'proc_net_igmp',
net_igmp6_p: 'proc_net_igmp6',
net_netlink_p: 'proc_net_netlink',
net_netstat_p: 'proc_net_netstat',
net_packet_p: 'proc_net_packet',
net_protocols_p: 'proc_net_protocols',
net_route_p: 'proc_net_route',
net_unix_p: 'proc_net_unix',
net_ipv6_route_p: 'proc_net_ipv6_route', # before net_dev_mcast
net_dev_mcast_p: 'proc_net_dev_mcast', # after net_ipv6_route
pid_fdinfo_p: 'proc_pid_fdinfo',
pid_io_p: 'proc_pid_io',
pid_mountinfo_p: 'proc_pid_mountinfo',
pid_numa_maps_p: 'proc_pid_numa_maps',
pid_stat_p: 'proc_pid_stat',
pid_statm_p: 'proc_pid_statm',
pid_status_p: 'proc_pid_status',
pid_smaps_p: 'proc_pid_smaps', # before pid_maps
pid_maps_p: 'proc_pid_maps', # after pid_smaps
scsi_device_info: 'proc_scsi_device_info',
scsi_scsi_p: 'proc_scsi_scsi'
}
for reg_pattern, parse_mod in procmap.items():
if reg_pattern.search(data):
try:
procparser = importlib.import_module('jc.parsers.' + parse_mod)
return procparser.parse(data, quiet=quiet, raw=raw)
except ModuleNotFoundError:
raise ParseError('Proc file type not yet implemented.')
raise ParseError('Proc file could not be identified.')

179
jc/parsers/proc_buddyinfo.py Normal file
View File

@ -0,0 +1,179 @@
"""jc - JSON Convert `/proc/buddyinfo` file parser
Usage (cli):
$ cat /proc/buddyinfo | jc --proc
or
$ jc /proc/buddyinfo
or
$ cat /proc/buddyinfo | jc --proc-buddyinfo
Usage (module):
import jc
result = jc.parse('proc', proc_buddyinfo_file)
or
import jc
result = jc.parse('proc_buddyinfo', proc_buddyinfo_file)
Schema:
All values are integers.
[
{
"node": integer,
"zone": string,
"free_chunks": [
integer # [0]
]
}
]
[0] array index correlates to the Order number.
E.g. free_chunks[0] is the value for Order 0
Examples:
$ cat /proc/buddyinfo | jc --proc -p
[
{
"node": 0,
"zone": "DMA",
"free_chunks": [
0,
0,
0,
1,
1,
1,
1,
1,
0,
1,
3
]
},
{
"node": 0,
"zone": "DMA32",
"free_chunks": [
78,
114,
82,
52,
38,
25,
13,
9,
3,
4,
629
]
},
{
"node": 0,
"zone": "Normal",
"free_chunks": [
0,
22,
8,
10,
1,
1,
2,
11,
13,
0,
0
]
}
]
"""
from typing import List, Dict
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/buddyinfo` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: List[Dict]) -> List[Dict]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured to conform to the schema.
"""
int_list = {'node'}
for entry in proc_data:
for key in entry:
if key in int_list:
entry[key] = jc.utils.convert_to_int(entry[key])
if 'free_chunks' in entry:
entry['free_chunks'] = [int(x) for x in entry['free_chunks']]
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List = []
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
buddy_list = line.split()
raw_output.append(
{
'node': buddy_list[1][:-1],
'zone': buddy_list[3],
'free_chunks': buddy_list[4:]
}
)
return raw_output if raw else _process(raw_output)

188
jc/parsers/proc_consoles.py Normal file
View File

@ -0,0 +1,188 @@
"""jc - JSON Convert `/proc/consoles` file parser
Usage (cli):
$ cat /proc/consoles | jc --proc
or
$ jc /proc/consoles
or
$ cat /proc/consoles | jc --proc-consoles
Usage (module):
import jc
result = jc.parse('proc', proc_consoles_file)
or
import jc
result = jc.parse('proc_consoles', proc_consoles_file)
Schema:
[
{
"device": string,
"operations": string,
"operations_list": [
string # [0]
],
"flags": string,
"flags_list": [
string # [1]
],
"major": integer,
"minor": integer
}
]
[0] Values: read, write, unblank
[1] Values: enabled, preferred, primary boot, prink buffer,
braille device, safe when CPU offline
Examples:
$ cat /proc/consoles | jc --proc -p
[
{
"device": "tty0",
"operations": "-WU",
"operations_list": [
"write",
"unblank"
],
"flags": "ECp",
"flags_list": [
"enabled",
"preferred",
"printk buffer"
],
"major": 4,
"minor": 7
},
{
"device": "ttyS0",
"operations": "-W-",
"operations_list": [
"write"
],
"flags": "Ep",
"flags_list": [
"enabled",
"printk buffer"
],
"major": 4,
"minor": 64
}
]
"""
import shlex
from typing import List, Dict
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/consoles` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: List[Dict]) -> List[Dict]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured to conform to the schema.
"""
int_list = {'major', 'minor'}
for entry in proc_data:
for key in entry:
if key in int_list:
entry[key] = jc.utils.convert_to_int(entry[key])
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List = []
operations_map = {
'R': 'read',
'W': 'write',
'U': 'unblank'
}
flags_map = {
'E': 'enabled',
'C': 'preferred',
'B': 'primary boot',
'p': 'printk buffer',
'b': 'braille device',
'a': 'safe when CPU offline'
}
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
# since parens are acting like quotation marks, use shlex.split()
# after converting parens to quotes.
line = line.replace('(', '"'). replace(')', '"')
device, operations, flags, maj_min = shlex.split(line)
operations_str = operations.replace('-', '')
operations_list = [operations_map[i] for i in operations_str]
flags_str = flags.replace (' ', '')
flags_list = [flags_map[i] for i in flags_str]
raw_output.append(
{
'device': device,
'operations': operations,
'operations_list': operations_list,
'flags': flags,
'flags_list': flags_list,
'major': maj_min.split(':')[0],
'minor': maj_min.split(':')[1]
}
)
return raw_output if raw else _process(raw_output)

329
jc/parsers/proc_cpuinfo.py Normal file
View File

@ -0,0 +1,329 @@
"""jc - JSON Convert `/proc/cpuinfo` file parser
Usage (cli):
$ cat /proc/cpuinfo | jc --proc
or
$ jc /proc/cpuinfo
or
$ cat /proc/cpuinfo | jc --proc-cpuinfo
Usage (module):
import jc
result = jc.parse('proc', proc_cpuinfo_file)
or
import jc
result = jc.parse('proc_cpuinfo', proc_cpuinfo_file)
Schema:
Integer, float, and boolean ("yes"/"no") conversions are attempted. Blank
strings are converted to `null`.
"Well-known" keys like `cache size`, `address types`, `bugs`, and `flags`
are processed into sensible data types. (see below)
If this is not desired, then use the `--raw` (CLI) or `raw=True` (Module)
option.
[
{
"processor": integer,
"address sizes": string,
"address_size_physical": integer, # in bits
"address_size_virtual": integer, # in bits
"cache size": string,
"cache_size_num": integer,
"cache_size_unit": string,
"flags": [
string
],
"bugs": [
string
],
"bogomips": float,
<key>: string/int/float/boolean/null
}
]
Examples:
$ cat /proc/cpuinfo | jc --proc -p
[
{
"processor": 0,
"vendor_id": "GenuineIntel",
"cpu family": 6,
"model": 142,
"model name": "Intel(R) Core(TM) i5-8279U CPU @ 2.40GHz",
"stepping": 10,
"cpu MHz": 2400.0,
"cache size": "6144 KB",
"physical id": 0,
"siblings": 1,
"core id": 0,
"cpu cores": 1,
"apicid": 0,
"initial apicid": 0,
"fpu": true,
"fpu_exception": true,
"cpuid level": 22,
"wp": true,
"bogomips": 4800.0,
"clflush size": 64,
"cache_alignment": 64,
"address sizes": "45 bits physical, 48 bits virtual",
"power management": null,
"address_size_physical": 45,
"address_size_virtual": 48,
"cache_size_num": 6144,
"cache_size_unit": "KB",
"flags": [
"fpu",
"vme",
"de",
"pse",
"tsc",
"msr",
"pae",
"mce",
"cx8",
"apic",
"sep",
"mtrr",
"pge",
"mca",
"cmov",
"pat",
"pse36",
"clflush",
"mmx",
"fxsr",
"sse",
"sse2",
"ss",
"syscall",
"nx",
"pdpe1gb",
"rdtscp",
"lm",
"constant_tsc",
"arch_perfmon",
"nopl",
"xtopology",
"tsc_reliable",
"nonstop_tsc",
"cpuid",
"pni",
"pclmulqdq",
"ssse3",
"fma",
"cx16",
"pcid",
"sse4_1",
"sse4_2",
"x2apic",
"movbe",
"popcnt",
"tsc_deadline_timer",
"aes",
"xsave",
"avx",
"f16c",
"rdrand",
"hypervisor",
"lahf_lm",
"abm",
"3dnowprefetch",
"cpuid_fault",
"invpcid_single",
"pti",
"ssbd",
"ibrs",
"ibpb",
"stibp",
"fsgsbase",
"tsc_adjust",
"bmi1",
"avx2",
"smep",
"bmi2",
"invpcid",
"rdseed",
"adx",
"smap",
"clflushopt",
"xsaveopt",
"xsavec",
"xgetbv1",
"xsaves",
"arat",
"md_clear",
"flush_l1d",
"arch_capabilities"
],
"bugs": [
"cpu_meltdown",
"spectre_v1",
"spectre_v2",
"spec_store_bypass",
"l1tf",
"mds",
"swapgs",
"itlb_multihit",
"srbds"
]
},
...
]
$ proc_cpuinfo | jc --proc_cpuinfo -p -r
[
{
"processor": "0",
"vendor_id": "GenuineIntel",
"cpu family": "6",
"model": "142",
"model name": "Intel(R) Core(TM) i5-8279U CPU @ 2.40GHz",
"stepping": "10",
"cpu MHz": "2400.000",
"cache size": "6144 KB",
"physical id": "0",
"siblings": "1",
"core id": "0",
"cpu cores": "1",
"apicid": "0",
"initial apicid": "0",
"fpu": "yes",
"fpu_exception": "yes",
"cpuid level": "22",
"wp": "yes",
"flags": "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge ...",
"bugs": "cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass ...",
"bogomips": "4800.00",
"clflush size": "64",
"cache_alignment": "64",
"address sizes": "45 bits physical, 48 bits virtual",
"power management": ""
},
...
]
"""
from typing import List, Dict
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/cpuinfo` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: List[Dict]) -> List[Dict]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured to conform to the schema.
"""
for entry in proc_data:
for key in entry:
if entry[key] == '':
entry[key] = None
try:
entry[key] = int(entry[key])
except Exception:
pass
try:
if isinstance(entry[key], str) and (entry[key] == 'yes' or entry[key] == 'no'):
entry[key] = jc.utils.convert_to_bool(entry[key])
except Exception:
pass
try:
if isinstance(entry[key], str) and '.' in entry[key]:
entry[key] = float(entry[key])
except Exception:
pass
if 'address sizes' in entry:
phy = int(entry['address sizes'].split()[0])
virt = int(entry['address sizes'].split()[3])
entry['address_size_physical'] = phy
entry['address_size_virtual'] = virt
if 'cache size' in entry:
cache_size_int, unit = entry['cache size'].split()
entry['cache_size_num'] = int(cache_size_int)
entry['cache_size_unit'] = unit
if 'flags' in entry:
entry['flags'] = entry['flags'].split()
if 'bugs' in entry:
entry['bugs'] = entry['bugs'].split()
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List = []
output_line: Dict = {}
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
if line.startswith('processor'):
if output_line:
raw_output.append(output_line)
output_line = {}
key, val = line.split(':', maxsplit=1)
output_line[key.strip()] = val.strip()
if output_line:
raw_output.append(output_line)
return raw_output if raw else _process(raw_output)

196
jc/parsers/proc_crypto.py Normal file
View File

@ -0,0 +1,196 @@
"""jc - JSON Convert `/proc/crypto` file parser
Usage (cli):
$ cat /proc/crypto | jc --proc
or
$ jc /proc/crypto
or
$ cat /proc/crypto | jc --proc-crypto
Usage (module):
import jc
result = jc.parse('proc', proc_crypto_file)
or
import jc
result = jc.parse('proc_crypto', proc_crypto_file)
Schema:
"Well-known" keys like `priority` and `refcnt` are converted to integers.
Also, keynames ending in "size" are converted to integers.
If this is not desired, then use the `--raw` (CLI) or `raw=True` (Module)
option.
[
{
"name": string,
"driver": string,
"module": string,
"priority": integer,
"refcnt": integer,
"selftest": string,
"internal": string,
"type": string,
"*size": integer
}
]
Examples:
$ cat /proc/crypto | jc --proc -p
[
{
"name": "ecdh",
"driver": "ecdh-generic",
"module": "ecdh_generic",
"priority": 100,
"refcnt": 1,
"selftest": "passed",
"internal": "no",
"type": "kpp"
},
{
"name": "blake2b-512",
"driver": "blake2b-512-generic",
"module": "blake2b_generic",
"priority": 100,
"refcnt": 1,
"selftest": "passed",
"internal": "no",
"type": "shash",
"blocksize": 128,
"digestsize": 64
},
...
]
$ proc_crypto | jc --proc_crypto -p -r
[
{
"name": "ecdh",
"driver": "ecdh-generic",
"module": "ecdh_generic",
"priority": "100",
"refcnt": "1",
"selftest": "passed",
"internal": "no",
"type": "kpp"
},
{
"name": "blake2b-512",
"driver": "blake2b-512-generic",
"module": "blake2b_generic",
"priority": "100",
"refcnt": "1",
"selftest": "passed",
"internal": "no",
"type": "shash",
"blocksize": "128",
"digestsize": "64"
},
{
"name": "blake2b-384",
"driver": "blake2b-384-generic",
"module": "blake2b_generic",
"priority": "100",
"refcnt": "1",
"selftest": "passed",
"internal": "no",
"type": "shash",
"blocksize": "128",
"digestsize": "48"
},
...
]
"""
from typing import List, Dict
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/crypto` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: List[Dict]) -> List[Dict]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured to conform to the schema.
"""
int_list = {'priority', 'refcnt'}
for entry in proc_data:
for key in entry:
if key in int_list or key.endswith('size'):
try:
entry[key] = int(entry[key])
except Exception:
pass
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List = []
output_line: Dict = {}
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
if line.startswith('name'):
if output_line:
raw_output.append(output_line)
output_line = {}
key, val = line.split(':', maxsplit=1)
output_line[key.strip()] = val.strip()
if output_line:
raw_output.append(output_line)
return raw_output if raw else _process(raw_output)

166
jc/parsers/proc_devices.py Normal file
View File

@ -0,0 +1,166 @@
"""jc - JSON Convert `/proc/devices` file parser
Usage (cli):
$ cat /proc/devices | jc --proc
or
$ jc /proc/devices
or
$ cat /proc/devices | jc --proc-devices
Usage (module):
import jc
result = jc.parse('proc', proc_devices_file)
or
import jc
result = jc.parse('proc_devices', proc_devices_file)
Schema:
Since devices can be members of multiple groups, the value for each device
is a list.
{
"character": {
"<device number>": [
string
]
},
"block": {
"<device number>": [
string
]
}
}
Examples:
$ cat /proc/devices | jc --proc -p
{
"character": {
"1": [
"mem"
],
"4": [
"/dev/vc/0",
"tty",
"ttyS"
],
"5": [
"/dev/tty",
"/dev/console",
"/dev/ptmx",
"ttyprintk"
],
"block": {
"7": [
"loop"
],
"8": [
"sd"
],
"9": [
"md"
]
}
}
"""
from typing import Dict
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/devices` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: Dict) -> Dict:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (Dictionary) raw structured data to process
Returns:
Dictionary. Structured to conform to the schema.
"""
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> Dict:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: Dict = {}
character: Dict = {}
block: Dict = {}
section = ''
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
if 'Character devices:' in line:
section = 'character'
continue
if 'Block devices:' in line:
section = 'block'
continue
devnum, group = line.split()
if section == 'character':
if not devnum in character:
character[devnum] = []
character[devnum].append(group)
continue
if section == 'block':
if not devnum in block:
block[devnum] = []
block[devnum].append(group)
continue
if character or block:
raw_output = {
'character': character,
'block': block
}
return raw_output if raw else _process(raw_output)

248
jc/parsers/proc_diskstats.py Normal file
View File

@ -0,0 +1,248 @@
"""jc - JSON Convert `/proc/diskstats` file parser
Usage (cli):
$ cat /proc/diskstats | jc --proc
or
$ jc /proc/diskstats
or
$ cat /proc/diskstats | jc --proc-diskstats
Usage (module):
import jc
result = jc.parse('proc', proc_diskstats_file)
or
import jc
result = jc.parse('proc_diskstats', proc_diskstats_file)
Schema:
[
{
"maj": integer,
"min": integer,
"device": string,
"reads_completed": integer,
"reads_merged": integer,
"sectors_read": integer,
"read_time_ms": integer,
"writes_completed": integer,
"writes_merged": integer,
"sectors_written": integer,
"write_time_ms": integer,
"io_in_progress": integer,
"io_time_ms": integer,
"weighted_io_time_ms": integer,
"discards_completed_successfully": integer,
"discards_merged": integer,
"sectors_discarded": integer,
"discarding_time_ms": integer,
"flush_requests_completed_successfully": integer,
"flushing_time_ms": integer
}
]
Examples:
$ cat /proc/diskstats | jc --proc -p
[
{
"maj": 7,
"min": 0,
"device": "loop0",
"reads_completed": 48,
"reads_merged": 0,
"sectors_read": 718,
"read_time_ms": 19,
"writes_completed": 0,
"writes_merged": 0,
"sectors_written": 0,
"write_time_ms": 0,
"io_in_progress": 0,
"io_time_ms": 36,
"weighted_io_time_ms": 19,
"discards_completed_successfully": 0,
"discards_merged": 0,
"sectors_discarded": 0,
"discarding_time_ms": 0,
"flush_requests_completed_successfully": 0,
"flushing_time_ms": 0
},
{
"maj": 7,
"min": 1,
"device": "loop1",
"reads_completed": 41,
"reads_merged": 0,
"sectors_read": 688,
"read_time_ms": 17,
"writes_completed": 0,
"writes_merged": 0,
"sectors_written": 0,
"write_time_ms": 0,
"io_in_progress": 0,
"io_time_ms": 28,
"weighted_io_time_ms": 17,
"discards_completed_successfully": 0,
"discards_merged": 0,
"sectors_discarded": 0,
"discarding_time_ms": 0,
"flush_requests_completed_successfully": 0,
"flushing_time_ms": 0
},
...
]
$ proc_diskstats | jc --proc_diskstats -p -r
[
{
"maj": "7",
"min": "0",
"device": "loop0",
"reads_completed": "48",
"reads_merged": "0",
"sectors_read": "718",
"read_time_ms": "19",
"writes_completed": "0",
"writes_merged": "0",
"sectors_written": "0",
"write_time_ms": "0",
"io_in_progress": "0",
"io_time_ms": "36",
"weighted_io_time_ms": "19",
"discards_completed_successfully": "0",
"discards_merged": "0",
"sectors_discarded": "0",
"discarding_time_ms": "0",
"flush_requests_completed_successfully": "0",
"flushing_time_ms": "0"
},
{
"maj": "7",
"min": "1",
"device": "loop1",
"reads_completed": "41",
"reads_merged": "0",
"sectors_read": "688",
"read_time_ms": "17",
"writes_completed": "0",
"writes_merged": "0",
"sectors_written": "0",
"write_time_ms": "0",
"io_in_progress": "0",
"io_time_ms": "28",
"weighted_io_time_ms": "17",
"discards_completed_successfully": "0",
"discards_merged": "0",
"sectors_discarded": "0",
"discarding_time_ms": "0",
"flush_requests_completed_successfully": "0",
"flushing_time_ms": "0"
},
{
"maj": "7",
"min": "2",
"device": "loop2",
"reads_completed": "119",
"reads_merged": "0",
"sectors_read": "2956",
"read_time_ms": "18",
"writes_completed": "0",
"writes_merged": "0",
"sectors_written": "0",
"write_time_ms": "0",
"io_in_progress": "0",
"io_time_ms": "56",
"weighted_io_time_ms": "18",
"discards_completed_successfully": "0",
"discards_merged": "0",
"sectors_discarded": "0",
"discarding_time_ms": "0",
"flush_requests_completed_successfully": "0",
"flushing_time_ms": "0"
},
...
]
"""
from typing import List, Dict
import jc.utils
from jc.parsers.universal import simple_table_parse
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/diskstats` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: List[Dict]) -> List[Dict]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured to conform to the schema.
"""
for entry in proc_data:
for key in entry:
if key != 'device':
entry[key] = int(entry[key])
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List = []
if jc.utils.has_data(data):
header = (
'maj min device reads_completed reads_merged sectors_read read_time_ms '
'writes_completed writes_merged sectors_written write_time_ms io_in_progress '
'io_time_ms weighted_io_time_ms discards_completed_successfully discards_merged '
'sectors_discarded discarding_time_ms flush_requests_completed_successfully '
'flushing_time_ms\n'
)
data = header + data
cleandata = filter(None, data.splitlines())
raw_output = simple_table_parse(cleandata)
return raw_output if raw else _process(raw_output)

173
jc/parsers/proc_driver_rtc.py Normal file
View File

@ -0,0 +1,173 @@
"""jc - JSON Convert `/proc/driver_rtc` file parser
Usage (cli):
$ cat /proc/driver_rtc | jc --proc
or
$ jc /proc/driver_rtc
or
$ cat /proc/driver_rtc | jc --proc-driver-rtc
Usage (module):
import jc
result = jc.parse('proc', proc_driver_rtc_file)
or
import jc
result = jc.parse('proc_driver_rtc', proc_driver_rtc_file)
Schema:
"yes" and "no" values are converted to `true`/`false`. Integer conversions
are attempted. If you do not want this behavior, then use `--raw` (cli) or
`raw=True` (module).
{
"rtc_time": string,
"rtc_date": string,
"alrm_time": string,
"alrm_date": string,
"alarm_IRQ": boolean,
"alrm_pending": boolean,
"update IRQ enabled": boolean,
"periodic IRQ enabled": boolean,
"periodic IRQ frequency": integer,
"max user IRQ frequency": integer,
"24hr": boolean,
"periodic_IRQ": boolean,
"update_IRQ": boolean,
"HPET_emulated": boolean,
"BCD": boolean,
"DST_enable": boolean,
"periodic_freq": integer,
"batt_status": string
}
Examples:
$ cat /proc/driver_rtc | jc --proc -p
{
"rtc_time": "16:09:21",
"rtc_date": "2022-09-03",
"alrm_time": "00:00:00",
"alrm_date": "2022-09-03",
"alarm_IRQ": false,
"alrm_pending": false,
"update IRQ enabled": false,
"periodic IRQ enabled": false,
"periodic IRQ frequency": 1024,
"max user IRQ frequency": 64,
"24hr": true,
"periodic_IRQ": false,
"update_IRQ": false,
"HPET_emulated": true,
"BCD": true,
"DST_enable": false,
"periodic_freq": 1024,
"batt_status": "okay"
}
$ cat /proc/driver_rtc | jc --proc -p -r
{
"rtc_time": "16:09:21",
"rtc_date": "2022-09-03",
"alrm_time": "00:00:00",
"alrm_date": "2022-09-03",
"alarm_IRQ": "no",
"alrm_pending": "no",
"update IRQ enabled": "no",
"periodic IRQ enabled": "no",
"periodic IRQ frequency": "1024",
"max user IRQ frequency": "64",
"24hr": "yes",
"periodic_IRQ": "no",
"update_IRQ": "no",
"HPET_emulated": "yes",
"BCD": "yes",
"DST_enable": "no",
"periodic_freq": "1024",
"batt_status": "okay"
}
"""
from typing import Dict
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/driver_rtc` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: Dict) -> Dict:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (Dictionary) raw structured data to process
Returns:
Dictionary. Structured to conform to the schema.
"""
for key, val in proc_data.items():
try:
proc_data[key] = int(val)
except:
pass
if val == 'yes':
proc_data[key] = True
if val == 'no':
proc_data[key] = False
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> Dict:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: Dict = {}
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
split_line = line.split(':', maxsplit=1)
key = split_line[0].strip()
val = split_line[1].rsplit(maxsplit=1)[0]
raw_output[key] = val
return raw_output if raw else _process(raw_output)

122
jc/parsers/proc_filesystems.py Normal file
View File

@ -0,0 +1,122 @@
"""jc - JSON Convert `/proc/filesystems` file parser
Usage (cli):
$ cat /proc/filesystems | jc --proc
or
$ jc /proc/filesystems
or
$ cat /proc/filesystems | jc --proc-filesystems
Usage (module):
import jc
result = jc.parse('proc', proc_filesystems_file)
or
import jc
result = jc.parse('proc_filesystems', proc_filesystems_file)
Schema:
[
{
"filesystem": string,
"nodev": boolean
}
]
Examples:
$ cat /proc/filesystems | jc --proc -p
[
{
"filesystem": "sysfs",
"nodev": true
},
{
"filesystem": "tmpfs",
"nodev": true
},
{
"filesystem": "bdev",
"nodev": true
},
...
]
"""
from typing import List, Dict
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/filesystems` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: List[Dict]) -> List[Dict]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured to conform to the schema.
"""
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List = []
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
split_line = line.split()
output_line = {'filesystem': split_line[-1]}
if len(split_line) == 2:
output_line['nodev'] = True # type: ignore
else:
output_line['nodev'] = False # type: ignore
raw_output.append(output_line)
return raw_output if raw else _process(raw_output)

207
jc/parsers/proc_interrupts.py Normal file
View File

@ -0,0 +1,207 @@
"""jc - JSON Convert `/proc/interrupts` file parser
Usage (cli):
$ cat /proc/interrupts | jc --proc
or
$ jc /proc/interrupts
or
$ cat /proc/interrupts | jc --proc-interrupts
Usage (module):
import jc
result = jc.parse('proc', proc_interrupts_file)
or
import jc
result = jc.parse('proc_interrupts', proc_interrupts_file)
Schema:
[
{
"irq": string,
"cpu_num": integer,
"interrupts": [
integer
],
"type": string,
"device": [
string
]
}
]
Examples:
$ cat /proc/interrupts | jc --proc -p
[
{
"irq": "0",
"cpu_num": 2,
"interrupts": [
18,
0
],
"type": "IO-APIC",
"device": [
"2-edge",
"timer"
]
},
{
"irq": "1",
"cpu_num": 2,
"interrupts": [
0,
73
],
"type": "IO-APIC",
"device": [
"1-edge",
"i8042"
]
},
...
]
$ proc_interrupts | jc --proc_interrupts -p -r
[
{
"irq": "0",
"cpu_num": 2,
"interrupts": [
"18",
"0"
],
"type": "IO-APIC",
"device": [
"2-edge",
"timer"
]
},
{
"irq": "1",
"cpu_num": 2,
"interrupts": [
"0",
"73"
],
"type": "IO-APIC",
"device": [
"1-edge",
"i8042"
]
},
...
]
"""
from typing import List, Dict
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/interrupts` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: List[Dict]) -> List[Dict]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured to conform to the schema.
"""
for entry in proc_data:
entry['interrupts'] = [int(x) for x in entry['interrupts']]
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List = []
if jc.utils.has_data(data):
data_lines = data.splitlines()
# get the number of cpus
cpu_num = len(data_lines[0].split())
for line in filter(None, data_lines):
# skip non-data lines
if not ':' in line:
continue
# process data lines
split_line = line.split()
irq = split_line.pop(0)[:-1]
interrupts = []
if irq == 'ERR' or irq == 'MIS':
interrupts.extend(split_line)
elif irq.isdigit():
for _ in range(cpu_num):
interrupts.append(split_line.pop(0))
interrupt_type = split_line.pop(0)
device = split_line
else:
for _ in range(cpu_num):
interrupts.append(split_line.pop(0))
interrupt_type = ' '.join(split_line)
device = []
raw_output.append(
{
'irq': irq,
'cpu_num': cpu_num,
'interrupts': interrupts,
'type': interrupt_type,
'device': device or None
}
)
return raw_output if raw else _process(raw_output)

130
jc/parsers/proc_iomem.py Normal file
View File

@ -0,0 +1,130 @@
"""jc - JSON Convert `/proc/iomem` file parser
Usage (cli):
$ cat /proc/iomem | jc --proc
or
$ jc /proc/iomem
or
$ cat /proc/iomem | jc --proc-iomem
Usage (module):
import jc
result = jc.parse('proc', proc_iomem_file)
or
import jc
result = jc.parse('proc_iomem', proc_iomem_file)
Schema:
[
{
"start": string,
"end": string,
"device": string
}
]
Examples:
$ cat /proc/iomem | jc --proc -p
[
{
"start": "00000000",
"end": "00000fff",
"device": "Reserved"
},
{
"start": "00001000",
"end": "0009e7ff",
"device": "System RAM"
},
{
"start": "0009e800",
"end": "0009ffff",
"device": "Reserved"
},
...
]
"""
from typing import List, Dict
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/iomem` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: List[Dict]) -> List[Dict]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured to conform to the schema.
"""
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List = []
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
colon_split = line.split(':', maxsplit=1)
device = colon_split[1].strip()
mem_split = colon_split[0].split('-', maxsplit=1)
start = mem_split[0].strip()
end = mem_split[1].strip()
raw_output.append(
{
'start': start,
'end': end,
'device': device
}
)
return raw_output if raw else _process(raw_output)

113
jc/parsers/proc_ioports.py Normal file
View File

@ -0,0 +1,113 @@
"""jc - JSON Convert `/proc/ioports` file parser
Usage (cli):
$ cat /proc/ioports | jc --proc
or
$ jc /proc/ioports
or
$ cat /proc/ioports | jc --proc-ioports
Usage (module):
import jc
result = jc.parse('proc', proc_ioports_file)
or
import jc
result = jc.parse('proc_ioports', proc_ioports_file)
Schema:
[
{
"start": string,
"end": string,
"device": string
}
]
Examples:
$ cat /proc/ioports | jc --proc -p
[
{
"start": "0000",
"end": "0cf7",
"device": "PCI Bus 0000:00"
},
{
"start": "0000",
"end": "001f",
"device": "dma1"
},
{
"start": "0020",
"end": "0021",
"device": "PNP0001:00"
},
...
]
"""
from typing import List, Dict
import jc.utils
import jc.parsers.proc_iomem as proc_iomem
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/ioports` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: List[Dict]) -> List[Dict]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured to conform to the schema.
"""
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List = proc_iomem.parse(data)
return raw_output if raw else _process(raw_output)

139
jc/parsers/proc_loadavg.py Normal file
View File

@ -0,0 +1,139 @@
"""jc - JSON Convert `/proc/loadavg` file parser
Usage (cli):
$ cat /proc/loadavg | jc --proc
or
$ jc /proc/loadavg
or
$ cat /proc/loadavg | jc --proc-loadavg
Usage (module):
import jc
result = jc.parse('proc', proc_loadavg_file)
or
import jc
result = jc.parse('proc_loadavg', proc_loadavg_file)
Schema:
All values are integers.
{
"load_1m": float,
"load_5m": float,
"load_15m": float,
"running": integer,
"available": integer,
"last_pid": integer
}
Examples:
$ cat /proc/loadavg | jc --proc -p
{
"load_1m": 0.0,
"load_5m": 0.01,
"load_15m": 0.03,
"running": 2,
"available": 111,
"last_pid": 2039
}
$ cat /proc/loadavg | jc --proc -p -r
{
"load_1m": "0.00",
"load_5m": "0.01",
"load_15m": "0.03",
"running": "2",
"available": "111",
"last_pid": "2039"
}
"""
from typing import Dict
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/loadavg` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: Dict) -> Dict:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (Dictionary) raw structured data to process
Returns:
Dictionary. Structured to conform to the schema.
"""
float_list = {'load_1m', 'load_5m', 'load_15m'}
int_list = {'running', 'available', 'last_pid'}
for key in proc_data:
if key in float_list:
proc_data[key] = float(proc_data[key])
if key in int_list:
proc_data[key] = int(proc_data[key])
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> Dict:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: Dict = {}
if jc.utils.has_data(data):
load_1m, load_5m, load_15m, runnable, last_pid = data.split()
running, available = runnable.split('/')
raw_output = {
'load_1m': load_1m,
'load_5m': load_5m,
'load_15m': load_15m,
'running': running,
'available': available,
'last_pid': last_pid
}
return raw_output if raw else _process(raw_output)

186
jc/parsers/proc_locks.py Normal file
View File

@ -0,0 +1,186 @@
"""jc - JSON Convert `/proc/locks` file parser
Usage (cli):
$ cat /proc/locks | jc --proc
or
$ jc /proc/locks
or
$ cat /proc/locks | jc --proc-locks
Usage (module):
import jc
result = jc.parse('proc', proc_locks_file)
or
import jc
result = jc.parse('proc_locks', proc_locks_file)
Schema:
[
{
"id": integer,
"class": string,
"type": string,
"access": string,
"pid": integer,
"maj": string,
"min": string,
"inode": integer,
"start": string,
"end": string
}
]
Examples:
$ cat /proc/locks | jc --proc -p
[
{
"id": 1,
"class": "POSIX",
"type": "ADVISORY",
"access": "WRITE",
"pid": 877,
"maj": "00",
"min": "19",
"inode": 812,
"start": "0",
"end": "EOF"
},
{
"id": 2,
"class": "FLOCK",
"type": "ADVISORY",
"access": "WRITE",
"pid": 854,
"maj": "00",
"min": "19",
"inode": 805,
"start": "0",
"end": "EOF"
},
...
]
$ proc_locks | jc --proc_locks -p -r
[
{
"id": "1",
"class": "POSIX",
"type": "ADVISORY",
"access": "WRITE",
"pid": "877",
"maj": "00",
"min": "19",
"inode": "812",
"start": "0",
"end": "EOF"
},
{
"id": "2",
"class": "FLOCK",
"type": "ADVISORY",
"access": "WRITE",
"pid": "854",
"maj": "00",
"min": "19",
"inode": "805",
"start": "0",
"end": "EOF"
},
...
]
"""
from typing import List, Dict
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/locks` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: List[Dict]) -> List[Dict]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured to conform to the schema.
"""
int_list = {'id', 'pid', 'inode'}
for entry in proc_data:
for key in entry:
if key in int_list:
entry[key] = int(entry[key])
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List = []
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
id, class_, type_, access, pid, file, start, end = line.split()
maj, min, inode = file.split(':')
raw_output.append(
{
'id': id[:-1],
'class': class_,
'type': type_,
'access': access,
'pid': pid,
'maj': maj,
'min': min,
'inode': inode,
'start': start,
'end': end
}
)
return raw_output if raw else _process(raw_output)

153
jc/parsers/proc_meminfo.py Normal file
View File

@ -0,0 +1,153 @@
"""jc - JSON Convert `/proc/meminfo` file parser
Usage (cli):
$ cat /proc/meminfo | jc --proc
or
$ jc /proc/meminfo
or
$ cat /proc/meminfo | jc --proc-meminfo
Usage (module):
import jc
result = jc.parse('proc', proc_meminfo_file)
or
import jc
result = jc.parse('proc_meminfo', proc_meminfo_file)
Schema:
All values are integers.
{
<keyName> integer
}
Examples:
$ cat /proc/meminfo | jc --proc -p
{
"MemTotal": 3997272,
"MemFree": 2760316,
"MemAvailable": 3386876,
"Buffers": 40452,
"Cached": 684856,
"SwapCached": 0,
"Active": 475816,
"Inactive": 322064,
"Active(anon)": 70216,
"Inactive(anon)": 148,
"Active(file)": 405600,
"Inactive(file)": 321916,
"Unevictable": 19476,
"Mlocked": 19476,
"SwapTotal": 3996668,
"SwapFree": 3996668,
"Dirty": 152,
"Writeback": 0,
"AnonPages": 92064,
"Mapped": 79464,
"Shmem": 1568,
"KReclaimable": 188216,
"Slab": 288096,
"SReclaimable": 188216,
"SUnreclaim": 99880,
"KernelStack": 5872,
"PageTables": 1812,
"NFS_Unstable": 0,
"Bounce": 0,
"WritebackTmp": 0,
"CommitLimit": 5995304,
"Committed_AS": 445240,
"VmallocTotal": 34359738367,
"VmallocUsed": 21932,
"VmallocChunk": 0,
"Percpu": 107520,
"HardwareCorrupted": 0,
"AnonHugePages": 0,
"ShmemHugePages": 0,
"ShmemPmdMapped": 0,
"FileHugePages": 0,
"FilePmdMapped": 0,
"HugePages_Total": 0,
"HugePages_Free": 0,
"HugePages_Rsvd": 0,
"HugePages_Surp": 0,
"Hugepagesize": 2048,
"Hugetlb": 0,
"DirectMap4k": 192320,
"DirectMap2M": 4001792,
"DirectMap1G": 2097152
}
"""
from typing import Dict
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/meminfo` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: Dict) -> Dict:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (Dictionary) raw structured data to process
Returns:
Dictionary. Structured to conform to the schema.
"""
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> Dict:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: Dict = {}
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
split_line = line.split(':', maxsplit=1)
key = split_line[0]
val = int(split_line[1].rsplit(maxsplit=1)[0])
raw_output[key] = val
return raw_output if raw else _process(raw_output)

184
jc/parsers/proc_modules.py Normal file
View File

@ -0,0 +1,184 @@
"""jc - JSON Convert `/proc/modules` file parser
Usage (cli):
$ cat /proc/modules | jc --proc
or
$ jc /proc/modules
or
$ cat /proc/modules | jc --proc-modules
Usage (module):
import jc
result = jc.parse('proc', proc_modules_file)
or
import jc
result = jc.parse('proc_modules', proc_modules_file)
Schema:
[
{
"module": string,
"size": integer,
"used": integer,
"used_by": [
string
],
"status": string,
"location": string
}
]
Examples:
$ cat /proc/modules | jc --proc -p
[
{
"module": "binfmt_misc",
"size": 24576,
"used": 1,
"used_by": [],
"status": "Live",
"location": "0xffffffffc0ab4000"
},
{
"module": "vsock_loopback",
"size": 16384,
"used": 0,
"used_by": [],
"status": "Live",
"location": "0xffffffffc0a14000"
},
{
"module": "vmw_vsock_virtio_transport_common",
"size": 36864,
"used": 1,
"used_by": [
"vsock_loopback"
],
"status": "Live",
"location": "0xffffffffc0a03000"
},
...
]
$ proc_modules | jc --proc_modules -p -r
[
{
"module": "binfmt_misc",
"size": "24576",
"used": "1",
"used_by": [],
"status": "Live",
"location": "0xffffffffc0ab4000"
},
{
"module": "vsock_loopback",
"size": "16384",
"used": "0",
"used_by": [],
"status": "Live",
"location": "0xffffffffc0a14000"
},
{
"module": "vmw_vsock_virtio_transport_common",
"size": "36864",
"used": "1",
"used_by": [
"vsock_loopback"
],
"status": "Live",
"location": "0xffffffffc0a03000"
},
...
]
"""
from typing import List, Dict
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`/proc/modules` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
hidden = True
__version__ = info.version
def _process(proc_data: List[Dict]) -> List[Dict]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured to conform to the schema.
"""
int_list = {'size', 'used'}
for entry in proc_data:
for key in entry:
if key in int_list:
entry[key] = jc.utils.convert_to_int(entry[key])
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List = []
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
module, size, used, used_by, status, location = line.split()
used_by_list = used_by.split(',')[:-1]
raw_output.append(
{
'module': module,
'size': size,
'used': used,
'used_by': used_by_list,
'status': status,
'location': location
}
)
return raw_output if raw else _process(raw_output)

113
man/jc.1
View File

@ -1,13 +1,26 @@
.TH jc 1 2022-08-29 1.21.2 "JSON Convert"
.TH jc 1 2022-09-13 1.21.2 "JSON Convert"
.SH NAME
\fBjc\fP \- JSON Convert JSONifies the output of many CLI tools and file-types
\fBjc\fP \- JSON Convert JSONifies the output of many CLI tools, file-types, and strings
.SH SYNOPSIS
COMMAND | jc PARSER [OPTIONS]
or "Magic" syntax:
Standard syntax:
.RS
COMMAND | \fBjc\fP [OPTIONS] PARSER
cat FILE | \fBjc\fP [OPTIONS] PARSER
echo STRING | \fBjc\fP [OPTIONS] PARSER
.RE
Magic syntax:
.RS
\fBjc\fP [OPTIONS] COMMAND
\fBjc\fP [OPTIONS] /proc/<path-to-procfile>
.RE
.SH DESCRIPTION
\fBjc\fP JSONifies the output of many CLI tools, file-types, and common strings for easier parsing in scripts. \fBjc\fP accepts piped input from \fBSTDIN\fP and outputs a JSON representation of the previous command's output to \fBSTDOUT\fP. Alternatively, the "Magic" syntax can be used by prepending \fBjc\fP to the command to be converted. Options can be passed to \fBjc\fP immediately before the command is given. (Note: "Magic" syntax does not support shell builtins or command aliases)
@ -392,6 +405,86 @@ PLIST file parser
\fB--postconf\fP
`postconf -M` command parser
.TP
.B
\fB--proc\fP
`/proc/` file parser
.TP
.B
\fB--proc-buddyinfo\fP
`/proc/buddyinfo` file parser
.TP
.B
\fB--proc-consoles\fP
`/proc/consoles` file parser
.TP
.B
\fB--proc-cpuinfo\fP
`/proc/cpuinfo` file parser
.TP
.B
\fB--proc-crypto\fP
`/proc/crypto` file parser
.TP
.B
\fB--proc-devices\fP
`/proc/devices` file parser
.TP
.B
\fB--proc-diskstats\fP
`/proc/diskstats` file parser
.TP
.B
\fB--proc-driver-rtc\fP
`/proc/driver_rtc` file parser
.TP
.B
\fB--proc-filesystems\fP
`/proc/filesystems` file parser
.TP
.B
\fB--proc-interrupts\fP
`/proc/interrupts` file parser
.TP
.B
\fB--proc-iomem\fP
`/proc/iomem` file parser
.TP
.B
\fB--proc-ioports\fP
`/proc/ioports` file parser
.TP
.B
\fB--proc-loadavg\fP
`/proc/loadavg` file parser
.TP
.B
\fB--proc-locks\fP
`/proc/locks` file parser
.TP
.B
\fB--proc-meminfo\fP
`/proc/meminfo` file parser
.TP
.B
\fB--proc-modules\fP
`/proc/modules` file parser
.TP
.B
\fB--ps\fP
@ -639,7 +732,7 @@ Debug - show traceback (use \fB-dd\fP for verbose traceback)
.TP
.B
\fB-h\fP, \fB--help\fP
Help (\fB--help --parser_name\fP for parser documentation)
Help (\fB--help --parser_name\fP for parser documentation). Use twice to show hidden parsers (e.g. \fB-hh\fP)
.TP
.B
\fB-m\fP, \fB--monochrome\fP
@ -872,17 +965,21 @@ If a UTC timezone can be detected in the text of the command output, the timesta
.SH EXAMPLES
Standard Syntax:
.RS
$ dig www.google.com | jc \fB--dig\fP \fB-p\fP
$ dig www.google.com | jc \fB-p\fP \fB--dig\fP
$ cat /proc/meminfo | jc \fB--pretty\fP \fB--proc\fP
.RE
Magic Syntax:
.RS
$ jc \fB-p\fP dig www.google.com
$ jc \fB--pretty\fP dig www.google.com
$ jc \fB--pretty\fP /proc/meminfo
.RE
For parser documentation:
.RS
$ jc \fB-h\fP \fB--dig\fP
$ jc \fB--help\fP \fB--dig\fP
.RE
.SH AUTHOR
Kelly Brazil (kellyjonbrazil@gmail.com)

5
readmegen.py
View File

@ -1,12 +1,15 @@
#!/usr/bin/env python3
# Genereate README.md from jc metadata using jinja2 templates
import jc.cli
import jc.lib
from jinja2 import Environment, FileSystemLoader
file_loader = FileSystemLoader('templates')
env = Environment(loader=file_loader)
template = env.get_template('readme_template')
output = template.render(jc=jc.cli.about_jc())
# output = template.render(jc=jc.cli.about_jc())
output = template.render(parsers=jc.lib.all_parser_info(),
info=jc.cli.about_jc())
with open('README.md', 'w') as f:
f.write(output)

31
templates/manpage_template
View File

@ -1,13 +1,26 @@
.TH jc 1 {{ today }} {{ jc.version}} "JSON Convert"
.SH NAME
\fBjc\fP \- JSON Convert JSONifies the output of many CLI tools and file-types
\fBjc\fP \- JSON Convert JSONifies the output of many CLI tools, file-types, and strings
.SH SYNOPSIS
COMMAND | jc PARSER [OPTIONS]
or "Magic" syntax:
Standard syntax:
.RS
COMMAND | \fBjc\fP [OPTIONS] PARSER
cat FILE | \fBjc\fP [OPTIONS] PARSER
echo STRING | \fBjc\fP [OPTIONS] PARSER
.RE
Magic syntax:
.RS
\fBjc\fP [OPTIONS] COMMAND
\fBjc\fP [OPTIONS] /proc/<path-to-procfile>
.RE
.SH DESCRIPTION
\fBjc\fP JSONifies the output of many CLI tools, file-types, and common strings for easier parsing in scripts. \fBjc\fP accepts piped input from \fBSTDIN\fP and outputs a JSON representation of the previous command's output to \fBSTDOUT\fP. Alternatively, the "Magic" syntax can be used by prepending \fBjc\fP to the command to be converted. Options can be passed to \fBjc\fP immediately before the command is given. (Note: "Magic" syntax does not support shell builtins or command aliases)
@ -44,7 +57,7 @@ Debug - show traceback (use \fB-dd\fP for verbose traceback)
.TP
.B
\fB-h\fP, \fB--help\fP
Help (\fB--help --parser_name\fP for parser documentation)
Help (\fB--help --parser_name\fP for parser documentation). Use twice to show hidden parsers (e.g. \fB-hh\fP)
.TP
.B
\fB-m\fP, \fB--monochrome\fP
@ -277,17 +290,21 @@ If a UTC timezone can be detected in the text of the command output, the timesta
.SH EXAMPLES
Standard Syntax:
.RS
$ dig www.google.com | jc \fB--dig\fP \fB-p\fP
$ dig www.google.com | jc \fB-p\fP \fB--dig\fP
$ cat /proc/meminfo | jc \fB--pretty\fP \fB--proc\fP
.RE
Magic Syntax:
.RS
$ jc \fB-p\fP dig www.google.com
$ jc \fB--pretty\fP dig www.google.com
$ jc \fB--pretty\fP /proc/meminfo
.RE
For parser documentation:
.RS
$ jc \fB-h\fP \fB--dig\fP
$ jc \fB--help\fP \fB--dig\fP
.RE
.SH AUTHOR
{{ jc.author }} ({{ jc.author_email }})

50
templates/readme_template
View File

@ -70,7 +70,7 @@ values are converted, and, in some cases, additional semantic context fields are
added.
To access the raw, pre-processed JSON, use the `-r` cli option or the `raw=True`
function parameter in `parse()`.
function parameter in `parse()` when using `jc` as a python library.
Schemas for each parser can be found at the documentation link beside each
[**Parser**](#parsers) below.
@ -133,14 +133,18 @@ on Github.
`jc` accepts piped input from `STDIN` and outputs a JSON representation of the
previous command's output to `STDOUT`.
```bash
COMMAND | jc PARSER [OPTIONS]
COMMAND | jc [OPTIONS] PARSER
cat FILE | jc [OPTIONS] PARSER
echo STRING | jc [OPTIONS] PARSER
```
Alternatively, the "magic" syntax can be used by prepending `jc` to the command
to be converted. Options can be passed to `jc` immediately before the command is
given. (Note: command aliases and shell builtins are not supported)
to be converted or in front of the absolute path for Proc files. Options can be
passed to `jc` immediately before the command or Proc file path is given.
(Note: command aliases and shell builtins are not supported)
```bash
jc [OPTIONS] COMMAND
jc [OPTIONS] /proc/<path-to-procfile>
```
The JSON output can be compact (default) or pretty formatted with the `-p`
@ -149,7 +153,7 @@ option.
### Parsers
| Argument | Command or Filetype | Documentation |
|-------------------|---------------------------------------------------------|----------------------------------------------------------------------------|{% for parser in jc.parsers %}
|-------------------|---------------------------------------------------------|----------------------------------------------------------------------------|{% for parser in parsers %}
| `{{ "{:>15}".format(parser.argument) }}` | {{ "{:<55}".format(parser.description) }} | {{ "{:<74}".format("[details](https://kellyjonbrazil.github.io/jc/docs/parsers/" + parser.name + ")") }} |{% endfor %}
### Options
@ -159,7 +163,7 @@ option.
| `-a` | `--about` | About `jc`. Prints information about `jc` and the parsers (in JSON or YAML, of course!) |
| `-C` | `--force-color` | Force color output even when using pipes (overrides `-m` and the `NO_COLOR` env variable) |
| `-d` | `--debug` | Debug mode. Prints trace messages if parsing issues are encountered (use`-dd` for verbose debugging) |
| `-h` | `--help` | Help. Use `jc -h --parser_name` for parser documentation |
| `-h` | `--help` | Help. Use `jc -h --parser_name` for parser documentation. Use twice to show hidden parsers (e.g. `-hh`) |
| `-m` | `--monochrome` | Monochrome output |
| `-M` | `--meta-out` | Add metadata to output including timestamp, parser name, magic command, magic command exit code, etc. | |
| `-p` | `--pretty` | Pretty format the JSON output |
@ -418,12 +422,12 @@ that case you can suppress the warning message with the `-q` cli option or the
macOS:
```bash
cat lsof.out | jc --lsof -q
cat lsof.out | jc -q --lsof
```
or Windows:
```bash
type lsof.out | jc --lsof -q
type lsof.out | jc -q --lsof
```
Tested on:
@ -467,7 +471,7 @@ documentation.
### arp
```bash
arp | jc --arp -p # or: jc -p arp
arp | jc -p --arp # or: jc -p arp
```
```json
[
@ -506,7 +510,7 @@ cat homes.csv
...
```
```bash
cat homes.csv | jc --csv -p
cat homes.csv | jc -p --csv
```
```json
[
@ -547,7 +551,7 @@ cat homes.csv | jc --csv -p
```
### /etc/hosts file
```bash
cat /etc/hosts | jc --hosts -p
cat /etc/hosts | jc -p --hosts
```
```json
[
@ -574,7 +578,7 @@ cat /etc/hosts | jc --hosts -p
```
### ifconfig
```bash
ifconfig | jc --ifconfig -p # or: jc -p ifconfig
ifconfig | jc -p --ifconfig # or: jc -p ifconfig
```
```json
[
@ -632,7 +636,7 @@ Port = 50022
ForwardX11 = no
```
```bash
cat example.ini | jc --ini -p
cat example.ini | jc -p --ini
```
```json
{
@ -654,7 +658,7 @@ cat example.ini | jc --ini -p
```
### ls
```bash
$ ls -l /usr/bin | jc --ls -p # or: jc -p ls -l /usr/bin
$ ls -l /usr/bin | jc -p --ls # or: jc -p ls -l /usr/bin
```
```json
[
@ -690,7 +694,7 @@ $ ls -l /usr/bin | jc --ls -p # or: jc -p ls -l /usr/bin
```
### netstat
```bash
netstat -apee | jc --netstat -p # or: jc -p netstat -apee
netstat -apee | jc -p --netstat # or: jc -p netstat -apee
```
```json
[
@ -778,7 +782,7 @@ netstat -apee | jc --netstat -p # or: jc -p netstat -apee
```
### /etc/passwd file
```bash
cat /etc/passwd | jc --passwd -p
cat /etc/passwd | jc -p --passwd
```
```json
[
@ -804,7 +808,7 @@ cat /etc/passwd | jc --passwd -p
```
### ping
```bash
ping 8.8.8.8 -c 3 | jc --ping -p # or: jc -p ping 8.8.8.8 -c 3
ping 8.8.8.8 -c 3 | jc -p --ping # or: jc -p ping 8.8.8.8 -c 3
```
```json
{
@ -857,7 +861,7 @@ ping 8.8.8.8 -c 3 | jc --ping -p # or: jc -p ping 8.8.8.8 -c 3
```
### ps
```bash
ps axu | jc --ps -p # or: jc -p ps axu
ps axu | jc -p --ps # or: jc -p ps axu
```
```json
[
@ -904,7 +908,7 @@ ps axu | jc --ps -p # or: jc -p ps axu
```
### traceroute
```bash
traceroute -m 2 8.8.8.8 | jc --traceroute -p
traceroute -m 2 8.8.8.8 | jc -p --traceroute
# or: jc -p traceroute -m 2 8.8.8.8
```
```json
@ -969,7 +973,7 @@ traceroute -m 2 8.8.8.8 | jc --traceroute -p
```
### uptime
```bash
uptime | jc --uptime -p # or: jc -p uptime
uptime | jc -p --uptime # or: jc -p uptime
```
```json
{
@ -1014,7 +1018,7 @@ cat cd_catalog.xml
...
```
```bash
cat cd_catalog.xml | jc --xml -p
cat cd_catalog.xml | jc -p --xml
```
```json
{
@ -1066,7 +1070,7 @@ spec:
mode: ISTIO_MUTUAL
```
```bash
cat istio.yaml | jc --yaml -p
cat istio.yaml | jc -p --yaml
```
```json
[
@ -1104,4 +1108,4 @@ cat istio.yaml | jc --yaml -p
]
```
{{ jc.copyright }}
{{ info.copyright }}

5
tests/test_jc_lib.py
View File

@ -53,6 +53,11 @@ class MyTests(unittest.TestCase):
def test_lib_all_parser_info_length(self):
self.assertGreaterEqual(len(jc.lib.all_parser_info()), 80)
def test_lib_all_parser_hidden_length(self):
reg_length = len(jc.lib.all_parser_info())
hidden_length = len(jc.lib.all_parser_info(show_hidden=True))
self.assertGreater(hidden_length, reg_length)
def test_lib_plugin_parser_mod_list_is_list(self):
self.assertIsInstance(jc.lib.plugin_parser_mod_list(), list)