1
0
mirror of https://github.com/kellyjonbrazil/jc.git synced 2026-04-03 17:44:07 +02:00

Compare commits

...

107 Commits

Author SHA1 Message Date
Kelly Brazil
7fd67fda13 Merge pull request #256 from kellyjonbrazil/dev
v1.20.1
2022-06-15 22:16:07 +00:00
Kelly Brazil
7583f315ce add long options tests 2022-06-15 13:21:37 -07:00
Kelly Brazil
7fd70d2088 doc update 2022-06-15 13:10:12 -07:00
Kelly Brazil
337ee73844 doc update 2022-06-15 12:51:42 -07:00
Kelly Brazil
43702a260b strip single quotes just like double quotes 2022-06-15 12:50:58 -07:00
Kelly Brazil
40208a9f76 doc update 2022-06-15 11:25:21 -07:00
Kelly Brazil
b958358389 update _snake_case comment 2022-06-15 11:15:26 -07:00
Kelly Brazil
c23aacedad doc update 2022-06-15 11:12:49 -07:00
Kelly Brazil
4f148469d7 preserve keyname case with -r 2022-06-15 11:12:43 -07:00
Kelly Brazil
247c43278c add postconf tests 2022-06-15 09:22:09 -07:00
Kelly Brazil
45f45e0511 add examples 2022-06-15 09:18:11 -07:00
Kelly Brazil
c0c469ae9b add comments 2022-06-14 17:03:54 -07:00
Kelly Brazil
4b86fd8d8a manipulate words and CURRENT to fix magic completions in Zsh 2022-06-14 16:51:45 -07:00
Kelly Brazil
8fba47f449 doc update 2022-06-14 12:42:30 -07:00
Kelly Brazil
f9ae964280 reuse arrays in zsh completions 2022-06-14 08:49:26 -07:00
Kelly Brazil
ca95615d7f partially working _normal completion for zsh 2022-06-14 07:42:55 -07:00
Kelly Brazil
747b255e34 zsh working same as bash except for magic command completions 2022-06-13 21:09:37 -07:00
Kelly Brazil
c0da9ebd6c only check previous words in bash completion 2022-06-10 17:25:19 -07:00
Kelly Brazil
5c40b38a05 fix help bash completions so they don't repeat parsers 2022-06-10 12:02:04 -07:00
Kelly Brazil
7bc03dcf06 add local vars to bash completion 2022-06-08 08:51:23 -07:00
Kelly Brazil
f62e6168fd fix help completions for bash 2022-06-08 08:50:20 -07:00
Kelly Brazil
39c1470ea6 add special options to bash completion 2022-06-07 11:57:48 -07:00
Kelly Brazil
e48b99f1c1 fix bash completion behavior 2022-06-07 11:02:05 -07:00
Kelly Brazil
71ae545907 better working bash completion 2022-06-06 16:24:52 -07:00
Kelly Brazil
b9a5eda187 formatting 2022-06-06 10:52:40 -07:00
Kelly Brazil
231a2039c2 update options 2022-06-06 10:49:17 -07:00
Kelly Brazil
a415bc23fa man page update 2022-06-06 10:32:18 -07:00
Kelly Brazil
79add35fc1 split long and short options 2022-06-05 18:09:44 -07:00
Kelly Brazil
901763fc39 fix hyphens 2022-06-05 18:06:38 -07:00
Kelly Brazil
1034cb1ea2 Merge branch 'dev' of https://github.com/kellyjonbrazil/jc into dev 2022-06-05 18:04:42 -07:00
Kelly Brazil
020093fd67 revert non-breaking hyphens 2022-06-05 18:04:35 -07:00
Kelly Brazil
f17b3fbd32 add non-breaking hyphens 2022-06-05 17:54:34 -07:00
Kelly Brazil
15b58e3a6b add long-options 2022-06-05 17:52:14 -07:00
Kelly Brazil
da6c98826b doc update 2022-06-05 17:40:05 -07:00
Kelly Brazil
26415e2978 formatting 2022-06-05 17:17:13 -07:00
Kelly Brazil
d849bd3b66 add bash and zsh completions to cli 2022-06-05 16:23:55 -07:00
Kelly Brazil
5996192455 remove indent at end of help text 2022-06-05 12:16:30 -07:00
Kelly Brazil
a38b6b5522 add long options 2022-06-05 12:12:58 -07:00
Kelly Brazil
1d6bc40bff fix no_wake_up_before_first_use to allow None 2022-06-03 10:35:49 -07:00
Kelly Brazil
437fa62cb1 update schema 2022-06-03 10:29:55 -07:00
Kelly Brazil
6cde26d9ed add process logic 2022-06-02 14:53:31 -07:00
Kelly Brazil
f6dd5a68cd doc update 2022-06-02 09:28:00 -07:00
Kelly Brazil
b1507dc576 initial postconf parser 2022-06-02 09:14:02 -07:00
Kelly Brazil
a5d5b1554f change variable name from list to my_list 2022-06-02 08:34:51 -07:00
Kelly Brazil
ec8efebc94 move _get_item outside of parse function 2022-06-02 08:31:28 -07:00
Kelly Brazil
f86cbf5527 Merge pull request #250 from pbrezina/id
id: parse output without name
2022-06-02 15:27:07 +00:00
Pavel Březina
57142d899c id: parse output without name
If a group does not exist only gid is present in the command output.
2022-06-02 12:51:11 +02:00
Kelly Brazil
504a04279e version bump 2022-06-01 12:13:50 -07:00
Kelly Brazil
3b4ef4a814 formatting 2022-06-01 12:13:41 -07:00
Kelly Brazil
a7e3d2fc86 Merge pull request #251 from kellyjonbrazil/master
sync to dev
2022-06-01 17:35:52 +00:00
Kelly Brazil
776ee66bae Merge pull request #249 from kellyjonbrazil/dev
Dev v1.20.0
2022-05-31 16:23:43 +00:00
Kelly Brazil
9bf2cd0691 doc update 2022-05-31 09:16:54 -07:00
Kelly Brazil
fec74cf305 doc formatting 2022-05-28 13:13:07 -07:00
Kelly Brazil
d3d7fbca61 doc formatting 2022-05-28 12:41:36 -07:00
Kelly Brazil
550862a415 doc update 2022-05-28 12:18:02 -07:00
Kelly Brazil
1763b530da add tests for new fields 2022-05-27 17:40:54 -07:00
Kelly Brazil
8307150cae add integer fields 2022-05-27 17:26:48 -07:00
Kelly Brazil
d2223c45d1 add new fields to schema 2022-05-27 17:20:54 -07:00
Kelly Brazil
472ee5e295 add new fields. allow for new unknown fields with warning 2022-05-27 17:07:39 -07:00
Kelly Brazil
1d0aebd836 add new fields 2022-05-27 16:54:49 -07:00
Kelly Brazil
8240626043 add idle status for newer top version support 2022-05-27 16:27:51 -07:00
Kelly Brazil
2cd14235c8 remove unnecessary pass line 2022-05-27 14:29:48 -07:00
Kelly Brazil
f774513554 add top-s tests 2022-05-27 14:29:33 -07:00
Kelly Brazil
388da9f003 remove try/except at end 2022-05-27 14:11:27 -07:00
Kelly Brazil
fa18243491 add check for top data 2022-05-27 13:48:13 -07:00
Kelly Brazil
5df1c1702b add top-s parser 2022-05-27 13:34:07 -07:00
Kelly Brazil
b6557802f4 change windows command prompt in docs 2022-05-27 08:54:37 -07:00
Kelly Brazil
15e3a511b6 doc update 2022-05-26 16:04:57 -07:00
Kelly Brazil
08fbde0e8f fix stdin stder quoting 2022-05-26 16:03:30 -07:00
Kelly Brazil
28ebb4e8dd fix notes 2022-05-26 15:54:39 -07:00
Kelly Brazil
04fda57cbe fix note 2022-05-26 15:11:58 -07:00
Kelly Brazil
b24495136c fix blockquotes 2022-05-26 15:07:54 -07:00
Kelly Brazil
7d3fa55571 doc update 2022-05-26 11:52:29 -07:00
Kelly Brazil
a76b5db8db doc update 2022-05-26 11:37:16 -07:00
Kelly Brazil
667dd01ac7 fix process table detection 2022-05-26 09:29:59 -07:00
Kelly Brazil
559bee962a add top tests 2022-05-26 09:19:47 -07:00
Kelly Brazil
9f977d06e0 fix asciitable parser for cases where centered rows cause misaligned fields 2022-05-26 08:57:35 -07:00
Kelly Brazil
f67a916940 add new parser counts to jc -a 2022-05-25 14:30:00 -07:00
Kelly Brazil
8a1308948f fix typo 2022-05-25 12:21:44 -07:00
Kelly Brazil
4e130d11a3 fix schema 2022-05-25 12:18:51 -07:00
Kelly Brazil
51aa5b268f fix for mem info in different units 2022-05-25 10:46:58 -07:00
Kelly Brazil
6869133f53 working top parser and doc update 2022-05-25 10:31:41 -07:00
Kelly Brazil
ad3d88e47f doc update 2022-05-25 08:14:14 -07:00
Kelly Brazil
6d29f8ba74 quiet option for warning messages 2022-05-25 07:37:21 -07:00
Kelly Brazil
c7cb89e91d more processing 2022-05-24 17:22:23 -07:00
Kelly Brazil
27f5118abd add top parser 2022-05-24 15:47:51 -07:00
Kelly Brazil
ac5a6e516d doc update 2022-05-23 15:12:14 -07:00
Kelly Brazil
5d5bfbc1d1 doc update 2022-05-23 13:58:24 -07:00
Kelly Brazil
9f4ba80000 refactor - create is_compatible function 2022-05-23 13:52:08 -07:00
Kelly Brazil
c21c334b73 version bump 2022-05-23 13:43:57 -07:00
Kelly Brazil
14093d9d43 Merge pull request #247 from kellyjonbrazil/yaml-dev
Yaml dev
2022-05-23 20:36:44 +00:00
Kelly Brazil
176ca2f75d docstring update 2022-05-23 13:16:57 -07:00
Kelly Brazil
cfb71c7dad add test for strings ending with colon 2022-05-22 09:13:32 -07:00
Kelly Brazil
722eab83b1 add Other Shell section 2022-05-21 07:00:43 -07:00
Kelly Brazil
d30edb2dae add yaml_out tests 2022-05-20 18:15:56 -07:00
Kelly Brazil
cba461ce8a remove JSONDecodeError detection 2022-05-20 17:48:52 -07:00
Kelly Brazil
f876505e25 add colors and ensure ascii on error 2022-05-20 17:22:56 -07:00
Kelly Brazil
5b9fcd5852 initial working yaml out 2022-05-20 14:51:23 -07:00
Kelly Brazil
8a122cd9e1 fix wiki language 2022-05-20 07:58:19 -07:00
Kelly Brazil
557e68225c add filtering capabilities for other shells 2022-05-19 11:16:25 -07:00
Kelly Brazil
dc0947b87e add other shell info. clean up parser table 2022-05-19 10:52:18 -07:00
Kelly Brazil
af2c06cd28 doc update 2022-05-15 22:28:12 -07:00
Kelly Brazil
67a4c6f797 add initial \n to first line of multiline fields 2022-05-15 22:23:00 -07:00
Kelly Brazil
0d1c857410 fix to keep first line of multiline field 2022-05-15 22:19:01 -07:00
Kelly Brazil
c7684dc94d allow multiline fields in pip-show 2022-05-15 22:06:43 -07:00
Kelly Brazil
a247572f64 changelog update 2022-05-15 20:49:41 -07:00
Kelly Brazil
306111303a Merge pull request #244 from kellyjonbrazil/master
Merge pull request #242 from kellyjonbrazil/dev sync to dev
2022-05-16 03:46:59 +00:00
131 changed files with 4657 additions and 618 deletions

View File

@@ -1,41 +1,58 @@
jc changelog
20220615 v1.20.1
- Add `postconf -M` parser tested on linux
- Update `asciitable` and `asciitable-m` parsers to preserve case in key
names when using the `-r` or `raw=True` options.
- Add long options (e.g. `--help`, `--about`, `--pretty`, etc.)
- Add shell completions for Bash and Zsh
- Fix `id` parser for cases where the user or group name is not present
20220531 v1.20.0
- Add YAML output option with `-y`
- Add `top -b` standard and streaming parsers tested on linux
- Add `plugin_parser_count`, `standard_parser_count`, and `streaming_parser_count`
keys to `jc -a` output
- Add `is_compatible` function to the `utils` module
- Fix `pip-show` parser for packages with a multi-line license field
- Fix ASCII Table parser for cases where centered headers cause mis-aligned fields
20220513 v1.19.0
- Add chage --list command parser tested on linux
- Add git log command streaming parser
- Fix git log standard parser for coner-cases where hash values are in messages
- Fix df command parser for rare instances when a newline is found at the end
- Add `chage --list` command parser tested on linux
- Add `git log` command streaming parser
- Fix `git log` standard parser for corner-cases where hash values are in messages
- Fix `df` command parser for rare instances when a newline is found at the end
- Allow jc to pip install on unsupported python version 3.6
- Fix asciitable-m parser to skip some rows that contain column separator
characters in cell data. A warning message will be printed to STDOUT
- Fix `asciitable-m` parser to skip some rows that contain column separator
characters in cell data. A warning message will be printed to STDERR
unless `-q` or `quiet=True` is used.
20220427 v1.18.8
- Fix update-alternatives --query parser for cases where `slaves` are not present
- Fix `update-alternatives --query` parser for cases where `slaves` are not present
- Fix UnicodeEncodeError on some systems where LANG=C is set and unicode
characters are in the output
- Update history parser: do not drop non-ASCII characters if the system
- Update `history` parser: do not drop non-ASCII characters if the system
is configured for UTF-8 encoding
- Enhance "magic syntax" to always use UTF-8 encoding
20220425 v1.18.7
- Add git log command parser
- Add update-alternatives --query parser
- Add update-alternatives --get-selections parser
- Add `git log` command parser
- Add `update-alternatives --query` parser
- Add `update-alternatives --get-selections` parser
- Fix key/value and ini parsers to allow duplicate keys
- Fix yaml file parser for files including timestamp objects
- Update xrandr parser: add a 'rotation' field
- Update `xrandr` parser: add a 'rotation' field
- Fix failing tests by moving template files
- Add python interpreter version and path to -v and -a output
20220325 v1.18.6
- Add pidstat command parser tested on linux
- Add pidstat command streaming parser tested on linux
- Add mpstat command parser tested on linux
- Add mpstat command streaming parser tested on linux
- Add `pidstat` command parser tested on linux
- Add `pidstat` command streaming parser tested on linux
- Add `mpstat` command parser tested on linux
- Add `mpstat` command streaming parser tested on linux
- Add single-line ASCII and Unicode table parser
- Add multi-line ASCII and Unicode table parser
- Add documentation option to parser_info() and all_parser_info()
- Add documentation option to `parser_info()` and `all_parser_info()`
20220305 v1.18.5
- Fix date parser to ensure AM/PM period string is always uppercase

View File

@@ -2728,6 +2728,36 @@ pip show wrapt wheel | jc --pip-show -p # or: jc -p pip show wrapt whe
}
]
```
### postconf -M
```bash
postconf -M | jc --postconf -p # or jc -p postconf -M
```
```json
[
{
"service_name": "smtp",
"service_type": "inet",
"private": false,
"unprivileged": null,
"chroot": true,
"wake_up_time": null,
"process_limit": null,
"command": "smtpd",
"no_wake_up_before_first_use": null
},
{
"service_name": "pickup",
"service_type": "unix",
"private": false,
"unprivileged": null,
"chroot": true,
"wake_up_time": 60,
"process_limit": 1,
"command": "pickup",
"no_wake_up_before_first_use": false
}
]
```
### ps
```bash
ps -ef | jc --ps -p # or: jc -p ps -ef
@@ -3465,6 +3495,105 @@ timedatectl | jc --timedatectl -p # or: jc -p timedatectl
"epoch_utc": 1583888001
}
```
### tob -b
```bash
top -b -n 1 | jc --top -p # or jc -p tob -b -n 1
```
```json
[
{
"time": "11:20:43",
"uptime": 118,
"users": 2,
"load_1m": 0.0,
"load_5m": 0.01,
"load_15m": 0.05,
"tasks_total": 108,
"tasks_running": 2,
"tasks_sleeping": 106,
"tasks_stopped": 0,
"tasks_zombie": 0,
"cpu_user": 5.6,
"cpu_sys": 11.1,
"cpu_nice": 0.0,
"cpu_idle": 83.3,
"cpu_wait": 0.0,
"cpu_hardware": 0.0,
"cpu_software": 0.0,
"cpu_steal": 0.0,
"mem_total": 3.7,
"mem_free": 3.3,
"mem_used": 0.2,
"mem_buff_cache": 0.2,
"swap_total": 2.0,
"swap_free": 2.0,
"swap_used": 0.0,
"mem_available": 3.3,
"processes": [
{
"pid": 2225,
"user": "kbrazil",
"priority": 20,
"nice": 0,
"virtual_mem": 158.1,
"resident_mem": 2.2,
"shared_mem": 1.6,
"status": "running",
"percent_cpu": 12.5,
"percent_mem": 0.1,
"time_hundredths": "0:00.02",
"command": "top",
"parent_pid": 1884,
"uid": 1000,
"real_uid": 1000,
"real_user": "kbrazil",
"saved_uid": 1000,
"saved_user": "kbrazil",
"gid": 1000,
"group": "kbrazil",
"pgrp": 2225,
"tty": "pts/0",
"tty_process_gid": 2225,
"session_id": 1884,
"thread_count": 1,
"last_used_processor": 0,
"time": "0:00",
"swap": 0.0,
"code": 0.1,
"data": 1.0,
"major_page_fault_count": 0,
"minor_page_fault_count": 736,
"dirty_pages_count": 0,
"sleeping_in_function": null,
"flags": "..4.2...",
"cgroups": "1:name=systemd:/user.slice/user-1000.+",
"supplementary_gids": [
10,
1000
],
"supplementary_groups": [
"wheel",
"kbrazil"
],
"thread_gid": 2225,
"environment_variables": [
"XDG_SESSION_ID=2",
"HOSTNAME=localhost"
],
"major_page_fault_count_delta": 0,
"minor_page_fault_count_delta": 4,
"used": 2.2,
"ipc_namespace_inode": 4026531839,
"mount_namespace_inode": 4026531840,
"net_namespace_inode": 4026531956,
"pid_namespace_inode": 4026531836,
"user_namespace_inode": 4026531837,
"nts_namespace_inode": 4026531838
}
]
}
]
```
### tracepath
```bash
tracepath6 3ffe:2400:0:109::2 | jc --tracepath -p

265
README.md
View File

@@ -144,125 +144,128 @@ option.
### Parsers
| Argument | Command or Filetype | Documentation |
|--------------|-------------------------|-------------------|
| `--acpi` | `acpi` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/acpi) |
| `--airport` | `airport -I` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/airport) |
| `--airport-s` | `airport -s` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/airport_s) |
| `--arp` | `arp` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/arp) |
| `--asciitable` | ASCII and Unicode table parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/asciitable) |
| `--asciitable-m` | multi-line ASCII and Unicode table parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/asciitable_m) |
| `--blkid` | `blkid` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/blkid) |
| `--chage` | `chage --list` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/chage) |
| `--cksum` | `cksum` and `sum` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/cksum) |
| `--crontab` | `crontab` command and file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab) |
| `--crontab-u` | `crontab` file parser with user support | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab_u) |
| `--csv` | CSV file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/csv) |
| `--csv-s` | CSV file streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/csv_s) |
| `--date` | `date` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/date) |
| `--df` | `df` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/df) |
| `--dig` | `dig` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/dig) |
| `--dir` | `dir` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/dir) |
| `--dmidecode` | `dmidecode` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/dmidecode) |
| `--dpkg-l` | `dpkg -l` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/dpkg_l) |
| `--du` | `du` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/du) |
| `--env` | `env` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/env) |
| `--file` | `file` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/file) |
| `--finger` | `finger` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/finger) |
| `--free` | `free` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/free) |
| `--fstab` | `/etc/fstab` file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/fstab) |
| `--git-log` | `git log` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/git_log) |
| `--git-log-s` | `git log` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/git_log_s) |
| `--group` | `/etc/group` file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/group) |
| `--gshadow` | `/etc/gshadow` file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/gshadow) |
| `--hash` | `hash` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/hash) |
| `--hashsum` | hashsum command parser (`md5sum`, `shasum`, etc.) | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/hashsum) |
| `--hciconfig` | `hciconfig` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/hciconfig) |
| `--history` | `history` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/history) |
| `--hosts` | `/etc/hosts` file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/hosts) |
| `--id` | `id` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/id) |
| `--ifconfig` | `ifconfig` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ifconfig) |
| `--ini` | INI file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ini) |
| `--iostat` | `iostat` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/iostat) |
| `--iostat-s` | `iostat` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/iostat_s) |
| `--iptables` | `iptables` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/iptables) |
| `--iw-scan` | `iw dev [device] scan` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/iw_scan) |
| `--jar-manifest` | MANIFEST.MF file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/jar_manifest) |
| `--jobs` | `jobs` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/jobs) |
| `--kv` | Key/Value file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/kv) |
| `--last` | `last` and `lastb` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/last) |
| `--ls` | `ls` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ls) |
| `--ls-s` | `ls` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ls_s) |
| `--lsblk` | `lsblk` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/lsblk) |
| `--lsmod` | `lsmod` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/lsmod) |
| `--lsof` | `lsof` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/lsof) |
| `--lsusb` | `lsusb` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/lsusb) |
| `--mount` | `mount` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/mount) |
| `--mpstat` | `mpstat` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/mpstat) |
| `--mpstat-s` | `mpstat` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/mpstat_s) |
| `--netstat` | `netstat` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/netstat) |
| `--nmcli` | `nmcli` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/nmcli) |
| `--ntpq` | `ntpq -p` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ntpq) |
| `--passwd` | `/etc/passwd` file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/passwd) |
| `--pidstat` | `pidstat` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/pidstat) |
| `--pidstat-s` | `pidstat` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/pidstat_s) |
| `--ping` | `ping` and `ping6` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ping) |
| `--ping-s` | `ping` and `ping6` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ping_s) |
| `--pip-list` | `pip list` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_list) |
| `--pip-show` | `pip show` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_show) |
| `--ps` | `ps` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ps) |
| `--route` | `route` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/route) |
| `--rpm-qi` | `rpm -qi` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/rpm_qi) |
| `--rsync` | `rsync` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/rsync) |
| `--rsync-s` | `rsync` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/rsync_s) |
| `--sfdisk` | `sfdisk` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/sfdisk) |
| `--shadow` | `/etc/shadow` file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/shadow) |
| `--ss` | `ss` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ss) |
| `--stat` | `stat` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/stat) |
| `--stat-s` | `stat` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/stat_s) |
| `--sysctl` | `sysctl` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/sysctl) |
| `--systemctl` | `systemctl` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl) |
| `--systemctl-lj` | `systemctl list-jobs` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_lj) |
| `--systemctl-ls` | `systemctl list-sockets` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_ls) |
| `--systemctl-luf` | `systemctl list-unit-files` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_luf) |
| `--systeminfo` | `systeminfo` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/systeminfo) |
| `--time` | `/usr/bin/time` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/time) |
| `--timedatectl` | `timedatectl status` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/timedatectl) |
| `--tracepath` | `tracepath` and `tracepath6` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/tracepath) |
| `--traceroute` | `traceroute` and `traceroute6` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/traceroute) |
| `--ufw` | `ufw status` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ufw) |
| `--ufw-appinfo` | `ufw app info [application]` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ufw_appinfo) |
| `--uname` | `uname -a` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/uname) |
| `--update-alt-gs` | `update-alternatives --get-selections` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/update_alt_gs) |
| `--update-alt-q` | `update-alternatives --query` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/update_alt_q) |
| `--upower` | `upower` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/upower) |
| `--uptime` | `uptime` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/uptime) |
| `--vmstat` | `vmstat` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/vmstat) |
| `--vmstat-s` | `vmstat` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/vmstat_s) |
| `--w` | `w` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/w) |
| `--wc` | `wc` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/wc) |
| `--who` | `who` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/who) |
| `--xml` | XML file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/xml) |
| `--xrandr` | `xrandr` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/xrandr) |
| `--yaml` | YAML file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/yaml) |
| `--zipinfo` | `zipinfo` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/zipinfo) |
| Argument | Command or Filetype | Documentation |
|-------------------|---------------------------------------------------------|-------------------------------------------------------------------------|
| ` --acpi` | `acpi` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/acpi) |
| ` --airport` | `airport -I` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/airport) |
| ` --airport-s` | `airport -s` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/airport_s) |
| ` --arp` | `arp` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/arp) |
| ` --asciitable` | ASCII and Unicode table parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/asciitable) |
| ` --asciitable-m` | multi-line ASCII and Unicode table parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/asciitable_m) |
| ` --blkid` | `blkid` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/blkid) |
| ` --chage` | `chage --list` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/chage) |
| ` --cksum` | `cksum` and `sum` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/cksum) |
| ` --crontab` | `crontab` command and file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab) |
| ` --crontab-u` | `crontab` file parser with user support | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab_u) |
| ` --csv` | CSV file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/csv) |
| ` --csv-s` | CSV file streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/csv_s) |
| ` --date` | `date` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/date) |
| ` --df` | `df` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/df) |
| ` --dig` | `dig` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/dig) |
| ` --dir` | `dir` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/dir) |
| ` --dmidecode` | `dmidecode` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/dmidecode) |
| ` --dpkg-l` | `dpkg -l` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/dpkg_l) |
| ` --du` | `du` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/du) |
| ` --env` | `env` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/env) |
| ` --file` | `file` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/file) |
| ` --finger` | `finger` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/finger) |
| ` --free` | `free` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/free) |
| ` --fstab` | `/etc/fstab` file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/fstab) |
| ` --git-log` | `git log` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/git_log) |
| ` --git-log-s` | `git log` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/git_log_s) |
| ` --group` | `/etc/group` file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/group) |
| ` --gshadow` | `/etc/gshadow` file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/gshadow) |
| ` --hash` | `hash` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/hash) |
| ` --hashsum` | hashsum command parser (`md5sum`, `shasum`, etc.) | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/hashsum) |
| ` --hciconfig` | `hciconfig` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/hciconfig) |
| ` --history` | `history` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/history) |
| ` --hosts` | `/etc/hosts` file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/hosts) |
| ` --id` | `id` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/id) |
| ` --ifconfig` | `ifconfig` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ifconfig) |
| ` --ini` | INI file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ini) |
| ` --iostat` | `iostat` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/iostat) |
| ` --iostat-s` | `iostat` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/iostat_s) |
| ` --iptables` | `iptables` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/iptables) |
| ` --iw-scan` | `iw dev [device] scan` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/iw_scan) |
| ` --jar-manifest` | MANIFEST.MF file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/jar_manifest) |
| ` --jobs` | `jobs` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/jobs) |
| ` --kv` | Key/Value file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/kv) |
| ` --last` | `last` and `lastb` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/last) |
| ` --ls` | `ls` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ls) |
| ` --ls-s` | `ls` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ls_s) |
| ` --lsblk` | `lsblk` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/lsblk) |
| ` --lsmod` | `lsmod` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/lsmod) |
| ` --lsof` | `lsof` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/lsof) |
| ` --lsusb` | `lsusb` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/lsusb) |
| ` --mount` | `mount` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/mount) |
| ` --mpstat` | `mpstat` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/mpstat) |
| ` --mpstat-s` | `mpstat` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/mpstat_s) |
| ` --netstat` | `netstat` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/netstat) |
| ` --nmcli` | `nmcli` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/nmcli) |
| ` --ntpq` | `ntpq -p` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ntpq) |
| ` --passwd` | `/etc/passwd` file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/passwd) |
| ` --pidstat` | `pidstat -h` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/pidstat) |
| ` --pidstat-s` | `pidstat -h` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/pidstat_s) |
| ` --ping` | `ping` and `ping6` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ping) |
| ` --ping-s` | `ping` and `ping6` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ping_s) |
| ` --pip-list` | `pip list` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_list) |
| ` --pip-show` | `pip show` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_show) |
| ` --postconf` | `postconf -M` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/postconf) |
| ` --ps` | `ps` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ps) |
| ` --route` | `route` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/route) |
| ` --rpm-qi` | `rpm -qi` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/rpm_qi) |
| ` --rsync` | `rsync` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/rsync) |
| ` --rsync-s` | `rsync` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/rsync_s) |
| ` --sfdisk` | `sfdisk` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/sfdisk) |
| ` --shadow` | `/etc/shadow` file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/shadow) |
| ` --ss` | `ss` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ss) |
| ` --stat` | `stat` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/stat) |
| ` --stat-s` | `stat` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/stat_s) |
| ` --sysctl` | `sysctl` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/sysctl) |
| ` --systemctl` | `systemctl` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl) |
| ` --systemctl-lj` | `systemctl list-jobs` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_lj) |
| ` --systemctl-ls` | `systemctl list-sockets` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_ls) |
| `--systemctl-luf` | `systemctl list-unit-files` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_luf) |
| ` --systeminfo` | `systeminfo` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/systeminfo) |
| ` --time` | `/usr/bin/time` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/time) |
| ` --timedatectl` | `timedatectl status` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/timedatectl) |
| ` --top` | `top -b` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/top) |
| ` --top-s` | `top -b` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/top_s) |
| ` --tracepath` | `tracepath` and `tracepath6` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/tracepath) |
| ` --traceroute` | `traceroute` and `traceroute6` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/traceroute) |
| ` --ufw` | `ufw status` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ufw) |
| ` --ufw-appinfo` | `ufw app info [application]` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/ufw_appinfo) |
| ` --uname` | `uname -a` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/uname) |
| `--update-alt-gs` | `update-alternatives --get-selections` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/update_alt_gs) |
| ` --update-alt-q` | `update-alternatives --query` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/update_alt_q) |
| ` --upower` | `upower` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/upower) |
| ` --uptime` | `uptime` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/uptime) |
| ` --vmstat` | `vmstat` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/vmstat) |
| ` --vmstat-s` | `vmstat` command streaming parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/vmstat_s) |
| ` --w` | `w` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/w) |
| ` --wc` | `wc` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/wc) |
| ` --who` | `who` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/who) |
| ` --xml` | XML file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/xml) |
| ` --xrandr` | `xrandr` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/xrandr) |
| ` --yaml` | YAML file parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/yaml) |
| ` --zipinfo` | `zipinfo` command parser | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/zipinfo) |
### Options
- `-a` about `jc`. Prints information about `jc` and the parsers (in JSON, of
course!)
- `-C` force color output even when using pipes (overrides `-m` and the
`NO_COLOR` env variable)
- `-d` debug mode. Prints trace messages if parsing issues are encountered (use
`-dd` for verbose debugging)
- `-h` help. Use `jc -h --parser_name` for parser documentation
- `-m` monochrome JSON output
- `-p` pretty format the JSON output
- `-q` quiet mode. Suppresses parser warning messages (use `-qq` to ignore
streaming parser errors)
- `-r` raw output. Provides a more literal JSON output, typically with string
values and no additional semantic processing
- `-u` unbuffer output
- `-v` version information
| Short | Long | Description |
|-------|-----------------|--------------------------------------------------------------------------------------------------------------|
| `-a` | `--about` | About `jc`. Prints information about `jc` and the parsers (in JSON or YAML, of course!) |
| `-C` | `--force-color` | Force color output even when using pipes (overrides `-m` and the `NO_COLOR` env variable) |
| `-d` | `--debug` | Debug mode. Prints trace messages if parsing issues are encountered (use`-dd` for verbose debugging) |
| `-h` | `--help` | Help. Use `jc -h --parser_name` for parser documentation |
| `-m` | `--monochrome` | Monochrome output |
| `-p` | `--pretty` | Pretty format the JSON output |
| `-q` | `--quiet` | Quiet mode. Suppresses parser warning messages (use `-qq` to ignore streaming parser errors) |
| `-r` | `--raw` | Raw output. Provides more literal output, typically with string values and no additional semantic processing |
| `-u` | `--unbuffer` | Unbuffer output |
| `-v` | `--version` | Version information |
| `-y` | `--yaml-out` | YAML output |
| `-B` | `--bash-comp` | Generate Bash shell completion script |
| `-Z` | `--zsh-comp` | Generate Zsh shell completion script |
### Exit Codes
Any fatal errors within `jc` will generate an exit code of `100`, otherwise the
@@ -309,11 +312,11 @@ color output will override both the `NO_COLOR` environment variable and the `-m`
option.
### Streaming Parsers
Most parsers load all of the data from STDIN, parse it, then output the entire
Most parsers load all of the data from `STDIN`, parse it, then output the entire
JSON document serially. There are some streaming parsers (e.g. `ls-s` and
`ping-s`) that immediately start processing and outputing the data line-by-line
as [JSON Lines](https://jsonlines.org/) (aka [NDJSON](http://ndjson.org/)) while
it is being received from STDIN. This can significantly reduce the amount of
it is being received from `STDIN`. This can significantly reduce the amount of
memory required to parse large amounts of command output (e.g. `ls -lR /`) and
can sometimes process the data more quickly. Streaming parsers have slightly
different behavior than standard parsers as outlined below.
@@ -442,15 +445,30 @@ If a UTC timezone can be detected in the text of the command output, the
timestamp will be timezone aware and have a `_utc` suffix on the key name.
(e.g. `epoch_utc`) No other timezones are supported for aware timestamps.
## Use In Other Shells
`jc` can be used in most any shell. Some modern shells have JSON deserialization
and filtering capabilities built-in which makes using `jc` even more convenient.
For example, the following is possible in [NGS](https://ngs-lang.org/)
(Next Generation Shell):
```bash
myvar = ``jc dig www.google.com``[0].answer[0].data
```
This runs `jc`, parses the output JSON, and assigs the resulting data structure
to a variable in a single line of code.
For more examples of how to use `jc` in other shells, see this
[wiki page](https://github.com/kellyjonbrazil/jc/wiki/Using-jc-With-Different-Shells).
## Compatibility
Some parsers like `dig`, `xml`, `csv`, etc. will work on any platform. Other
parsers that convert platform-specific output will generate a warning message if
they are run on an unsupported platform. To see all parser information,
including compatibility, run `jc -ap`.
You may still use a parser on an unsupported platform - for example, you may want
to parse a file with linux `lsof` output on an macOS or Windows laptop. In that
case you can suppress the warning message with the `-q` cli option or the
You may still use a parser on an unsupported platform - for example, you may
want to parse a file with linux `lsof` output on an macOS or Windows laptop. In
that case you can suppress the warning message with the `-q` cli option or the
`quiet=True` function parameter in `parse()`:
macOS:
@@ -479,7 +497,8 @@ Tested on:
## Contributions
Feel free to add/improve code or parsers! You can use the
[`jc/parsers/foo.py`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py)
or [`jc/parsers/foo_s.py (streaming)`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo_s.py) parsers as a template and submit your parser with a pull request.
or [`jc/parsers/foo_s.py (streaming)`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo_s.py) parsers as a template and submit your parser with a pull
request.
Please see the [Contributing Guidelines](https://github.com/kellyjonbrazil/jc/blob/master/CONTRIBUTING.md) for more information.

View File

@@ -8,7 +8,12 @@ jc - JSON Convert `asciitable` parser
This parser converts ASCII and Unicode text tables with single-line rows.
Column headers must be at least two spaces apart from each other and must
be unique.
be unique. For best results, column headers should be left-justified. If
column separators are present, then non-left-justified headers will be fixed
automatically.
Row separators are optional and are ignored. Each non-row-separator line is
considered a separate row in the table.
For example:
@@ -54,6 +59,9 @@ etc...
Headers (keys) are converted to snake-case. All values are returned as
strings, except empty strings, which are converted to None/null.
> Note: To preserve the case of the keys use the `-r` cli option or
> `raw=True` argument in `parse()`.
Usage (cli):
$ cat table.txt | jc --asciitable
@@ -136,4 +144,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.2 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -29,10 +29,14 @@ Headers (keys) are converted to snake-case and newlines between multi-line
headers are joined with an underscore. All values are returned as strings,
except empty strings, which are converted to None/null.
> Note: To preserve the case of the keys use the `-r` cli option or
> `raw=True` argument in `parse()`.
> Note: table column separator characters (e.g. `|`) cannot be present
inside the cell data. If detected, a warning message will be printed to
STDERR and the line will be skipped. The warning message can be suppressed
by using the `-q` command option or by setting `quiet=True` in `parse()`.
> inside the cell data. If detected, a warning message will be printed to
> `STDERR` and the line will be skipped. The warning message can be
> suppressed by using the `-q` command option or by setting `quiet=True` in
> `parse()`.
Usage (cli):
@@ -125,4 +129,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.1 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.2 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -6,14 +6,14 @@
jc - JSON Convert `csv` file streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
The `csv` streaming parser will attempt to automatically detect the
delimiter character. If the delimiter cannot be detected it will default
to comma. The first row of the file must be a header row.
Note: The first 100 rows are read into memory to enable delimiter detection,
then the rest of the rows are loaded lazily.
> Note: The first 100 rows are read into memory to enable delimiter
> detection, then the rest of the rows are loaded lazily.
Usage (cli):

View File

@@ -19,7 +19,7 @@ time of the system the parser is run on)
Usage (cli):
C:> dir | jc --dir
C:\> dir | jc --dir
Usage (module):
@@ -42,7 +42,7 @@ Schema:
Examples:
C:> dir | jc --dir -p
C:\> dir | jc --dir -p
[
{
"date": "03/24/2021",
@@ -83,7 +83,7 @@ Examples:
...
]
C:> dir | jc --dir -p -r
C:\> dir | jc --dir -p -r
[
{
"date": "03/24/2021",

View File

@@ -43,8 +43,8 @@ Schema:
"author": string,
"author_email": string,
"date": string,
"epoch": integer, [0]
"epoch_utc": integer, [1]
"epoch": integer, # [0]
"epoch_utc": integer, # [1]
"commit_by": string,
"commit_by_email": string,
"commit_by_date": string,

View File

@@ -6,7 +6,7 @@
jc - JSON Convert `git log` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Can be used with the following format options:
- `oneline`
@@ -44,8 +44,8 @@ Schema:
"author": string,
"author_email": string,
"date": string,
"epoch": integer, [0]
"epoch_utc": integer, [1]
"epoch": integer, # [0]
"epoch_utc": integer, # [1]
"commit_by": string,
"commit_by_email": string,
"commit_by_date": string,

View File

@@ -128,4 +128,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, aix, freebsd
Version 1.4 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.5 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -5,7 +5,7 @@
jc - JSON Convert `ifconfig` command output parser
Note: No `ifconfig` options are supported.
> Note: No `ifconfig` options are supported.
Usage (cli):

View File

@@ -11,9 +11,10 @@ Parses standard `INI` files and files containing simple key/value pairs.
- Comment prefix can be `#` or `;`. Comments must be on their own line.
- If duplicate keys are found, only the last value will be used.
> Note: Values starting and ending with quotation marks will have the marks
removed. If you would like to keep the quotation marks, use the `-r`
command-line argument or the `raw=True` argument in `parse()`.
> Note: Values starting and ending with double or single quotation marks
> will have the marks removed. If you would like to keep the quotation
> marks, use the `-r` command-line argument or the `raw=True` argument in
> `parse()`.
Usage (cli):
@@ -26,8 +27,8 @@ Usage (module):
Schema:
ini or key/value document converted to a dictionary - see the
configparser standard library documentation for more details.
ini or key/value document converted to a dictionary - see the configparser
standard library documentation for more details.
{
"key1": string,
@@ -91,4 +92,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.6 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.7 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -5,7 +5,7 @@
jc - JSON Convert `iostat` command output parser
Note: `iostat` version 11 and higher include a JSON output option
> Note: `iostat` version 11 and higher include a JSON output option
Usage (cli):

View File

@@ -6,20 +6,20 @@
jc - JSON Convert `iostat` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Note: `iostat` version 11 and higher include a JSON output option
> Note: `iostat` version 11 and higher include a JSON output option
Usage (cli):
$ iostat | jc --iostat-s
> Note: When piping `jc` converted `iostat` output to other processes it may
appear the output is hanging due to the OS pipe buffers. This is because
`iostat` output is too small to quickly fill up the buffer. Use the `-u`
option to unbuffer the `jc` output if you would like immediate output. See
the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
for more information.
> appear the output is hanging due to the OS pipe buffers. This is because
> `iostat` output is too small to quickly fill up the buffer. Use the `-u`
> option to unbuffer the `jc` output if you would like immediate output. See
> the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
> for more information.
Usage (module):

View File

@@ -12,8 +12,8 @@ Supports files containing simple key/value pairs.
- If duplicate keys are found, only the last value will be used.
> Note: Values starting and ending with quotation marks will have the marks
removed. If you would like to keep the quotation marks, use the `-r`
command-line argument or the `raw=True` argument in `parse()`.
> removed. If you would like to keep the quotation marks, use the `-r`
> command-line argument or the `raw=True` argument in `parse()`.
Usage (cli):
@@ -26,8 +26,8 @@ Usage (module):
Schema:
key/value document converted to a dictionary - see the
configparser standard library documentation for more details.
key/value document converted to a dictionary - see the configparser standard
library documentation for more details.
{
"key1": string,

View File

@@ -9,11 +9,12 @@ Options supported:
- `lbaR1`
- `--time-style=full-iso`
Note: The `-1`, `-l`, or `-b` option of `ls` should be used to correctly
parse filenames that include newline characters. Since `ls` does not encode
newlines in filenames when outputting to a pipe it will cause `jc` to see
multiple files instead of a single file if `-1`, `-l`, or `-b` is not used.
Alternatively, `vdir` can be used, which is the same as running `ls -lb`.
> Note: The `-1`, `-l`, or `-b` option of `ls` should be used to correctly
> parse filenames that include newline characters. Since `ls` does not
> encode newlines in filenames when outputting to a pipe it will cause `jc`
> to see multiple files instead of a single file if `-1`, `-l`, or `-b` is
> not used. Alternatively, `vdir` can be used, which is the same as running
> `ls -lb`.
The `epoch` calculated timestamp field is naive. (i.e. based on the local
time of the system the parser is run on)

View File

@@ -6,7 +6,7 @@
jc - JSON Convert `ls` and `vdir` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Requires the `-l` option to be used on `ls`. If there are newline characters
in the filename, then make sure to use the `-b` option on `ls`.

View File

@@ -22,9 +22,9 @@ Usage (module):
Schema:
Note: <item> object keynames are assigned directly from the lsusb
output. If there are duplicate <item> names in a section, only the
last one is converted.
> Note: <item> object keynames are assigned directly from the lsusb
> output. If there are duplicate <item> names in a section, only the
> last one is converted.
[
{

View File

@@ -5,7 +5,7 @@
jc - JSON Convert `mpstat` command output parser
Note: Latest versions of `mpstat` support JSON output (v11.5.1+)
> Note: Latest versions of `mpstat` support JSON output (v11.5.1+)
Usage (cli):

View File

@@ -6,9 +6,9 @@
jc - JSON Convert `mpstat` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Note: Latest versions of `mpstat` support JSON output (v11.5.1+)
> Note: Latest versions of `mpstat` support JSON output (v11.5.1+)
Usage (cli):

View File

@@ -29,18 +29,18 @@ Usage (module):
Schema:
Because there are so many options, the schema is not strictly defined.
Integer and Float value conversions are attempted and the original
values are kept if they fail. If you don't want automatic conversion,
then use the -r or raw=True option to disable it.
Because there are so many options, the schema is not strictly defined.
Integer and Float value conversions are attempted and the original
values are kept if they fail. If you don't want automatic conversion,
then use the `-r` or `raw=True` option to disable it.
The structure is flat, for the most part, but there are a couple of
"well-known" keys that are further parsed into objects for convenience.
These are documented below.
The structure is flat, for the most part, but there are a couple of
"well-known" keys that are further parsed into objects for convenience.
These are documented below.
[
{
"<key>": string/integer/float, [0]
"<key>": string/integer/float, # [0]
"dhcp4_option_x": {
"name": string,
"value": string/integer/float,

View File

@@ -3,7 +3,7 @@
# jc.parsers.pidstat
jc - JSON Convert `pidstat` command output parser
jc - JSON Convert `pidstat -h` command output parser
Must use the `-h` option in `pidstat`. All other `pidstat` options are
supported in combination with `-h`.

View File

@@ -3,10 +3,10 @@
# jc.parsers.pidstat\_s
jc - JSON Convert `pidstat` command output streaming parser
jc - JSON Convert `pidstat -h` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Must use the `-h` option in `pidstat`. All other `pidstat` options are
supported in combination with `-h`.
@@ -16,11 +16,11 @@ Usage (cli):
$ pidstat | jc --pidstat-s
> Note: When piping `jc` converted `pidstat` output to other processes it
may appear the output is hanging due to the OS pipe buffers. This is
because `pidstat` output is too small to quickly fill up the buffer. Use
the `-u` option to unbuffer the `jc` output if you would like immediate
output. See the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
for more information.
> may appear the output is hanging due to the OS pipe buffers. This is
> because `pidstat` output is too small to quickly fill up the buffer. Use
> the `-u` option to unbuffer the `jc` output if you would like immediate
> output. See the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
> for more information.
Usage (module):

View File

@@ -9,8 +9,8 @@ Supports `ping` and `ping6` output.
Usage (cli):
Note: Use the ping `-c` (count) option, otherwise data will not be
piped to `jc`.
> Note: Use the ping `-c` (count) option, otherwise data will not be
> piped to `jc`.
$ ping -c 3 1.2.3.4 | jc --ping

View File

@@ -6,20 +6,20 @@
jc - JSON Convert `ping` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Supports `ping` and `ping6` output.
Usage (cli):
$ ping | jc --ping-s
$ ping 1.2.3.4 | jc --ping-s
> Note: When piping `jc` converted `ping` output to other processes it may
appear the output is hanging due to the OS pipe buffers. This is because
`ping` output is too small to quickly fill up the buffer. Use the `-u`
option to unbuffer the `jc` output if you would like immediate output.
See the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
for more information.
> appear the output is hanging due to the OS pipe buffers. This is because
> `ping` output is too small to quickly fill up the buffer. Use the `-u`
> option to unbuffer the `jc` output if you would like immediate output.
> See the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
> for more information.
Usage (module):

View File

@@ -70,7 +70,7 @@ Examples:
### parse
```python
def parse(data, raw=False, quiet=False)
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
@@ -88,4 +88,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.3 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.4 by Kelly Brazil (kellyjonbrazil@gmail.com)

115
docs/parsers/postconf.md Normal file
View File

@@ -0,0 +1,115 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.postconf"></a>
# jc.parsers.postconf
jc - JSON Convert `postconf -M` command output parser
Usage (cli):
$ postconf -M | jc --postconf
or
$ jc postconf -M
Usage (module):
import jc
result = jc.parse('postconf', postconf_command_output)
Schema:
[
{
"service_name": string,
"service_type": string,
"private": boolean/null, # [0]
"unprivileged": boolean/null, # [0]
"chroot": boolean/null, # [0]
"wake_up_time": integer/null, # [0]
"no_wake_up_before_first_use": boolean/null, # [1]
"process_limit": integer/null, # [0]
"command": string
}
]
[0] '-' converted to null/None
[1] null/None if `wake_up_time` is null/None
Examples:
$ postconf -M | jc --postconf -p
[
{
"service_name": "smtp",
"service_type": "inet",
"private": false,
"unprivileged": null,
"chroot": true,
"wake_up_time": null,
"process_limit": null,
"command": "smtpd",
"no_wake_up_before_first_use": null
},
{
"service_name": "pickup",
"service_type": "unix",
"private": false,
"unprivileged": null,
"chroot": true,
"wake_up_time": 60,
"process_limit": 1,
"command": "pickup",
"no_wake_up_before_first_use": false
}
]
$ postconf -M | jc --postconf -p -r
[
{
"service_name": "smtp",
"service_type": "inet",
"private": "n",
"unprivileged": "-",
"chroot": "y",
"wake_up_time": "-",
"process_limit": "-",
"command": "smtpd"
},
{
"service_name": "pickup",
"service_type": "unix",
"private": "n",
"unprivileged": "-",
"chroot": "y",
"wake_up_time": "60",
"process_limit": "1",
"command": "pickup"
}
]
<a id="jc.parsers.postconf.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -22,20 +22,20 @@ Schema:
[
{
"destination": string,
"gateway": string,
"genmask": string,
"flags": string,
"destination": string,
"gateway": string,
"genmask": string,
"flags": string,
"flags_pretty": [
string
string
]
"metric": integer,
"ref": integer,
"use": integer,
"mss": integer,
"window": integer,
"irtt": integer,
"iface": string
"metric": integer,
"ref": integer,
"use": integer,
"mss": integer,
"window": integer,
"irtt": integer,
"iface": string
}
]

View File

@@ -10,7 +10,7 @@ Works with `rpm -qi [package]` or `rpm -qia`.
The `..._epoch` calculated timestamp fields are naive. (i.e. based on the
local time of the system the parser is run on)
The `..._epoch_utc` calculated timestamp fields are timezone-aware and is
The `..._epoch_utc` calculated timestamp fields are timezone-aware and are
only available if the timezone field is UTC.
Usage (cli):

View File

@@ -6,7 +6,7 @@
jc - JSON Convert `rsync` command output parser
Supports the `-i` or `--itemize-changes` options with all levels of
verbosity. This parser will process the STDOUT output or a log file
verbosity. This parser will process the `STDOUT` output or a log file
generated with the `--log-file` option.
Usage (cli):
@@ -51,8 +51,8 @@ Schema:
"time": string,
"process": integer,
"metadata": string,
"update_type": string/null, [0]
"file_type": string/null, [1]
"update_type": string/null, # [0]
"file_type": string/null, # [1]
"checksum_or_value_different": bool/null,
"size_different": bool/null,
"modification_time_different": bool/null,
@@ -61,7 +61,7 @@ Schema:
"group_different": bool/null,
"acl_different": bool/null,
"extended_attribute_different": bool/null,
"epoch": integer, [2]
"epoch": integer, # [2]
}
]
}

View File

@@ -6,10 +6,10 @@
jc - JSON Convert `rsync` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Supports the `-i` or `--itemize-changes` options with all levels of
verbosity. This parser will process the STDOUT output or a log file
verbosity. This parser will process the `STDOUT` output or a log file
generated with the `--log-file` option.
Usage (cli):
@@ -49,8 +49,8 @@ Schema:
"time": string,
"process": integer,
"metadata": string,
"update_type": string/null, [0]
"file_type": string/null, [1]
"update_type": string/null, # [0]
"file_type": string/null, # [1]
"checksum_or_value_different": bool/null,
"size_different": bool/null,
"modification_time_different": bool/null,
@@ -59,7 +59,7 @@ Schema:
"group_different": bool/null,
"acl_different": bool/null,
"extended_attribute_different": bool/null,
"epoch": integer, [2]
"epoch": integer, # [2]
# below object only exists if using -qq or ignore_exceptions=True
"_jc_meta": {

View File

@@ -5,8 +5,8 @@
jc - JSON Convert `ss` command output parser
Extended information options like -e and -p are not supported and may cause
parsing irregularities.
Extended information options like `-e` and `-p` are not supported and may
cause parsing irregularities.
Usage (cli):
@@ -23,8 +23,8 @@ Usage (module):
Schema:
Information from https://www.cyberciti.biz/files/ss.html used to define
field names
Information from https://www.cyberciti.biz/files/ss.html used to define
field names
[
{

View File

@@ -6,7 +6,7 @@
jc - JSON Convert `stat` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
The `xxx_epoch` calculated timestamp fields are naive. (i.e. based on the
local time of the system the parser is run on).

View File

@@ -5,10 +5,10 @@
jc - JSON Convert `sysctl -a` command output parser
Note: Since `sysctl` output is not easily parsable only a very simple
key/value object will be output. An attempt is made to convert obvious
integers and floats. If no conversion is desired, use the `-r`
command-line argument or the `raw=True` argument in `parse()`.
> Note: Since `sysctl` output is not easily parsable only a very simple
> key/value object will be output. An attempt is made to convert obvious
> integers and floats. If no conversion is desired, use the `-r`
> command-line argument or the `raw=True` argument in `parse()`.
Usage (cli):

View File

@@ -11,8 +11,8 @@ used to redirect the output to a file that can be read by `jc`.
Alternatively, the output from `/usr/bin/time` can be redirected to `STDOUT`
so `jc` can receive it.
Note: `/usr/bin/time` is similar but different from the Bash builtin
`time` command.
> Note: `/usr/bin/time` is similar but different from the Bash builtin
> `time` command.
Usage (cli):
@@ -26,8 +26,8 @@ Usage (module):
Schema:
Source: https://www.freebsd.org/cgi/man.cgi?query=getrusage
https://man7.org/linux/man-pages/man1/time.1.html
Source: https://www.freebsd.org/cgi/man.cgi?query=getrusage,
https://man7.org/linux/man-pages/man1/time.1.html
{
"real_time": float,

339
docs/parsers/top.md Normal file
View File

@@ -0,0 +1,339 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.top"></a>
# jc.parsers.top
jc - JSON Convert `top -b` command output parser
Requires batch mode (`-b`). The `-n` option must also be used to limit
the number of times `top` is run.
Warning messages will be printed to `STDERR` if truncated fields are
detected. These warnings can be suppressed with the `-q` or `quiet=True`
option.
Usage (cli):
$ top -b -n 3 | jc --top
or
$ jc top -b -n 3
Usage (module):
import jc
result = jc.parse('top', top_command_output)
Schema:
All `-` values are converted to `null`
[
{
"time": string,
"uptime": integer,
"users": integer,
"load_1m": float,
"load_5m": float,
"load_15m": float,
"tasks_total": integer,
"tasks_running": integer,
"tasks_sleeping": integer,
"tasks_stopped": integer,
"tasks_zombie": integer,
"cpu_user": float,
"cpu_sys": float,
"cpu_nice": float,
"cpu_idle": float,
"cpu_wait": float,
"cpu_hardware": float,
"cpu_software": float,
"cpu_steal": float,
"mem_total": float, # [0]
"mem_free": float, # [0]
"mem_used": float, # [0]
"mem_buff_cache": float, # [0]
"swap_total": float, # [0]
"swap_free": float, # [0]
"swap_used": float, # [0]
"mem_available": float, # [0]
"processes": [
{
"pid": integer,
"user": string,
"priority": integer,
"nice": integer,
"virtual_mem": float, # [1]
"resident_mem": float, # [1]
"shared_mem": float, # [1]
"status": string,
"percent_cpu": float,
"percent_mem": float,
"time_hundredths": string,
"command": string,
"parent_pid": integer,
"uid": integer,
"real_uid": integer,
"real_user": string,
"saved_uid": integer,
"saved_user": string,
"gid": integer,
"group": string,
"pgrp": integer,
"tty": string,
"tty_process_gid": integer,
"session_id": integer,
"thread_count": integer,
"last_used_processor": integer,
"time": string,
"swap": float, # [1]
"code": float, # [1]
"data": float, # [1]
"major_page_fault_count": integer,
"minor_page_fault_count": integer,
"dirty_pages_count": integer,
"sleeping_in_function": string,
"flags": string,
"cgroups": string,
"supplementary_gids": [
integer
],
"supplementary_groups": [
string
],
"thread_gid": integer,
"environment_variables": [
string
]
"major_page_fault_count_delta": integer,
"minor_page_fault_count_delta": integer,
"used": float, # [1]
"ipc_namespace_inode": integer,
"mount_namespace_inode": integer,
"net_namespace_inode": integer,
"pid_namespace_inode": integer,
"user_namespace_inode": integer,
"nts_namespace_inode": integer,
"control_group_name": string,
"lxc_container_name": string,
"numa_node": integer,
"out_of_mem_adjustment": integer,
"out_of_mem_score": integer,
"resident_anon_mem": integer,
"resident_file_backed_mem": integer,
"resident_locked_mem": integer,
"resident_shared_mem": integer
}
]
}
]
[0] Values are in the units output by `top`
[1] Unit suffix stripped during float conversion
Examples:
$ top -b -n 3 | jc --top -p
[
{
"time": "11:20:43",
"uptime": 118,
"users": 2,
"load_1m": 0.0,
"load_5m": 0.01,
"load_15m": 0.05,
"tasks_total": 108,
"tasks_running": 2,
"tasks_sleeping": 106,
"tasks_stopped": 0,
"tasks_zombie": 0,
"cpu_user": 5.6,
"cpu_sys": 11.1,
"cpu_nice": 0.0,
"cpu_idle": 83.3,
"cpu_wait": 0.0,
"cpu_hardware": 0.0,
"cpu_software": 0.0,
"cpu_steal": 0.0,
"mem_total": 3.7,
"mem_free": 3.3,
"mem_used": 0.2,
"mem_buff_cache": 0.2,
"swap_total": 2.0,
"swap_free": 2.0,
"swap_used": 0.0,
"mem_available": 3.3,
"processes": [
{
"pid": 2225,
"user": "kbrazil",
"priority": 20,
"nice": 0,
"virtual_mem": 158.1,
"resident_mem": 2.2,
"shared_mem": 1.6,
"status": "running",
"percent_cpu": 12.5,
"percent_mem": 0.1,
"time_hundredths": "0:00.02",
"command": "top",
"parent_pid": 1884,
"uid": 1000,
"real_uid": 1000,
"real_user": "kbrazil",
"saved_uid": 1000,
"saved_user": "kbrazil",
"gid": 1000,
"group": "kbrazil",
"pgrp": 2225,
"tty": "pts/0",
"tty_process_gid": 2225,
"session_id": 1884,
"thread_count": 1,
"last_used_processor": 0,
"time": "0:00",
"swap": 0.0,
"code": 0.1,
"data": 1.0,
"major_page_fault_count": 0,
"minor_page_fault_count": 736,
"dirty_pages_count": 0,
"sleeping_in_function": null,
"flags": "..4.2...",
"cgroups": "1:name=systemd:/user.slice/user-1000.+",
"supplementary_gids": [
10,
1000
],
"supplementary_groups": [
"wheel",
"kbrazil"
],
"thread_gid": 2225,
"environment_variables": [
"XDG_SESSION_ID=2",
"HOSTNAME=localhost"
],
"major_page_fault_count_delta": 0,
"minor_page_fault_count_delta": 4,
"used": 2.2,
"ipc_namespace_inode": 4026531839,
"mount_namespace_inode": 4026531840,
"net_namespace_inode": 4026531956,
"pid_namespace_inode": 4026531836,
"user_namespace_inode": 4026531837,
"nts_namespace_inode": 4026531838
},
...
]
}
]
$ top -b -n 3 | jc --top -p -r
[
{
"time": "11:20:43",
"uptime": "1:18",
"users": "2",
"load_1m": "0.00",
"load_5m": "0.01",
"load_15m": "0.05",
"tasks_total": "108",
"tasks_running": "2",
"tasks_sleeping": "106",
"tasks_stopped": "0",
"tasks_zombie": "0",
"cpu_user": "5.6",
"cpu_sys": "11.1",
"cpu_nice": "0.0",
"cpu_idle": "83.3",
"cpu_wait": "0.0",
"cpu_hardware": "0.0",
"cpu_software": "0.0",
"cpu_steal": "0.0",
"swap_total": "2.0",
"swap_free": "2.0",
"swap_used": "0.0",
"mem_available": "3.3",
"processes": [
{
"PID": "2225",
"USER": "kbrazil",
"PR": "20",
"NI": "0",
"VIRT": "158.1m",
"RES": "2.2m",
"SHR": "1.6m",
"S": "R",
"%CPU": "12.5",
"%MEM": "0.1",
"TIME+": "0:00.02",
"COMMAND": "top",
"PPID": "1884",
"UID": "1000",
"RUID": "1000",
"RUSER": "kbrazil",
"SUID": "1000",
"SUSER": "kbrazil",
"GID": "1000",
"GROUP": "kbrazil",
"PGRP": "2225",
"TTY": "pts/0",
"TPGID": "2225",
"SID": "1884",
"nTH": "1",
"P": "0",
"TIME": "0:00",
"SWAP": "0.0m",
"CODE": "0.1m",
"DATA": "1.0m",
"nMaj": "0",
"nMin": "736",
"nDRT": "0",
"WCHAN": "-",
"Flags": "..4.2...",
"CGROUPS": "1:name=systemd:/user.slice/user-1000.+",
"SUPGIDS": "10,1000",
"SUPGRPS": "wheel,kbrazil",
"TGID": "2225",
"ENVIRON": "XDG_SESSION_ID=2 HOSTNAME=localhost S+",
"vMj": "0",
"vMn": "4",
"USED": "2.2m",
"nsIPC": "4026531839",
"nsMNT": "4026531840",
"nsNET": "4026531956",
"nsPID": "4026531836",
"nsUSER": "4026531837",
"nsUTS": "4026531838"
},
...
]
}
]
<a id="jc.parsers.top.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

180
docs/parsers/top_s.md Normal file
View File

@@ -0,0 +1,180 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.top_s"></a>
# jc.parsers.top\_s
jc - JSON Convert `top -b` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
> Dictionaries (module)
Requires batch mode (`-b`).
Warning messages will be printed to `STDERR` if truncated fields are
detected. These warnings can be suppressed with the `-q` or `quiet=True`
option.
Usage (cli):
$ top -b | jc --top-s
Usage (module):
import jc
result = jc.parse('top_s', top_command_output.splitlines())
for item in result:
# do something
Schema:
{
"time": string,
"uptime": integer,
"users": integer,
"load_1m": float,
"load_5m": float,
"load_15m": float,
"tasks_total": integer,
"tasks_running": integer,
"tasks_sleeping": integer,
"tasks_stopped": integer,
"tasks_zombie": integer,
"cpu_user": float,
"cpu_sys": float,
"cpu_nice": float,
"cpu_idle": float,
"cpu_wait": float,
"cpu_hardware": float,
"cpu_software": float,
"cpu_steal": float,
"mem_total": float, # [0]
"mem_free": float, # [0]
"mem_used": float, # [0]
"mem_buff_cache": float, # [0]
"swap_total": float, # [0]
"swap_free": float, # [0]
"swap_used": float, # [0]
"mem_available": float, # [0]
"processes": [
{
"pid": integer,
"user": string,
"priority": integer,
"nice": integer,
"virtual_mem": float, # [1]
"resident_mem": float, # [1]
"shared_mem": float, # [1]
"status": string,
"percent_cpu": float,
"percent_mem": float,
"time_hundredths": string,
"command": string,
"parent_pid": integer,
"uid": integer,
"real_uid": integer,
"real_user": string,
"saved_uid": integer,
"saved_user": string,
"gid": integer,
"group": string,
"pgrp": integer,
"tty": string,
"tty_process_gid": integer,
"session_id": integer,
"thread_count": integer,
"last_used_processor": integer,
"time": string,
"swap": float, # [1]
"code": float, # [1]
"data": float, # [1]
"major_page_fault_count": integer,
"minor_page_fault_count": integer,
"dirty_pages_count": integer,
"sleeping_in_function": string,
"flags": string,
"cgroups": string,
"supplementary_gids": [
integer
],
"supplementary_groups": [
string
],
"thread_gid": integer,
"environment_variables": [
string
]
"major_page_fault_count_delta": integer,
"minor_page_fault_count_delta": integer,
"used": float, # [1]
"ipc_namespace_inode": integer,
"mount_namespace_inode": integer,
"net_namespace_inode": integer,
"pid_namespace_inode": integer,
"user_namespace_inode": integer,
"nts_namespace_inode": integer,
"control_group_name": string,
"lxc_container_name": string,
"numa_node": integer,
"out_of_mem_adjustment": integer,
"out_of_mem_score": integer,
"resident_anon_mem": integer,
"resident_file_backed_mem": integer,
"resident_locked_mem": integer,
"resident_shared_mem": integer
}
],
# below object only exists if using -qq or ignore_exceptions=True
"_jc_meta": {
"success": boolean, # false if error parsing
"error": string, # exists if "success" is false
"line": string # exists if "success" is false
}
}
[0] Values are in the units output by `top`
[1] Unit suffix stripped during float conversion
Examples:
$ top -b | jc --top-s
{"time":"11:24:50","uptime":2,"users":2,"load_1m":0.23,"load_5m":...}
...
$ top -b | jc --top-s -r
{"time":"11:24:50","uptime":"2 min","users":"2","load_1m":"0.23","lo...}
...
<a id="jc.parsers.top_s.parse"></a>
### parse
```python
@add_jc_meta
def parse(data: Iterable[str],
raw: bool = False,
quiet: bool = False,
ignore_exceptions: bool = False) -> Union[Iterable[Dict], tuple]
```
Main text parsing generator function. Returns an iterable object.
Parameters:
data: (iterable) line-based text data to parse
(e.g. sys.stdin or str.splitlines())
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
ignore_exceptions: (boolean) ignore parsing exceptions if True
Returns:
Iterable of Dictionaries
### Parser Information
Compatibility: linux
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -7,12 +7,12 @@ jc - JSON Convert `traceroute` command output parser
Supports `traceroute` and `traceroute6` output.
Note: On some operating systems you will need to redirect `STDERR` to
`STDOUT` for destination info since the header line is sent to
`STDERR`. A warning message will be printed to `STDERR` if the
header row is not found.
e.g. `$ traceroute 8.8.8.8 2>&1 | jc --traceroute`
> Note: On some operating systems you will need to redirect `STDERR` to
> `STDOUT` for destination info since the header line is sent to
> `STDERR`. A warning message will be printed to `STDERR` if the
> header row is not found.
>
> e.g. `$ traceroute 8.8.8.8 2>&1 | jc --traceroute`
Usage (cli):

View File

@@ -5,7 +5,7 @@
jc - JSON Convert `uname -a` command output parser
Note: Must use `uname -a`
> Note: Must use `uname -a`
Usage (cli):

View File

@@ -6,7 +6,7 @@
jc - JSON Convert `vmstat` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Options supported: `-a`, `-w`, `-d`, `-t`
@@ -21,11 +21,11 @@ Usage (cli):
$ vmstat | jc --vmstat-s
> Note: When piping `jc` converted `vmstat` output to other processes it may
appear the output is hanging due to the OS pipe buffers. This is because
`vmstat` output is too small to quickly fill up the buffer. Use the `-u`
option to unbuffer the `jc` output if you would like immediate output. See
the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
for more information.
> appear the output is hanging due to the OS pipe buffers. This is because
> `vmstat` output is too small to quickly fill up the buffer. Use the `-u`
> option to unbuffer the `jc` output if you would like immediate output. See
> the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
> for more information.
Usage (module):
@@ -75,9 +75,9 @@ Schema:
# below object only exists if using -qq or ignore_exceptions=True
"_jc_meta": {
"success": boolean, # [2]
"error": string, # [3]
"line": string # [3]
"success": boolean, # [2]
"error": string, # [3]
"line": string # [3]
}
}

View File

@@ -16,8 +16,8 @@ Usage (module):
Schema:
XML Document converted to a Dictionary
See https://github.com/martinblech/xmltodict for details
XML Document converted to a Dictionary. See https://github.com/martinblech/xmltodict
for details.
{
"key1": string/object,

View File

@@ -5,7 +5,7 @@
jc - JSON Convert `YAML` file parser
Note: datetime objects will be converted to strings.
> Note: `datetime` objects will be converted to strings.
Usage (cli):

View File

@@ -5,10 +5,7 @@
jc - JSON Convert `zipinfo` command output parser
Options supported:
- none
Note: The default listing format.
> Note: No `zipinfo` options are supported.
Usage (cli):

View File

@@ -3,6 +3,7 @@
* [jc.utils](#jc.utils)
* [warning\_message](#jc.utils.warning_message)
* [error\_message](#jc.utils.error_message)
* [is\_compatible](#jc.utils.is_compatible)
* [compatibility](#jc.utils.compatibility)
* [has\_data](#jc.utils.has_data)
* [convert\_to\_int](#jc.utils.convert_to_int)
@@ -26,8 +27,8 @@ jc - JSON Convert utils
def warning_message(message_lines: List[str]) -> None
```
Prints warning message for non-fatal issues. The first line is
prepended with 'jc: Warning - ' and subsequent lines are indented.
Prints warning message to `STDERR` for non-fatal issues. The first line
is prepended with 'jc: Warning - ' and subsequent lines are indented.
Wraps text as needed based on the terminal width.
Parameters:
@@ -46,7 +47,7 @@ Returns:
def error_message(message_lines: List[str]) -> None
```
Prints an error message for fatal issues. The first line is
Prints an error message to `STDERR` for fatal issues. The first line is
prepended with 'jc: Error - ' and subsequent lines are indented.
Wraps text as needed based on the terminal width.
@@ -58,6 +59,16 @@ Returns:
None - just prints output to STDERR
<a id="jc.utils.is_compatible"></a>
### is\_compatible
```python
def is_compatible(compatible: List) -> bool
```
Returns True if the parser is compatible with the running OS platform.
<a id="jc.utils.compatibility"></a>
### compatibility
@@ -68,8 +79,8 @@ def compatibility(mod_name: str,
quiet: bool = False) -> None
```
Checks for the parser's compatibility with the running OS
platform.
Checks for the parser's compatibility with the running OS platform and
prints a warning message to `STDERR` if not compatible and quiet=False.
Parameters:

239
jc/cli.py
View File

@@ -2,16 +2,20 @@
JC cli module
"""
import io
import sys
import os
import textwrap
import signal
import shlex
import subprocess
import json
from typing import List, Dict
from .lib import (__version__, parser_info, all_parser_info, parsers,
_get_parser, _parser_is_streaming)
_get_parser, _parser_is_streaming, standard_parser_mod_list,
plugin_parser_mod_list, streaming_parser_mod_list)
from . import utils
from .cli_data import long_options_map
from .shell_completions import bash_completion, zsh_completion
from . import tracebackplus
from .exceptions import LibraryNotInstalled, ParseError
@@ -21,13 +25,12 @@ try:
from pygments import highlight
from pygments.style import Style
from pygments.token import (Name, Number, String, Keyword)
from pygments.lexers import JsonLexer
from pygments.lexers.data import JsonLexer, YamlLexer
from pygments.formatters import Terminal256Formatter
PYGMENTS_INSTALLED = True
except Exception:
PYGMENTS_INSTALLED = False
JC_ERROR_EXIT = 100
@@ -134,7 +137,7 @@ def set_env_colors(env_colors=None):
def piped_output(force_color):
"""
Return False if stdout is a TTY. True if output is being piped to
Return False if `STDOUT` is a TTY. True if output is being piped to
another program and foce_color is True. This allows forcing of ANSI
color codes even when using pipes.
"""
@@ -166,6 +169,22 @@ def parsers_text(indent=0, pad=0):
return ptext
def options_text(indent=0, pad=0):
"""Return the argument and description information from each option"""
otext = ''
padding_char = ' '
for option in long_options_map:
o_short = '-' + long_options_map[option][0]
o_desc = long_options_map[option][1]
o_combined = o_short + ', ' + option
padding = pad - len(o_combined)
indent_text = padding_char * indent
padding_text = padding_char * padding
otext += indent_text + o_combined + padding_text + o_desc + '\n'
return otext
def about_jc():
"""Return jc info and the contents of each parser.info as a dictionary"""
return {
@@ -180,48 +199,44 @@ def about_jc():
'python_version': '.'.join((str(sys.version_info.major), str(sys.version_info.minor), str(sys.version_info.micro))),
'python_path': sys.executable,
'parser_count': len(all_parser_info()),
'standard_parser_count': len(standard_parser_mod_list()),
'streaming_parser_count': len(streaming_parser_mod_list()),
'plugin_parser_count': len(plugin_parser_mod_list()),
'parsers': all_parser_info()
}
def helptext():
"""Return the help text with the list of parsers"""
parsers_string = parsers_text(indent=12, pad=17)
parsers_string = parsers_text(indent=4, pad=20)
options_string = options_text(indent=4, pad=20)
helptext_string = f'''\
jc converts the output of many commands and file-types to JSON
jc converts the output of many commands and file-types to JSON or YAML
Usage: COMMAND | jc PARSER [OPTIONS]
Usage:
COMMAND | jc PARSER [OPTIONS]
or magic syntax:
or magic syntax:
jc [OPTIONS] COMMAND
jc [OPTIONS] COMMAND
Parsers:
Parsers:
{parsers_string}
Options:
-a about jc
-C force color output even when using pipes (overrides -m)
-d debug (-dd for verbose debug)
-h help (-h --parser_name for parser documentation)
-m monochrome output
-p pretty print output
-q quiet - suppress parser warnings (-qq to ignore streaming errors)
-r raw JSON output
-u unbuffer output
-v version info
Options:
{options_string}
Examples:
Standard Syntax:
$ dig www.google.com | jc --dig --pretty
Examples:
Standard Syntax:
$ dig www.google.com | jc --dig -p
Magic Syntax:
$ jc --pretty dig www.google.com
Magic Syntax:
$ jc -p dig www.google.com
Parser Documentation:
$ jc --help --dig
'''
Parser Documentation:
$ jc -h --dig
'''
return textwrap.dedent(helptext_string)
return helptext_string
def help_doc(options):
@@ -263,11 +278,52 @@ def versiontext():
return textwrap.dedent(versiontext_string)
def yaml_out(data, pretty=False, env_colors=None, mono=False, piped_out=False, ascii_only=False):
"""
Return a YAML formatted string. String may include color codes. If the
YAML library is not installed, output will fall back to JSON with a
warning message to STDERR"""
# make ruamel.yaml import optional
try:
from ruamel.yaml import YAML
YAML_INSTALLED = True
except Exception:
YAML_INSTALLED = False
if YAML_INSTALLED:
y_string_buf = io.BytesIO()
# monkey patch to disable plugins since we don't use them and in
# ruamel.yaml versions prior to 0.17.0 the use of __file__ in the
# plugin code is incompatible with the pyoxidizer packager
YAML.official_plug_ins = lambda a: []
yaml = YAML()
yaml.default_flow_style = False
yaml.explicit_start = True
yaml.allow_unicode = not ascii_only
yaml.encoding = 'utf-8'
yaml.dump(data, y_string_buf)
y_string = y_string_buf.getvalue().decode('utf-8')[:-1]
if not mono and not piped_out:
# set colors
class JcStyle(Style):
styles = set_env_colors(env_colors)
return str(highlight(y_string, YamlLexer(), Terminal256Formatter(style=JcStyle))[0:-1])
return y_string
utils.warning_message(['YAML Library not installed. Reverting to JSON output.'])
return json_out(data, pretty=pretty, env_colors=env_colors, mono=mono, piped_out=piped_out, ascii_only=ascii_only)
def json_out(data, pretty=False, env_colors=None, mono=False, piped_out=False, ascii_only=False):
"""
Return a JSON formatted string. String may include color codes or be
pretty printed.
"""
import json
separators = (',', ':')
indent = None
@@ -287,24 +343,42 @@ def json_out(data, pretty=False, env_colors=None, mono=False, piped_out=False, a
return j_string
def safe_print_json(list_or_dict, pretty=None, env_colors=None, mono=None,
piped_out=None, flush=None):
"""Safely prints JSON output in both UTF-8 and ASCII systems"""
try:
print(json_out(list_or_dict,
pretty=pretty,
env_colors=env_colors,
mono=mono,
piped_out=piped_out),
flush=flush)
except UnicodeEncodeError:
print(json_out(list_or_dict,
pretty=pretty,
env_colors=env_colors,
mono=mono,
piped_out=piped_out,
ascii_only=True),
flush=flush)
def safe_print_out(list_or_dict, pretty=None, env_colors=None, mono=None,
piped_out=None, flush=None, yaml=None):
"""Safely prints JSON or YAML output in both UTF-8 and ASCII systems"""
if yaml:
try:
print(yaml_out(list_or_dict,
pretty=pretty,
env_colors=env_colors,
mono=mono,
piped_out=piped_out),
flush=flush)
except UnicodeEncodeError:
print(yaml_out(list_or_dict,
pretty=pretty,
env_colors=env_colors,
mono=mono,
piped_out=piped_out,
ascii_only=True),
flush=flush)
else:
try:
print(json_out(list_or_dict,
pretty=pretty,
env_colors=env_colors,
mono=mono,
piped_out=piped_out),
flush=flush)
except UnicodeEncodeError:
print(json_out(list_or_dict,
pretty=pretty,
env_colors=env_colors,
mono=mono,
piped_out=piped_out,
ascii_only=True),
flush=flush)
def magic_parser(args):
@@ -318,7 +392,7 @@ def magic_parser(args):
jc_options (list) list of jc options
"""
# bail immediately if there are no args or a parser is defined
if len(args) <= 1 or args[1].startswith('--'):
if len(args) <= 1 or (args[1].startswith('--') and args[1] not in long_options_map):
return False, None, None, []
args_given = args[1:]
@@ -326,6 +400,12 @@ def magic_parser(args):
# find the options
for arg in list(args_given):
# long option found - populate option list
if arg in long_options_map:
options.extend(long_options_map[arg][0])
args_given.pop(0)
continue
# parser found - use standard syntax
if arg.startswith('--'):
return False, None, None, []
@@ -420,6 +500,9 @@ def main():
# find options if magic_parser did not find a command
if not valid_command:
for opt in sys.argv:
if opt in long_options_map:
options.extend(long_options_map[opt][0])
if opt.startswith('-') and not opt.startswith('--'):
options.extend(opt[1:])
@@ -435,6 +518,9 @@ def main():
raw = 'r' in options
unbuffer = 'u' in options
version_info = 'v' in options
yaml_out = 'y' in options
bash_comp = 'B' in options
zsh_comp = 'Z' in options
if verbose_debug:
tracebackplus.enable(context=11)
@@ -443,11 +529,12 @@ def main():
mono = True
if about:
safe_print_json(about_jc(),
pretty=pretty,
env_colors=jc_colors,
mono=mono,
piped_out=piped_output(force_color))
safe_print_out(about_jc(),
pretty=pretty,
env_colors=jc_colors,
mono=mono,
piped_out=piped_output(force_color),
yaml=yaml_out)
sys.exit(0)
if help_me:
@@ -458,6 +545,14 @@ def main():
utils._safe_print(versiontext())
sys.exit(0)
if bash_comp:
utils._safe_print(bash_completion())
sys.exit(0)
if zsh_comp:
utils._safe_print(zsh_completion())
sys.exit(0)
# if magic syntax used, try to run the command and error if it's not found, etc.
magic_stdout, magic_stderr, magic_exit_code = None, None, 0
if run_command:
@@ -534,12 +629,13 @@ def main():
quiet=quiet,
ignore_exceptions=ignore_exceptions)
for line in result:
safe_print_json(line,
pretty=pretty,
env_colors=jc_colors,
mono=mono,
piped_out=piped_output(force_color),
flush=unbuffer)
safe_print_out(line,
pretty=pretty,
env_colors=jc_colors,
mono=mono,
piped_out=piped_output(force_color),
flush=unbuffer,
yaml=yaml_out)
sys.exit(combined_exit_code(magic_exit_code, 0))
@@ -550,12 +646,13 @@ def main():
raw=raw,
quiet=quiet)
safe_print_json(result,
pretty=pretty,
env_colors=jc_colors,
mono=mono,
piped_out=piped_output(force_color),
flush=unbuffer)
safe_print_out(result,
pretty=pretty,
env_colors=jc_colors,
mono=mono,
piped_out=piped_output(force_color),
flush=unbuffer,
yaml=yaml_out)
sys.exit(combined_exit_code(magic_exit_code, 0))
@@ -570,14 +667,6 @@ def main():
])
sys.exit(combined_exit_code(magic_exit_code, JC_ERROR_EXIT))
except json.JSONDecodeError:
if debug:
raise
utils.error_message(['There was an issue generating the JSON output.',
'For details use the -d or -dd option.'])
sys.exit(combined_exit_code(magic_exit_code, JC_ERROR_EXIT))
except Exception:
if debug:
raise

18
jc/cli_data.py Normal file
View File

@@ -0,0 +1,18 @@
"""jc - JSON Convert cli_data module"""
from typing import List, Dict
long_options_map: Dict[str, List[str]] = {
'--about': ['a', 'about jc'],
'--force-color': ['C', 'force color output even when using pipes (overrides -m)'],
'--debug': ['d', 'debug (double for verbose debug)'],
'--help': ['h', 'help (--help --parser_name for parser documentation)'],
'--monochrome': ['m', 'monochrome output'],
'--pretty': ['p', 'pretty print output'],
'--quiet': ['q', 'suppress warnings (double to ignore streaming errors)'],
'--raw': ['r', 'raw output'],
'--unbuffer': ['u', 'unbuffer output'],
'--version': ['v', 'version info'],
'--yaml-out': ['y', 'YAML output'],
'--bash-comp': ['B', 'gen Bash completion: jc -B > /etc/bash_completion.d/jc'],
'--zsh-comp': ['Z', 'gen Zsh completion: jc -Z > "${fpath[1]}/_jc"']
}

View File

@@ -6,7 +6,7 @@ import importlib
from typing import Dict, List, Iterable, Union, Iterator
from jc import appdirs
__version__ = '1.19.0'
__version__ = '1.20.1'
parsers = [
'acpi',
@@ -73,6 +73,7 @@ parsers = [
'ping-s',
'pip-list',
'pip-show',
'postconf',
'ps',
'route',
'rpm-qi',
@@ -91,6 +92,8 @@ parsers = [
'systeminfo',
'time',
'timedatectl',
'top',
'top-s',
'tracepath',
'traceroute',
'ufw',

View File

@@ -3,7 +3,12 @@
This parser converts ASCII and Unicode text tables with single-line rows.
Column headers must be at least two spaces apart from each other and must
be unique.
be unique. For best results, column headers should be left-justified. If
column separators are present, then non-left-justified headers will be fixed
automatically.
Row separators are optional and are ignored. Each non-row-separator line is
considered a separate row in the table.
For example:
@@ -49,6 +54,9 @@ etc...
Headers (keys) are converted to snake-case. All values are returned as
strings, except empty strings, which are converted to None/null.
> Note: To preserve the case of the keys use the `-r` cli option or
> `raw=True` argument in `parse()`.
Usage (cli):
$ cat table.txt | jc --asciitable
@@ -117,7 +125,7 @@ from jc.parsers.universal import sparse_table_parse
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
version = '1.2'
description = 'ASCII and Unicode table parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -139,6 +147,12 @@ def _process(proc_data: List[Dict]) -> List[Dict]:
List of Dictionaries. Structured to conform to the schema.
"""
# normalize keys: convert to lowercase
for item in proc_data:
for key in item.copy():
k_new = key.lower()
item[k_new] = item.pop(key)
return proc_data
@@ -222,18 +236,19 @@ def _is_separator(line: str) -> bool:
def _snake_case(line: str) -> str:
"""
replace spaces between words and special characters with an underscore
and set to lowercase
Replace spaces between words and special characters with an underscore.
Ignore the replacement char (�) used for header padding.
"""
line = re.sub(r'[^a-zA-Z0-9 ]', '_', line)
return re.sub(r'\b \b', '_', line).lower()
line = re.sub(r'[^a-zA-Z0-9 ]', '_', line) # special characters
line = re.sub(r'\b \b', '_', line) # spaces between words
return line
def _normalize_rows(table: str) -> List[str]:
"""
returns a List of row strings. Header is snake-cased
"""
result = []
result: List[str] = []
for line in table.splitlines():
# skip blank lines
if not line.strip():
@@ -243,7 +258,36 @@ def _normalize_rows(table: str) -> List[str]:
if _is_separator(line):
continue
# data row - remove column separators
# header or data row found - remove column separators
if not result: # this is the header row
# normalize the separator
line = line.replace('', '|')\
.replace('', '|')\
.replace('', '|')\
.replace('', '|')\
.replace('', '|')\
.replace('', '|')\
.replace('', '|')\
.replace('', '|')\
.replace('', '|')
# find the number of chars to pad in front of headers that are too
# far away from the separator. Replace spaces with unicode char: �
# we will remove this char from headers after sparse_table_parse
problem_header_pattern = re.compile(r'(?:\| )( +)([^|]+)')
problem_headers = problem_header_pattern.findall(line)
if problem_headers:
for p_header in problem_headers:
old_header = p_header[0] + p_header[1]
sub_chars = '' * len(p_header[0])
new_header = sub_chars + p_header[1]
line = line.replace(old_header, new_header)
line = line.replace('|', ' ')
result.append(_snake_case(line))
continue
# this is a data row
line = line.replace('|', ' ')\
.replace('', ' ')\
.replace('', ' ')\
@@ -256,7 +300,6 @@ def _normalize_rows(table: str) -> List[str]:
.replace('', ' ')
result.append(line)
result[0] = _snake_case(result[0])
return result
@@ -266,7 +309,8 @@ def _fixup_headers(table: List[Dict]) -> List[Dict]:
for row in table:
new_row = row.copy()
for k in row:
k_new = k
# remove replacement character
k_new = k.replace('', '')
# remove consecutive underscores
k_new = re.sub(r'__+', '_', k_new)
# remove trailing underscores

View File

@@ -24,10 +24,14 @@ Headers (keys) are converted to snake-case and newlines between multi-line
headers are joined with an underscore. All values are returned as strings,
except empty strings, which are converted to None/null.
> Note: To preserve the case of the keys use the `-r` cli option or
> `raw=True` argument in `parse()`.
> Note: table column separator characters (e.g. `|`) cannot be present
inside the cell data. If detected, a warning message will be printed to
STDERR and the line will be skipped. The warning message can be suppressed
by using the `-q` command option or by setting `quiet=True` in `parse()`.
> inside the cell data. If detected, a warning message will be printed to
> `STDERR` and the line will be skipped. The warning message can be
> suppressed by using the `-q` command option or by setting `quiet=True` in
> `parse()`.
Usage (cli):
@@ -106,7 +110,7 @@ from jc.exceptions import ParseError
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.1'
version = '1.2'
description = 'multi-line ASCII and Unicode table parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -128,6 +132,12 @@ def _process(proc_data: List[Dict]) -> List[Dict]:
List of Dictionaries. Structured to conform to the schema.
"""
# normalize keys: convert to lowercase
for item in proc_data:
for key in item.copy():
k_new = key.lower()
item[k_new] = item.pop(key)
return proc_data
@@ -232,12 +242,11 @@ def _is_separator(line: str) -> bool:
def _snake_case(line: str) -> str:
"""
replace spaces between words and special characters with an underscore
and set to lowercase
replace spaces between words and special characters with an underscore.
"""
# must include all column separator characters in regex
line = re.sub(r'[^a-zA-Z0-9 |│┃┆┇┊┋╎╏║]', '_', line)
return re.sub(r'\b \b', '_', line).lower()
return re.sub(r'\b \b', '_', line)
def _fixup_separators(line: str) -> str:

View File

@@ -1,14 +1,14 @@
"""jc - JSON Convert `csv` file streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
The `csv` streaming parser will attempt to automatically detect the
delimiter character. If the delimiter cannot be detected it will default
to comma. The first row of the file must be a header row.
Note: The first 100 rows are read into memory to enable delimiter detection,
then the rest of the rows are loaded lazily.
> Note: The first 100 rows are read into memory to enable delimiter
> detection, then the rest of the rows are loaded lazily.
Usage (cli):

View File

@@ -14,7 +14,7 @@ time of the system the parser is run on)
Usage (cli):
C:> dir | jc --dir
C:\> dir | jc --dir
Usage (module):
@@ -37,7 +37,7 @@ Schema:
Examples:
C:> dir | jc --dir -p
C:\> dir | jc --dir -p
[
{
"date": "03/24/2021",
@@ -78,7 +78,7 @@ Examples:
...
]
C:> dir | jc --dir -p -r
C:\> dir | jc --dir -p -r
[
{
"date": "03/24/2021",

View File

@@ -1,7 +1,7 @@
"""jc - JSON Convert `foo` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
<<Short foo description and caveats>>

View File

@@ -38,8 +38,8 @@ Schema:
"author": string,
"author_email": string,
"date": string,
"epoch": integer, [0]
"epoch_utc": integer, [1]
"epoch": integer, # [0]
"epoch_utc": integer, # [1]
"commit_by": string,
"commit_by_email": string,
"commit_by_date": string,

View File

@@ -1,7 +1,7 @@
"""jc - JSON Convert `git log` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Can be used with the following format options:
- `oneline`
@@ -39,8 +39,8 @@ Schema:
"author": string,
"author_email": string,
"date": string,
"epoch": integer, [0]
"epoch_utc": integer, [1]
"epoch": integer, # [0]
"epoch_utc": integer, # [1]
"commit_by": string,
"commit_by_email": string,
"commit_by_date": string,

View File

@@ -105,7 +105,7 @@ import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.4'
version = '1.5'
description = '`id` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -144,6 +144,13 @@ def _process(proc_data):
return proc_data
def _get_item(my_list, index, default=None):
if index < len(my_list):
return my_list[index]
return default
def parse(data, raw=False, quiet=False):
"""
Main text parsing function
@@ -174,14 +181,14 @@ def parse(data, raw=False, quiet=False):
uid_parsed = uid_parsed.split('=')
raw_output['uid'] = {}
raw_output['uid']['id'] = uid_parsed[1]
raw_output['uid']['name'] = uid_parsed[2]
raw_output['uid']['name'] = _get_item(uid_parsed, 2)
if section.startswith('gid'):
gid_parsed = section.replace('(', '=').replace(')', '=')
gid_parsed = gid_parsed.split('=')
raw_output['gid'] = {}
raw_output['gid']['id'] = gid_parsed[1]
raw_output['gid']['name'] = gid_parsed[2]
raw_output['gid']['name'] = _get_item(gid_parsed, 2)
if section.startswith('groups'):
groups_parsed = section.replace('(', '=').replace(')', '=')
@@ -193,7 +200,7 @@ def parse(data, raw=False, quiet=False):
group_dict = {}
grp_parsed = group.split('=')
group_dict['id'] = grp_parsed[0]
group_dict['name'] = grp_parsed[1]
group_dict['name'] = _get_item(grp_parsed, 1)
raw_output['groups'].append(group_dict)
if section.startswith('context'):

View File

@@ -1,6 +1,6 @@
"""jc - JSON Convert `ifconfig` command output parser
Note: No `ifconfig` options are supported.
> Note: No `ifconfig` options are supported.
Usage (cli):

View File

@@ -6,9 +6,10 @@ Parses standard `INI` files and files containing simple key/value pairs.
- Comment prefix can be `#` or `;`. Comments must be on their own line.
- If duplicate keys are found, only the last value will be used.
> Note: Values starting and ending with quotation marks will have the marks
removed. If you would like to keep the quotation marks, use the `-r`
command-line argument or the `raw=True` argument in `parse()`.
> Note: Values starting and ending with double or single quotation marks
> will have the marks removed. If you would like to keep the quotation
> marks, use the `-r` command-line argument or the `raw=True` argument in
> `parse()`.
Usage (cli):
@@ -21,8 +22,8 @@ Usage (module):
Schema:
ini or key/value document converted to a dictionary - see the
configparser standard library documentation for more details.
ini or key/value document converted to a dictionary - see the configparser
standard library documentation for more details.
{
"key1": string,
@@ -69,7 +70,7 @@ import configparser
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.6'
version = '1.7'
description = 'INI file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -99,17 +100,21 @@ def _process(proc_data):
for key, value in proc_data[heading].items():
if value is not None and value.startswith('"') and value.endswith('"'):
proc_data[heading][key] = value.lstrip('"').rstrip('"')
elif value is not None and value.startswith("'") and value.endswith("'"):
proc_data[heading][key] = value.lstrip("'").rstrip("'")
elif value is None:
proc_data[heading][key] = ''
# simple key/value files with no headers
else:
if (proc_data[heading] is not None and
proc_data[heading].startswith('"') and
proc_data[heading].endswith('"')):
if proc_data[heading] is not None and proc_data[heading].startswith('"') and proc_data[heading].endswith('"'):
proc_data[heading] = proc_data[heading].lstrip('"').rstrip('"')
elif proc_data[heading] is not None and proc_data[heading].startswith("'") and proc_data[heading].endswith("'"):
proc_data[heading] = proc_data[heading].lstrip("'").rstrip("'")
elif proc_data[heading] is None:
proc_data[heading] = ''

View File

@@ -1,6 +1,6 @@
"""jc - JSON Convert `iostat` command output parser
Note: `iostat` version 11 and higher include a JSON output option
> Note: `iostat` version 11 and higher include a JSON output option
Usage (cli):

View File

@@ -1,20 +1,20 @@
"""jc - JSON Convert `iostat` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Note: `iostat` version 11 and higher include a JSON output option
> Note: `iostat` version 11 and higher include a JSON output option
Usage (cli):
$ iostat | jc --iostat-s
> Note: When piping `jc` converted `iostat` output to other processes it may
appear the output is hanging due to the OS pipe buffers. This is because
`iostat` output is too small to quickly fill up the buffer. Use the `-u`
option to unbuffer the `jc` output if you would like immediate output. See
the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
for more information.
> appear the output is hanging due to the OS pipe buffers. This is because
> `iostat` output is too small to quickly fill up the buffer. Use the `-u`
> option to unbuffer the `jc` output if you would like immediate output. See
> the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
> for more information.
Usage (module):

View File

@@ -7,8 +7,8 @@ Supports files containing simple key/value pairs.
- If duplicate keys are found, only the last value will be used.
> Note: Values starting and ending with quotation marks will have the marks
removed. If you would like to keep the quotation marks, use the `-r`
command-line argument or the `raw=True` argument in `parse()`.
> removed. If you would like to keep the quotation marks, use the `-r`
> command-line argument or the `raw=True` argument in `parse()`.
Usage (cli):
@@ -21,8 +21,8 @@ Usage (module):
Schema:
key/value document converted to a dictionary - see the
configparser standard library documentation for more details.
key/value document converted to a dictionary - see the configparser standard
library documentation for more details.
{
"key1": string,

View File

@@ -4,11 +4,12 @@ Options supported:
- `lbaR1`
- `--time-style=full-iso`
Note: The `-1`, `-l`, or `-b` option of `ls` should be used to correctly
parse filenames that include newline characters. Since `ls` does not encode
newlines in filenames when outputting to a pipe it will cause `jc` to see
multiple files instead of a single file if `-1`, `-l`, or `-b` is not used.
Alternatively, `vdir` can be used, which is the same as running `ls -lb`.
> Note: The `-1`, `-l`, or `-b` option of `ls` should be used to correctly
> parse filenames that include newline characters. Since `ls` does not
> encode newlines in filenames when outputting to a pipe it will cause `jc`
> to see multiple files instead of a single file if `-1`, `-l`, or `-b` is
> not used. Alternatively, `vdir` can be used, which is the same as running
> `ls -lb`.
The `epoch` calculated timestamp field is naive. (i.e. based on the local
time of the system the parser is run on)

View File

@@ -1,7 +1,7 @@
"""jc - JSON Convert `ls` and `vdir` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Requires the `-l` option to be used on `ls`. If there are newline characters
in the filename, then make sure to use the `-b` option on `ls`.

View File

@@ -17,9 +17,9 @@ Usage (module):
Schema:
Note: <item> object keynames are assigned directly from the lsusb
output. If there are duplicate <item> names in a section, only the
last one is converted.
> Note: <item> object keynames are assigned directly from the lsusb
> output. If there are duplicate <item> names in a section, only the
> last one is converted.
[
{

View File

@@ -1,6 +1,6 @@
"""jc - JSON Convert `mpstat` command output parser
Note: Latest versions of `mpstat` support JSON output (v11.5.1+)
> Note: Latest versions of `mpstat` support JSON output (v11.5.1+)
Usage (cli):

View File

@@ -1,9 +1,9 @@
"""jc - JSON Convert `mpstat` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Note: Latest versions of `mpstat` support JSON output (v11.5.1+)
> Note: Latest versions of `mpstat` support JSON output (v11.5.1+)
Usage (cli):

View File

@@ -24,18 +24,18 @@ Usage (module):
Schema:
Because there are so many options, the schema is not strictly defined.
Integer and Float value conversions are attempted and the original
values are kept if they fail. If you don't want automatic conversion,
then use the -r or raw=True option to disable it.
Because there are so many options, the schema is not strictly defined.
Integer and Float value conversions are attempted and the original
values are kept if they fail. If you don't want automatic conversion,
then use the `-r` or `raw=True` option to disable it.
The structure is flat, for the most part, but there are a couple of
"well-known" keys that are further parsed into objects for convenience.
These are documented below.
The structure is flat, for the most part, but there are a couple of
"well-known" keys that are further parsed into objects for convenience.
These are documented below.
[
{
"<key>": string/integer/float, [0]
"<key>": string/integer/float, # [0]
"dhcp4_option_x": {
"name": string,
"value": string/integer/float,

View File

@@ -1,4 +1,4 @@
"""jc - JSON Convert `pidstat` command output parser
"""jc - JSON Convert `pidstat -h` command output parser
Must use the `-h` option in `pidstat`. All other `pidstat` options are
supported in combination with `-h`.
@@ -129,7 +129,7 @@ from jc.exceptions import ParseError
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`pidstat` command parser'
description = '`pidstat -h` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']

View File

@@ -1,7 +1,7 @@
"""jc - JSON Convert `pidstat` command output streaming parser
"""jc - JSON Convert `pidstat -h` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Must use the `-h` option in `pidstat`. All other `pidstat` options are
supported in combination with `-h`.
@@ -11,11 +11,11 @@ Usage (cli):
$ pidstat | jc --pidstat-s
> Note: When piping `jc` converted `pidstat` output to other processes it
may appear the output is hanging due to the OS pipe buffers. This is
because `pidstat` output is too small to quickly fill up the buffer. Use
the `-u` option to unbuffer the `jc` output if you would like immediate
output. See the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
for more information.
> may appear the output is hanging due to the OS pipe buffers. This is
> because `pidstat` output is too small to quickly fill up the buffer. Use
> the `-u` option to unbuffer the `jc` output if you would like immediate
> output. See the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
> for more information.
Usage (module):
@@ -84,7 +84,7 @@ from jc.exceptions import ParseError
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`pidstat` command streaming parser'
description = '`pidstat -h` command streaming parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']

View File

@@ -4,8 +4,8 @@ Supports `ping` and `ping6` output.
Usage (cli):
Note: Use the ping `-c` (count) option, otherwise data will not be
piped to `jc`.
> Note: Use the ping `-c` (count) option, otherwise data will not be
> piped to `jc`.
$ ping -c 3 1.2.3.4 | jc --ping

View File

@@ -1,20 +1,20 @@
"""jc - JSON Convert `ping` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Supports `ping` and `ping6` output.
Usage (cli):
$ ping | jc --ping-s
$ ping 1.2.3.4 | jc --ping-s
> Note: When piping `jc` converted `ping` output to other processes it may
appear the output is hanging due to the OS pipe buffers. This is because
`ping` output is too small to quickly fill up the buffer. Use the `-u`
option to unbuffer the `jc` output if you would like immediate output.
See the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
for more information.
> appear the output is hanging due to the OS pipe buffers. This is because
> `ping` output is too small to quickly fill up the buffer. Use the `-u`
> option to unbuffer the `jc` output if you would like immediate output.
> See the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
> for more information.
Usage (module):

View File

@@ -60,12 +60,13 @@ Examples:
}
]
"""
from typing import List, Dict, Optional
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.3'
version = '1.4'
description = '`pip show` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -76,7 +77,7 @@ class info():
__version__ = info.version
def _process(proc_data):
def _process(proc_data: List[Dict]) -> List[Dict]:
"""
Final processing to conform to the schema.
@@ -92,7 +93,11 @@ def _process(proc_data):
return proc_data
def parse(data, raw=False, quiet=False):
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
@@ -109,8 +114,10 @@ def parse(data, raw=False, quiet=False):
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output = []
package = {}
raw_output: List = []
package: Dict = {}
last_key: str = ''
last_key_data: List = []
# Clear any blank lines
cleandata = list(filter(None, data.splitlines()))
@@ -119,21 +126,38 @@ def parse(data, raw=False, quiet=False):
for row in cleandata:
if row.startswith('---'):
if last_key_data:
package[last_key] = package[last_key] + '\n' + '\n'.join(last_key_data)
raw_output.append(package)
package = {}
last_key = ''
last_key_data = []
continue
item_key = row.split(': ', maxsplit=1)[0].lower().replace('-', '_')
item_value = row.split(': ', maxsplit=1)[1]
if not row.startswith(' '):
item_key = row.split(': ', maxsplit=1)[0].lower().replace('-', '_')
item_value: Optional[str] = row.split(': ', maxsplit=1)[1]
if item_value == '':
item_value = None
if item_value == '':
item_value = None
package.update({item_key: item_value})
if last_key_data and last_key != item_key:
package[last_key] = package[last_key] + '\n' + '\n'.join(last_key_data)
last_key_data = []
raw_output.append(package)
package[item_key] = item_value
last_key = item_key
continue
if raw:
return raw_output
else:
return _process(raw_output)
if row.startswith(' '):
last_key_data.append(row.strip())
continue
if package:
if last_key_data:
package[last_key] = package[last_key] + '\n' + '\n'.join(last_key_data)
raw_output.append(package)
return raw_output if raw else _process(raw_output)

173
jc/parsers/postconf.py Normal file
View File

@@ -0,0 +1,173 @@
"""jc - JSON Convert `postconf -M` command output parser
Usage (cli):
$ postconf -M | jc --postconf
or
$ jc postconf -M
Usage (module):
import jc
result = jc.parse('postconf', postconf_command_output)
Schema:
[
{
"service_name": string,
"service_type": string,
"private": boolean/null, # [0]
"unprivileged": boolean/null, # [0]
"chroot": boolean/null, # [0]
"wake_up_time": integer/null, # [0]
"no_wake_up_before_first_use": boolean/null, # [1]
"process_limit": integer/null, # [0]
"command": string
}
]
[0] '-' converted to null/None
[1] null/None if `wake_up_time` is null/None
Examples:
$ postconf -M | jc --postconf -p
[
{
"service_name": "smtp",
"service_type": "inet",
"private": false,
"unprivileged": null,
"chroot": true,
"wake_up_time": null,
"process_limit": null,
"command": "smtpd",
"no_wake_up_before_first_use": null
},
{
"service_name": "pickup",
"service_type": "unix",
"private": false,
"unprivileged": null,
"chroot": true,
"wake_up_time": 60,
"process_limit": 1,
"command": "pickup",
"no_wake_up_before_first_use": false
}
]
$ postconf -M | jc --postconf -p -r
[
{
"service_name": "smtp",
"service_type": "inet",
"private": "n",
"unprivileged": "-",
"chroot": "y",
"wake_up_time": "-",
"process_limit": "-",
"command": "smtpd"
},
{
"service_name": "pickup",
"service_type": "unix",
"private": "n",
"unprivileged": "-",
"chroot": "y",
"wake_up_time": "60",
"process_limit": "1",
"command": "pickup"
}
]
"""
from typing import List, Dict
import jc.utils
from jc.parsers.universal import simple_table_parse
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`postconf -M` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
magic_commands = ['postconf -M']
__version__ = info.version
def _process(proc_data: List[Dict]) -> List[Dict]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured to conform to the schema.
"""
keys = ['private', 'unprivileged', 'chroot', 'wake_up_time', 'process_limit']
bools = ['private', 'unprivileged', 'chroot']
integers = ['wake_up_time', 'process_limit']
for item in proc_data:
if item['wake_up_time'].endswith('?'):
item['no_wake_up_before_first_use'] = True
elif item['wake_up_time'] == '-':
item['no_wake_up_before_first_use'] = None
else:
item['no_wake_up_before_first_use'] = False
for key in keys:
if item[key] == '-':
item[key] = None
for key in bools:
if item[key] is not None:
item[key] = jc.utils.convert_to_bool(item[key])
for key in integers:
if item[key] is not None:
item[key] = jc.utils.convert_to_int(item[key])
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List = []
if jc.utils.has_data(data):
table = ['service_name service_type private unprivileged chroot wake_up_time process_limit command']
data_list = list(filter(None, data.splitlines()))
table.extend(data_list)
raw_output = simple_table_parse(table)
return raw_output if raw else _process(raw_output)

View File

@@ -17,20 +17,20 @@ Schema:
[
{
"destination": string,
"gateway": string,
"genmask": string,
"flags": string,
"destination": string,
"gateway": string,
"genmask": string,
"flags": string,
"flags_pretty": [
string
string
]
"metric": integer,
"ref": integer,
"use": integer,
"mss": integer,
"window": integer,
"irtt": integer,
"iface": string
"metric": integer,
"ref": integer,
"use": integer,
"mss": integer,
"window": integer,
"irtt": integer,
"iface": string
}
]

View File

@@ -5,7 +5,7 @@ Works with `rpm -qi [package]` or `rpm -qia`.
The `..._epoch` calculated timestamp fields are naive. (i.e. based on the
local time of the system the parser is run on)
The `..._epoch_utc` calculated timestamp fields are timezone-aware and is
The `..._epoch_utc` calculated timestamp fields are timezone-aware and are
only available if the timezone field is UTC.
Usage (cli):

View File

@@ -1,7 +1,7 @@
"""jc - JSON Convert `rsync` command output parser
Supports the `-i` or `--itemize-changes` options with all levels of
verbosity. This parser will process the STDOUT output or a log file
verbosity. This parser will process the `STDOUT` output or a log file
generated with the `--log-file` option.
Usage (cli):
@@ -46,8 +46,8 @@ Schema:
"time": string,
"process": integer,
"metadata": string,
"update_type": string/null, [0]
"file_type": string/null, [1]
"update_type": string/null, # [0]
"file_type": string/null, # [1]
"checksum_or_value_different": bool/null,
"size_different": bool/null,
"modification_time_different": bool/null,
@@ -56,7 +56,7 @@ Schema:
"group_different": bool/null,
"acl_different": bool/null,
"extended_attribute_different": bool/null,
"epoch": integer, [2]
"epoch": integer, # [2]
}
]
}

View File

@@ -1,10 +1,10 @@
"""jc - JSON Convert `rsync` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Supports the `-i` or `--itemize-changes` options with all levels of
verbosity. This parser will process the STDOUT output or a log file
verbosity. This parser will process the `STDOUT` output or a log file
generated with the `--log-file` option.
Usage (cli):
@@ -44,8 +44,8 @@ Schema:
"time": string,
"process": integer,
"metadata": string,
"update_type": string/null, [0]
"file_type": string/null, [1]
"update_type": string/null, # [0]
"file_type": string/null, # [1]
"checksum_or_value_different": bool/null,
"size_different": bool/null,
"modification_time_different": bool/null,
@@ -54,7 +54,7 @@ Schema:
"group_different": bool/null,
"acl_different": bool/null,
"extended_attribute_different": bool/null,
"epoch": integer, [2]
"epoch": integer, # [2]
# below object only exists if using -qq or ignore_exceptions=True
"_jc_meta": {

View File

@@ -1,7 +1,7 @@
"""jc - JSON Convert `ss` command output parser
Extended information options like -e and -p are not supported and may cause
parsing irregularities.
Extended information options like `-e` and `-p` are not supported and may
cause parsing irregularities.
Usage (cli):
@@ -18,8 +18,8 @@ Usage (module):
Schema:
Information from https://www.cyberciti.biz/files/ss.html used to define
field names
Information from https://www.cyberciti.biz/files/ss.html used to define
field names
[
{

View File

@@ -1,7 +1,7 @@
"""jc - JSON Convert `stat` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
The `xxx_epoch` calculated timestamp fields are naive. (i.e. based on the
local time of the system the parser is run on).

View File

@@ -1,9 +1,9 @@
"""jc - JSON Convert `sysctl -a` command output parser
Note: Since `sysctl` output is not easily parsable only a very simple
key/value object will be output. An attempt is made to convert obvious
integers and floats. If no conversion is desired, use the `-r`
command-line argument or the `raw=True` argument in `parse()`.
> Note: Since `sysctl` output is not easily parsable only a very simple
> key/value object will be output. An attempt is made to convert obvious
> integers and floats. If no conversion is desired, use the `-r`
> command-line argument or the `raw=True` argument in `parse()`.
Usage (cli):

View File

@@ -6,8 +6,8 @@ used to redirect the output to a file that can be read by `jc`.
Alternatively, the output from `/usr/bin/time` can be redirected to `STDOUT`
so `jc` can receive it.
Note: `/usr/bin/time` is similar but different from the Bash builtin
`time` command.
> Note: `/usr/bin/time` is similar but different from the Bash builtin
> `time` command.
Usage (cli):
@@ -21,8 +21,8 @@ Usage (module):
Schema:
Source: https://www.freebsd.org/cgi/man.cgi?query=getrusage
https://man7.org/linux/man-pages/man1/time.1.html
Source: https://www.freebsd.org/cgi/man.cgi?query=getrusage,
https://man7.org/linux/man-pages/man1/time.1.html
{
"real_time": float,

640
jc/parsers/top.py Normal file
View File

@@ -0,0 +1,640 @@
"""jc - JSON Convert `top -b` command output parser
Requires batch mode (`-b`). The `-n` option must also be used to limit
the number of times `top` is run.
Warning messages will be printed to `STDERR` if truncated fields are
detected. These warnings can be suppressed with the `-q` or `quiet=True`
option.
Usage (cli):
$ top -b -n 3 | jc --top
or
$ jc top -b -n 3
Usage (module):
import jc
result = jc.parse('top', top_command_output)
Schema:
All `-` values are converted to `null`
[
{
"time": string,
"uptime": integer,
"users": integer,
"load_1m": float,
"load_5m": float,
"load_15m": float,
"tasks_total": integer,
"tasks_running": integer,
"tasks_sleeping": integer,
"tasks_stopped": integer,
"tasks_zombie": integer,
"cpu_user": float,
"cpu_sys": float,
"cpu_nice": float,
"cpu_idle": float,
"cpu_wait": float,
"cpu_hardware": float,
"cpu_software": float,
"cpu_steal": float,
"mem_total": float, # [0]
"mem_free": float, # [0]
"mem_used": float, # [0]
"mem_buff_cache": float, # [0]
"swap_total": float, # [0]
"swap_free": float, # [0]
"swap_used": float, # [0]
"mem_available": float, # [0]
"processes": [
{
"pid": integer,
"user": string,
"priority": integer,
"nice": integer,
"virtual_mem": float, # [1]
"resident_mem": float, # [1]
"shared_mem": float, # [1]
"status": string,
"percent_cpu": float,
"percent_mem": float,
"time_hundredths": string,
"command": string,
"parent_pid": integer,
"uid": integer,
"real_uid": integer,
"real_user": string,
"saved_uid": integer,
"saved_user": string,
"gid": integer,
"group": string,
"pgrp": integer,
"tty": string,
"tty_process_gid": integer,
"session_id": integer,
"thread_count": integer,
"last_used_processor": integer,
"time": string,
"swap": float, # [1]
"code": float, # [1]
"data": float, # [1]
"major_page_fault_count": integer,
"minor_page_fault_count": integer,
"dirty_pages_count": integer,
"sleeping_in_function": string,
"flags": string,
"cgroups": string,
"supplementary_gids": [
integer
],
"supplementary_groups": [
string
],
"thread_gid": integer,
"environment_variables": [
string
]
"major_page_fault_count_delta": integer,
"minor_page_fault_count_delta": integer,
"used": float, # [1]
"ipc_namespace_inode": integer,
"mount_namespace_inode": integer,
"net_namespace_inode": integer,
"pid_namespace_inode": integer,
"user_namespace_inode": integer,
"nts_namespace_inode": integer,
"control_group_name": string,
"lxc_container_name": string,
"numa_node": integer,
"out_of_mem_adjustment": integer,
"out_of_mem_score": integer,
"resident_anon_mem": integer,
"resident_file_backed_mem": integer,
"resident_locked_mem": integer,
"resident_shared_mem": integer
}
]
}
]
[0] Values are in the units output by `top`
[1] Unit suffix stripped during float conversion
Examples:
$ top -b -n 3 | jc --top -p
[
{
"time": "11:20:43",
"uptime": 118,
"users": 2,
"load_1m": 0.0,
"load_5m": 0.01,
"load_15m": 0.05,
"tasks_total": 108,
"tasks_running": 2,
"tasks_sleeping": 106,
"tasks_stopped": 0,
"tasks_zombie": 0,
"cpu_user": 5.6,
"cpu_sys": 11.1,
"cpu_nice": 0.0,
"cpu_idle": 83.3,
"cpu_wait": 0.0,
"cpu_hardware": 0.0,
"cpu_software": 0.0,
"cpu_steal": 0.0,
"mem_total": 3.7,
"mem_free": 3.3,
"mem_used": 0.2,
"mem_buff_cache": 0.2,
"swap_total": 2.0,
"swap_free": 2.0,
"swap_used": 0.0,
"mem_available": 3.3,
"processes": [
{
"pid": 2225,
"user": "kbrazil",
"priority": 20,
"nice": 0,
"virtual_mem": 158.1,
"resident_mem": 2.2,
"shared_mem": 1.6,
"status": "running",
"percent_cpu": 12.5,
"percent_mem": 0.1,
"time_hundredths": "0:00.02",
"command": "top",
"parent_pid": 1884,
"uid": 1000,
"real_uid": 1000,
"real_user": "kbrazil",
"saved_uid": 1000,
"saved_user": "kbrazil",
"gid": 1000,
"group": "kbrazil",
"pgrp": 2225,
"tty": "pts/0",
"tty_process_gid": 2225,
"session_id": 1884,
"thread_count": 1,
"last_used_processor": 0,
"time": "0:00",
"swap": 0.0,
"code": 0.1,
"data": 1.0,
"major_page_fault_count": 0,
"minor_page_fault_count": 736,
"dirty_pages_count": 0,
"sleeping_in_function": null,
"flags": "..4.2...",
"cgroups": "1:name=systemd:/user.slice/user-1000.+",
"supplementary_gids": [
10,
1000
],
"supplementary_groups": [
"wheel",
"kbrazil"
],
"thread_gid": 2225,
"environment_variables": [
"XDG_SESSION_ID=2",
"HOSTNAME=localhost"
],
"major_page_fault_count_delta": 0,
"minor_page_fault_count_delta": 4,
"used": 2.2,
"ipc_namespace_inode": 4026531839,
"mount_namespace_inode": 4026531840,
"net_namespace_inode": 4026531956,
"pid_namespace_inode": 4026531836,
"user_namespace_inode": 4026531837,
"nts_namespace_inode": 4026531838
},
...
]
}
]
$ top -b -n 3 | jc --top -p -r
[
{
"time": "11:20:43",
"uptime": "1:18",
"users": "2",
"load_1m": "0.00",
"load_5m": "0.01",
"load_15m": "0.05",
"tasks_total": "108",
"tasks_running": "2",
"tasks_sleeping": "106",
"tasks_stopped": "0",
"tasks_zombie": "0",
"cpu_user": "5.6",
"cpu_sys": "11.1",
"cpu_nice": "0.0",
"cpu_idle": "83.3",
"cpu_wait": "0.0",
"cpu_hardware": "0.0",
"cpu_software": "0.0",
"cpu_steal": "0.0",
"swap_total": "2.0",
"swap_free": "2.0",
"swap_used": "0.0",
"mem_available": "3.3",
"processes": [
{
"PID": "2225",
"USER": "kbrazil",
"PR": "20",
"NI": "0",
"VIRT": "158.1m",
"RES": "2.2m",
"SHR": "1.6m",
"S": "R",
"%CPU": "12.5",
"%MEM": "0.1",
"TIME+": "0:00.02",
"COMMAND": "top",
"PPID": "1884",
"UID": "1000",
"RUID": "1000",
"RUSER": "kbrazil",
"SUID": "1000",
"SUSER": "kbrazil",
"GID": "1000",
"GROUP": "kbrazil",
"PGRP": "2225",
"TTY": "pts/0",
"TPGID": "2225",
"SID": "1884",
"nTH": "1",
"P": "0",
"TIME": "0:00",
"SWAP": "0.0m",
"CODE": "0.1m",
"DATA": "1.0m",
"nMaj": "0",
"nMin": "736",
"nDRT": "0",
"WCHAN": "-",
"Flags": "..4.2...",
"CGROUPS": "1:name=systemd:/user.slice/user-1000.+",
"SUPGIDS": "10,1000",
"SUPGRPS": "wheel,kbrazil",
"TGID": "2225",
"ENVIRON": "XDG_SESSION_ID=2 HOSTNAME=localhost S+",
"vMj": "0",
"vMn": "4",
"USED": "2.2m",
"nsIPC": "4026531839",
"nsMNT": "4026531840",
"nsNET": "4026531956",
"nsPID": "4026531836",
"nsUSER": "4026531837",
"nsUTS": "4026531838"
},
...
]
}
]
"""
from typing import List, Dict
import jc.utils
from jc.parsers.uptime import parse as parse_uptime
from jc.parsers.universal import sparse_table_parse as parse_table
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`top -b` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
magic_commands = ['top -b']
__version__ = info.version
def _safe_split(string: str, path: str, delim: str = ' ', quiet=False) -> List[str]:
split_string = string.split(delim)
split_string = [x for x in split_string if not x.endswith('+')]
if string.endswith('+') and not quiet:
jc.utils.warning_message([f'{path} list was truncated by top'])
return split_string
def _process(proc_data: List[Dict], quiet=False) -> List[Dict]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured to conform to the schema.
"""
key_map: Dict = {
'%CPU': 'percent_cpu',
'%MEM': 'percent_mem',
'CGNAME': 'control_group_name',
'CGROUPS': 'cgroups',
'CODE': 'code',
'COMMAND': 'command',
'DATA': 'data',
'ENVIRON': 'environment_variables',
'Flags': 'flags',
'GID': 'gid',
'GROUP': 'group',
'LXC': 'lxc_container_name',
'NI': 'nice',
'NU': 'numa_node',
'OOMa': 'out_of_mem_adjustment',
'OOMs': 'out_of_mem_score',
'P': 'last_used_processor',
'PGRP': 'pgrp',
'PID': 'pid',
'PPID': 'parent_pid',
'PR': 'priority',
'RES': 'resident_mem',
'RSan': 'resident_anon_mem',
'RSfd': 'resident_file_backed_mem',
'RSlk': 'resident_locked_mem',
'RSsh': 'resident_shared_mem',
'RUID': 'real_uid',
'RUSER': 'real_user',
'S': 'status',
'SHR': 'shared_mem',
'SID': 'session_id',
'SUID': 'saved_uid',
'SUPGIDS': 'supplementary_gids',
'SUPGRPS': 'supplementary_groups',
'SUSER': 'saved_user',
'SWAP': 'swap',
'TGID': 'thread_gid',
'TIME': 'time',
'TIME+': 'time_hundredths',
'TPGID': 'tty_process_gid',
'TTY': 'tty',
'UID': 'uid',
'USED': 'used',
'USER': 'user',
'VIRT': 'virtual_mem',
'WCHAN': 'sleeping_in_function',
'nDRT': 'dirty_pages_count',
'nMaj': 'major_page_fault_count',
'nMin': 'minor_page_fault_count',
'nTH': 'thread_count',
'nsIPC': 'ipc_namespace_inode',
'nsMNT': 'mount_namespace_inode',
'nsNET': 'net_namespace_inode',
'nsPID': 'pid_namespace_inode',
'nsUSER': 'user_namespace_inode',
'nsUTS': 'nts_namespace_inode',
'vMj': 'major_page_fault_count_delta',
'vMn': 'minor_page_fault_count_delta'
}
status_map: Dict = {
'D': 'uninterruptible sleep',
'I': 'idle',
'R': 'running',
'S': 'sleeping',
'T': 'stopped by job control signal',
't': 'stopped by debugger during trace',
'Z': 'zombie'
}
int_list: List = [
'uptime', 'users', 'tasks_total', 'tasks_running', 'tasks_sleeping', 'tasks_stopped',
'tasks_zombie', 'pid', 'priority', 'nice', 'parent_pid', 'uid', 'real_uid', 'saved_uid',
'gid', 'pgrp', 'tty_process_gid', 'session_id', 'thread_count', 'last_used_processor',
'major_page_fault_count', 'minor_page_fault_count', 'dirty_pages_count', 'thread_gid',
'major_page_fault_count_delta', 'minor_page_fault_count_delta', 'ipc_namespace_inode',
'mount_namespace_inode', 'net_namespace_inode', 'pid_namespace_inode',
'user_namespace_inode', 'nts_namespace_inode', 'numa_node', 'out_of_mem_adjustment',
'out_of_mem_score', 'resident_anon_mem', 'resident_file_backed_mem', 'resident_locked_mem',
'resident_shared_mem'
]
float_list: List = [
'load_1m', 'load_5m', 'load_15m', 'cpu_user', 'cpu_sys', 'cpu_nice', 'cpu_idle', 'cpu_wait',
'cpu_hardware', 'cpu_software', 'cpu_steal', 'percent_cpu', 'percent_mem', 'mem_total',
'mem_free', 'mem_used', 'mem_buff_cache', 'swap_total', 'swap_free', 'swap_used',
'mem_available', 'virtual_mem', 'resident_mem', 'shared_mem', 'swap', 'code', 'data', 'used'
]
for idx, item in enumerate(proc_data):
for key in item:
# root truncation warnings
if isinstance(item[key], str) and item[key].endswith('+') and not quiet:
jc.utils.warning_message([f'item[{idx}]["{key}"] was truncated by top'])
# root int and float conversions
if key in int_list:
item[key] = jc.utils.convert_to_int(item[key])
if key in float_list:
item[key] = jc.utils.convert_to_float(item[key])
for p_idx, proc in enumerate(item['processes']):
# rename processes keys to conform to schema
proc_copy = proc.copy()
for old_key in proc_copy.keys():
if old_key in proc:
proc[key_map[old_key]] = proc.pop(old_key)
else:
jc.utils.warning_message([f'Unknown field detected at item[{idx}]["processes"]: {old_key}'])
# cleanup values
for key in proc.keys():
# set dashes to nulls
if proc[key] == '-':
proc[key] = None
# because of ambiguous column spacing (right-justified numbers
# with left-justified dashes for null values) there are some hanging
# dashes that need to be cleaned up in some values. Seems the correct
# values are kept in the assigned columns, so this should not affect
# data integrity.
if proc[key] and proc[key].endswith(' -'):
new_val = proc[key][::-1]
new_val = new_val.replace('- ', '')
new_val = new_val[::-1]
proc[key] = new_val
# do int/float conversions for the process objects
if proc[key]:
if key in int_list:
proc[key] = jc.utils.convert_to_int(proc[key])
if key in float_list:
proc[key] = jc.utils.convert_to_float(proc[key])
# set status string
if proc.get('status'):
proc['status'] = status_map[proc['status']]
# split supplementary_gids to a list of integers
if proc.get('supplementary_gids'):
proc['supplementary_gids'] = _safe_split(
proc['supplementary_gids'],
f'item[{idx}]["processes"][{p_idx}]["supplementary_gids"]',
',', quiet=quiet
)
proc['supplementary_gids'] = [jc.utils.convert_to_int(x) for x in proc['supplementary_gids']]
# split supplementary_groups to a list of strings
if proc.get('supplementary_groups'):
proc['supplementary_groups'] = _safe_split(
proc['supplementary_groups'],
f'item[{idx}]["processes"][{p_idx}]["supplementary_groups"]',
',', quiet=quiet
)
# split environment_variables to a list of strings
if proc.get('environment_variables'):
proc['environment_variables'] = _safe_split(
proc['environment_variables'],
f'item[{idx}]["processes"][{p_idx}]["environment_variables"]',
quiet=quiet
)
for key in proc.keys():
# print final warnings for truncated string values
if isinstance(proc[key], str) and proc[key].endswith('+') and not quiet:
jc.utils.warning_message([f'item[{idx}]["processes"][{p_idx}]["{key}"] was truncated by top'])
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[Dict]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List = []
item_obj: Dict = {}
process_table = False
process_list: List = []
if jc.utils.has_data(data):
for line in data.splitlines():
if line.startswith('top - '):
if item_obj:
if process_list:
item_obj['processes'] = parse_table(process_list)
raw_output.append(item_obj)
process_table = False
process_list = []
item_obj = {}
uptime_str = line[6:]
item_obj.update(parse_uptime(uptime_str, raw=True))
continue
if line.startswith('Tasks:'):
# Tasks: 112 total, 1 running, 111 sleeping, 0 stopped, 0 zombie
line_list = line.split()
item_obj.update(
{
'tasks_total': line_list[1],
'tasks_running': line_list[3],
'tasks_sleeping': line_list[5],
'tasks_stopped': line_list[7],
'tasks_zombie': line_list[9]
}
)
continue
if line.startswith('%Cpu(s):'):
# %Cpu(s): 5.9 us, 5.9 sy, 0.0 ni, 88.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
line_list = line.split()
item_obj.update(
{
'cpu_user': line_list[1],
'cpu_sys': line_list[3],
'cpu_nice': line_list[5],
'cpu_idle': line_list[7],
'cpu_wait': line_list[9],
'cpu_hardware': line_list[11],
'cpu_software': line_list[13],
'cpu_steal': line_list[15]
}
)
continue
if line[1:].startswith('iB Mem :'):
# KiB Mem : 3861332 total, 3446476 free, 216940 used, 197916 buff/cache
line_list = line.split()
item_obj.update(
{
'mem_total': line_list[3],
'mem_free': line_list[5],
'mem_used': line_list[7],
'mem_buff_cache': line_list[9]
}
)
continue
if line[1:].startswith('iB Swap:'):
# KiB Swap: 2097148 total, 2097148 free, 0 used. 3419356 avail Mem
line_list = line.split()
item_obj.update(
{
'swap_total': line_list[2],
'swap_free': line_list[4],
'swap_used': line_list[6],
'mem_available': line_list[8]
}
)
continue
if not process_table and line == '':
process_table = True
continue
if process_table and not line == '':
process_list.append(line)
continue
if item_obj:
if process_list:
item_obj['processes'] = parse_table(process_list)
raw_output.append(item_obj)
return raw_output if raw else _process(raw_output, quiet=quiet)

490
jc/parsers/top_s.py Normal file
View File

@@ -0,0 +1,490 @@
"""jc - JSON Convert `top -b` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
> Dictionaries (module)
Requires batch mode (`-b`).
Warning messages will be printed to `STDERR` if truncated fields are
detected. These warnings can be suppressed with the `-q` or `quiet=True`
option.
Usage (cli):
$ top -b | jc --top-s
Usage (module):
import jc
result = jc.parse('top_s', top_command_output.splitlines())
for item in result:
# do something
Schema:
{
"time": string,
"uptime": integer,
"users": integer,
"load_1m": float,
"load_5m": float,
"load_15m": float,
"tasks_total": integer,
"tasks_running": integer,
"tasks_sleeping": integer,
"tasks_stopped": integer,
"tasks_zombie": integer,
"cpu_user": float,
"cpu_sys": float,
"cpu_nice": float,
"cpu_idle": float,
"cpu_wait": float,
"cpu_hardware": float,
"cpu_software": float,
"cpu_steal": float,
"mem_total": float, # [0]
"mem_free": float, # [0]
"mem_used": float, # [0]
"mem_buff_cache": float, # [0]
"swap_total": float, # [0]
"swap_free": float, # [0]
"swap_used": float, # [0]
"mem_available": float, # [0]
"processes": [
{
"pid": integer,
"user": string,
"priority": integer,
"nice": integer,
"virtual_mem": float, # [1]
"resident_mem": float, # [1]
"shared_mem": float, # [1]
"status": string,
"percent_cpu": float,
"percent_mem": float,
"time_hundredths": string,
"command": string,
"parent_pid": integer,
"uid": integer,
"real_uid": integer,
"real_user": string,
"saved_uid": integer,
"saved_user": string,
"gid": integer,
"group": string,
"pgrp": integer,
"tty": string,
"tty_process_gid": integer,
"session_id": integer,
"thread_count": integer,
"last_used_processor": integer,
"time": string,
"swap": float, # [1]
"code": float, # [1]
"data": float, # [1]
"major_page_fault_count": integer,
"minor_page_fault_count": integer,
"dirty_pages_count": integer,
"sleeping_in_function": string,
"flags": string,
"cgroups": string,
"supplementary_gids": [
integer
],
"supplementary_groups": [
string
],
"thread_gid": integer,
"environment_variables": [
string
]
"major_page_fault_count_delta": integer,
"minor_page_fault_count_delta": integer,
"used": float, # [1]
"ipc_namespace_inode": integer,
"mount_namespace_inode": integer,
"net_namespace_inode": integer,
"pid_namespace_inode": integer,
"user_namespace_inode": integer,
"nts_namespace_inode": integer,
"control_group_name": string,
"lxc_container_name": string,
"numa_node": integer,
"out_of_mem_adjustment": integer,
"out_of_mem_score": integer,
"resident_anon_mem": integer,
"resident_file_backed_mem": integer,
"resident_locked_mem": integer,
"resident_shared_mem": integer
}
],
# below object only exists if using -qq or ignore_exceptions=True
"_jc_meta": {
"success": boolean, # false if error parsing
"error": string, # exists if "success" is false
"line": string # exists if "success" is false
}
}
[0] Values are in the units output by `top`
[1] Unit suffix stripped during float conversion
Examples:
$ top -b | jc --top-s
{"time":"11:24:50","uptime":2,"users":2,"load_1m":0.23,"load_5m":...}
...
$ top -b | jc --top-s -r
{"time":"11:24:50","uptime":"2 min","users":"2","load_1m":"0.23","lo...}
...
"""
from typing import List, Dict, Iterable, Union
import jc.utils
from jc.streaming import (
add_jc_meta, streaming_input_type_check, streaming_line_input_type_check, raise_or_yield
)
from jc.exceptions import ParseError
from jc.parsers.uptime import parse as parse_uptime
from jc.parsers.universal import sparse_table_parse as parse_table
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`top -b` command streaming parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
streaming = True
__version__ = info.version
def _safe_split(string: str, path: str, delim: str = ' ', quiet=False) -> List[str]:
split_string = string.split(delim)
split_string = [x for x in split_string if not x.endswith('+')]
if string.endswith('+') and not quiet:
jc.utils.warning_message([f'{path} list was truncated by top'])
return split_string
def _process(proc_data: Dict, idx=0, quiet=False) -> Dict:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (Dictionary) raw structured data to process
Returns:
Dictionary. Structured data to conform to the schema.
"""
key_map: Dict = {
'%CPU': 'percent_cpu',
'%MEM': 'percent_mem',
'CGNAME': 'control_group_name',
'CGROUPS': 'cgroups',
'CODE': 'code',
'COMMAND': 'command',
'DATA': 'data',
'ENVIRON': 'environment_variables',
'Flags': 'flags',
'GID': 'gid',
'GROUP': 'group',
'LXC': 'lxc_container_name',
'NI': 'nice',
'NU': 'numa_node',
'OOMa': 'out_of_mem_adjustment',
'OOMs': 'out_of_mem_score',
'P': 'last_used_processor',
'PGRP': 'pgrp',
'PID': 'pid',
'PPID': 'parent_pid',
'PR': 'priority',
'RES': 'resident_mem',
'RSan': 'resident_anon_mem',
'RSfd': 'resident_file_backed_mem',
'RSlk': 'resident_locked_mem',
'RSsh': 'resident_shared_mem',
'RUID': 'real_uid',
'RUSER': 'real_user',
'S': 'status',
'SHR': 'shared_mem',
'SID': 'session_id',
'SUID': 'saved_uid',
'SUPGIDS': 'supplementary_gids',
'SUPGRPS': 'supplementary_groups',
'SUSER': 'saved_user',
'SWAP': 'swap',
'TGID': 'thread_gid',
'TIME': 'time',
'TIME+': 'time_hundredths',
'TPGID': 'tty_process_gid',
'TTY': 'tty',
'UID': 'uid',
'USED': 'used',
'USER': 'user',
'VIRT': 'virtual_mem',
'WCHAN': 'sleeping_in_function',
'nDRT': 'dirty_pages_count',
'nMaj': 'major_page_fault_count',
'nMin': 'minor_page_fault_count',
'nTH': 'thread_count',
'nsIPC': 'ipc_namespace_inode',
'nsMNT': 'mount_namespace_inode',
'nsNET': 'net_namespace_inode',
'nsPID': 'pid_namespace_inode',
'nsUSER': 'user_namespace_inode',
'nsUTS': 'nts_namespace_inode',
'vMj': 'major_page_fault_count_delta',
'vMn': 'minor_page_fault_count_delta'
}
status_map: Dict = {
'D': 'uninterruptible sleep',
'I': 'idle',
'R': 'running',
'S': 'sleeping',
'T': 'stopped by job control signal',
't': 'stopped by debugger during trace',
'Z': 'zombie'
}
int_list: List = [
'uptime', 'users', 'tasks_total', 'tasks_running', 'tasks_sleeping', 'tasks_stopped',
'tasks_zombie', 'pid', 'priority', 'nice', 'parent_pid', 'uid', 'real_uid', 'saved_uid',
'gid', 'pgrp', 'tty_process_gid', 'session_id', 'thread_count', 'last_used_processor',
'major_page_fault_count', 'minor_page_fault_count', 'dirty_pages_count', 'thread_gid',
'major_page_fault_count_delta', 'minor_page_fault_count_delta', 'ipc_namespace_inode',
'mount_namespace_inode', 'net_namespace_inode', 'pid_namespace_inode',
'user_namespace_inode', 'nts_namespace_inode', 'numa_node', 'out_of_mem_adjustment',
'out_of_mem_score', 'resident_anon_mem', 'resident_file_backed_mem', 'resident_locked_mem',
'resident_shared_mem'
]
float_list: List = [
'load_1m', 'load_5m', 'load_15m', 'cpu_user', 'cpu_sys', 'cpu_nice', 'cpu_idle', 'cpu_wait',
'cpu_hardware', 'cpu_software', 'cpu_steal', 'percent_cpu', 'percent_mem', 'mem_total',
'mem_free', 'mem_used', 'mem_buff_cache', 'swap_total', 'swap_free', 'swap_used',
'mem_available', 'virtual_mem', 'resident_mem', 'shared_mem', 'swap', 'code', 'data', 'used'
]
for key in proc_data:
# root truncation warnings
if isinstance(proc_data[key], str) and proc_data[key].endswith('+') and not quiet:
jc.utils.warning_message([f'item[{idx}]["{key}"] was truncated by top'])
# root int and float conversions
if key in int_list:
proc_data[key] = jc.utils.convert_to_int(proc_data[key])
if key in float_list:
proc_data[key] = jc.utils.convert_to_float(proc_data[key])
for p_idx, proc in enumerate(proc_data['processes']):
# rename processes keys to conform to schema
proc_copy = proc.copy()
for old_key in proc_copy.keys():
if old_key in proc:
proc[key_map[old_key]] = proc.pop(old_key)
else:
jc.utils.warning_message([f'Unknown field detected at item[{idx}]["processes"]: {old_key}'])
# cleanup values
for key in proc.keys():
# set dashes to nulls
if proc[key] == '-':
proc[key] = None
# because of ambiguous column spacing (right-justified numbers
# with left-justified dashes for null values) there are some hanging
# dashes that need to be cleaned up in some values. Seems the correct
# values are kept in the assigned columns, so this should not affect
# data integrity.
if proc[key] and proc[key].endswith(' -'):
new_val = proc[key][::-1]
new_val = new_val.replace('- ', '')
new_val = new_val[::-1]
proc[key] = new_val
# do int/float conversions for the process objects
if proc[key]:
if key in int_list:
proc[key] = jc.utils.convert_to_int(proc[key])
if key in float_list:
proc[key] = jc.utils.convert_to_float(proc[key])
# set status string
if proc.get('status'):
proc['status'] = status_map[proc['status']]
# split supplementary_gids to a list of integers
if proc.get('supplementary_gids'):
proc['supplementary_gids'] = _safe_split(
proc['supplementary_gids'],
f'item[{idx}]["processes"][{p_idx}]["supplementary_gids"]',
',', quiet=quiet
)
proc['supplementary_gids'] = [jc.utils.convert_to_int(x) for x in proc['supplementary_gids']]
# split supplementary_groups to a list of strings
if proc.get('supplementary_groups'):
proc['supplementary_groups'] = _safe_split(
proc['supplementary_groups'],
f'item[{idx}]["processes"][{p_idx}]["supplementary_groups"]',
',', quiet=quiet
)
# split environment_variables to a list of strings
if proc.get('environment_variables'):
proc['environment_variables'] = _safe_split(
proc['environment_variables'],
f'item[{idx}]["processes"][{p_idx}]["environment_variables"]',
quiet=quiet
)
for key in proc.keys():
# print final warnings for truncated string values
if isinstance(proc[key], str) and proc[key].endswith('+') and not quiet:
jc.utils.warning_message([f'item[{idx}]["processes"][{p_idx}]["{key}"] was truncated by top'])
return proc_data
@add_jc_meta
def parse(
data: Iterable[str],
raw: bool = False,
quiet: bool = False,
ignore_exceptions: bool = False
) -> Union[Iterable[Dict], tuple]:
"""
Main text parsing generator function. Returns an iterable object.
Parameters:
data: (iterable) line-based text data to parse
(e.g. sys.stdin or str.splitlines())
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
ignore_exceptions: (boolean) ignore parsing exceptions if True
Returns:
Iterable of Dictionaries
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
streaming_input_type_check(data)
output_line: Dict = {}
process_table = False
process_list: List = []
idx = 0
for line in data:
try:
streaming_line_input_type_check(line)
if line.startswith('top - '):
if output_line:
if process_list:
output_line['processes'] = parse_table(process_list)
yield output_line if raw else _process(output_line, idx=idx, quiet=quiet)
process_table = False
process_list = []
output_line = {}
idx += 1
uptime_str = line[6:]
output_line.update(parse_uptime(uptime_str, raw=True))
continue
if line.startswith('Tasks:'):
# Tasks: 112 total, 1 running, 111 sleeping, 0 stopped, 0 zombie
line_list = line.split()
output_line.update(
{
'tasks_total': line_list[1],
'tasks_running': line_list[3],
'tasks_sleeping': line_list[5],
'tasks_stopped': line_list[7],
'tasks_zombie': line_list[9]
}
)
continue
if line.startswith('%Cpu(s):'):
# %Cpu(s): 5.9 us, 5.9 sy, 0.0 ni, 88.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
line_list = line.split()
output_line.update(
{
'cpu_user': line_list[1],
'cpu_sys': line_list[3],
'cpu_nice': line_list[5],
'cpu_idle': line_list[7],
'cpu_wait': line_list[9],
'cpu_hardware': line_list[11],
'cpu_software': line_list[13],
'cpu_steal': line_list[15]
}
)
continue
if line[1:].startswith('iB Mem :'):
# KiB Mem : 3861332 total, 3446476 free, 216940 used, 197916 buff/cache
line_list = line.split()
output_line.update(
{
'mem_total': line_list[3],
'mem_free': line_list[5],
'mem_used': line_list[7],
'mem_buff_cache': line_list[9]
}
)
continue
if line[1:].startswith('iB Swap:'):
# KiB Swap: 2097148 total, 2097148 free, 0 used. 3419356 avail Mem
line_list = line.split()
output_line.update(
{
'swap_total': line_list[2],
'swap_free': line_list[4],
'swap_used': line_list[6],
'mem_available': line_list[8]
}
)
continue
if not process_table and line.rstrip() == '':
process_table = True
continue
if process_table and not line.rstrip() == '':
process_list.append(line.rstrip())
continue
if process_table and line.rstrip() == '':
continue
raise ParseError('Not top data')
except Exception as e:
yield raise_or_yield(ignore_exceptions, e, line)
if output_line:
if process_list:
output_line['processes'] = parse_table(process_list)
yield output_line if raw else _process(output_line, idx=idx, quiet=quiet)

View File

@@ -2,12 +2,12 @@
Supports `traceroute` and `traceroute6` output.
Note: On some operating systems you will need to redirect `STDERR` to
`STDOUT` for destination info since the header line is sent to
`STDERR`. A warning message will be printed to `STDERR` if the
header row is not found.
e.g. `$ traceroute 8.8.8.8 2>&1 | jc --traceroute`
> Note: On some operating systems you will need to redirect `STDERR` to
> `STDOUT` for destination info since the header line is sent to
> `STDERR`. A warning message will be printed to `STDERR` if the
> header row is not found.
>
> e.g. `$ traceroute 8.8.8.8 2>&1 | jc --traceroute`
Usage (cli):

View File

@@ -1,6 +1,6 @@
"""jc - JSON Convert `uname -a` command output parser
Note: Must use `uname -a`
> Note: Must use `uname -a`
Usage (cli):

View File

@@ -1,7 +1,7 @@
"""jc - JSON Convert `vmstat` command output streaming parser
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
Dictionaries (module)
> Dictionaries (module)
Options supported: `-a`, `-w`, `-d`, `-t`
@@ -16,11 +16,11 @@ Usage (cli):
$ vmstat | jc --vmstat-s
> Note: When piping `jc` converted `vmstat` output to other processes it may
appear the output is hanging due to the OS pipe buffers. This is because
`vmstat` output is too small to quickly fill up the buffer. Use the `-u`
option to unbuffer the `jc` output if you would like immediate output. See
the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
for more information.
> appear the output is hanging due to the OS pipe buffers. This is because
> `vmstat` output is too small to quickly fill up the buffer. Use the `-u`
> option to unbuffer the `jc` output if you would like immediate output. See
> the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
> for more information.
Usage (module):
@@ -70,9 +70,9 @@ Schema:
# below object only exists if using -qq or ignore_exceptions=True
"_jc_meta": {
"success": boolean, # [2]
"error": string, # [3]
"line": string # [3]
"success": boolean, # [2]
"error": string, # [3]
"line": string # [3]
}
}

View File

@@ -11,8 +11,8 @@ Usage (module):
Schema:
XML Document converted to a Dictionary
See https://github.com/martinblech/xmltodict for details
XML Document converted to a Dictionary. See https://github.com/martinblech/xmltodict
for details.
{
"key1": string/object,

View File

@@ -1,6 +1,6 @@
"""jc - JSON Convert `YAML` file parser
Note: datetime objects will be converted to strings.
> Note: `datetime` objects will be converted to strings.
Usage (cli):

View File

@@ -1,9 +1,6 @@
"""jc - JSON Convert `zipinfo` command output parser
Options supported:
- none
Note: The default listing format.
> Note: No `zipinfo` options are supported.
Usage (cli):

348
jc/shell_completions.py Normal file
View File

@@ -0,0 +1,348 @@
"""jc - JSON Convert shell_completions module"""
from string import Template
from .cli_data import long_options_map
from .lib import all_parser_info
bash_template = Template('''\
_jc()
{
local cur prev words cword jc_commands jc_parsers jc_options \\
jc_about_options jc_about_mod_options jc_help_options jc_special_options
jc_commands=(${bash_commands})
jc_parsers=(${bash_parsers})
jc_options=(${bash_options})
jc_about_options=(${bash_about_options})
jc_about_mod_options=(${bash_about_mod_options})
jc_help_options=(${bash_help_options})
jc_special_options=(${bash_special_options})
COMPREPLY=()
_get_comp_words_by_ref cur prev words cword
# if jc_about_options are found anywhere in the line, then only complete from jc_about_mod_options
for i in "$${words[@]::$${#words[@]}-1}"; do
if [[ " $${jc_about_options[*]} " =~ " $${i} " ]]; then
COMPREPLY=( $$( compgen -W "$${jc_about_mod_options[*]}" \\
-- "$${cur}" ) )
return 0
fi
done
# if jc_help_options and a parser are found anywhere in the line, then no more completions
if
(
for i in "$${words[@]::$${#words[@]}-1}"; do
if [[ " $${jc_help_options[*]} " =~ " $${i} " ]]; then
return 0
fi
done
return 1
) && (
for i in "$${words[@]::$${#words[@]}-1}"; do
if [[ " $${jc_parsers[*]} " =~ " $${i} " ]]; then
return 0
fi
done
return 1
); then
return 0
fi
# if jc_help_options are found anywhere in the line, then only complete with parsers
for i in "$${words[@]::$${#words[@]}-1}"; do
if [[ " $${jc_help_options[*]} " =~ " $${i} " ]]; then
COMPREPLY=( $$( compgen -W "$${jc_parsers[*]}" \\
-- "$${cur}" ) )
return 0
fi
done
# if special options are found anywhere in the line, then no more completions
for i in "$${words[@]::$${#words[@]}-1}"; do
if [[ " $${jc_special_options[*]} " =~ " $${i} " ]]; then
return 0
fi
done
# if magic command is found anywhere in the line, use called command's autocompletion
for i in "$${words[@]::$${#words[@]}-1}"; do
if [[ " $${jc_commands[*]} " =~ " $${i} " ]]; then
_command
return 0
fi
done
# if a parser arg is found anywhere in the line, only show options and help options
for i in "$${words[@]::$${#words[@]}-1}"; do
if [[ " $${jc_parsers[*]} " =~ " $${i} " ]]; then
COMPREPLY=( $$( compgen -W "$${jc_options[*]} $${jc_help_options[*]}" \\
-- "$${cur}" ) )
return 0
fi
done
# default completion
COMPREPLY=( $$( compgen -W "$${jc_options[*]} $${jc_about_options[*]} $${jc_help_options[*]} $${jc_special_options[*]} $${jc_parsers[*]} $${jc_commands[*]}" \\
-- "$${cur}" ) )
} &&
complete -F _jc jc
''')
zsh_template = Template('''\
#compdef jc
_jc() {
local -a jc_commands jc_commands_describe \\
jc_parsers jc_parsers_describe \\
jc_options jc_options_describe \\
jc_about_options jc_about_options_describe \\
jc_about_mod_options jc_about_mod_options_describe \\
jc_help_options jc_help_options_describe \\
jc_special_options jc_special_options_describe
jc_commands=(${zsh_commands})
jc_commands_describe=(
${zsh_commands_describe}
)
jc_parsers=(${zsh_parsers})
jc_parsers_describe=(
${zsh_parsers_describe}
)
jc_options=(${zsh_options})
jc_options_describe=(
${zsh_options_describe}
)
jc_about_options=(${zsh_about_options})
jc_about_options_describe=(
${zsh_about_options_describe}
)
jc_about_mod_options=(${zsh_about_mod_options})
jc_about_mod_options_describe=(
${zsh_about_mod_options_describe}
)
jc_help_options=(${zsh_help_options})
jc_help_options_describe=(
${zsh_help_options_describe}
)
jc_special_options=(${zsh_special_options})
jc_special_options_describe=(
${zsh_special_options_describe}
)
# if jc_about_options are found anywhere in the line, then only complete from jc_about_mod_options
for i in $${words:0:-1}; do
if (( $$jc_about_options[(Ie)$${i}] )); then
_describe 'commands' jc_about_mod_options_describe
return 0
fi
done
# if jc_help_options and a parser are found anywhere in the line, then no more completions
if
(
for i in $${words:0:-1}; do
if (( $$jc_help_options[(Ie)$${i}] )); then
return 0
fi
done
return 1
) && (
for i in $${words:0:-1}; do
if (( $$jc_parsers[(Ie)$${i}] )); then
return 0
fi
done
return 1
); then
return 0
fi
# if jc_help_options are found anywhere in the line, then only complete with parsers
for i in $${words:0:-1}; do
if (( $$jc_help_options[(Ie)$${i}] )); then
_describe 'commands' jc_parsers_describe
return 0
fi
done
# if special options are found anywhere in the line, then no more completions
for i in $${words:0:-1}; do
if (( $$jc_special_options[(Ie)$${i}] )); then
return 0
fi
done
# if magic command is found anywhere in the line, use called command's autocompletion
for i in $${words:0:-1}; do
if (( $$jc_commands[(Ie)$${i}] )); then
# hack to remove options between jc and the magic command
shift $$(( $${#words} - 2 )) words
words[1,0]=(jc)
CURRENT=$${#words}
# run the magic command's completions
_arguments '*::arguments:_normal'
return 0
fi
done
# if a parser arg is found anywhere in the line, only show options and help options
for i in $${words:0:-1}; do
if (( $$jc_parsers[(Ie)$${i}] )); then
_describe 'commands' jc_options_describe -- jc_help_options_describe
return 0
fi
done
# default completion
_describe 'commands' jc_options_describe -- jc_about_options_describe -- jc_help_options_describe -- jc_special_options_describe -- jc_parsers_describe -- jc_commands_describe
}
_jc
''')
about_options = ['--about', '-a']
about_mod_options = ['--pretty', '-p', '--yaml-out', '-y', '--monochrome', '-m', '--force-color', '-C']
help_options = ['--help', '-h']
special_options = ['--version', '-v', '--bash-comp', '-B', '--zsh-comp', '-Z']
def get_commands():
command_list = []
for cmd in all_parser_info():
if 'magic_commands' in cmd:
command_list.extend(cmd['magic_commands'])
return sorted(list(set([i.split()[0] for i in command_list])))
def get_options():
options_list = []
for opt in long_options_map:
options_list.append(opt)
options_list.append('-' + long_options_map[opt][0])
return options_list
def get_parsers():
p_list = []
for cmd in all_parser_info():
if 'argument' in cmd:
p_list.append(cmd['argument'])
return p_list
def get_parsers_descriptions():
pd_list = []
for p in all_parser_info():
if 'description' in p:
pd_list.append(f"'{p['argument']}:{p['description']}'")
return pd_list
def get_zsh_command_descriptions(command_list):
zsh_commands = []
for cmd in command_list:
zsh_commands.append(f"""'{cmd}:run "{cmd}" command with magic syntax.'""")
return zsh_commands
def get_descriptions(opt_list):
"""Return a list of options:description items."""
opt_desc_list = []
for item in opt_list:
# get long options
if item in long_options_map:
opt_desc_list.append(f"'{item}:{long_options_map[item][1]}'")
continue
# get short options
for k, v in long_options_map.items():
if item[1:] == v[0]:
opt_desc_list.append(f"'{item}:{v[1]}'")
continue
return opt_desc_list
def bash_completion():
parsers_str = ' '.join(get_parsers())
opts_no_special = get_options()
for s_option in special_options:
opts_no_special.remove(s_option)
for a_option in about_options:
opts_no_special.remove(a_option)
for h_option in help_options:
opts_no_special.remove(h_option)
options_str = ' '.join(opts_no_special)
about_options_str = ' '.join(about_options)
about_mod_options_str = ' '.join(about_mod_options)
help_options_str = ' '.join(help_options)
special_options_str = ' '.join(special_options)
commands_str = ' '.join(get_commands())
return bash_template.substitute(
bash_parsers=parsers_str,
bash_special_options=special_options_str,
bash_about_options=about_options_str,
bash_about_mod_options=about_mod_options_str,
bash_help_options=help_options_str,
bash_options=options_str,
bash_commands=commands_str
)
def zsh_completion():
parsers_str = ' '.join(get_parsers())
parsers_describe = '\n '.join(get_parsers_descriptions())
opts_no_special = get_options()
for s_option in special_options:
opts_no_special.remove(s_option)
for a_option in about_options:
opts_no_special.remove(a_option)
for h_option in help_options:
opts_no_special.remove(h_option)
options_str = ' '.join(opts_no_special)
options_describe = '\n '.join(get_descriptions(opts_no_special))
about_options_str = ' '.join(about_options)
about_options_describe = '\n '.join(get_descriptions(about_options))
about_mod_options_str = ' '.join(about_mod_options)
about_mod_options_describe = '\n '.join(get_descriptions(about_mod_options))
help_options_str = ' '.join(help_options)
help_options_describe = '\n '.join(get_descriptions(help_options))
special_options_str = ' '.join(special_options)
special_options_describe = '\n '.join(get_descriptions(special_options))
commands_str = ' '.join(get_commands())
commands_describe = '\n '.join(get_zsh_command_descriptions(get_commands()))
return zsh_template.substitute(
zsh_parsers=parsers_str,
zsh_parsers_describe=parsers_describe,
zsh_special_options=special_options_str,
zsh_special_options_describe=special_options_describe,
zsh_about_options=about_options_str,
zsh_about_options_describe=about_options_describe,
zsh_about_mod_options=about_mod_options_str,
zsh_about_mod_options_describe=about_mod_options_describe,
zsh_help_options=help_options_str,
zsh_help_options_describe=help_options_describe,
zsh_options=options_str,
zsh_options_describe=options_describe,
zsh_commands=commands_str,
zsh_commands_describe=commands_describe
)

View File

@@ -31,8 +31,8 @@ def _safe_print(string: str, sep=' ', end='\n', file=sys.stdout, flush=False) ->
def warning_message(message_lines: List[str]) -> None:
"""
Prints warning message for non-fatal issues. The first line is
prepended with 'jc: Warning - ' and subsequent lines are indented.
Prints warning message to `STDERR` for non-fatal issues. The first line
is prepended with 'jc: Warning - ' and subsequent lines are indented.
Wraps text as needed based on the terminal width.
Parameters:
@@ -67,7 +67,7 @@ def warning_message(message_lines: List[str]) -> None:
def error_message(message_lines: List[str]) -> None:
"""
Prints an error message for fatal issues. The first line is
Prints an error message to `STDERR` for fatal issues. The first line is
prepended with 'jc: Error - ' and subsequent lines are indented.
Wraps text as needed based on the terminal width.
@@ -97,10 +97,24 @@ def error_message(message_lines: List[str]) -> None:
_safe_print(message, file=sys.stderr)
def is_compatible(compatible: List) -> bool:
"""
Returns True if the parser is compatible with the running OS platform.
"""
platform_found = False
for platform in compatible:
if sys.platform.startswith(platform):
platform_found = True
break
return platform_found
def compatibility(mod_name: str, compatible: List, quiet: bool = False) -> None:
"""
Checks for the parser's compatibility with the running OS
platform.
Checks for the parser's compatibility with the running OS platform and
prints a warning message to `STDERR` if not compatible and quiet=False.
Parameters:
@@ -116,21 +130,13 @@ def compatibility(mod_name: str, compatible: List, quiet: bool = False) -> None:
None - just prints output to STDERR
"""
if not quiet:
platform_found = False
for platform in compatible:
if sys.platform.startswith(platform):
platform_found = True
break
if not platform_found:
mod = mod_name.split('.')[-1]
compat_list = ', '.join(compatible)
warning_message([
f'{mod} parser is not compatible with your OS ({sys.platform}).',
f'Compatible platforms: {compat_list}'
])
if not quiet and not is_compatible(compatible):
mod = mod_name.split('.')[-1]
compat_list = ', '.join(compatible)
warning_message([
f'{mod} parser is not compatible with your OS ({sys.platform}).',
f'Compatible platforms: {compat_list}'
])
def has_data(data: str) -> bool:

View File

@@ -1,4 +1,4 @@
.TH jc 1 2022-05-13 1.19.0 "JSON Convert"
.TH jc 1 2022-06-15 1.20.1 "JSON Convert"
.SH NAME
\fBjc\fP \- JSON Convert JSONifies the output of many CLI tools and file-types
.SH SYNOPSIS
@@ -310,12 +310,12 @@ Key/Value file parser
.TP
.B
\fB--pidstat\fP
`pidstat` command parser
`pidstat -h` command parser
.TP
.B
\fB--pidstat-s\fP
`pidstat` command streaming parser
`pidstat -h` command streaming parser
.TP
.B
@@ -337,6 +337,11 @@ Key/Value file parser
\fB--pip-show\fP
`pip show` command parser
.TP
.B
\fB--postconf\fP
`postconf -M` command parser
.TP
.B
\fB--ps\fP
@@ -427,6 +432,16 @@ Key/Value file parser
\fB--timedatectl\fP
`timedatectl status` command parser
.TP
.B
\fB--top\fP
`top -b` command parser
.TP
.B
\fB--top-s\fP
`top -b` command streaming parser
.TP
.B
\fB--tracepath\fP
@@ -526,44 +541,56 @@ Options:
.TP
.B
\fB-a\fP
about \fBjc\fP (JSON output)
\fB-a\fP, \fB--about\fP
About \fBjc\fP (JSON or YAML output)
.TP
.B
\fB-C\fP
force color output even when using pipes (overrides \fB-m\fP and the \fBNO_COLOR\fP env variable)
\fB-C\fP, \fB--force-color\fP
Force color output even when using pipes (overrides \fB-m\fP and the \fBNO_COLOR\fP env variable)
.TP
.B
\fB-d\fP
debug - show traceback (use \fB-dd\fP for verbose traceback)
\fB-d\fP, \fB--debug\fP
Debug - show traceback (use \fB-dd\fP for verbose traceback)
.TP
.B
\fB-h\fP
help (\fB-h --parser_name\fP for parser documentation)
\fB-h\fP, \fB--help\fP
Help (\fB--help --parser_name\fP for parser documentation)
.TP
.B
\fB-m\fP
monochrome output
\fB-m\fP, \fB--monochrome\fP
Monochrome output
.TP
.B
\fB-p\fP
pretty print output
\fB-p\fP, \fB--pretty\fP
Pretty print output
.TP
.B
\fB-q\fP
quiet - suppress warnings (use \fB-qq\fP to ignore streaming parser errors)
\fB-q\fP, \fB--quiet\fP
Quiet mode. Suppresses parser warning messages (use -qq to ignore streaming parser errors)
.TP
.B
\fB-r\fP
raw JSON output
\fB-r\fP, \fB--raw\fP
Raw output. Provides more literal output, typically with string values and no additional semantic processing
.TP
.B
\fB-u\fP
unbuffer output (useful for slow streaming data with streaming parsers)
\fB-u\fP, \fB--unbuffer\fP
Unbuffer output (useful for slow streaming data with streaming parsers)
.TP
.B
\fB-v\fP
version information
\fB-v\fP, \fB--version\fP
Version information
.TP
.B
\fB-y\fP, \fB--yaml-out\fP
YAML output
.TP
.B
\fB-B\fP, \fB--bash-comp\fP
Generate Bash shell completion script
.TP
.B
\fB-Z\fP, \fB--zsh-comp\fP
Generate Zsh shell completion script
.SH EXIT CODES
Any fatal errors within \fBjc\fP will generate an exit code of \fB100\fP, otherwise the exit code will be \fB0\fP. When using the "Magic" syntax (e.g. \fBjc ifconfig eth0\fP), \fBjc\fP will store the exit code of the program being parsed and add it to the \fBjc\fP exit code. This way it is easier to determine if an error was from the parsed program or \fBjc\fP.

View File

@@ -5,7 +5,7 @@ with open('README.md', 'r') as f:
setuptools.setup(
name='jc',
version='1.19.0',
version='1.20.1',
author='Kelly Brazil',
author_email='kellyjonbrazil@gmail.com',
description='Converts the output of popular command-line tools and file-types to JSON.',

View File

@@ -31,44 +31,56 @@ Options:
.TP
.B
\fB-a\fP
about \fBjc\fP (JSON output)
\fB-a\fP, \fB--about\fP
About \fBjc\fP (JSON or YAML output)
.TP
.B
\fB-C\fP
force color output even when using pipes (overrides \fB-m\fP and the \fBNO_COLOR\fP env variable)
\fB-C\fP, \fB--force-color\fP
Force color output even when using pipes (overrides \fB-m\fP and the \fBNO_COLOR\fP env variable)
.TP
.B
\fB-d\fP
debug - show traceback (use \fB-dd\fP for verbose traceback)
\fB-d\fP, \fB--debug\fP
Debug - show traceback (use \fB-dd\fP for verbose traceback)
.TP
.B
\fB-h\fP
help (\fB-h --parser_name\fP for parser documentation)
\fB-h\fP, \fB--help\fP
Help (\fB--help --parser_name\fP for parser documentation)
.TP
.B
\fB-m\fP
monochrome output
\fB-m\fP, \fB--monochrome\fP
Monochrome output
.TP
.B
\fB-p\fP
pretty print output
\fB-p\fP, \fB--pretty\fP
Pretty print output
.TP
.B
\fB-q\fP
quiet - suppress warnings (use \fB-qq\fP to ignore streaming parser errors)
\fB-q\fP, \fB--quiet\fP
Quiet mode. Suppresses parser warning messages (use -qq to ignore streaming parser errors)
.TP
.B
\fB-r\fP
raw JSON output
\fB-r\fP, \fB--raw\fP
Raw output. Provides more literal output, typically with string values and no additional semantic processing
.TP
.B
\fB-u\fP
unbuffer output (useful for slow streaming data with streaming parsers)
\fB-u\fP, \fB--unbuffer\fP
Unbuffer output (useful for slow streaming data with streaming parsers)
.TP
.B
\fB-v\fP
version information
\fB-v\fP, \fB--version\fP
Version information
.TP
.B
\fB-y\fP, \fB--yaml-out\fP
YAML output
.TP
.B
\fB-B\fP, \fB--bash-comp\fP
Generate Bash shell completion script
.TP
.B
\fB-Z\fP, \fB--zsh-comp\fP
Generate Zsh shell completion script
.SH EXIT CODES
Any fatal errors within \fBjc\fP will generate an exit code of \fB100\fP, otherwise the exit code will be \fB0\fP. When using the "Magic" syntax (e.g. \fBjc ifconfig eth0\fP), \fBjc\fP will store the exit code of the program being parsed and add it to the \fBjc\fP exit code. This way it is easier to determine if an error was from the parsed program or \fBjc\fP.

View File

@@ -144,26 +144,26 @@ option.
### Parsers
| Argument | Command or Filetype | Documentation |
|--------------|-------------------------|-------------------|{% for parser in jc.parsers %}
| `{{ parser.argument }}` | {{ parser.description }} | [📃](https://kellyjonbrazil.github.io/jc/docs/parsers/{{ parser.name }}) |{% endfor %}
| Argument | Command or Filetype | Documentation |
|-------------------|---------------------------------------------------------|-------------------------------------------------------------------------|{% for parser in jc.parsers %}
| `{{ "{:>15}".format(parser.argument) }}` | {{ "{:<55}".format(parser.description) }} | {{ "{:<70}".format("[📃](https://kellyjonbrazil.github.io/jc/docs/parsers/" + parser.name + ")") }} |{% endfor %}
### Options
- `-a` about `jc`. Prints information about `jc` and the parsers (in JSON, of
course!)
- `-C` force color output even when using pipes (overrides `-m` and the
`NO_COLOR` env variable)
- `-d` debug mode. Prints trace messages if parsing issues are encountered (use
`-dd` for verbose debugging)
- `-h` help. Use `jc -h --parser_name` for parser documentation
- `-m` monochrome JSON output
- `-p` pretty format the JSON output
- `-q` quiet mode. Suppresses parser warning messages (use `-qq` to ignore
streaming parser errors)
- `-r` raw output. Provides a more literal JSON output, typically with string
values and no additional semantic processing
- `-u` unbuffer output
- `-v` version information
| Short | Long | Description |
|-------|-----------------|--------------------------------------------------------------------------------------------------------------|
| `-a` | `--about` | About `jc`. Prints information about `jc` and the parsers (in JSON or YAML, of course!) |
| `-C` | `--force-color` | Force color output even when using pipes (overrides `-m` and the `NO_COLOR` env variable) |
| `-d` | `--debug` | Debug mode. Prints trace messages if parsing issues are encountered (use`-dd` for verbose debugging) |
| `-h` | `--help` | Help. Use `jc -h --parser_name` for parser documentation |
| `-m` | `--monochrome` | Monochrome output |
| `-p` | `--pretty` | Pretty format the JSON output |
| `-q` | `--quiet` | Quiet mode. Suppresses parser warning messages (use `-qq` to ignore streaming parser errors) |
| `-r` | `--raw` | Raw output. Provides more literal output, typically with string values and no additional semantic processing |
| `-u` | `--unbuffer` | Unbuffer output |
| `-v` | `--version` | Version information |
| `-y` | `--yaml-out` | YAML output |
| `-B` | `--bash-comp` | Generate Bash shell completion script |
| `-Z` | `--zsh-comp` | Generate Zsh shell completion script |
### Exit Codes
Any fatal errors within `jc` will generate an exit code of `100`, otherwise the
@@ -210,11 +210,11 @@ color output will override both the `NO_COLOR` environment variable and the `-m`
option.
### Streaming Parsers
Most parsers load all of the data from STDIN, parse it, then output the entire
Most parsers load all of the data from `STDIN`, parse it, then output the entire
JSON document serially. There are some streaming parsers (e.g. `ls-s` and
`ping-s`) that immediately start processing and outputing the data line-by-line
as [JSON Lines](https://jsonlines.org/) (aka [NDJSON](http://ndjson.org/)) while
it is being received from STDIN. This can significantly reduce the amount of
it is being received from `STDIN`. This can significantly reduce the amount of
memory required to parse large amounts of command output (e.g. `ls -lR /`) and
can sometimes process the data more quickly. Streaming parsers have slightly
different behavior than standard parsers as outlined below.
@@ -343,15 +343,30 @@ If a UTC timezone can be detected in the text of the command output, the
timestamp will be timezone aware and have a `_utc` suffix on the key name.
(e.g. `epoch_utc`) No other timezones are supported for aware timestamps.
## Use In Other Shells
`jc` can be used in most any shell. Some modern shells have JSON deserialization
and filtering capabilities built-in which makes using `jc` even more convenient.
For example, the following is possible in [NGS](https://ngs-lang.org/)
(Next Generation Shell):
```bash
myvar = ``jc dig www.google.com``[0].answer[0].data
```
This runs `jc`, parses the output JSON, and assigs the resulting data structure
to a variable in a single line of code.
For more examples of how to use `jc` in other shells, see this
[wiki page](https://github.com/kellyjonbrazil/jc/wiki/Using-jc-With-Different-Shells).
## Compatibility
Some parsers like `dig`, `xml`, `csv`, etc. will work on any platform. Other
parsers that convert platform-specific output will generate a warning message if
they are run on an unsupported platform. To see all parser information,
including compatibility, run `jc -ap`.
You may still use a parser on an unsupported platform - for example, you may want
to parse a file with linux `lsof` output on an macOS or Windows laptop. In that
case you can suppress the warning message with the `-q` cli option or the
You may still use a parser on an unsupported platform - for example, you may
want to parse a file with linux `lsof` output on an macOS or Windows laptop. In
that case you can suppress the warning message with the `-q` cli option or the
`quiet=True` function parameter in `parse()`:
macOS:
@@ -380,7 +395,8 @@ Tested on:
## Contributions
Feel free to add/improve code or parsers! You can use the
[`jc/parsers/foo.py`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py)
or [`jc/parsers/foo_s.py (streaming)`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo_s.py) parsers as a template and submit your parser with a pull request.
or [`jc/parsers/foo_s.py (streaming)`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo_s.py) parsers as a template and submit your parser with a pull
request.
Please see the [Contributing Guidelines](https://github.com/kellyjonbrazil/jc/blob/master/CONTRIBUTING.md) for more information.

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,115 @@
top - 11:20:43 up 1:18, 2 users, load average: 0.00, 0.01, 0.05
Tasks: 108 total, 2 running, 106 sleeping, 0 stopped, 0 zombie
%Cpu(s): 5.6 us, 11.1 sy, 0.0 ni, 83.3 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
GiB Mem : 3.7 total, 3.3 free, 0.2 used, 0.2 buff/cache
GiB Swap: 2.0 total, 2.0 free, 0.0 used. 3.3 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND PPID UID RUID RUSER SUID SUSER GID GROUP PGRP TTY TPGID SID nTH P TIME SWAP CODE DATA nMaj nMin nDRT WCHAN Flags CGROUPS SUPGIDS SUPGRPS TGID ENVIRON vMj vMn USED nsIPC nsMNT nsNET nsPID nsUSER nsUTS
2225 kbrazil 20 0 158.1m 2.2m 1.6m R 12.5 0.1 0:00.02 top 1884 1000 1000 kbrazil 1000 kbrazil 1000 kbrazil 2225 pts/0 2225 1884 1 0 0:00 0.0m 0.1m 1.0m 0 736 0 - ..4.2... 1:name=systemd:/user.slice/user-1000.+ 10,1000 wheel,kbrazil 2225 XDG_SESSION_ID=2 HOSTNAME=localhost S+ 0 4 2.2m 4026531839 4026531840 4026531956 4026531836 4026531837 4026531838
1 root 20 0 125.1m 6.5m 4.1m S 0.0 0.2 0:01.47 systemd 0 0 0 root 0 root 0 root 1 ? -1 1 1 0 0:01 0.0m 1.4m 82.4m 63 13k 0 ep_poll ..4.21.. - - - 1 - 0 0 6.5m - - - - - -
2 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kthreadd 0 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 kthreadd ..2.a.4. - - - 2 - 0 0 0.0m - - - - - -
4 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kworker/0:0H 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 worker_th+ .42.a.6. - - - 4 - 0 0 0.0m - - - - - -
6 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.10 ksoftirqd/0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 smpboot_t+ .42.a.4. - - - 6 - 0 0 0.0m - - - - - -
7 root rt 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 migration/0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 smpboot_t+ .42.a.4. - - - 7 - 0 0 0.0m - - - - - -
8 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 rcu_bh 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rcu_gp_kt+ ..2.a.4. - - - 8 - 0 0 0.0m - - - - - -
9 root 20 0 0.0m 0.0m 0.0m R 0.0 0.0 0:00.49 rcu_sched 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 ? ..2.a.4. - - - 9 - 0 0 0.0m - - - - - -
10 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 lru-add-drain 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 10 - 0 0 0.0m - - - - - -
11 root rt 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.08 watchdog/0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 smpboot_t+ .42.a14. - - - 11 - 0 0 0.0m - - - - - -
13 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kdevtmpfs 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 devtmpfsd ..2.a14. - - - 13 - 0 0 0.0m - - - - - -
14 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 netns 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 14 - 0 0 0.0m - - - - - -
15 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 khungtaskd 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 watchdog ..2.a.4. - - - 15 - 0 0 0.0m - - - - - -
16 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 writeback 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 16 - 0 0 0.0m - - - - - -
17 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kintegrityd 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 17 - 0 0 0.0m - - - - - -
18 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 bioset 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 18 - 0 0 0.0m - - - - - -
19 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 bioset 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 19 - 0 0 0.0m - - - - - -
20 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 bioset 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 20 - 0 0 0.0m - - - - - -
21 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kblockd 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 21 - 0 0 0.0m - - - - - -
22 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 md 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 22 - 0 0 0.0m - - - - - -
23 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 edac-poller 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 23 - 0 0 0.0m - - - - - -
24 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 watchdogd 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 24 - 0 0 0.0m - - - - - -
30 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kswapd0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 kswapd ..a4284. - - - 30 - 0 0 0.0m - - - - - -
31 root 25 5 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 ksmd 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 ksm_scan_+ 4.2.2.4. - - - 31 - 0 0 0.0m - - - - - -
32 root 39 19 0.0m 0.0m 0.0m S 0.0 0.0 0:00.03 khugepaged 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 khugepaged 4.2.2.4. - - - 32 - 0 0 0.0m - - - - - -
33 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 crypto 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 33 - 0 0 0.0m - - - - - -
41 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kthrotld 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 41 - 0 0 0.0m - - - - - -
43 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kmpath_rdacd 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 43 - 0 0 0.0m - - - - - -
44 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kaluad 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 44 - 0 0 0.0m - - - - - -
45 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kpsmoused 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 45 - 0 0 0.0m - - - - - -
47 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 ipv6_addrconf 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 47 - 0 0 0.0m - - - - - -
60 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 deferwq 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 60 - 0 0 0.0m - - - - - -
95 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kauditd 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 kauditd_t+ ..2.2.4. - - - 95 - 0 0 0.0m - - - - - -
272 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 nfit 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 272 - 0 0 0.0m - - - - - -
273 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 mpt_poll_0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 273 - 0 0 0.0m - - - - - -
274 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 mpt/0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 274 - 0 0 0.0m - - - - - -
275 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 ata_sff 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 275 - 0 0 0.0m - - - - - -
321 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 scsi_eh_0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 scsi_erro+ ..2.a.4. - - - 321 - 0 0 0.0m - - - - - -
326 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 scsi_tmf_0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 326 - 0 0 0.0m - - - - - -
345 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.01 scsi_eh_1 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 scsi_erro+ ..2.a.4. - - - 345 - 0 0 0.0m - - - - - -
346 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 scsi_tmf_1 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 346 - 0 0 0.0m - - - - - -
347 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.01 scsi_eh_2 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 scsi_erro+ ..2.a.4. - - - 347 - 0 0 0.0m - - - - - -
350 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 scsi_tmf_2 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 350 - 0 0 0.0m - - - - - -
356 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kworker/u256:4 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 worker_th+ .42.a.6. - - - 356 - 0 0 0.0m - - - - - -
359 root -51 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.02 irq/16-vmwgfx 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 irq_thread ..2.a14. - - - 359 - 0 0 0.0m - - - - - -
360 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 ttm_swap 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 360 - 0 0 0.0m - - - - - -
431 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kdmflush 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 431 - 0 0 0.0m - - - - - -
432 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 bioset 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 432 - 0 0 0.0m - - - - - -
442 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kdmflush 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 442 - 0 0 0.0m - - - - - -
443 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 bioset 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 443 - 0 0 0.0m - - - - - -
455 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 bioset 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 455 - 0 0 0.0m - - - - - -
456 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfsalloc 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 456 - 0 0 0.0m - - - - - -
457 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs_mru_cache 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 457 - 0 0 0.0m - - - - - -
458 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-buf/dm-0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 458 - 0 0 0.0m - - - - - -
459 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-data/dm-0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 459 - 0 0 0.0m - - - - - -
460 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-conv/dm-0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 460 - 0 0 0.0m - - - - - -
461 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-cil/dm-0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 461 - 0 0 0.0m - - - - - -
462 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-reclaim/dm- 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 462 - 0 0 0.0m - - - - - -
463 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-log/dm-0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 463 - 0 0 0.0m - - - - - -
464 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-eofblocks/d 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 464 - 0 0 0.0m - - - - - -
465 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.81 xfsaild/dm-0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 xfsaild 4.2.284. - - - 465 - 0 0 0.0m - - - - - -
466 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.03 kworker/0:1H 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 worker_th+ .42.a.6. - - - 466 - 0 0 0.0m - - - - - -
544 root 20 0 38.2m 3.2m 2.9m S 0.0 0.1 0:00.22 systemd-journal 1 0 0 root 0 root 0 root 544 ? -1 544 1 0 0:00 0.0m 0.3m 0.4m 7 3534 0 ep_poll ..4.21.. 3:devices:/system.slice/systemd-journ+ - - 544 - 0 0 3.2m - - - - - -
565 root 20 0 196.4m 4.1m 2.5m S 0.0 0.1 0:00.01 lvmetad 1 0 0 root 0 root 0 root 565 ? -1 565 1 0 0:00 0.0m 0.1m 153.5m 1 1536 0 poll_sche+ 4.4.21.. 3:devices:/system.slice/lvm2-lvmetad.+ - - 565 - 0 0 4.1m - - - - - -
575 root 20 0 47.0m 5.1m 2.8m S 0.0 0.1 0:00.20 systemd-udevd 1 0 0 root 0 root 0 root 575 ? -1 575 1 0 0:00 0.0m 0.4m 2.4m 3 3107 0 ep_poll ..4.21.. 3:devices:/system.slice/systemd-udevd+ - - 575 - 0 0 5.1m - - - - - -
631 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kworker/u257:0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 worker_th+ .42.a.6. - - - 631 - 0 0 0.0m - - - - - -
635 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 hci0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 635 - 0 0 0.0m - - - - - -
636 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 hci0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 636 - 0 0 0.0m - - - - - -
637 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kworker/u257:1 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 worker_th+ .42.a.6. - - - 637 - 0 0 0.0m - - - - - -
649 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-buf/sda1 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 649 - 0 0 0.0m - - - - - -
654 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-data/sda1 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 654 - 0 0 0.0m - - - - - -
659 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-conv/sda1 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 659 - 0 0 0.0m - - - - - -
660 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-cil/sda1 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 660 - 0 0 0.0m - - - - - -
665 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-reclaim/sda 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 665 - 0 0 0.0m - - - - - -
668 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-log/sda1 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 668 - 0 0 0.0m - - - - - -
673 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-eofblocks/s 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 rescuer_t+ .42.a.6. - - - 673 - 0 0 0.0m - - - - - -
675 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfsaild/sda1 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 xfsaild 4.2.284. - - - 675 - 0 0 0.0m - - - - - -
757 root 16 -4 54.2m 0.9m 0.5m S 0.0 0.0 0:00.01 auditd 1 0 0 root 0 root 0 root 757 ? -1 757 2 0 0:00 0.0m 0.1m 8.5m 0 218 0 ep_poll ..4.214. 3:devices:/system.slice/auditd.servic+ - - 757 - 0 0 0.9m - - - - - -
784 root 20 0 56.2m 2.5m 2.1m S 0.0 0.1 0:00.01 bluetoothd 1 0 0 root 0 root 0 root 784 ? -1 784 1 0 0:00 0.0m 0.9m 0.4m 14 979 0 poll_sche+ 4.4.21.. 3:devices:/system.slice/bluetooth.ser+ - - 784 - 0 0 2.5m - - - - - -
785 root 20 0 25.8m 1.8m 1.4m S 0.0 0.0 0:00.08 systemd-logind 1 0 0 root 0 root 0 root 785 ? -1 785 1 0 0:00 0.0m 0.6m 0.4m 4 820 0 ep_poll ..4.21.. 3:devices:/system.slice/systemd-login+ - - 785 - 0 0 1.8m - - - - - -
788 root 20 0 51.6m 2.7m 1.9m S 0.0 0.1 0:00.02 smartd 1 0 0 root 0 root 0 root 788 ? -1 788 1 0 0:00 0.0m 0.6m 0.6m 9 1004 0 hrtimer_n+ 4.4.21.. 3:devices:/system.slice/smartd.servic+ - - 788 - 0 0 2.7m - - - - - -
789 polkitd 20 0 598.6m 12.7m 4.8m S 0.0 0.3 0:00.11 polkitd 1 999 999 polkitd 999 polkitd 998 polkitd 789 ? -1 789 7 0 0:00 0.0m 0.1m 512.3m 48 2646 0 poll_sche+ 4.4.21.. 3:devices:/system.slice/polkit.servic+ 998 polkitd 789 - 0 0 12.7m - - - - - -
790 dbus 20 0 64.9m 2.6m 1.8m S 0.0 0.1 0:00.21 dbus-daemon 1 81 81 dbus 81 dbus 81 dbus 790 ? -1 790 2 0 0:00 0.0m 0.2m 8.7m 2 869 0 ep_poll ..4.21.. 3:devices:/system.slice/dbus.service,+ - - 790 - 0 0 2.6m - - - - - -
797 chrony 20 0 115.0m 1.8m 1.3m S 0.0 0.0 0:00.08 chronyd 1 998 998 chrony 998 chrony 996 chrony 796 ? -1 796 1 0 0:00 0.0m 0.2m 72.4m 3 462 0 poll_sche+ 4.4.214. 3:devices:/system.slice/chronyd.servi+ - - 797 - 0 0 1.8m - - - - - -
838 root 20 0 123.3m 1.6m 1.0m S 0.0 0.0 0:00.50 crond 1 0 0 root 0 root 0 root 838 ? -1 838 1 0 0:00 0.0m 0.1m 1.3m 1 771 0 hrtimer_n+ 4.4.21.. 3:devices:/system.slice/crond.service+ - - 838 - 0 0 1.6m - - - - - -
858 root 20 0 350.3m 28.8m 6.9m S 0.0 0.8 0:00.64 firewalld 1 0 0 root 0 root 0 root 858 ? -1 858 2 0 0:00 0.0m 0.0m 93.1m 64 53k 0 poll_sche+ 4.4.21.. 3:devices:/system.slice/firewalld.ser+ - - 858 - 0 0 28.8m - - - - - -
859 root 20 0 107.5m 0.8m 0.7m S 0.0 0.0 0:00.00 agetty 1 0 0 root 0 root 0 root 859 ttyS0 859 859 1 0 0:00 0.0m 0.0m 0.3m 1 394 0 n_tty_read ..4.21.. 3:devices:/system.slice/system-serial+ - - 859 - 0 0 0.8m - - - - - -
860 root 20 0 94.3m 2.5m 1.8m S 0.0 0.1 0:00.13 login 1 0 0 root 0 root 1000 kbrazil 860 ? -1 860 1 0 0:00 0.0m 0.0m 0.8m 9 1265 0 do_wait ..4.21.. 1:name=systemd:/user.slice/user-1000.+ 10,1000 wheel,kbrazil 860 - 0 0 2.5m - - - - - -
883 root 20 0 813.4m 8.8m 6.7m S 0.0 0.2 0:00.28 NetworkManager 1 0 0 root 0 root 0 root 883 ? -1 883 3 0 0:00 0.0m 2.7m 561.5m 51 2944 0 poll_sche+ 4.4.21.. 3:devices:/system.slice/NetworkManage+ - - 883 - 0 0 8.8m - - - - - -
1033 root 20 0 100.5m 5.3m 3.3m S 0.0 0.1 0:00.02 dhclient 883 0 0 root 0 root 0 root 1033 ? -1 883 1 0 0:00 0.0m 0.4m 1.7m 15 1840 0 poll_sche+ 4.4.21.. 3:devices:/system.slice/NetworkManage+ - - 1033 - 0 0 5.3m - - - - - -
1224 root 20 0 560.7m 17.0m 6.0m S 0.0 0.5 0:00.97 tuned 1 0 0 root 0 root 0 root 1224 ? -1 1224 5 0 0:00 0.0m 0.0m 297.9m 5 9147 0 poll_sche+ 4.4.21.. 3:devices:/system.slice/tuned.service+ - - 1224 - 0 0 17.0m - - - - - -
1225 root 20 0 565.2m 25.1m 12.9m S 0.0 0.7 0:06.57 dockerd-current 1 0 0 root 0 root 0 root 1225 ? -1 1225 9 0 0:06 0.0m 31.1m 464.2m 234 7475 0 do_wait ..4.21.. 3:devices:/system.slice/docker.servic+ - - 1225 - 0 0 25.1m - - - - - -
1227 root 20 0 110.3m 4.2m 3.2m S 0.0 0.1 0:00.01 sshd 1 0 0 root 0 root 0 root 1227 ? -1 1227 1 0 0:00 0.0m 0.8m 0.7m 8 1323 0 poll_sche+ 4.4.21.. 3:devices:/system.slice/sshd.service,+ - - 1227 - 0 0 4.2m - - - - - -
1228 root 20 0 213.4m 4.3m 3.3m S 0.0 0.1 0:00.65 rsyslogd 1 0 0 root 0 root 0 root 1228 ? -1 1228 3 0 0:00 0.0m 0.6m 145.3m 9 1533 0 poll_sche+ 4.4.21.. 3:devices:/system.slice/rsyslog.servi+ - - 1228 - 0 0 4.3m - - - - - -
1272 root 20 0 280.5m 10.9m 5.1m S 0.0 0.3 0:03.09 docker-containe 1225 0 0 root 0 root 0 root 1272 ? -1 1272 8 0 0:03 0.0m 6.5m 255.8m 46 3932 0 futex_wai+ 4.4.21.. 3:devices:/system.slice/docker.servic+ - - 1272 - 0 0 10.9m - - - - - -
1501 root 20 0 87.6m 2.1m 1.1m S 0.0 0.1 0:00.04 master 1 0 0 root 0 root 0 root 1501 ? -1 1501 1 0 0:00 0.0m 0.2m 0.8m 0 423 0 ep_poll ..4.214. 3:devices:/system.slice/postfix.servi+ 0 root 1501 - 0 0 2.1m - - - - - -
1511 postfix 20 0 87.7m 4.0m 3.0m S 0.0 0.1 0:00.01 pickup 1501 89 89 postfix 89 postfix 89 postfix 1501 ? -1 1501 1 0 0:00 0.0m 0.3m 0.8m 1 1166 0 ep_poll ..4.21.. 3:devices:/system.slice/postfix.servi+ 12,89 mail,postfix 1511 - 0 0 4.0m - - - - - -
1512 postfix 20 0 87.8m 4.0m 3.0m S 0.0 0.1 0:00.01 qmgr 1501 89 89 postfix 89 postfix 89 postfix 1501 ? -1 1501 1 0 0:00 0.0m 0.3m 0.8m 1 1195 0 ep_poll ..4.21.. 3:devices:/system.slice/postfix.servi+ 12,89 mail,postfix 1512 - 0 0 4.0m - - - - - -
1859 kbrazil 20 0 113.0m 2.3m 1.5m S 0.0 0.1 0:00.01 bash 860 1000 1000 kbrazil 1000 kbrazil 1000 kbrazil 1859 tty1 1859 1859 1 0 0:00 0.0m 0.9m 0.7m 1 1226 0 n_tty_read ..4.21.. 1:name=systemd:/user.slice/user-1000.+ 10,1000 wheel,kbrazil 1859 HOME=/home/kbrazil USER=kbrazil SHELL+ 0 0 2.3m 4026531839 4026531840 4026531956 4026531836 4026531837 4026531838
1879 root 20 0 155.2m 5.6m 4.3m S 0.0 0.1 0:00.16 sshd 1227 0 0 root 0 root 0 root 1879 ? -1 1879 1 0 0:00 0.0m 0.8m 0.9m 0 1849 0 poll_sche+ 4.4.21.. 1:name=systemd:/user.slice/user-1000.+ - - 1879 - 0 0 5.6m - - - - - -
1883 kbrazil 20 0 155.2m 2.4m 1.1m S 0.0 0.1 0:01.29 sshd 1879 1000 1000 kbrazil 1000 kbrazil 1000 kbrazil 1879 ? -1 1879 1 0 0:01 0.0m 0.8m 0.9m 0 362 0 poll_sche+ 4.4.214. 1:name=systemd:/user.slice/user-1000.+ 10,1000 wheel,kbrazil 1883 - 0 0 2.4m - - - - - -
1884 kbrazil 20 0 112.9m 2.2m 1.6m S 0.0 0.1 0:00.07 bash 1883 1000 1000 kbrazil 1000 kbrazil 1000 kbrazil 1884 pts/0 2225 1884 1 0 0:00 0.0m 0.9m 0.6m 0 2593 0 do_wait ..4.2... 1:name=systemd:/user.slice/user-1000.+ 10,1000 wheel,kbrazil 1884 LANG=en_US.UTF-8 USER=kbrazil LOGNAME+ 0 0 2.2m 4026531839 4026531840 4026531956 4026531836 4026531837 4026531838
2019 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.11 kworker/u256:0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 worker_th+ .42.a.6. - - - 2019 - 0 0 0.0m - - - - - -
2117 root 20 0 122.4m 1.1m 0.9m S 0.0 0.0 0:00.00 anacron 1 0 0 root 0 root 0 root 2117 ? -1 2117 1 0 0:00 0.0m 0.0m 0.4m 0 281 0 sigsuspend ..4.2.4. 1:name=systemd:/user.slice/user-0.sli+ 0 root 2117 - 0 0 1.1m - - - - - -
2139 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.36 kworker/0:2 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 worker_th+ .42.a.6. - - - 2139 - 0 0 0.0m - - - - - -
2197 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.01 kworker/0:3 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 worker_th+ .42.a.6. - - - 2197 - 0 0 0.0m - - - - - -
2205 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.01 kworker/0:0 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 worker_th+ .42.a.6. - - - 2205 - 0 0 0.0m - - - - - -
2213 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kworker/0:1 2 0 0 root 0 root 0 root 0 ? -1 0 1 0 0:00 0.0m 0.0m 0.0m 0 0 0 worker_th+ .42.a.6. - - - 2213 - 0 0 0.0m - - 5678 - 1234 -

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,115 @@
top - 11:10:47 up 1:08, 2 users, load average: 0.00, 0.01, 0.05
Tasks: 108 total, 2 running, 106 sleeping, 0 stopped, 0 zombie
%Cpu(s): 5.9 us, 5.9 sy, 0.0 ni, 88.2 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
GiB Mem : 3.7 total, 3.3 free, 0.2 used, 0.2 buff/cache
GiB Swap: 2.0 total, 2.0 free, 0.0 used. 3.3 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 20 0 125.1m 6.5m 4.1m S 0.0 0.2 0:01.47 systemd
2 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kthreadd
4 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kworker/0:0H
6 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.10 ksoftirqd/0
7 root rt 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 migration/0
8 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 rcu_bh
9 root 20 0 0.0m 0.0m 0.0m R 0.0 0.0 0:00.49 rcu_sched
10 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 lru-add-drain
11 root rt 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.07 watchdog/0
13 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kdevtmpfs
14 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 netns
15 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 khungtaskd
16 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 writeback
17 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kintegrityd
18 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 bioset
19 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 bioset
20 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 bioset
21 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kblockd
22 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 md
23 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 edac-poller
24 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 watchdogd
30 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kswapd0
31 root 25 5 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 ksmd
32 root 39 19 0.0m 0.0m 0.0m S 0.0 0.0 0:00.02 khugepaged
33 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 crypto
41 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kthrotld
43 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kmpath_rdacd
44 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kaluad
45 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kpsmoused
47 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 ipv6_addrconf
60 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 deferwq
95 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kauditd
272 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 nfit
273 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 mpt_poll_0
274 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 mpt/0
275 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 ata_sff
321 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 scsi_eh_0
326 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 scsi_tmf_0
345 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.01 scsi_eh_1
346 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 scsi_tmf_1
347 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.01 scsi_eh_2
350 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 scsi_tmf_2
356 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kworker/u256:4
359 root -51 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.02 irq/16-vmwgfx
360 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 ttm_swap
431 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kdmflush
432 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 bioset
442 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kdmflush
443 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 bioset
455 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 bioset
456 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfsalloc
457 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs_mru_cache
458 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-buf/dm-0
459 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-data/dm-0
460 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-conv/dm-0
461 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-cil/dm-0
462 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-reclaim/dm-
463 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-log/dm-0
464 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-eofblocks/d
465 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.61 xfsaild/dm-0
466 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.03 kworker/0:1H
544 root 20 0 38.2m 3.2m 2.9m S 0.0 0.1 0:00.22 systemd-journal
565 root 20 0 196.4m 4.1m 2.5m S 0.0 0.1 0:00.01 lvmetad
575 root 20 0 47.0m 5.1m 2.8m S 0.0 0.1 0:00.20 systemd-udevd
631 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kworker/u257:0
635 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 hci0
636 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 hci0
637 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kworker/u257:1
649 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-buf/sda1
654 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-data/sda1
659 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-conv/sda1
660 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-cil/sda1
665 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-reclaim/sda
668 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-log/sda1
673 root 0 -20 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfs-eofblocks/s
675 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 xfsaild/sda1
757 root 16 -4 54.2m 0.9m 0.5m S 0.0 0.0 0:00.01 auditd
784 root 20 0 56.2m 2.5m 2.1m S 0.0 0.1 0:00.01 bluetoothd
785 root 20 0 25.8m 1.8m 1.4m S 0.0 0.0 0:00.08 systemd-logind
788 root 20 0 51.6m 2.7m 1.9m S 0.0 0.1 0:00.02 smartd
789 polkitd 20 0 598.6m 12.7m 4.8m S 0.0 0.3 0:00.11 polkitd
790 dbus 20 0 64.9m 2.6m 1.8m S 0.0 0.1 0:00.20 dbus-daemon
797 chrony 20 0 115.0m 1.7m 1.3m S 0.0 0.0 0:00.08 chronyd
838 root 20 0 123.3m 1.6m 1.0m S 0.0 0.0 0:00.49 crond
858 root 20 0 350.3m 28.8m 6.9m S 0.0 0.8 0:00.64 firewalld
859 root 20 0 107.5m 0.8m 0.7m S 0.0 0.0 0:00.00 agetty
860 root 20 0 94.3m 2.5m 1.8m S 0.0 0.1 0:00.13 login
883 root 20 0 813.4m 8.8m 6.7m S 0.0 0.2 0:00.27 NetworkManager
1033 root 20 0 100.5m 5.3m 3.3m S 0.0 0.1 0:00.02 dhclient
1224 root 20 0 560.7m 17.0m 6.0m S 0.0 0.5 0:00.86 tuned
1225 root 20 0 565.2m 25.1m 12.9m S 0.0 0.7 0:05.79 dockerd-current
1227 root 20 0 110.3m 4.2m 3.2m S 0.0 0.1 0:00.01 sshd
1228 root 20 0 213.4m 4.3m 3.3m S 0.0 0.1 0:00.58 rsyslogd
1272 root 20 0 280.5m 10.9m 5.1m S 0.0 0.3 0:02.70 docker-containe
1501 root 20 0 87.6m 2.1m 1.1m S 0.0 0.1 0:00.03 master
1511 postfix 20 0 87.7m 4.0m 3.0m S 0.0 0.1 0:00.01 pickup
1512 postfix 20 0 87.8m 4.0m 3.0m S 0.0 0.1 0:00.01 qmgr
1859 kbrazil 20 0 113.0m 2.3m 1.5m S 0.0 0.1 0:00.01 bash
1879 root 20 0 155.2m 5.6m 4.3m S 0.0 0.1 0:00.16 sshd
1883 kbrazil 20 0 155.2m 2.4m 1.1m S 0.0 0.1 0:00.99 sshd
1884 kbrazil 20 0 112.9m 2.1m 1.6m S 0.0 0.1 0:00.03 bash
1984 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.59 kworker/0:1
2019 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.07 kworker/u256:0
2080 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.01 kworker/0:0
2117 root 20 0 122.4m 1.1m 0.9m S 0.0 0.0 0:00.00 anacron
2139 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.03 kworker/0:2
2197 root 20 0 0.0m 0.0m 0.0m S 0.0 0.0 0:00.00 kworker/0:3
2198 kbrazil 20 0 158.1m 2.0m 1.5m R 0.0 0.1 0:00.01 top

Some files were not shown because too many files have changed in this diff Show More