python/jc
1
0
Fork 0
mirror of https://github.com/kellyjonbrazil/jc.git synced 2025-07-17 01:32:37 +02:00
Code Issues Releases Activity

Merge pull request #114 from kellyjonbrazil/dev

Browse Source 
Dev v1.15.0
This commit is contained in:
Kelly Brazil
2021-04-07 07:54:41 -07:00
committed by GitHub
parent 577811f00b a73fdb7478
commit 275f3860d5
227 changed files with 19978 additions and 889 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
.github/workflows/pythonapp.yml vendored
View File

@ -18,6 +18,12 @@ jobs:
steps:
- uses: actions/checkout@v2
- name: "Set up timezone to America/Los_Angeles"
uses: szenius/set-timezone@v1.0
with:
timezoneLinux: "America/Los_Angeles"
timezoneMacos: "America/Los_Angeles"
timezoneWindows: "Pacific Standard Time"
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v1
with:

31
CHANGELOG
View File

@ -1,5 +1,36 @@
jc changelog
20210407 v1.15.0
- Add acpi command parser tested on linux
- Add upower command parser tested on linux
- Add /usr/bin/time command parser tested on linux and macOS
- Add dpkg -l command parser tested on linux
- Add rpm -qai command parser tested on linux
- Add finger command parser tested on linux and macOS
- Add dir command parser tested on Windows 10
- Update date parser: complete rewrite (v2.0) providing many enhancements:
- Make weekday numbering ISO 8601 compliant
- Add a calculated naive timestamp field
- Add a calculated UTC timestamp field (only if date output is in UTC)
- Add several fields, including: hour_24, utc_offset, day_of_year, week_of_year, iso, and timezone_aware
- Update uptime parser to add uptime_days, uptime_hours, uptime_minutes, uptime_total_seconds, time_hour,
time_minute, and time_second fields
- Update last parser to use new timestamp function
- Update stat parser to add access_time_epoch, access_time_epoch_utc, modify_time_epoch, modify_time_epoch_utc,
change_time_epoch, change_time_epoch_utc, birth_time_epoch, birth_time_epoch_utc fields
- Update timedatectl parser to add epoch_utc field
- Update who parser to add epoch field
- Update dig parser to add when_epoch and when_epoch_utc fields
- Update ls parser to add epoch and epoch_utc fields
- Add -h option to display the help text. Piping errors no longer show the help text.
- Add -v option to display version information.
- Add contributing information to project root
- Make all external python library dependencies optional: pygments, ruamel.yaml, xmltodict
- JSON output now supports unencoded unicode characters
- JSON output is now more compact unless the -p (pretty) option is used
- Developer scripts added and enhanced to automate documentation and man page creation
- Enhanced man page
20210305 v1.14.4
- Packaging fix only for binaries and RPMs hosted on https://github.com/kellyjonbrazil/jc-packaging.
Packages from PyPi and OS repositories are not affected. This fixes an issue that kept the YAML

83
CONTRIBUTING.md Normal file
View File

@ -0,0 +1,83 @@
# Contributing to jc
We love your input! We want to make contributing to this project as easy and transparent as possible, whether it's:
- Reporting a bug
- Discussing the current state of the code
- Submitting a fix
- Proposing new features
- Proposing a new parser
## We Develop with Github
We use github to host code, to track issues and feature requests, as well as accept pull requests.
## We Use Github Flow, So All Code Changes Happen Through Pull Requests
Pull requests are the best way to propose changes to the codebase (we use [Github Flow](https://guides.github.com/introduction/flow/index.html)). We actively welcome your pull requests:
1. Open an issue to discuss the new feature, bug fix, or parser before opening a pull request. For new parsers, it is important to agree upon a schema before developing the parser.
2. Fork the repo and create your branch from `dev`, if available, otherwise `master`.
3. If you've added code that should be tested, add tests. All new parsers should have several sample outputs and tests.
4. Documentation is auto-generated from docstrings, so ensure they are clear and accurate.
5. Ensure the test suite passes.
6. Make sure your code lints.
7. Issue that pull request!
## Parser Schema Guidelines
- Try to keep the schema as flat as possible - typically a list of flat dictionaries
- Keys should be lowercase, contain no special characters, and spaces should be converted to underscores
- Keys should be static, if possible. If they have to be dynamic, then they should not contain lists or dictionaries
This will make it easier to use tools like `jq` without requiring escaping of special characters, encapsulating key names in [""], keeps paths predictable, and makes iterating and searching for values easier.
**Examples**
Bad:
```
{
"Interface 1": [
192.168.1.1,
172.16.1.1
],
"Wifi Interface 1": [
10.1.1.1
]
}
```
Good:
```
[
{
"interface": "Interface 1",
"ip_addresses": [
192.168.1.1,
172.16.1.1
]
},
{
"interface": "Wifi Interface 1",
"ip_addresses": [
10.1.1.1
]
}
]
```
## Any contributions you make will be under the MIT Software License
In short, when you submit code changes, your submissions are understood to be under the same [MIT License](http://choosealicense.com/licenses/mit/) that covers the project. Feel free to contact the maintainers if that's a concern.
## Report bugs using Github's Issues
We use GitHub issues to track public bugs. Report a bug by [opening a new issue](https://github.com/kellyjonbrazil/jc/issues); it's that easy!
## Write bug reports with detail, background, and sample code
**Great Bug Reports** tend to have:
- A quick summary and/or background
- Steps to reproduce
- Be specific!
- Give sample code if you can.
- What you expected would happen
- What actually happens
- Notes (possibly including why you think this might be happening, or stuff you tried that didn't work)
## Use a Consistent Coding Style
* 4 spaces for indentation rather than tabs
* Use a Python linter that will enforce PEP 8 and other best practices

579
EXAMPLES.md
View File

@ -1,4 +1,94 @@
## JC Examples
### acpi
```bash
acpi -V | jc --acpi -p # or: jc -p acpi -V
```
```json
[
{
"type": "Battery",
"id": 0,
"state": "Charging",
"charge_percent": 71,
"until_charged": "00:29:20",
"design_capacity_mah": 2110,
"last_full_capacity": 2271,
"last_full_capacity_percent": 100,
"until_charged_hours": 0,
"until_charged_minutes": 29,
"until_charged_seconds": 20,
"until_charged_total_seconds": 1760
},
{
"type": "Adapter",
"id": 0,
"on-line": true
},
{
"type": "Thermal",
"id": 0,
"mode": "ok",
"temperature": 46.0,
"temperature_unit": "C",
"trip_points": [
{
"id": 0,
"switches_to_mode": "critical",
"temperature": 127.0,
"temperature_unit": "C"
},
{
"id": 1,
"switches_to_mode": "hot",
"temperature": 127.0,
"temperature_unit": "C"
}
]
},
{
"type": "Cooling",
"id": 0,
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": 1,
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": 2,
"messages": [
"x86_pkg_temp no state information available"
]
},
{
"type": "Cooling",
"id": 3,
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": 4,
"messages": [
"intel_powerclamp no state information available"
]
},
{
"type": "Cooling",
"id": 5,
"messages": [
"Processor 0 of 10"
]
}
]
```
### airport -I
```bash
airport -I | jc --airport -p # or: jc -p airport -I
@ -406,17 +496,25 @@ date | jc --date -p # or: jc -p date
```
```json
{
"year": 2020,
"month_num": 7,
"day": 31,
"hour": 16,
"minute": 48,
"second": 11,
"period": null,
"month": "Jul",
"weekday": "Fri",
"weekday_num": 6,
"timezone": "PDT"
"year": 2021,
"month": "Mar",
"month_num": 3,
"day": 25,
"weekday": "Thu",
"weekday_num": 4,
"hour": 2,
"hour_24": 2,
"minute": 2,
"second": 26,
"period": "AM",
"timezone": "UTC",
"utc_offset": "+0000",
"day_of_year": 84,
"week_of_year": 12,
"iso": "2021-03-25T02:02:26+00:00",
"epoch": 1616662946,
"epoch_utc": 1616637746,
"timezone_aware": true
}
```
### df
@ -450,7 +548,7 @@ dig cnn.com www.cnn.com @205.251.194.64 | jc --dig -p # or: jc -p dig
```json
[
{
"id": 5509,
"id": 52172,
"opcode": "QUERY",
"status": "NOERROR",
"flags": [
@ -472,38 +570,40 @@ dig cnn.com www.cnn.com @205.251.194.64 | jc --dig -p # or: jc -p dig
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 60,
"ttl": 27,
"data": "151.101.65.67"
},
{
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 27,
"data": "151.101.129.67"
},
{
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 60,
"data": "151.101.193.67"
},
{
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 60,
"ttl": 27,
"data": "151.101.1.67"
},
{
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 60,
"data": "151.101.65.67"
"ttl": 27,
"data": "151.101.193.67"
}
],
"query_time": 28,
"query_time": 38,
"server": "2600",
"when": "Tue Nov 12 07:13:03 PST 2019",
"rcvd": 100
"when": "Tue Mar 30 20:07:59 PDT 2021",
"rcvd": 100,
"when_epoch": 1617160079,
"when_epoch_utc": null
},
{
"id": 62696,
"id": 36292,
"opcode": "QUERY",
"status": "NOERROR",
"flags": [
@ -559,10 +659,12 @@ dig cnn.com www.cnn.com @205.251.194.64 | jc --dig -p # or: jc -p dig
"data": "ns-576.awsdns-08.net."
}
],
"query_time": 29,
"query_time": 27,
"server": "205.251.194.64#53(205.251.194.64)",
"when": "Tue Nov 12 07:13:03 PST 2019",
"rcvd": 212
"when": "Tue Mar 30 20:07:59 PDT 2021",
"rcvd": 212,
"when_epoch": 1617160079,
"when_epoch_utc": null
}
]
```
@ -572,7 +674,7 @@ dig -x 1.1.1.1 | jc --dig -p # or: jc -p dig -x 1.1.1.1
```json
[
{
"id": 50324,
"id": 22191,
"opcode": "QUERY",
"status": "NOERROR",
"flags": [
@ -594,14 +696,61 @@ dig -x 1.1.1.1 | jc --dig -p # or: jc -p dig -x 1.1.1.1
"name": "1.1.1.1.in-addr.arpa.",
"class": "IN",
"type": "PTR",
"ttl": 1634,
"ttl": 1800,
"data": "one.one.one.one."
}
],
"query_time": 36,
"query_time": 44,
"server": "2600",
"when": "Tue Nov 12 07:13:49 PST 2019",
"rcvd": 78
"when": "Tue Mar 30 20:10:34 PDT 2021",
"rcvd": 78,
"when_epoch": 1617160234,
"when_epoch_utc": null
}
]
```
### dir
```bash
dir | jc --dir -p # or: jc -p dir
```
```json
[
{
"date": "03/24/2021",
"time": "03:15 PM",
"dir": true,
"size": null,
"filename": ".",
"parent": "C:\\Program Files\\Internet Explorer",
"epoch": 1616624100
},
{
"date": "03/24/2021",
"time": "03:15 PM",
"dir": true,
"size": null,
"filename": "..",
"parent": "C:\\Program Files\\Internet Explorer",
"epoch": 1616624100
},
{
"date": "12/07/2019",
"time": "02:49 AM",
"dir": true,
"size": null,
"filename": "en-US",
"parent": "C:\\Program Files\\Internet Explorer",
"epoch": 1575715740
},
{
"date": "12/07/2019",
"time": "02:09 AM",
"dir": false,
"size": 54784,
"filename": "ExtExport.exe",
"parent": "C:\\Program Files\\Internet Explorer",
"epoch": 1575713340
}
]
```
@ -652,6 +801,60 @@ dmidecode | jc --dmidecode -p # or: jc -p dmidecode
}
]
```
### dpkg -l
```bash
dpkg -l | jc --dpkg-l -p # or: jc -p dpkg -l
```
```json
[
{
"codes": "ii",
"name": "accountsservice",
"version": "0.6.45-1ubuntu1.3",
"architecture": "amd64",
"description": "query and manipulate user account information",
"desired": "install",
"status": "installed"
},
{
"codes": "rc",
"name": "acl",
"version": "2.2.52-3build1",
"architecture": "amd64",
"description": "Access control list utilities",
"desired": "remove",
"status": "config-files"
},
{
"codes": "uWR",
"name": "acpi",
"version": "1.7-1.1",
"architecture": "amd64",
"description": "displays information on ACPI devices",
"desired": "unknown",
"status": "trigger await",
"error": "reinstall required"
},
{
"codes": "rh",
"name": "acpid",
"version": "1:2.0.28-1ubuntu1",
"architecture": "amd64",
"description": "Advanced Configuration and Power Interface event daemon",
"desired": "remove",
"status": "half installed"
},
{
"codes": "pn",
"name": "adduser",
"version": "3.116ubuntu1",
"architecture": "all",
"description": "add and remove users and groups",
"desired": "purge",
"status": "not installed"
}
]
```
### du
```bash
du /usr | jc --du -p # or: jc -p du /usr
@ -748,6 +951,39 @@ file * | jc --file -p # or: jc -p file *
}
]
```
### finger
```bash
finger | jc --finger -p # or: jc -p finger
```
```json
[
{
"login": "jdoe",
"name": "John Doe",
"tty": "tty1",
"idle": "14d",
"login_time": "Mar 22 21:14",
"tty_writeable": false,
"idle_minutes": 0,
"idle_hours": 0,
"idle_days": 14,
"total_idle_minutes": 20160
},
{
"login": "jdoe",
"name": "John Doe",
"tty": "pts/0",
"idle": null,
"login_time": "Apr 5 15:33",
"details": "(192.168.1.22)",
"tty_writeable": true,
"idle_minutes": 0,
"idle_hours": 0,
"idle_days": 0,
"total_idle_minutes": 0
}
]
```
### free
```bash
free | jc --free -p # or: jc -p free
@ -1475,32 +1711,36 @@ cat keyvalue.txt | jc --kv -p
```
### last and lastb
```bash
last | jc --last -p # or: jc -p last
last -F | jc --last -p # or: jc -p last -F
```
```json
[
{
"user": "joeuser",
"tty": "ttys002",
"hostname": null,
"login": "Thu Feb 27 14:31",
"logout": "still logged in"
"user": "kbrazil",
"tty": "pts/0",
"hostname": "kbrazil-mac.attlocal.net",
"login": "Tue Jan 5 14:29:24 2021",
"logout": "still logged in",
"login_epoch": 1609885764
},
{
"user": "joeuser",
"tty": "ttys003",
"user": "kbrazil",
"tty": "tty1",
"hostname": null,
"login": "Thu Feb 27 10:38",
"logout": "10:38",
"duration": "00:00"
"login": "Tue Jan 5 14:28:41 2021",
"logout": "still logged in",
"login_epoch": 1609885721
},
{
"user": "joeuser",
"tty": "ttys003",
"hostname": null,
"login": "Thu Feb 27 10:18",
"logout": "10:18",
"duration": "00:00"
"user": "reboot",
"tty": "system boot",
"hostname": "3.10.0-1062.1.2.el7.x86_64",
"login": "Tue Jan 5 14:28:28 2021",
"logout": "Tue Jan 5 14:29:36 2021",
"duration": "00:01",
"login_epoch": 1609885708,
"logout_epoch": 1609885776,
"duration_seconds": 68
}
]
```
@ -2239,6 +2479,59 @@ route -ee | jc --route -p # or: jc -p route -ee
}
]
```
### rpm -qai
```bash
rpm_qia | jc --rpm_qi -p # or: jc -p rpm -qia
```
```json
[
{
"name": "make",
"epoch": 1,
"version": "3.82",
"release": "24.el7",
"architecture": "x86_64",
"install_date": "Wed 16 Oct 2019 09:21:42 AM PDT",
"group": "Development/Tools",
"size": 1160660,
"license": "GPLv2+",
"signature": "RSA/SHA256, Thu 22 Aug 2019 02:34:59 PM PDT, Key ID 24c6a8a7f4a80eb5",
"source_rpm": "make-3.82-24.el7.src.rpm",
"build_date": "Thu 08 Aug 2019 05:47:25 PM PDT",
"build_host": "x86-01.bsys.centos.org",
"relocations": "(not relocatable)",
"packager": "CentOS BuildSystem <http://bugs.centos.org>",
"vendor": "CentOS",
"url": "http://www.gnu.org/software/make/",
"summary": "A GNU tool which simplifies the build process for users",
"description": "A GNU tool for controlling the generation of executables and other non-source...",
"build_epoch": 1565311645,
"build_epoch_utc": null
},
{
"name": "kbd-legacy",
"version": "1.15.5",
"release": "15.el7",
"architecture": "noarch",
"install_date": "Thu 15 Aug 2019 10:53:08 AM PDT",
"group": "System Environment/Base",
"size": 503608,
"license": "GPLv2+",
"signature": "RSA/SHA256, Mon 12 Nov 2018 07:17:49 AM PST, Key ID 24c6a8a7f4a80eb5",
"source_rpm": "kbd-1.15.5-15.el7.src.rpm",
"build_date": "Tue 30 Oct 2018 03:40:00 PM PDT",
"build_host": "x86-01.bsys.centos.org",
"relocations": "(not relocatable)",
"packager": "CentOS BuildSystem <http://bugs.centos.org>",
"vendor": "CentOS",
"url": "http://ftp.altlinux.org/pub/people/legion/kbd",
"summary": "Legacy data for kbd package",
"description": "The kbd-legacy package contains original keymaps for kbd package. Please note...",
"build_epoch": 1540939200,
"build_epoch_utc": null
}
]
```
### /etc/shadow file
```bash
cat /etc/shadow | jc --shadow -p
@ -2420,7 +2713,15 @@ stat /bin/* | jc --stat -p # or: jc -p stat /bin/*
"access_time": "2019-11-14 08:18:03.509681766 +0000",
"modify_time": "2019-06-06 22:28:15.000000000 +0000",
"change_time": "2019-08-12 17:21:29.521945390 +0000",
"birth_time": null
"birth_time": null,
"access_time_epoch": 1573748283,
"access_time_epoch_utc": 1573719483,
"modify_time_epoch": 1559885295,
"modify_time_epoch_utc": 1559860095,
"change_time_epoch": 1565655689,
"change_time_epoch_utc": 1565630489,
"birth_time_epoch": null,
"birth_time_epoch_utc": null
},
{
"file": "/bin/btrfs",
@ -2440,7 +2741,15 @@ stat /bin/* | jc --stat -p # or: jc -p stat /bin/*
"access_time": "2019-11-14 08:18:28.990834276 +0000",
"modify_time": "2018-03-12 23:04:27.000000000 +0000",
"change_time": "2019-08-12 17:21:29.545944399 +0000",
"birth_time": null
"birth_time": null,
"access_time_epoch": 1573748308,
"access_time_epoch_utc": 1573719508,
"modify_time_epoch": 1520921067,
"modify_time_epoch_utc": 1520895867,
"change_time_epoch": 1565655689,
"change_time_epoch_utc": 1565630489,
"birth_time_epoch": null,
"birth_time_epoch_utc": null
}
]
```
@ -2557,6 +2866,42 @@ systemctl list-unit-files | jc --systemctl-luf -p # or: jc -p systemct
}
]
```
### /usr/bin/time
```bash
/usr/bin/time --verbose -o timefile.out sleep 2.5; cat timefile.out | jc --time -p
```
```json
{
"command_being_timed": "sleep 2.5",
"user_time": 0.0,
"system_time": 0.0,
"cpu_percent": 0,
"elapsed_time": "0:02.50",
"average_shared_text_size": 0,
"average_unshared_data_size": 0,
"average_stack_size": 0,
"average_total_size": 0,
"maximum_resident_set_size": 2084,
"average_resident_set_size": 0,
"major_pagefaults": 0,
"minor_pagefaults": 72,
"voluntary_context_switches": 2,
"involuntary_context_switches": 1,
"swaps": 0,
"block_input_operations": 0,
"block_output_operations": 0,
"messages_sent": 0,
"messages_received": 0,
"signals_delivered": 0,
"page_size": 4096,
"exit_status": 0,
"elapsed_time_hours": 0,
"elapsed_time_minutes": 0,
"elapsed_time_seconds": 2,
"elapsed_time_centiseconds": 50,
"elapsed_time_total_seconds": 2.5
}
```
### timedatectl status
```bash
timedatectl | jc --timedatectl -p # or: jc -p timedatectl
@ -2570,7 +2915,8 @@ timedatectl | jc --timedatectl -p # or: jc -p timedatectl
"ntp_enabled": true,
"ntp_synchronized": true,
"rtc_in_local_tz": false,
"dst_active": true
"dst_active": true,
"epoch_utc": 1583888001
}
```
### tracepath
@ -2706,18 +3052,88 @@ uname -a | jc --uname -p # or: jc -p uname -a
"kernel_version": "#74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019"
}
```
### upower
```bash
upower -i /org/freedesktop/UPower/devices/battery | jc --upower -p # or jc -p upower -i /org/freedesktop/UPower/devices/battery
```
```json
[
{
"native_path": "/sys/devices/LNXSYSTM:00/device:00/PNP0C0A:00/power_supply/BAT0",
"vendor": "NOTEBOOK",
"model": "BAT",
"serial": "0001",
"power_supply": true,
"updated": "Thu 11 Mar 2021 06:28:08 PM UTC",
"has_history": true,
"has_statistics": true,
"detail": {
"type": "battery",
"present": true,
"rechargeable": true,
"state": "charging",
"energy": 22.3998,
"energy_empty": 0.0,
"energy_full": 52.6473,
"energy_full_design": 62.16,
"energy_rate": 31.6905,
"voltage": 12.191,
"time_to_full": 57.3,
"percentage": 42.5469,
"capacity": 84.6964,
"technology": "lithium-ion",
"energy_unit": "Wh",
"energy_empty_unit": "Wh",
"energy_full_unit": "Wh",
"energy_full_design_unit": "Wh",
"energy_rate_unit": "W",
"voltage_unit": "V",
"time_to_full_unit": "minutes"
},
"history_charge": [
{
"time": 1328809335,
"percent_charged": 42.547,
"status": "charging"
},
{
"time": 1328809305,
"percent_charged": 42.02,
"status": "charging"
}
],
"history_rate": [
{
"time": 1328809335,
"percent_charged": 31.691,
"status": "charging"
}
],
"updated_seconds_ago": 441975,
"updated_epoch": 1615516088,
"updated_epoch_utc": 1615487288
}
]
```
### uptime
```bash
uptime | jc --uptime -p # or: jc -p uptime
```
```json
{
"time": "11:30:44",
"uptime": "1 day, 21:17",
"users": 1,
"load_1m": 0.01,
"load_5m": 0.04,
"load_15m": 0.05
"time": "11:35",
"uptime": "3 days, 4:03",
"users": 5,
"load_1m": 1.88,
"load_5m": 2.0,
"load_15m": 1.94,
"time_hour": 11,
"time_minute": 35,
"time_second": null,
"uptime_days": 3,
"uptime_hours": 4,
"uptime_minutes": 3,
"uptime_total_seconds": 273780
}
```
### w
@ -2793,13 +3209,15 @@ who | jc --who -p # or: jc -p who
{
"user": "joeuser",
"tty": "ttyS0",
"time": "2020-03-02 02:52"
"time": "2020-03-02 02:52",
"epoch": 1583146320
},
{
"user": "joeuser",
"tty": "pts/0",
"time": "2020-03-02 05:15",
"from": "192.168.71.1"
"from": "192.168.71.1",
"epoch": 1583154900
}
]
```
@ -2811,32 +3229,8 @@ who -a | jc --who -p # or: jc -p who -a
{
"event": "reboot",
"time": "Feb 7 23:31",
"pid": 1
},
{
"user": "joeuser",
"writeable_tty": "-",
"tty": "console",
"time": "Feb 7 23:32",
"idle": "old",
"pid": 105
},
{
"user": "joeuser",
"writeable_tty": "+",
"tty": "ttys000",
"time": "Feb 13 16:44",
"idle": ".",
"pid": 51217,
"comment": "term=0 exit=0"
},
{
"user": "joeuser",
"writeable_tty": "?",
"tty": "ttys003",
"time": "Feb 28 08:59",
"idle": "01:36",
"pid": 41402
"pid": 1,
"epoch": null
},
{
"user": "joeuser",
@ -2845,7 +3239,8 @@ who -a | jc --who -p # or: jc -p who -a
"time": "Mar 1 16:35",
"idle": ".",
"pid": 15679,
"from": "192.168.1.5"
"from": "192.168.1.5",
"epoch": null
}
]
```
@ -2963,4 +3358,6 @@ cat istio.yaml | jc --yaml -p
}
}
]
```
```
© 2019-2021 Kelly Brazil

194
README.md
View File

@ -68,9 +68,13 @@ The `jc` parsers can also be used as python modules. In this case the output wil
```
Two representations of the data are possible. The default representation uses a strict schema per parser and converts known numbers to int/float JSON values. Certain known values of `None` are converted to JSON `null`, known boolean values are converted, and, in some cases, additional semantic context fields are added.
> Note: Some parsers have calculated epoch timestamp fields added to the output. Unless a timestamp field name has a `_utc` suffix it is considered naive. (i.e. based on the local timezone of the system the `jc` parser was run on).
>
> If a UTC timezone can be detected in the text of the command output, the timestamp will be timezone aware and have a `_utc` suffix on the key name. (e.g. `epoch_utc`) No other timezones are supported for aware timestamps.
To access the raw, pre-processed JSON, use the `-r` cli option or the `raw=True` function parameter in `parse()`.
Schemas for each parser can be found in the [`docs/parsers`](https://github.com/kellyjonbrazil/jc/tree/master/docs/parsers) folder.
Schemas for each parser can be found at the documentation link beside each parser below.
Release notes can be found [here](https://blog.kellybrazil.com/category/jc-news/).
@ -98,7 +102,7 @@ pip3 install jc
| Fedora linux | `dnf install jc` |
| openSUSE linux | `zypper install jc` |
| Arch linux | `pacman -S jc` |
| NixOS linux | `nix-env -iA nixpkgs.jc` |
| NixOS linux | `nix-env -iA nixpkgs.jc` or `nix-env -iA nixos.jc` |
| Guix System linux | `guix install jc` |
| MacOS | `brew install jc` |
| FreeBSD | `portsnap fetch update && cd /usr/ports/textproc/py-jc && make install clean` |
@ -120,77 +124,87 @@ The JSON output can be compact (default) or pretty formatted with the `-p` optio
> Note: For best results set the `LANG` locale environment variable to `C`. For example, either by setting directly on the command-line: `$ LANG=C date | jc --date`, or by exporting to the environment before running commands: `$ export LANG=C`.
### Parsers
- `--airport` enables the `airport -I` command parser (OSX)
- `--airport-s` enables the `airport -s` command parser (OSX)
- `--arp` enables the `arp` command parser
- `--blkid` enables the `blkid` command parser
- `--cksum` enables the `cksum` and `sum` command parser
- `--crontab` enables the `crontab` command and file parser
- `--crontab-u` enables the `crontab` file parser with user support
- `--csv` enables the `CSV` file parser
- `--date` enables the `date` command parser
- `--df` enables the `df` command parser
- `--dig` enables the `dig` command parser
- `--dmidecode` enables the `dmidecode` command parser
- `--du` enables the `du` command parser
- `--env` enables the `env` and `printenv` command parser
- `--file` enables the `file` command parser
- `--free` enables the `free` command parser
- `--fstab` enables the `/etc/fstab` file parser
- `--group` enables the `/etc/group` file parser
- `--gshadow` enables the `/etc/gshadow` file parser
- `--hash` enables the `hash` command parser
- `--hashsum` enables the `hashsum` command parser (`md5`, `md5sum`, `shasum`, `sha1sum`, `sha224sum`, `sha256sum`, `sha384sum`, `sha512sum`)
- `--hciconfig` enables the `hciconfig` command parser
- `--history` enables the `history` command parser
- `--hosts` enables the `/etc/hosts` file parser
- `--id` enables the `id` command parser
- `--ifconfig` enables the `ifconfig` command parser
- `--ini` enables the `INI` file parser
- `--iptables` enables the `iptables` command parser
- `--iw-scan` enables the `iw dev <device> scan` command parser (beta)
- `--jobs` enables the `jobs` command parser
- `--kv` enables the `Key/Value` file parser
- `--last` enables the `last` and `lastb` command parser
- `--ls` enables the `ls` and `vdir` command parser
- `--lsblk` enables the `lsblk` command parser
- `--lsmod` enables the `lsmod` command parser
- `--lsof` enables the `lsof` command parser
- `--mount` enables the `mount` command parser
- `--netstat` enables the `netstat` command parser
- `--ntpq` enables the `ntpq -p` command parser
- `--passwd` enables the `/etc/passwd` file parser
- `--ping` enables the `ping` and `ping6` command parser
- `--pip-list` enables the `pip list` command parser
- `--pip-show` enables the `pip show` command parser
- `--ps` enables the `ps` command parser
- `--route` enables the `route` command parser
- `--shadow` enables the `/etc/shadow` file parser
- `--ss` enables the `ss` command parser
- `--stat` enables the `stat` command parser
- `--sysctl` enables the `sysctl -a` command parser
- `--systemctl` enables the `systemctl` command parser
- `--systemctl-lj` enables the `systemctl list-jobs` command parser
- `--systemctl-ls` enables the `systemctl list-sockets` command parser
- `--systemctl-luf` enables the `systemctl list-unit-files` command parser
- `--timedatectl` enables the `timedatectl status` command parser
- `--tracepath` enables the `tracepath` and `tracepath6` command parser
- `--traceroute` enables the `traceroute` and `traceroute6` command parser
- `--uname` enables the `uname -a` command parser
- `--uptime` enables the `uptime` command parser
- `--w` enables the `w` command parser
- `--wc` enables the `wc` command parser
- `--who` enables the `who` command parser
- `--xml` enables the `XML` file parser
- `--yaml` enables the `YAML` file parser
- `--acpi` enables the `acpi` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/acpi))
- `--airport` enables the `airport -I` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/airport))
- `--airport-s` enables the `airport -s` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/airport_s))
- `--arp` enables the `arp` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/arp))
- `--blkid` enables the `blkid` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/blkid))
- `--cksum` enables the `cksum` and `sum` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/cksum))
- `--crontab` enables the `crontab` command and file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab))
- `--crontab-u` enables the `crontab` file parser with user support ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab_u))
- `--csv` enables the CSV file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/csv))
- `--date` enables the `date` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/date))
- `--df` enables the `df` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/df))
- `--dig` enables the `dig` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/dig))
- `--dir` enables the `dir` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/dir))
- `--dmidecode` enables the `dmidecode` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/dmidecode))
- `--dpkg-l` enables the `dpkg -l` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/dpkg_l))
- `--du` enables the `du` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/du))
- `--env` enables the `env` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/env))
- `--file` enables the `file` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/file))
- `--finger` enables the `finger` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/finger))
- `--free` enables the `free` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/free))
- `--fstab` enables the `/etc/fstab` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/fstab))
- `--group` enables the `/etc/group` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/group))
- `--gshadow` enables the `/etc/gshadow` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/gshadow))
- `--hash` enables the `hash` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/hash))
- `--hashsum` enables the hashsum command parser (`md5sum`, `shasum`, etc.) ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/hashsum))
- `--hciconfig` enables the `hciconfig` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/hciconfig))
- `--history` enables the `history` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/history))
- `--hosts` enables the `/etc/hosts` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/hosts))
- `--id` enables the `id` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/id))
- `--ifconfig` enables the `ifconfig` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ifconfig))
- `--ini` enables the INI file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ini))
- `--iptables` enables the `iptables` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/iptables))
- `--iw-scan` enables the `iw dev [device] scan` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/iw_scan))
- `--jobs` enables the `jobs` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/jobs))
- `--kv` enables the Key/Value file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/kv))
- `--last` enables the `last` and `lastb` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/last))
- `--ls` enables the `ls` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ls))
- `--lsblk` enables the `lsblk` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/lsblk))
- `--lsmod` enables the `lsmod` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/lsmod))
- `--lsof` enables the `lsof` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/lsof))
- `--mount` enables the `mount` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/mount))
- `--netstat` enables the `netstat` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/netstat))
- `--ntpq` enables the `ntpq -p` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ntpq))
- `--passwd` enables the `/etc/passwd` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/passwd))
- `--ping` enables the `ping` and `ping6` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ping))
- `--pip-list` enables the `pip list` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_list))
- `--pip-show` enables the `pip show` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_show))
- `--ps` enables the `ps` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ps))
- `--route` enables the `route` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/route))
- `--rpm_qi` enables the `rpm -qi` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/rpm_qi))
- `--shadow` enables the `/etc/shadow` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/shadow))
- `--ss` enables the `ss` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ss))
- `--stat` enables the `stat` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/stat))
- `--sysctl` enables the `sysctl` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/sysctl))
- `--systemctl` enables the `systemctl` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl))
- `--systemctl-lj` enables the `systemctl list-jobs` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_lj))
- `--systemctl-ls` enables the `systemctl list-sockets` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_ls))
- `--systemctl-luf` enables the `systemctl list-unit-files` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_luf))
- `--time` enables the `/usr/bin/time` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/time))
- `--timedatectl` enables the `timedatectl status` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/timedatectl))
- `--tracepath` enables the `tracepath` and `tracepath6` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/tracepath))
- `--traceroute` enables the `traceroute` and `traceroute6` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/traceroute))
- `--uname` enables the `uname -a` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/uname))
- `--upower` enables the `upower` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/upower))
- `--uptime` enables the `uptime` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/uptime))
- `--w` enables the `w` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/w))
- `--wc` enables the `wc` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/wc))
- `--who` enables the `who` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/who))
- `--xml` enables the XML file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/xml))
- `--yaml` enables the YAML file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/yaml))
### Options
- `-a` about `jc`. Prints information about `jc` and the parsers (in JSON, of course!)
- `-d` debug mode. Prints trace messages if parsing issues are encountered (use `-dd` for verbose debugging)
- `-h` `jc` help
- `-m` monochrome JSON output
- `-p` pretty format the JSON output
- `-q` quiet mode. Suppresses parser warning messages
- `-r` raw output. Provides a more literal JSON output with all values as strings and no additional semantic processing
- `-r` raw output. Provides a more literal JSON output, typically with string values and no additional semantic processing
- `-v` version information
### Setting Custom Colors via Environment Variable
You can specify custom colors via the `JC_COLORS` environment variable. The `JC_COLORS` environment variable takes four comma separated string values in the following format:
@ -224,7 +238,7 @@ Local plugin filenames must be valid python module names, therefore must consist
## Compatibility
Some parsers like `ls`, `ps`, `dig`, etc. will work on any platform. Other parsers that are platform-specific will generate a warning message if they are used on an unsupported platform. To see all parser information, including compatibility, run `jc -ap`.
You may still use a parser on an unsupported platform - for example, you may want to parse a file with linux `lsof` output on an OSX laptop. In that case you can suppress the warning message with the `-q` cli option or the `quiet=True` function parameter in `parse()`:
You may still use a parser on an unsupported platform - for example, you may want to parse a file with linux `lsof` output on an macOS laptop. In that case you can suppress the warning message with the `-q` cli option or the `quiet=True` function parameter in `parse()`:
```bash
cat lsof.out | jc --lsof -q
@ -235,14 +249,17 @@ Tested on:
- Ubuntu 18.04
- Ubuntu 20.04
- Fedora32
- OSX 10.11.6
- OSX 10.14.6
- macOS 10.11.6
- macOS 10.14.6
- NixOS
- FreeBSD12
- Windows 10
## Contributions
Feel free to add/improve code or parsers! You can use the [`jc/parsers/foo.py`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py) parser as a template and submit your parser with a pull request.
Please see the [Contributing Guidelines](https://github.com/kellyjonbrazil/jc/blob/master/CONTRIBUTING.md) for more information.
## Acknowledgments
- Local parser plugin feature contributed by [Dean Serenevy](https://github.com/duelafn)
- CI automation and code optimizations by [philippeitis](https://github.com/philippeitis)
@ -254,7 +271,7 @@ Feel free to add/improve code or parsers! You can use the [`jc/parsers/foo.py`](
- Excellent constructive feedback from [Ilya Sher](https://github.com/ilyash-b)
## Examples
Here are some examples of `jc` output. For more examples, see [EXAMPLES.md](https://github.com/kellyjonbrazil/jc/blob/master/EXAMPLES.md) or the [parser documentation](https://github.com/kellyjonbrazil/jc/tree/master/docs/parsers).
Here are some examples of `jc` output. For more examples, see [here](https://kellyjonbrazil.github.io/jc/EXAMPLES) or the parser documentation.
### arp
```bash
arp | jc --arp -p # or: jc -p arp
@ -342,7 +359,7 @@ dig cnn.com @205.251.194.64 | jc --dig -p # or: jc -p dig cnn.com @205
```json
[
{
"id": 5509,
"id": 52172,
"opcode": "QUERY",
"status": "NOERROR",
"flags": [
@ -364,14 +381,16 @@ dig cnn.com @205.251.194.64 | jc --dig -p # or: jc -p dig cnn.com @205
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 60,
"data": "151.101.129.67"
"ttl": 27,
"data": "151.101.65.67"
}
],
"query_time": 28,
"query_time": 38,
"server": "2600",
"when": "Tue Nov 12 07:13:03 PST 2019",
"rcvd": 100
"when": "Tue Mar 30 20:07:59 PDT 2021",
"rcvd": 100,
"when_epoch": 1617160079,
"when_epoch_utc": null
}
]
```
@ -802,12 +821,19 @@ uptime | jc --uptime -p # or: jc -p uptime
```
```json
{
"time": "11:30:44",
"uptime": "1 day, 21:17",
"users": 1,
"load_1m": 0.01,
"load_5m": 0.04,
"load_15m": 0.05
"time": "11:35",
"uptime": "3 days, 4:03",
"users": 5,
"load_1m": 1.88,
"load_5m": 2.0,
"load_15m": 1.94,
"time_hour": 11,
"time_minute": 35,
"time_second": null,
"uptime_days": 3,
"uptime_hours": 4,
"uptime_minutes": 3,
"uptime_total_seconds": 273780
}
```
### XML files
@ -924,4 +950,6 @@ cat istio.yaml | jc --yaml -p
}
}
]
```
```
© 2019-2021 Kelly Brazil

76
docgen.sh
View File

@ -3,68 +3,18 @@
# requires pydoc-markdown 2.1.0.post1
cd jc
echo Building docs for: package
pydocmd simple jc+ > ../docs/readme.md
echo Building docs for: utils
pydocmd simple utils+ > ../docs/utils.md
pydocmd simple jc.parsers.airport+ > ../docs/parsers/airport.md
pydocmd simple jc.parsers.airport_s+ > ../docs/parsers/airport_s.md
pydocmd simple jc.parsers.arp+ > ../docs/parsers/arp.md
pydocmd simple jc.parsers.blkid+ > ../docs/parsers/blkid.md
pydocmd simple jc.parsers.cksum+ > ../docs/parsers/cksum.md
pydocmd simple jc.parsers.crontab+ > ../docs/parsers/crontab.md
pydocmd simple jc.parsers.crontab_u+ > ../docs/parsers/crontab_u.md
pydocmd simple jc.parsers.csv+ > ../docs/parsers/csv.md
pydocmd simple jc.parsers.date+ > ../docs/parsers/date.md
pydocmd simple jc.parsers.df+ > ../docs/parsers/df.md
pydocmd simple jc.parsers.dig+ > ../docs/parsers/dig.md
pydocmd simple jc.parsers.dmidecode+ > ../docs/parsers/dmidecode.md
pydocmd simple jc.parsers.du+ > ../docs/parsers/du.md
pydocmd simple jc.parsers.env+ > ../docs/parsers/env.md
pydocmd simple jc.parsers.file+ > ../docs/parsers/file.md
pydocmd simple jc.parsers.free+ > ../docs/parsers/free.md
pydocmd simple jc.parsers.fstab+ > ../docs/parsers/fstab.md
pydocmd simple jc.parsers.group+ > ../docs/parsers/group.md
pydocmd simple jc.parsers.gshadow+ > ../docs/parsers/gshadow.md
pydocmd simple jc.parsers.hash+ > ../docs/parsers/hash.md
pydocmd simple jc.parsers.hashsum+ > ../docs/parsers/hashsum.md
pydocmd simple jc.parsers.hciconfig+ > ../docs/parsers/hciconfig.md
pydocmd simple jc.parsers.history+ > ../docs/parsers/history.md
pydocmd simple jc.parsers.hosts+ > ../docs/parsers/hosts.md
pydocmd simple jc.parsers.id+ > ../docs/parsers/id.md
pydocmd simple jc.parsers.ifconfig+ > ../docs/parsers/ifconfig.md
pydocmd simple jc.parsers.ini+ > ../docs/parsers/ini.md
pydocmd simple jc.parsers.iptables+ > ../docs/parsers/iptables.md
pydocmd simple jc.parsers.iw_scan+ > ../docs/parsers/iw_scan.md
pydocmd simple jc.parsers.jobs+ > ../docs/parsers/jobs.md
pydocmd simple jc.parsers.kv+ > ../docs/parsers/kv.md
pydocmd simple jc.parsers.last+ > ../docs/parsers/last.md
pydocmd simple jc.parsers.ls+ > ../docs/parsers/ls.md
pydocmd simple jc.parsers.lsblk+ > ../docs/parsers/lsblk.md
pydocmd simple jc.parsers.lsmod+ > ../docs/parsers/lsmod.md
pydocmd simple jc.parsers.lsof+ > ../docs/parsers/lsof.md
pydocmd simple jc.parsers.mount+ > ../docs/parsers/mount.md
pydocmd simple jc.parsers.netstat+ > ../docs/parsers/netstat.md
pydocmd simple jc.parsers.ntpq+ > ../docs/parsers/ntpq.md
pydocmd simple jc.parsers.passwd+ > ../docs/parsers/passwd.md
pydocmd simple jc.parsers.ping+ > ../docs/parsers/ping.md
pydocmd simple jc.parsers.pip_list+ > ../docs/parsers/pip_list.md
pydocmd simple jc.parsers.pip_show+ > ../docs/parsers/pip_show.md
pydocmd simple jc.parsers.ps+ > ../docs/parsers/ps.md
pydocmd simple jc.parsers.route+ > ../docs/parsers/route.md
pydocmd simple jc.parsers.shadow+ > ../docs/parsers/shadow.md
pydocmd simple jc.parsers.ss+ > ../docs/parsers/ss.md
pydocmd simple jc.parsers.stat+ > ../docs/parsers/stat.md
pydocmd simple jc.parsers.sysctl+ > ../docs/parsers/sysctl.md
pydocmd simple jc.parsers.systemctl+ > ../docs/parsers/systemctl.md
pydocmd simple jc.parsers.systemctl_lj+ > ../docs/parsers/systemctl_lj.md
pydocmd simple jc.parsers.systemctl_ls+ > ../docs/parsers/systemctl_ls.md
pydocmd simple jc.parsers.systemctl_luf+ > ../docs/parsers/systemctl_luf.md
pydocmd simple jc.parsers.timedatectl+ > ../docs/parsers/timedatectl.md
pydocmd simple jc.parsers.tracepath+ > ../docs/parsers/tracepath.md
pydocmd simple jc.parsers.traceroute+ > ../docs/parsers/traceroute.md
pydocmd simple jc.parsers.uname+ > ../docs/parsers/uname.md
pydocmd simple jc.parsers.uptime+ > ../docs/parsers/uptime.md
pydocmd simple jc.parsers.w+ > ../docs/parsers/w.md
pydocmd simple jc.parsers.wc+ > ../docs/parsers/wc.md
pydocmd simple jc.parsers.who+ > ../docs/parsers/who.md
pydocmd simple jc.parsers.xml+ > ../docs/parsers/xml.md
pydocmd simple jc.parsers.yaml+ > ../docs/parsers/yaml.md
# a bit of inception here... jc is being used to help
# automate the generation of its own documentation. :)
parsers=$(jc -a | jq -r .parsers[].name)
for parser in $parsers
do
echo Building docs for: $parser
pydocmd simple jc.parsers.${parser}+ > ../docs/parsers/${parser}.md
done

268
docs/parsers/acpi.md Normal file
View File

@ -0,0 +1,268 @@
# jc.parsers.acpi
jc - JSON CLI output utility `acpi` command output parser
Usage (cli):
$ acpi -V | jc --acpi
or
$ jc acpi -V
Usage (module):
import jc.parsers.acpi
result = jc.parsers.acpi.parse(acpi_command_output)
Compatibility:
'linux'
Examples:
$ acpi -V | jc --acpi -p
[
{
"type": "Battery",
"id": 0,
"state": "Charging",
"charge_percent": 71,
"until_charged": "00:29:20",
"design_capacity_mah": 2110,
"last_full_capacity": 2271,
"last_full_capacity_percent": 100,
"until_charged_hours": 0,
"until_charged_minutes": 29,
"until_charged_seconds": 20,
"until_charged_total_seconds": 1760
},
{
"type": "Adapter",
"id": 0,
"on-line": true
},
{
"type": "Thermal",
"id": 0,
"mode": "ok",
"temperature": 46.0,
"temperature_unit": "C",
"trip_points": [
{
"id": 0,
"switches_to_mode": "critical",
"temperature": 127.0,
"temperature_unit": "C"
},
{
"id": 1,
"switches_to_mode": "hot",
"temperature": 127.0,
"temperature_unit": "C"
}
]
},
{
"type": "Cooling",
"id": 0,
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": 1,
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": 2,
"messages": [
"x86_pkg_temp no state information available"
]
},
{
"type": "Cooling",
"id": 3,
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": 4,
"messages": [
"intel_powerclamp no state information available"
]
},
{
"type": "Cooling",
"id": 5,
"messages": [
"Processor 0 of 10"
]
}
]
$ acpi -V | jc --acpi -p -r
[
{
"type": "Battery",
"id": "0",
"state": "Charging",
"charge_percent": "71",
"until_charged": "00:29:20",
"design_capacity_mah": "2110",
"last_full_capacity": "2271",
"last_full_capacity_percent": "100"
},
{
"type": "Adapter",
"id": "0",
"on-line": true
},
{
"type": "Thermal",
"id": "0",
"mode": "ok",
"temperature": "46.0",
"temperature_unit": "C",
"trip_points": [
{
"id": "0",
"switches_to_mode": "critical",
"temperature": "127.0",
"temperature_unit": "C"
},
{
"id": "1",
"switches_to_mode": "hot",
"temperature": "127.0",
"temperature_unit": "C"
}
]
},
{
"type": "Cooling",
"id": "0",
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": "1",
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": "2",
"messages": [
"x86_pkg_temp no state information available"
]
},
{
"type": "Cooling",
"id": "3",
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": "4",
"messages": [
"intel_powerclamp no state information available"
]
},
{
"type": "Cooling",
"id": "5",
"messages": [
"Processor 0 of 10"
]
}
]
## info
```python
info()
```
## process
```python
process(proc_data)
```
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured data with the following schema:
[
{
"type": string,
"id": integer,
"state": string,
"charge_percent": integer,
"until_charged": string,
"until_charged_hours": integer,
"until_charged_minuts": integer,
"until_charged_seconds": integer,
"until_charged_total_seconds": integer,
"charge_remaining": string,
"charge_remaining_hours": integer,
"charge_remaining_minutes": integer,
"charge_remaining_seconds": integer,
"charge_remaining_total_seconds": integer,
"design_capacity_mah": integer,
"last_full_capacity": integer,
"last_full_capacity_percent": integer,
"on-line": boolean,
"mode": string,
"temperature": float,
"temperature_unit": string,
"trip_points": [
{
"id": integer,
"switches_to_mode": string,
"temperature": float,
"temperature_unit": string
}
],
"messages": [
string
]
}
]
## parse
```python
parse(data, raw=False, quiet=False)
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.

77
docs/parsers/date.md
View File

@ -2,6 +2,10 @@
# jc.parsers.date
jc - JSON CLI output utility `date` command output parser
The `epoch` calculated timestamp field is naive. (i.e. based on the local time of the system the parser is run on)
The `epoch_utc` calculated timestamp field is timezone-aware and is only available if the timezone field is UTC.
Usage (cli):
$ date | jc --date
@ -23,29 +27,25 @@ Examples:
$ date | jc --date -p
{
"year": 2020,
"month_num": 7,
"day": 31,
"hour": 16,
"minute": 48,
"second": 11,
"period": null,
"month": "Jul",
"weekday": "Fri",
"weekday_num": 6,
"timezone": "PDT"
}
$ date | jc --date -p -r
{
"year": "2020",
"month": "Jul",
"day": "31",
"weekday": "Fri",
"hour": "16",
"minute": "50",
"second": "01",
"timezone": "PDT"
"year": 2021,
"month": "Mar",
"month_num": 3,
"day": 25,
"weekday": "Thu",
"weekday_num": 4,
"hour": 2,
"hour_24": 2,
"minute": 2,
"second": 26,
"period": "AM",
"timezone": "UTC",
"utc_offset": "+0000",
"day_of_year": 84,
"week_of_year": 12,
"iso": "2021-03-25T02:02:26+00:00",
"epoch": 1616662946,
"epoch_utc": 1616637746,
"timezone_aware": true
}
@ -69,19 +69,26 @@ Parameters:
Returns:
Dictionary. Structured data with the following schema:
{
"year": integer,
"month_num": integer,
"day": integer,
"hour": integer,
"minute": integer,
"second": integer,
"period": string,
"month": string,
"weekday": string,
"weekday_num": integer,
"timezone": string
"year": integer,
"month": string,
"month_num": integer,
"day": integer,
"weekday": string,
"weekday_num": integer,
"hour": integer,
"hour_24": integer,
"minute": integer,
"second": integer,
"period": string,
"timezone": string,
"utc_offset": string, # null if timezone field is not UTC
"day_of_year": integer,
"week_of_year": integer,
"iso": string,
"epoch": integer, # naive timestamp
"epoch_utc": integer, # timezone-aware timestamp. Only available if timezone field is UTC
"timezone_aware": boolean # if true, all fields are correctly based on UTC
}

50
docs/parsers/dig.md
View File

@ -2,6 +2,10 @@
# jc.parsers.dig
jc - JSON CLI output utility `dig` command output parser
The `when_epoch` calculated timestamp field is naive (i.e. based on the local time of the system the parser is run on)
The `when_epoch_utc` calculated timestamp field is timezone-aware and is only available if the timezone field is UTC.
Usage (cli):
$ dig example.com | jc --dig
@ -24,7 +28,7 @@ Examples:
$ dig cnn.com www.cnn.com @205.251.194.64 | jc --dig -p
[
{
"id": 34128,
"id": 52172,
"opcode": "QUERY",
"status": "NOERROR",
"flags": [
@ -46,38 +50,40 @@ Examples:
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 60,
"ttl": 27,
"data": "151.101.65.67"
},
{
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 60,
"data": "151.101.193.67"
"ttl": 27,
"data": "151.101.129.67"
},
{
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 60,
"ttl": 27,
"data": "151.101.1.67"
},
{
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 60,
"data": "151.101.129.67"
"ttl": 27,
"data": "151.101.193.67"
}
],
"query_time": 37,
"query_time": 38,
"server": "2600",
"when": "Tue Nov 12 07:14:42 PST 2019",
"rcvd": 100
"when": "Tue Mar 30 20:07:59 PDT 2021",
"rcvd": 100,
"when_epoch": 1617160079,
"when_epoch_utc": null
},
{
"id": 15273,
"id": 36292,
"opcode": "QUERY",
"status": "NOERROR",
"flags": [
@ -133,10 +139,12 @@ Examples:
"data": "ns-576.awsdns-08.net."
}
],
"query_time": 23,
"query_time": 27,
"server": "205.251.194.64#53(205.251.194.64)",
"when": "Tue Nov 12 07:14:42 PST 2019",
"rcvd": 212
"when": "Tue Mar 30 20:07:59 PDT 2021",
"rcvd": 212,
"when_epoch": 1617160079,
"when_epoch_utc": null
}
]
@ -262,7 +270,7 @@ Examples:
$ dig -x 1.1.1.1 | jc --dig -p
[
{
"id": 34898,
"id": 22191,
"opcode": "QUERY",
"status": "NOERROR",
"flags": [
@ -284,14 +292,16 @@ Examples:
"name": "1.1.1.1.in-addr.arpa.",
"class": "IN",
"type": "PTR",
"ttl": 952,
"ttl": 1800,
"data": "one.one.one.one."
}
],
"query_time": 103,
"query_time": 44,
"server": "2600",
"when": "Tue Nov 12 07:15:33 PST 2019",
"rcvd": 78
"when": "Tue Mar 30 20:10:34 PDT 2021",
"rcvd": 78,
"when_epoch": 1617160234,
"when_epoch_utc": null
}
]
@ -400,6 +410,8 @@ Returns:
"query_time": integer, # in msec
"server": string,
"when": string,
"when_epoch": integer, # naive timestamp if when field is parsable, else null
"when_epoch_utc": integer, # timezone aware timestamp availabe for UTC, else null
"rcvd": integer
"size": string
}

161
docs/parsers/dir.md Normal file
View File

@ -0,0 +1,161 @@
# jc.parsers.dir
jc - JSON CLI output utility `dir` command output parser
Options supported:
- `/T timefield`
- `/O sortorder`
- `/C, /-C`
- `/S`
The `epoch` calculated timestamp field is naive (i.e. based on the local time of the system the parser is run on)
Usage (cli):
$ dir | jc --dir
or
$ jc dir
Usage (module):
import jc.parsers.dir
result = jc.parsers.dir.parse(dir_command_output)
Compatibility:
'win32'
Examples:
$ dir | jc --dir -p
[
{
"date": "03/24/2021",
"time": "03:15 PM",
"dir": true,
"size": null,
"filename": ".",
"parent": "C:\Program Files\Internet Explorer",
"epoch": 1616624100
},
{
"date": "03/24/2021",
"time": "03:15 PM",
"dir": true,
"size": null,
"filename": "..",
"parent": "C:\Program Files\Internet Explorer",
"epoch": 1616624100
},
{
"date": "12/07/2019",
"time": "02:49 AM",
"dir": true,
"size": null,
"filename": "en-US",
"parent": "C:\Program Files\Internet Explorer",
"epoch": 1575715740
},
{
"date": "12/07/2019",
"time": "02:09 AM",
"dir": false,
"size": 54784,
"filename": "ExtExport.exe",
"parent": "C:\Program Files\Internet Explorer",
"epoch": 1575713340
},
...
]
$ dir | jc --dir -p -r
[
{
"date": "03/24/2021",
"time": "03:15 PM",
"dir": true,
"size": null,
"filename": ".",
"parent": "C:\Program Files\Internet Explorer"
},
{
"date": "03/24/2021",
"time": "03:15 PM",
"dir": true,
"size": null,
"filename": "..",
"parent": "C:\Program Files\Internet Explorer"
},
{
"date": "12/07/2019",
"time": "02:49 AM",
"dir": true,
"size": null,
"filename": "en-US",
"parent": "C:\Program Files\Internet Explorer"
},
{
"date": "12/07/2019",
"time": "02:09 AM",
"dir": false,
"size": "54,784",
"filename": "ExtExport.exe",
"parent": "C:\Program Files\Internet Explorer"
},
...
]
## info
```python
info()
```
## process
```python
process(proc_data)
```
Final processing to conform to the schema.
Parameters:
proc_data: (Dictionary of Lists) raw structured data to process
Returns:
List of Dictionaries. Structured data with the following schema:
[
{
"date": string,
"time": string,
"epoch": integer, # naive timestamp
"dir": boolean,
"size": integer,
"filename: string,
"parent": string
}
]
## parse
```python
parse(data, raw=False, quiet=False)
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.

171
docs/parsers/dpkg_l.md Normal file
View File

@ -0,0 +1,171 @@
# jc.parsers.dpkg_l
jc - JSON CLI output utility `dpkg -l` command output parser
Set the `COLUMNS` environment variable to a large value to avoid field truncation. For example:
$ COLUMNS=500 dpkg -l | jc --dpkg-l
Usage (cli):
$ dpkg -l | jc --dpkg-l
or
$ jc dpkg -l
Usage (module):
import jc.parsers.dpkg
result = jc.parsers.dpkg.parse(dpkg_command_output)
Compatibility:
'linux'
Examples:
$ dpkg -l | jc --dpkg-l -p
[
{
"codes": "ii",
"name": "accountsservice",
"version": "0.6.45-1ubuntu1.3",
"architecture": "amd64",
"description": "query and manipulate user account information",
"desired": "install",
"status": "installed"
},
{
"codes": "rc",
"name": "acl",
"version": "2.2.52-3build1",
"architecture": "amd64",
"description": "Access control list utilities",
"desired": "remove",
"status": "config-files"
},
{
"codes": "uWR",
"name": "acpi",
"version": "1.7-1.1",
"architecture": "amd64",
"description": "displays information on ACPI devices",
"desired": "unknown",
"status": "trigger await",
"error": "reinstall required"
},
{
"codes": "rh",
"name": "acpid",
"version": "1:2.0.28-1ubuntu1",
"architecture": "amd64",
"description": "Advanced Configuration and Power Interface event daemon",
"desired": "remove",
"status": "half installed"
},
{
"codes": "pn",
"name": "adduser",
"version": "3.116ubuntu1",
"architecture": "all",
"description": "add and remove users and groups",
"desired": "purge",
"status": "not installed"
},
...
]
$ dpkg -l | jc --dpkg-l -p -r
[
{
"codes": "ii",
"name": "accountsservice",
"version": "0.6.45-1ubuntu1.3",
"architecture": "amd64",
"description": "query and manipulate user account information"
},
{
"codes": "rc",
"name": "acl",
"version": "2.2.52-3build1",
"architecture": "amd64",
"description": "Access control list utilities"
},
{
"codes": "uWR",
"name": "acpi",
"version": "1.7-1.1",
"architecture": "amd64",
"description": "displays information on ACPI devices"
},
{
"codes": "rh",
"name": "acpid",
"version": "1:2.0.28-1ubuntu1",
"architecture": "amd64",
"description": "Advanced Configuration and Power Interface event daemon"
},
{
"codes": "pn",
"name": "adduser",
"version": "3.116ubuntu1",
"architecture": "all",
"description": "add and remove users and groups"
},
...
]
## info
```python
info()
```
## process
```python
process(proc_data)
```
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured data with the following schema:
[
{
"codes": string,
"name": string,
"version": string,
"architecture": string,
"description": string,
"desired": string,
"status": string,
"error": string
}
]
## parse
```python
parse(data, raw=False, quiet=False)
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.

131
docs/parsers/finger.md Normal file
View File

@ -0,0 +1,131 @@
# jc.parsers.finger
jc - JSON CLI output utility `finger` command output parser
Supports `-s` output option. Does not support the `-l` detail option.
Usage (cli):
$ finger | jc --finger
or
$ jc finger
Usage (module):
import jc.parsers.finger
result = jc.parsers.finger.parse(finger_command_output)
Compatibility:
'linux', 'darwin', 'cygwin', freebsd'
Examples:
$ finger | jc --finger -p
[
{
"login": "jdoe",
"name": "John Doe",
"tty": "tty1",
"idle": "14d",
"login_time": "Mar 22 21:14",
"tty_writeable": false,
"idle_minutes": 0,
"idle_hours": 0,
"idle_days": 14,
"total_idle_minutes": 20160
},
{
"login": "jdoe",
"name": "John Doe",
"tty": "pts/0",
"idle": null,
"login_time": "Apr 5 15:33",
"details": "(192.168.1.22)",
"tty_writeable": true,
"idle_minutes": 0,
"idle_hours": 0,
"idle_days": 0,
"total_idle_minutes": 0
},
...
]
$ finger | jc --finger -p -r
[
{
"login": "jdoe",
"name": "John Doe",
"tty": "*tty1",
"idle": "14d",
"login_time": "Mar 22 21:14"
},
{
"login": "jdoe",
"name": "John Doe",
"tty": "pts/0",
"idle": null,
"login_time": "Apr 5 15:33",
"details": "(192.168.1.22)"
},
...
]
## info
```python
info()
```
## process
```python
process(proc_data)
```
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured data with the following schema:
[
{
"login": string,
"name": string,
"tty": string,
"idle": string, # null if empty
"login_time": string,
"details": string,
"tty_writeable": boolean,
"idle_minutes": integer,
"idle_hours": integer,
"idle_days": integer,
"total_idle_minutes": integer
}
]
## parse
```python
parse(data, raw=False, quiet=False)
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.

2
docs/parsers/iw_scan.md
View File

@ -2,7 +2,7 @@
# jc.parsers.iw_scan
jc - JSON CLI output utility `iw dev <device> scan` command output parser
This parser is considered beta quality. Not all fields are parsed.
This parser is considered beta quality. Not all fields are parsed and there are not enough samples to test.
Usage (cli):

8
docs/parsers/last.md
View File

@ -2,7 +2,9 @@
# jc.parsers.last
jc - JSON CLI output utility `last` and `lastb` command output parser
Supports -w and -F options.
Supports `-w` and `-F` options.
Calculated epoch time fields are naive (i.e. based on the local time of the system the parser is run on) since there is no timezone information in the `last` command output.
Usage (cli):
@ -116,8 +118,8 @@ Returns:
"login": string,
"logout": string,
"duration": string,
"login_epoch": integer, # available with last -F option
"logout_epoch": integer, # available with last -F option
"login_epoch": integer, # (naive) available with last -F option
"logout_epoch": integer, # (naive) available with last -F option
"duration_seconds": integer # available with last -F option
}
]

26
docs/parsers/ls.md
View File

@ -3,11 +3,15 @@
jc - JSON CLI output utility `ls` and `vdir` command output parser
Options supported:
- `lbaR`
- `lbaR1`
- `--time-style=full-iso`
- `-h`: File sizes will be available in text form with `-r` but larger file sizes with human readable suffixes will be converted to `Null` in the default view since the parser attempts to convert this field to an integer.
Note: The `-l` or `-b` option of `ls` should be used to correctly parse filenames that include newline characters. Since `ls` does not encode newlines in filenames when outputting to a pipe it will cause `jc` to see multiple files instead of a single file if `-l` or `-b` is not used. Alternatively, `vdir` can be used, which is the same as running `ls -lb`.
Note: The `-1`, `-l`, or `-b` option of `ls` should be used to correctly parse filenames that include newline characters. Since `ls` does not encode newlines in filenames when outputting to a pipe it will cause `jc` to see multiple files instead of a single file if `-1`, `-l`, or `-b` is not used. Alternatively, `vdir` can be used, which is the same as running `ls -lb`.
The `epoch` calculated timestamp field is naive (i.e. based on the local time of the system the parser is run on)
The `epoch_utc` calculated timestamp field is timezone-aware and is only available if the timezone field is UTC.
Usage (cli):
@ -174,14 +178,16 @@ Returns:
[
{
"filename": string,
"flags": string,
"links": integer,
"parent": string,
"owner": string,
"group": string,
"size": integer,
"date": string
"filename": string,
"flags": string,
"links": integer,
"parent": string,
"owner": string,
"group": string,
"size": integer,
"date": string,
"epoch": integer, # naive timestamp if date field exists and can be converted
"epoch_utc": integer # timezone aware timestamp if date field is in UTC and can be converted
}
]

191
docs/parsers/rpm_qai.md Normal file
View File

@ -0,0 +1,191 @@
# jc.parsers.rpm_qai
jc - JSON CLI output utility `rpm -qai` command output parser
Works with `rpm -qi [package]` or `rpm -qai`.
The `build_epoch` calculated timestamp field is naive (i.e. based on the local time of the system the parser is run on)
The `build_epoch_utc` calculated timestamp field is timezone-aware and is only available if the timezone field is UTC.
Usage (cli):
$ rpm -qai | jc --rpm_qai
or
$ jc rpm -qai
Usage (module):
import jc.parsers.rpm_qai
result = jc.parsers.rpm_qai.parse(rpm_qai_command_output)
Compatibility:
'linux'
Examples:
$ rpm_qai | jc --rpm_qai -p
[
{
"name": "make",
"epoch": 1,
"version": "3.82",
"release": "24.el7",
"architecture": "x86_64",
"install_date": "Wed 16 Oct 2019 09:21:42 AM PDT",
"group": "Development/Tools",
"size": 1160660,
"license": "GPLv2+",
"signature": "RSA/SHA256, Thu 22 Aug 2019 02:34:59 PM PDT, Key ID 24c6a8a7f4a80eb5",
"source_rpm": "make-3.82-24.el7.src.rpm",
"build_date": "Thu 08 Aug 2019 05:47:25 PM PDT",
"build_host": "x86-01.bsys.centos.org",
"relocations": "(not relocatable)",
"packager": "CentOS BuildSystem <http://bugs.centos.org>",
"vendor": "CentOS",
"url": "http://www.gnu.org/software/make/",
"summary": "A GNU tool which simplifies the build process for users",
"description": "A GNU tool for controlling the generation of executables and other non-source...",
"build_epoch": 1565311645,
"build_epoch_utc": null
},
{
"name": "kbd-legacy",
"version": "1.15.5",
"release": "15.el7",
"architecture": "noarch",
"install_date": "Thu 15 Aug 2019 10:53:08 AM PDT",
"group": "System Environment/Base",
"size": 503608,
"license": "GPLv2+",
"signature": "RSA/SHA256, Mon 12 Nov 2018 07:17:49 AM PST, Key ID 24c6a8a7f4a80eb5",
"source_rpm": "kbd-1.15.5-15.el7.src.rpm",
"build_date": "Tue 30 Oct 2018 03:40:00 PM PDT",
"build_host": "x86-01.bsys.centos.org",
"relocations": "(not relocatable)",
"packager": "CentOS BuildSystem <http://bugs.centos.org>",
"vendor": "CentOS",
"url": "http://ftp.altlinux.org/pub/people/legion/kbd",
"summary": "Legacy data for kbd package",
"description": "The kbd-legacy package contains original keymaps for kbd package. Please note...",
"build_epoch": 1540939200,
"build_epoch_utc": null
},
...
]
$ rpm -qai | jc --rpm_qai -p -r
[
{
"name": "make",
"epoch": "1",
"version": "3.82",
"release": "24.el7",
"architecture": "x86_64",
"install_date": "Wed 16 Oct 2019 09:21:42 AM PDT",
"group": "Development/Tools",
"size": "1160660",
"license": "GPLv2+",
"signature": "RSA/SHA256, Thu 22 Aug 2019 02:34:59 PM PDT, Key ID 24c6a8a7f4a80eb5",
"source_rpm": "make-3.82-24.el7.src.rpm",
"build_date": "Thu 08 Aug 2019 05:47:25 PM PDT",
"build_host": "x86-01.bsys.centos.org",
"relocations": "(not relocatable)",
"packager": "CentOS BuildSystem <http://bugs.centos.org>",
"vendor": "CentOS",
"url": "http://www.gnu.org/software/make/",
"summary": "A GNU tool which simplifies the build process for users",
"description": "A GNU tool for controlling the generation of executables and other..."
},
{
"name": "kbd-legacy",
"version": "1.15.5",
"release": "15.el7",
"architecture": "noarch",
"install_date": "Thu 15 Aug 2019 10:53:08 AM PDT",
"group": "System Environment/Base",
"size": "503608",
"license": "GPLv2+",
"signature": "RSA/SHA256, Mon 12 Nov 2018 07:17:49 AM PST, Key ID 24c6a8a7f4a80eb5",
"source_rpm": "kbd-1.15.5-15.el7.src.rpm",
"build_date": "Tue 30 Oct 2018 03:40:00 PM PDT",
"build_host": "x86-01.bsys.centos.org",
"relocations": "(not relocatable)",
"packager": "CentOS BuildSystem <http://bugs.centos.org>",
"vendor": "CentOS",
"url": "http://ftp.altlinux.org/pub/people/legion/kbd",
"summary": "Legacy data for kbd package",
"description": "The kbd-legacy package contains original keymaps for kbd package..."
},
...
]
## info
```python
info()
```
## process
```python
process(proc_data)
```
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured data with the following schema:
[
{
"name": string,
"epoch": integer,
"version": string,
"release": string,
"architecture": string,
"install_date": string,
"group": string,
"size": integer,
"license": string,
"signature": string,
"source_rpm": string,
"build_date": string,
"build_epoch": integer, # naive timestamp
"build_epoch_utc": integer, # Aware timestamp if timezone is UTC
"build_host": string,
"relocations": string,
"packager": string,
"vendor": string,
"url": string,
"summary": string,
"description": string
}
]
## parse
```python
parse(data, raw=False, quiet=False)
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.

191
docs/parsers/rpm_qi.md Normal file
View File

@ -0,0 +1,191 @@
# jc.parsers.rpm_qi
jc - JSON CLI output utility `rpm -qi` command output parser
Works with `rpm -qi [package]` or `rpm -qia`.
The `build_epoch` calculated timestamp field is naive (i.e. based on the local time of the system the parser is run on)
The `build_epoch_utc` calculated timestamp field is timezone-aware and is only available if the timezone field is UTC.
Usage (cli):
$ rpm -qia | jc --rpm_qi
or
$ jc rpm -qia
Usage (module):
import jc.parsers.rpm_qi
result = jc.parsers.rpm_qi.parse(rpm_qi_command_output)
Compatibility:
'linux'
Examples:
$ rpm -qia | jc --rpm_qi -p
[
{
"name": "make",
"epoch": 1,
"version": "3.82",
"release": "24.el7",
"architecture": "x86_64",
"install_date": "Wed 16 Oct 2019 09:21:42 AM PDT",
"group": "Development/Tools",
"size": 1160660,
"license": "GPLv2+",
"signature": "RSA/SHA256, Thu 22 Aug 2019 02:34:59 PM PDT, Key ID 24c6a8a7f4a80eb5",
"source_rpm": "make-3.82-24.el7.src.rpm",
"build_date": "Thu 08 Aug 2019 05:47:25 PM PDT",
"build_host": "x86-01.bsys.centos.org",
"relocations": "(not relocatable)",
"packager": "CentOS BuildSystem <http://bugs.centos.org>",
"vendor": "CentOS",
"url": "http://www.gnu.org/software/make/",
"summary": "A GNU tool which simplifies the build process for users",
"description": "A GNU tool for controlling the generation of executables and other non-source...",
"build_epoch": 1565311645,
"build_epoch_utc": null
},
{
"name": "kbd-legacy",
"version": "1.15.5",
"release": "15.el7",
"architecture": "noarch",
"install_date": "Thu 15 Aug 2019 10:53:08 AM PDT",
"group": "System Environment/Base",
"size": 503608,
"license": "GPLv2+",
"signature": "RSA/SHA256, Mon 12 Nov 2018 07:17:49 AM PST, Key ID 24c6a8a7f4a80eb5",
"source_rpm": "kbd-1.15.5-15.el7.src.rpm",
"build_date": "Tue 30 Oct 2018 03:40:00 PM PDT",
"build_host": "x86-01.bsys.centos.org",
"relocations": "(not relocatable)",
"packager": "CentOS BuildSystem <http://bugs.centos.org>",
"vendor": "CentOS",
"url": "http://ftp.altlinux.org/pub/people/legion/kbd",
"summary": "Legacy data for kbd package",
"description": "The kbd-legacy package contains original keymaps for kbd package. Please note...",
"build_epoch": 1540939200,
"build_epoch_utc": null
},
...
]
$ rpm -qia | jc --rpm_qi -p -r
[
{
"name": "make",
"epoch": "1",
"version": "3.82",
"release": "24.el7",
"architecture": "x86_64",
"install_date": "Wed 16 Oct 2019 09:21:42 AM PDT",
"group": "Development/Tools",
"size": "1160660",
"license": "GPLv2+",
"signature": "RSA/SHA256, Thu 22 Aug 2019 02:34:59 PM PDT, Key ID 24c6a8a7f4a80eb5",
"source_rpm": "make-3.82-24.el7.src.rpm",
"build_date": "Thu 08 Aug 2019 05:47:25 PM PDT",
"build_host": "x86-01.bsys.centos.org",
"relocations": "(not relocatable)",
"packager": "CentOS BuildSystem <http://bugs.centos.org>",
"vendor": "CentOS",
"url": "http://www.gnu.org/software/make/",
"summary": "A GNU tool which simplifies the build process for users",
"description": "A GNU tool for controlling the generation of executables and other..."
},
{
"name": "kbd-legacy",
"version": "1.15.5",
"release": "15.el7",
"architecture": "noarch",
"install_date": "Thu 15 Aug 2019 10:53:08 AM PDT",
"group": "System Environment/Base",
"size": "503608",
"license": "GPLv2+",
"signature": "RSA/SHA256, Mon 12 Nov 2018 07:17:49 AM PST, Key ID 24c6a8a7f4a80eb5",
"source_rpm": "kbd-1.15.5-15.el7.src.rpm",
"build_date": "Tue 30 Oct 2018 03:40:00 PM PDT",
"build_host": "x86-01.bsys.centos.org",
"relocations": "(not relocatable)",
"packager": "CentOS BuildSystem <http://bugs.centos.org>",
"vendor": "CentOS",
"url": "http://ftp.altlinux.org/pub/people/legion/kbd",
"summary": "Legacy data for kbd package",
"description": "The kbd-legacy package contains original keymaps for kbd package..."
},
...
]
## info
```python
info()
```
## process
```python
process(proc_data)
```
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured data with the following schema:
[
{
"name": string,
"epoch": integer,
"version": string,
"release": string,
"architecture": string,
"install_date": string,
"group": string,
"size": integer,
"license": string,
"signature": string,
"source_rpm": string,
"build_date": string,
"build_epoch": integer, # naive timestamp
"build_epoch_utc": integer, # Aware timestamp if timezone is UTC
"build_host": string,
"relocations": string,
"packager": string,
"vendor": string,
"url": string,
"summary": string,
"description": string
}
]
## parse
```python
parse(data, raw=False, quiet=False)
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.

80
docs/parsers/stat.md
View File

@ -2,6 +2,10 @@
# jc.parsers.stat
jc - JSON CLI output utility `stat` command output parser
The `xxx_epoch` calculated timestamp fields are naive (i.e. based on the local time of the system the parser is run on)
The `xxx_epoch_utc` calculated timestamp fields are timezone-aware and are only available if the timezone field is UTC.
Usage (cli):
$ stat * | jc --stat
@ -41,7 +45,15 @@ Examples:
"access_time": "2019-11-14 08:18:03.509681766 +0000",
"modify_time": "2019-06-06 22:28:15.000000000 +0000",
"change_time": "2019-08-12 17:21:29.521945390 +0000",
"birth_time": null
"birth_time": null,
"access_time_epoch": 1573748283,
"access_time_epoch_utc": 1573719483,
"modify_time_epoch": 1559885295,
"modify_time_epoch_utc": 1559860095,
"change_time_epoch": 1565655689,
"change_time_epoch_utc": 1565630489,
"birth_time_epoch": null,
"birth_time_epoch_utc": null
},
{
"file": "/bin/btrfs",
@ -61,7 +73,15 @@ Examples:
"access_time": "2019-11-14 08:18:28.990834276 +0000",
"modify_time": "2018-03-12 23:04:27.000000000 +0000",
"change_time": "2019-08-12 17:21:29.545944399 +0000",
"birth_time": null
"birth_time": null,
"access_time_epoch": 1573748308,
"access_time_epoch_utc": 1573719508,
"modify_time_epoch": 1520921067,
"modify_time_epoch_utc": 1520895867,
"change_time_epoch": 1565655689,
"change_time_epoch_utc": 1565630489,
"birth_time_epoch": null,
"birth_time_epoch_utc": null
},
...
]
@ -108,7 +128,7 @@ Examples:
"change_time": "2019-08-12 17:21:29.545944399 +0000",
"birth_time": null
},
..
...
]
@ -135,29 +155,37 @@ Returns:
[
{
"file": string,
"link_to" string,
"size": integer,
"blocks": integer,
"io_blocks": integer,
"type": string,
"device": string,
"inode": integer,
"links": integer,
"access": string,
"flags": string,
"uid": integer,
"user": string,
"gid": integer,
"group": string,
"access_time": string, # - = null
"modify_time": string, # - = null
"change_time": string, # - = null
"birth_time": string, # - = null
"unix_device": integer,
"rdev": integer,
"block_size": integer,
"unix_flags": string
"file": string,
"link_to" string,
"size": integer,
"blocks": integer,
"io_blocks": integer,
"type": string,
"device": string,
"inode": integer,
"links": integer,
"access": string,
"flags": string,
"uid": integer,
"user": string,
"gid": integer,
"group": string,
"access_time": string, # - = null
"access_time_epoch": integer, # naive timestamp
"access_time_epoch_utc": integer, # timezone-aware timestamp
"modify_time": string, # - = null
"modify_time_epoch": integer, # naive timestamp
"modify_time_epoch_utc": integer, # timezone-aware timestamp
"change_time": string, # - = null
"change_time_epoch": integer, # naive timestamp
"change_time_epoch_utc": integer, # timezone-aware timestamp
"birth_time": string, # - = null
"birth_time_epoch": integer, # naive timestamp
"birth_time_epoch_utc": integer, # timezone-aware timestamp
"unix_device": integer,
"rdev": integer,
"block_size": integer,
"unix_flags": string
}
]

164
docs/parsers/time.md Normal file
View File

@ -0,0 +1,164 @@
# jc.parsers.time
jc - JSON CLI output utility `/usr/bin/time` command output parser
Output from `/usr/bin/time` is sent to `STDERR`, so the `-o` option can be used to redirect the output to a file that can be read by `jc`.
Alternatively, the output from `/usr/bin/time` can be redirected to `STDOUT` so `jc` can receive it.
Note: `/usr/bin/time` is similar but different from the Bash builtin `time` command.
Usage (cli):
$ /usr/bin/time -o timefile.out sleep 2.5; cat timefile.out | jc --time -p
Usage (module):
import jc.parsers.time
result = jc.parsers.time.parse(time_command_output)
Compatibility:
'linux', 'darwin', 'cygwin', 'aix', 'freebsd'
Examples:
$ /usr/bin/time --verbose -o timefile.out sleep 2.5; cat timefile.out | jc --time -p
{
"command_being_timed": "sleep 2.5",
"user_time": 0.0,
"system_time": 0.0,
"cpu_percent": 0,
"elapsed_time": "0:02.50",
"average_shared_text_size": 0,
"average_unshared_data_size": 0,
"average_stack_size": 0,
"average_total_size": 0,
"maximum_resident_set_size": 2084,
"average_resident_set_size": 0,
"major_pagefaults": 0,
"minor_pagefaults": 72,
"voluntary_context_switches": 2,
"involuntary_context_switches": 1,
"swaps": 0,
"block_input_operations": 0,
"block_output_operations": 0,
"messages_sent": 0,
"messages_received": 0,
"signals_delivered": 0,
"page_size": 4096,
"exit_status": 0,
"elapsed_time_hours": 0,
"elapsed_time_minutes": 0,
"elapsed_time_seconds": 2,
"elapsed_time_centiseconds": 50,
"elapsed_time_total_seconds": 2.5
}
$ /usr/bin/time --verbose -o timefile.out sleep 2.5; cat timefile.out | jc --time -p -r
{
"command_being_timed": ""sleep 2.5"",
"user_time": "0.00",
"system_time": "0.00",
"cpu_percent": "0",
"elapsed_time": "0:02.50",
"average_shared_text_size": "0",
"average_unshared_data_size": "0",
"average_stack_size": "0",
"average_total_size": "0",
"maximum_resident_set_size": "2084",
"average_resident_set_size": "0",
"major_pagefaults": "0",
"minor_pagefaults": "72",
"voluntary_context_switches": "2",
"involuntary_context_switches": "0",
"swaps": "0",
"block_input_operations": "0",
"block_output_operations": "0",
"messages_sent": "0",
"messages_received": "0",
"signals_delivered": "0",
"page_size": "4096",
"exit_status": "0"
}
## info
```python
info()
```
## process
```python
process(proc_data)
```
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
Dictionary. Structured data with the following schema:
Source: https://www.freebsd.org/cgi/man.cgi?query=getrusage
https://man7.org/linux/man-pages/man1/time.1.html
{
"real_time": float,
"user_time": float,
"system_time": float,
"elapsed_time": string,
"elapsed_time_hours": integer,
"elapsed_time_minutes": integer,
"elapsed_time_seconds": integer,
"elapsed_time_centiseconds": integer,
"elapsed_time_total_seconds": float,
"cpu_percent": integer, # null if ?
"average_shared_text_size": integer,
"average_unshared_data_size": integer,
"average_unshared_stack_size": integer,
"average_shared_memory_size": integer,
"maximum_resident_set_size": integer,
"block_input_operations": integer, # aka File system inputs
"block_output_operations": integer, # aka File system outputs
"major_pagefaults": integer,
"minor_pagefaults": integer,
"swaps": integer,
"page_reclaims": integer,
"page_faults": integer,
"messages_sent": integer,
"messages_received": integer,
"signals_received": integer,
"voluntary_context_switches": integer,
"involuntary_context_switches": integer
"command_being_timed": string,
"average_stack_size": integer,
"average_total_size": integer,
"average_resident_set_size": integer,
"signals_delivered": integer,
"page_size": integer,
"exit_status": integer
}
## parse
```python
parse(data, raw=False, quiet=False)
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.

6
docs/parsers/timedatectl.md
View File

@ -2,6 +2,8 @@
# jc.parsers.timedatectl
jc - JSON CLI output utility `timedatectl` command output parser
The `epoch_utc` calculated timestamp field is timezone-aware and is only available if the universal_time field is available.
Usage (cli):
$ timedatectl | jc --timedatectl
@ -30,7 +32,8 @@ Examples:
"ntp_enabled": true,
"ntp_synchronized": true,
"rtc_in_local_tz": false,
"dst_active": true
"dst_active": true,
"epoch_utc": 1583888001
}
$ timedatectl | jc --timedatectl -p -r
@ -70,6 +73,7 @@ Returns:
{
"local_time": string,
"universal_time": string,
"epoch_utc": integer, # timezone-aware timestamp
"rtc_time": string,
"time_zone": string,
"ntp_enabled": boolean,

235
docs/parsers/upower.md Normal file
View File

@ -0,0 +1,235 @@
# jc.parsers.upower
jc - JSON CLI output utility `upower` command output parser
The `updated_epoch` calculated timestamp field is naive (i.e. based on the local time of the system the parser is run on)
The `updated_epoch_utc` calculated timestamp field is timezone-aware and is only available if the timezone field is UTC.
Usage (cli):
$ upower -d | jc --upower
or
$ jc upower -d
Usage (module):
import jc.parsers.upower
result = jc.parsers.upower.parse(upower_command_output)
Compatibility:
'linux'
Examples:
$ upower -i /org/freedesktop/UPower/devices/battery | jc --upower -p
[
{
"native_path": "/sys/devices/LNXSYSTM:00/device:00/PNP0C0A:00/power_supply/BAT0",
"vendor": "NOTEBOOK",
"model": "BAT",
"serial": "0001",
"power_supply": true,
"updated": "Thu 11 Mar 2021 06:28:08 PM UTC",
"has_history": true,
"has_statistics": true,
"detail": {
"type": "battery",
"present": true,
"rechargeable": true,
"state": "charging",
"energy": 22.3998,
"energy_empty": 0.0,
"energy_full": 52.6473,
"energy_full_design": 62.16,
"energy_rate": 31.6905,
"voltage": 12.191,
"time_to_full": 57.3,
"percentage": 42.5469,
"capacity": 84.6964,
"technology": "lithium-ion",
"energy_unit": "Wh",
"energy_empty_unit": "Wh",
"energy_full_unit": "Wh",
"energy_full_design_unit": "Wh",
"energy_rate_unit": "W",
"voltage_unit": "V",
"time_to_full_unit": "minutes"
},
"history_charge": [
{
"time": 1328809335,
"percent_charged": 42.547,
"status": "charging"
},
{
"time": 1328809305,
"percent_charged": 42.02,
"status": "charging"
}
],
"history_rate": [
{
"time": 1328809335,
"percent_charged": 31.691,
"status": "charging"
}
],
"updated_seconds_ago": 441975,
"updated_epoch": 1615516088,
"updated_epoch_utc": 1615487288
}
]
$ upower -i /org/freedesktop/UPower/devices/battery | jc --upower -p -r
[
{
"native_path": "/sys/devices/LNXSYSTM:00/device:00/PNP0C0A:00/power_supply/BAT0",
"vendor": "NOTEBOOK",
"model": "BAT",
"serial": "0001",
"power_supply": "yes",
"updated": "Thu 11 Mar 2021 06:28:08 PM UTC (441975 seconds ago)",
"has_history": "yes",
"has_statistics": "yes",
"detail": {
"type": "battery",
"present": "yes",
"rechargeable": "yes",
"state": "charging",
"energy": "22.3998 Wh",
"energy_empty": "0 Wh",
"energy_full": "52.6473 Wh",
"energy_full_design": "62.16 Wh",
"energy_rate": "31.6905 W",
"voltage": "12.191 V",
"time_to_full": "57.3 minutes",
"percentage": "42.5469%",
"capacity": "84.6964%",
"technology": "lithium-ion"
},
"history_charge": [
{
"time": "1328809335",
"percent_charged": "42.547",
"status": "charging"
},
{
"time": "1328809305",
"percent_charged": "42.020",
"status": "charging"
}
],
"history_rate": [
{
"time": "1328809335",
"percent_charged": "31.691",
"status": "charging"
}
]
}
]
## info
```python
info()
```
## process
```python
process(proc_data)
```
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured data with the following schema:
[
{
"type": string,
"device_name": string,
"native_path": string,
"power_supply": boolean,
"updated": string,
"updated_epoch": integer, # null if date-time conversion fails
"updated_epoch_utc": integer, # null if date-time conversion fails
"updated_seconds_ago": integer,
"has_history": boolean,
"has_statistics": boolean,
"detail": {
"type": string,
"warning_level": string, # null if none
"online": boolean,
"icon_name": string
"present": boolean,
"rechargeable": boolean,
"state": string,
"energy": float,
"energy_unit": string,
"energy_empty": float,
"energy_empty_unit": string,
"energy_full": float,
"energy_full_unit": string,
"energy_full_design": float,
"energy_full_design_unit": string,
"energy_rate": float,
"energy_rate_unit": string,
"voltage": float,
"voltage_unit": string,
"time_to_full": float,
"time_to_full_unit": string,
"percentage": float,
"capacity": float,
"technology": string
},
"history_charge": [
{
"time": integer,
"percent_charged": float,
"status": string
}
],
"history_rate":[
{
"time": integer,
"percent_charged": float,
"status": string
}
],
"daemon_version": string,
"on_battery": boolean,
"lid_is_closed": boolean,
"lid_is_present": boolean,
"critical_action": string
}
]
## parse
```python
parse(data, raw=False, quiet=False)
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.

50
docs/parsers/uptime.md
View File

@ -23,22 +23,29 @@ Example:
$ uptime | jc --uptime -p
{
"time": "11:30:44",
"uptime": "1 day, 21:17",
"users": 1,
"load_1m": 0.01,
"load_5m": 0.04,
"load_15m": 0.05
"time": "11:35",
"uptime": "3 days, 4:03",
"users": 5,
"load_1m": 1.88,
"load_5m": 2.0,
"load_15m": 1.94,
"time_hour": 11,
"time_minute": 35,
"time_second": null,
"uptime_days": 3,
"uptime_hours": 4,
"uptime_minutes": 3,
"uptime_total_seconds": 273780
}
$ uptime | jc --uptime -p -r
{
"time": "11:31:09",
"uptime": "1 day, 21:17",
"users": "1",
"load_1m": "0.00",
"load_5m": "0.04",
"load_15m": "0.05"
"time": "11:36",
"uptime": "3 days, 4:04",
"users": "5",
"load_1m": "1.88",
"load_5m": "1.99",
"load_15m": "1.94"
}
@ -64,12 +71,19 @@ Returns:
Dictionary. Structured data with the following schema:
{
"time": string,
"uptime": string,
"users": integer,
"load_1m": float,
"load_5m": float,
"load_15m": float
"time": string,
"time_hour": integer,
"time_minute": integer,
"time_second": integer, # null if not displayed
"uptime": string,
"uptime_days": integer,
"uptime_hours": integer,
"uptime_minutes": integer,
"uptime_total_seconds": integer,
"users": integer,
"load_1m": float,
"load_5m": float,
"load_15m": float
}

18
docs/parsers/who.md
View File

@ -4,6 +4,8 @@ jc - JSON CLI output utility `who` command output parser
Accepts any of the following who options (or no options): `-aTH`
The `epoch` calculated timestamp field is naive (i.e. based on the local time of the system the parser is run on)
Usage (cli):
$ who | jc --who
@ -28,7 +30,8 @@ Examples:
{
"event": "reboot",
"time": "Feb 7 23:31",
"pid": 1
"pid": 1,
"epoch": null
},
{
"user": "joeuser",
@ -36,7 +39,8 @@ Examples:
"tty": "console",
"time": "Feb 7 23:32",
"idle": "old",
"pid": 105
"pid": 105,
"epoch": null
},
{
"user": "joeuser",
@ -45,7 +49,8 @@ Examples:
"time": "Feb 13 16:44",
"idle": ".",
"pid": 51217,
"comment": "term=0 exit=0"
"comment": "term=0 exit=0",
"epoch": null
},
{
"user": "joeuser",
@ -53,7 +58,8 @@ Examples:
"tty": "ttys003",
"time": "Feb 28 08:59",
"idle": "01:36",
"pid": 41402
"pid": 41402,
"epoch": null
},
{
"user": "joeuser",
@ -62,7 +68,8 @@ Examples:
"time": "Mar 1 16:35",
"idle": ".",
"pid": 15679,
"from": "192.168.1.5"
"from": "192.168.1.5",
"epoch": null
}
]
@ -138,6 +145,7 @@ Returns:
"writeable_tty": string,
"tty": string,
"time": string,
"epoch": integer, # naive timestamp. null if time cannot be converted
"idle": string,
"pid": integer,
"from": string,

25
docs/utils.md
View File

@ -15,7 +15,7 @@ Parameters:
Returns:
no return, just prints output to STDERR
None - just prints output to STDERR
## error_message
@ -31,7 +31,7 @@ Parameters:
Returns:
no return, just prints output to STDERR
None - just prints output to STDERR
## compatibility
@ -50,7 +50,7 @@ Parameters:
Returns:
no return, just prints output to STDERR
None - just prints output to STDERR
## has_data
@ -68,3 +68,22 @@ Returns:
Boolean True if input string (data) contains non-whitespace characters, otherwise False
## timestamp
```python
timestamp(datetime_string)
```
Input a date-time text string of several formats and convert to a naive or timezone-aware epoch timestamp in UTC
Parameters:
datetime_string: (str) a string representation of a date-time in several supported formats
Attributes:
string (str) the input datetime string
format (int) the format rule that was used to decode the datetime string
naive (int) timestamp based on locally configured timezone. None if conversion fails
utc (int) aware timestamp only if UTC timezone detected in datetime string. None if conversion fails

3
jc/__init__.py
View File

@ -44,7 +44,7 @@ CLI Example:
Module Example:
>>> import jc.parsers.ls
>>>
>>>
>>> data='''-rwxr-xr-x 1 root wheel 23648 May 3 22:26 cat
... -rwxr-xr-x 1 root wheel 30016 May 3 22:26 chmod
... -rwxr-xr-x 1 root wheel 29024 May 3 22:26 cp
@ -69,3 +69,4 @@ Module Example:
"""
name = 'jc'
__version__ = '1.15.0'

171
jc/cli.py
View File

@ -11,25 +11,35 @@ import importlib
import textwrap
import signal
import json
import pygments
from pygments import highlight
from pygments.style import Style
from pygments.token import (Name, Number, String, Keyword)
from pygments.lexers import JsonLexer
from pygments.formatters import Terminal256Formatter
import jc
import jc.appdirs as appdirs
# make pygments import optional
try:
import pygments
from pygments import highlight
from pygments.style import Style
from pygments.token import (Name, Number, String, Keyword)
from pygments.lexers import JsonLexer
from pygments.formatters import Terminal256Formatter
pygments_installed = True
except Exception:
pygments_installed = False
class info():
version = '1.14.4'
version = jc.__version__
description = 'JSON CLI output utility'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
website = 'https://github.com/kellyjonbrazil/jc'
copyright = '© 2019-2021 Kelly Brazil'
license = 'MIT License'
__version__ = info.version
parsers = [
'acpi',
'airport',
'airport-s',
'arp',
@ -41,10 +51,13 @@ parsers = [
'date',
'df',
'dig',
'dir',
'dmidecode',
'dpkg-l',
'du',
'env',
'file',
'finger',
'free',
'fstab',
'group',
@ -75,6 +88,7 @@ parsers = [
'pip-show',
'ps',
'route',
'rpm_qi',
'shadow',
'ss',
'stat',
@ -83,10 +97,12 @@ parsers = [
'systemctl-lj',
'systemctl-ls',
'systemctl-luf',
'time',
'timedatectl',
'tracepath',
'traceroute',
'uname',
'upower',
'uptime',
'w',
'wc',
@ -112,44 +128,45 @@ if os.path.isdir(local_parsers_dir):
# We only support 2.3.0+, pygments changed color names in 2.4.0.
# startswith is sufficient and avoids potential exceptions from split and int.
if pygments.__version__.startswith('2.3.'):
PYGMENT_COLOR = {
'black': '#ansiblack',
'red': '#ansidarkred',
'green': '#ansidarkgreen',
'yellow': '#ansibrown',
'blue': '#ansidarkblue',
'magenta': '#ansipurple',
'cyan': '#ansiteal',
'gray': '#ansilightgray',
'brightblack': '#ansidarkgray',
'brightred': '#ansired',
'brightgreen': '#ansigreen',
'brightyellow': '#ansiyellow',
'brightblue': '#ansiblue',
'brightmagenta': '#ansifuchsia',
'brightcyan': '#ansiturquoise',
'white': '#ansiwhite',
}
else:
PYGMENT_COLOR = {
'black': 'ansiblack',
'red': 'ansired',
'green': 'ansigreen',
'yellow': 'ansiyellow',
'blue': 'ansiblue',
'magenta': 'ansimagenta',
'cyan': 'ansicyan',
'gray': 'ansigray',
'brightblack': 'ansibrightblack',
'brightred': 'ansibrightred',
'brightgreen': 'ansibrightgreen',
'brightyellow': 'ansibrightyellow',
'brightblue': 'ansibrightblue',
'brightmagenta': 'ansibrightmagenta',
'brightcyan': 'ansibrightcyan',
'white': 'ansiwhite',
}
if pygments_installed:
if pygments.__version__.startswith('2.3.'):
PYGMENT_COLOR = {
'black': '#ansiblack',
'red': '#ansidarkred',
'green': '#ansidarkgreen',
'yellow': '#ansibrown',
'blue': '#ansidarkblue',
'magenta': '#ansipurple',
'cyan': '#ansiteal',
'gray': '#ansilightgray',
'brightblack': '#ansidarkgray',
'brightred': '#ansired',
'brightgreen': '#ansigreen',
'brightyellow': '#ansiyellow',
'brightblue': '#ansiblue',
'brightmagenta': '#ansifuchsia',
'brightcyan': '#ansiturquoise',
'white': '#ansiwhite',
}
else:
PYGMENT_COLOR = {
'black': 'ansiblack',
'red': 'ansired',
'green': 'ansigreen',
'yellow': 'ansiyellow',
'blue': 'ansiblue',
'magenta': 'ansimagenta',
'cyan': 'ansicyan',
'gray': 'ansigray',
'brightblack': 'ansibrightblack',
'brightred': 'ansibrightred',
'brightgreen': 'ansibrightgreen',
'brightyellow': 'ansibrightyellow',
'brightblue': 'ansibrightblue',
'brightmagenta': 'ansibrightmagenta',
'brightcyan': 'ansibrightcyan',
'white': 'ansiwhite',
}
def set_env_colors(env_colors=None):
@ -187,7 +204,7 @@ def set_env_colors(env_colors=None):
# if there is an issue with the env variable, just set all colors to default and move on
if input_error:
print('jc: Warning: could not parse JC_COLORS environment variable\n', file=sys.stderr)
jc.utils.warning_message('could not parse JC_COLORS environment variable')
color_list = ['default', 'default', 'default', 'default']
# Try the color set in the JC_COLORS env variable first. If it is set to default, then fall back to default colors
@ -277,17 +294,20 @@ def about_jc():
'description': info.description,
'author': info.author,
'author_email': info.author_email,
'website': info.website,
'copyright': info.copyright,
'license': info.license,
'parser_count': len(parser_list),
'parsers': parser_list
}
def helptext(message):
def helptext():
"""Return the help text with the list of parsers"""
parsers_string = parsers_text(indent=12, pad=17)
helptext_string = f'''
jc: {message}
helptext_string = f'''\
jc converts the output of many commands and file-types to JSON
Usage: COMMAND | jc PARSER [OPTIONS]
@ -300,10 +320,12 @@ def helptext(message):
Options:
-a about jc
-d debug - show traceback (-dd for verbose traceback)
-h help
-m monochrome output
-p pretty print output
-q quiet - suppress parser warnings
-r raw JSON output
-v version info
Example:
ls -al | jc --ls -p
@ -315,6 +337,15 @@ def helptext(message):
return textwrap.dedent(helptext_string)
def versiontext():
"""Return the version text"""
versiontext_string = f'''\
jc version {info.version}
{info.website}
{info.copyright}'''
return textwrap.dedent(versiontext_string)
def json_out(data, pretty=False, env_colors=None, mono=False, piped_out=False):
"""Return a JSON formatted string. String may include color codes or be pretty printed."""
if not mono and not piped_out:
@ -323,14 +354,16 @@ def json_out(data, pretty=False, env_colors=None, mono=False, piped_out=False):
styles = set_env_colors(env_colors)
if pretty:
return str(highlight(json.dumps(data, indent=2), JsonLexer(), Terminal256Formatter(style=JcStyle))[0:-1])
return str(highlight(json.dumps(data, indent=2, ensure_ascii=False),
JsonLexer(), Terminal256Formatter(style=JcStyle))[0:-1])
else:
return str(highlight(json.dumps(data), JsonLexer(), Terminal256Formatter(style=JcStyle))[0:-1])
return str(highlight(json.dumps(data, separators=(',', ':'), ensure_ascii=False),
JsonLexer(), Terminal256Formatter(style=JcStyle))[0:-1])
else:
if pretty:
return json.dumps(data, indent=2)
return json.dumps(data, indent=2, ensure_ascii=False)
else:
return json.dumps(data)
return json.dumps(data, separators=(',', ':'), ensure_ascii=False)
def generate_magic_command(args):
@ -398,11 +431,13 @@ def magic():
elif run_command is None:
return
else:
print(helptext(f'parser not found for "{run_command}"'), file=sys.stderr)
jc.utils.error_message(f'parser not found for "{run_command}". Use "jc -h" for help.')
sys.exit(1)
def main():
import jc.utils
# break on ctrl-c keyboard interrupt
signal.signal(signal.SIGINT, ctrlc)
@ -424,23 +459,37 @@ def main():
if opt.startswith('-') and not opt.startswith('--'):
options.extend(opt[1:])
about = 'a' in options
debug = 'd' in options
verbose_debug = True if options.count('d') > 1 else False
mono = 'm' in options
help_me = 'h' in options
pretty = 'p' in options
quiet = 'q' in options
raw = 'r' in options
version_info = 'v' in options
if not pygments_installed:
mono = True
if about:
print(json_out(about_jc(), pretty=pretty, env_colors=jc_colors, mono=mono, piped_out=piped_output()))
sys.exit(0)
if help_me:
print(helptext())
sys.exit(0)
if version_info:
print(versiontext())
sys.exit(0)
if verbose_debug:
import jc.tracebackplus
jc.tracebackplus.enable(context=11)
if 'a' in options:
print(json_out(about_jc(), pretty=pretty, env_colors=jc_colors, mono=mono, piped_out=piped_output()))
sys.exit(0)
if sys.stdin.isatty():
print(helptext('missing piped data'), file=sys.stderr)
jc.utils.error_message('Missing piped data. Use "jc -h" for help.')
sys.exit(1)
data = sys.stdin.read()
@ -465,11 +514,11 @@ def main():
import jc.utils
jc.utils.error_message(
f'{parser_name} parser could not parse the input data. Did you use the correct parser?\n'
' For details use the -d or -dd option.')
' For details use the -d or -dd option. Use "jc -h" for help.')
sys.exit(1)
if not found:
print(helptext('missing or incorrect arguments'), file=sys.stderr)
jc.utils.error_message('Missing or incorrect arguments. Use "jc -h" for help.')
sys.exit(1)
print(json_out(result, pretty=pretty, env_colors=jc_colors, mono=mono, piped_out=piped_output()))

409
jc/parsers/acpi.py Normal file
View File

@ -0,0 +1,409 @@
"""jc - JSON CLI output utility `acpi` command output parser
Usage (cli):
$ acpi -V | jc --acpi
or
$ jc acpi -V
Usage (module):
import jc.parsers.acpi
result = jc.parsers.acpi.parse(acpi_command_output)
Compatibility:
'linux'
Examples:
$ acpi -V | jc --acpi -p
[
{
"type": "Battery",
"id": 0,
"state": "Charging",
"charge_percent": 71,
"until_charged": "00:29:20",
"design_capacity_mah": 2110,
"last_full_capacity": 2271,
"last_full_capacity_percent": 100,
"until_charged_hours": 0,
"until_charged_minutes": 29,
"until_charged_seconds": 20,
"until_charged_total_seconds": 1760
},
{
"type": "Adapter",
"id": 0,
"on-line": true
},
{
"type": "Thermal",
"id": 0,
"mode": "ok",
"temperature": 46.0,
"temperature_unit": "C",
"trip_points": [
{
"id": 0,
"switches_to_mode": "critical",
"temperature": 127.0,
"temperature_unit": "C"
},
{
"id": 1,
"switches_to_mode": "hot",
"temperature": 127.0,
"temperature_unit": "C"
}
]
},
{
"type": "Cooling",
"id": 0,
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": 1,
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": 2,
"messages": [
"x86_pkg_temp no state information available"
]
},
{
"type": "Cooling",
"id": 3,
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": 4,
"messages": [
"intel_powerclamp no state information available"
]
},
{
"type": "Cooling",
"id": 5,
"messages": [
"Processor 0 of 10"
]
}
]
$ acpi -V | jc --acpi -p -r
[
{
"type": "Battery",
"id": "0",
"state": "Charging",
"charge_percent": "71",
"until_charged": "00:29:20",
"design_capacity_mah": "2110",
"last_full_capacity": "2271",
"last_full_capacity_percent": "100"
},
{
"type": "Adapter",
"id": "0",
"on-line": true
},
{
"type": "Thermal",
"id": "0",
"mode": "ok",
"temperature": "46.0",
"temperature_unit": "C",
"trip_points": [
{
"id": "0",
"switches_to_mode": "critical",
"temperature": "127.0",
"temperature_unit": "C"
},
{
"id": "1",
"switches_to_mode": "hot",
"temperature": "127.0",
"temperature_unit": "C"
}
]
},
{
"type": "Cooling",
"id": "0",
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": "1",
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": "2",
"messages": [
"x86_pkg_temp no state information available"
]
},
{
"type": "Cooling",
"id": "3",
"messages": [
"Processor 0 of 10"
]
},
{
"type": "Cooling",
"id": "4",
"messages": [
"intel_powerclamp no state information available"
]
},
{
"type": "Cooling",
"id": "5",
"messages": [
"Processor 0 of 10"
]
}
]
"""
import jc.utils
class info():
version = '1.0'
description = '`acpi` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# compatible options: linux, darwin, cygwin, win32, aix, freebsd
compatible = ['linux']
magic_commands = ['acpi']
__version__ = info.version
def process(proc_data):
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured data with the following schema:
[
{
"type": string,
"id": integer,
"state": string,
"charge_percent": integer,
"until_charged": string,
"until_charged_hours": integer,
"until_charged_minuts": integer,
"until_charged_seconds": integer,
"until_charged_total_seconds": integer,
"charge_remaining": string,
"charge_remaining_hours": integer,
"charge_remaining_minutes": integer,
"charge_remaining_seconds": integer,
"charge_remaining_total_seconds": integer,
"design_capacity_mah": integer,
"last_full_capacity": integer,
"last_full_capacity_percent": integer,
"on-line": boolean,
"mode": string,
"temperature": float,
"temperature_unit": string,
"trip_points": [
{
"id": integer,
"switches_to_mode": string,
"temperature": float,
"temperature_unit": string
}
],
"messages": [
string
]
}
]
"""
int_list = ['id', 'charge_percent', 'design_capacity_mah', 'last_full_capacity', 'last_full_capacity_percent']
float_list = ['temperature']
for entry in proc_data:
for key in int_list:
if key in entry:
try:
entry[key] = int(entry[key])
except (ValueError):
entry[key] = None
if 'trip_points' in entry:
for tp in entry['trip_points']:
for key in int_list:
if key in tp:
try:
tp[key] = int(tp[key])
except (ValueError):
tp[key] = None
for entry in proc_data:
for key in float_list:
if key in entry:
try:
entry[key] = float(entry[key])
except (ValueError):
entry[key] = None
if 'trip_points' in entry:
for tp in entry['trip_points']:
for key in float_list:
if key in tp:
try:
tp[key] = float(tp[key])
except (ValueError):
tp[key] = None
for entry in proc_data:
if 'until_charged' in entry:
entry['until_charged_hours'] = int(entry['until_charged'].split(':')[0])
entry['until_charged_minutes'] = int(entry['until_charged'].split(':')[1])
entry['until_charged_seconds'] = int(entry['until_charged'].split(':')[2])
entry['until_charged_total_seconds'] = (entry['until_charged_hours'] * 3600) + \
(entry['until_charged_minutes'] * 60) + entry['until_charged_seconds']
if 'charge_remaining' in entry:
entry['charge_remaining_hours'] = int(entry['charge_remaining'].split(':')[0])
entry['charge_remaining_minutes'] = int(entry['charge_remaining'].split(':')[1])
entry['charge_remaining_seconds'] = int(entry['charge_remaining'].split(':')[2])
entry['charge_remaining_total_seconds'] = (entry['charge_remaining_hours'] * 3600) + \
(entry['charge_remaining_minutes'] * 60) + entry['charge_remaining_seconds']
return proc_data
def parse(data, raw=False, quiet=False):
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
if not quiet:
jc.utils.compatibility(__name__, info.compatible)
raw_output = []
output_line = {}
line_state = ''
last_line_state = ''
obj_type = ''
obj_id = ''
trip_points_list = []
trip_points_dict = {}
messages_list = []
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
obj_type = line.split()[0]
obj_id = line.split()[1][:-1]
line_state = obj_type + obj_id
if line_state != last_line_state:
if output_line:
raw_output.append(output_line)
output_line = {}
trip_points_list = []
messages_list = []
if obj_type == 'Battery':
output_line['type'] = obj_type
output_line['id'] = obj_id
if 'Charging' in line or 'Discharging' in line or 'Full' in line:
output_line['state'] = line.split()[2][:-1]
output_line['charge_percent'] = line.split()[3].rstrip('%,')
if 'rate information unavailable' not in line:
if 'Charging' in line:
output_line['until_charged'] = line.split()[4]
if 'Discharging' in line:
output_line['charge_remaining'] = line.split()[4]
if 'design capacity' in line:
output_line['design_capacity_mah'] = line.split()[4]
output_line['last_full_capacity'] = line.split()[9]
output_line['last_full_capacity_percent'] = line.split()[-1][:-1]
if obj_type == 'Adapter':
output_line['type'] = obj_type
output_line['id'] = obj_id
if 'on-line' in line:
output_line['on-line'] = True
else:
output_line['on-line'] = False
if obj_type == 'Thermal':
output_line['type'] = obj_type
output_line['id'] = obj_id
if 'trip point' not in line:
output_line['mode'] = line.split()[2][:-1]
output_line['temperature'] = line.split()[3]
output_line['temperature_unit'] = line.split()[-1]
else:
trip_points_dict = {
"id": line.split()[4],
"switches_to_mode": line.split()[8],
"temperature": line.split()[11],
"temperature_unit": line.split()[-1]
}
trip_points_list.append(trip_points_dict)
output_line['trip_points'] = trip_points_list
if obj_type == 'Cooling':
output_line['type'] = obj_type
output_line['id'] = obj_id
messages_list.append(line.split(maxsplit=2)[2])
output_line['messages'] = messages_list
last_line_state = line_state
if output_line:
raw_output.append(output_line)
if raw:
return raw_output
else:
return process(raw_output)

2
jc/parsers/airport.py
View File

@ -64,7 +64,7 @@ import jc.utils
class info():
version = '1.1'
description = 'airport -I command parser'
description = '`airport -I` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'

2
jc/parsers/airport_s.py
View File

@ -97,7 +97,7 @@ import jc.parsers.universal
class info():
version = '1.2'
description = 'airport -s command parser'
description = '`airport -s` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'

2
jc/parsers/arp.py
View File

@ -107,7 +107,7 @@ import jc.parsers.universal
class info():
version = '1.6'
description = 'arp command parser'
description = '`arp` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/blkid.py
View File

@ -89,7 +89,7 @@ import jc.utils
class info():
version = '1.2'
description = 'blkid command parser'
description = '`blkid` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'

3
jc/parsers/cksum.py
View File

@ -48,10 +48,9 @@ import jc.utils
class info():
version = '1.0'
description = 'cksum command parser'
description = '`cksum` and `sum` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
details = 'Parses cksum and sum program output'
# compatible options: linux, darwin, cygwin, win32, aix, freebsd
compatible = ['linux', 'darwin', 'cygwin', 'aix', 'freebsd']

2
jc/parsers/crontab.py
View File

@ -144,7 +144,7 @@ import jc.parsers.universal
class info():
version = '1.4'
description = 'crontab command and file parser'
description = '`crontab` command and file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'

2
jc/parsers/crontab_u.py
View File

@ -141,7 +141,7 @@ import jc.parsers.universal
class info():
version = '1.5'
description = 'crontab file parser with user support'
description = '`crontab` file parser with user support'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'

214
jc/parsers/date.py
View File

@ -1,5 +1,9 @@
"""jc - JSON CLI output utility `date` command output parser
The `epoch` calculated timestamp field is naive. (i.e. based on the local time of the system the parser is run on)
The `epoch_utc` calculated timestamp field is timezone-aware and is only available if the timezone field is UTC.
Usage (cli):
$ date | jc --date
@ -21,37 +25,34 @@ Examples:
$ date | jc --date -p
{
"year": 2020,
"month_num": 7,
"day": 31,
"hour": 16,
"minute": 48,
"second": 11,
"period": null,
"month": "Jul",
"weekday": "Fri",
"weekday_num": 6,
"timezone": "PDT"
}
$ date | jc --date -p -r
{
"year": "2020",
"month": "Jul",
"day": "31",
"weekday": "Fri",
"hour": "16",
"minute": "50",
"second": "01",
"timezone": "PDT"
"year": 2021,
"month": "Mar",
"month_num": 3,
"day": 25,
"weekday": "Thu",
"weekday_num": 4,
"hour": 2,
"hour_24": 2,
"minute": 2,
"second": 26,
"period": "AM",
"timezone": "UTC",
"utc_offset": "+0000",
"day_of_year": 84,
"week_of_year": 12,
"iso": "2021-03-25T02:02:26+00:00",
"epoch": 1616662946,
"epoch_utc": 1616637746,
"timezone_aware": true
}
"""
from datetime import datetime, timezone
import jc.utils
class info():
version = '1.1'
description = 'date command parser'
version = '2.0'
description = '`date` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@ -74,62 +75,30 @@ def process(proc_data):
Returns:
Dictionary. Structured data with the following schema:
{
"year": integer,
"month_num": integer,
"day": integer,
"hour": integer,
"minute": integer,
"second": integer,
"period": string,
"month": string,
"weekday": string,
"weekday_num": integer,
"timezone": string
"year": integer,
"month": string,
"month_num": integer,
"day": integer,
"weekday": string,
"weekday_num": integer,
"hour": integer,
"hour_24": integer,
"minute": integer,
"second": integer,
"period": string,
"timezone": string,
"utc_offset": string, # null if timezone field is not UTC
"day_of_year": integer,
"week_of_year": integer,
"iso": string,
"epoch": integer, # naive timestamp
"epoch_utc": integer, # timezone-aware timestamp. Only available if timezone field is UTC
"timezone_aware": boolean # if true, all fields are correctly based on UTC
}
"""
month_map = {
"Jan": 1,
"Feb": 2,
"Mar": 3,
"Apr": 4,
"May": 5,
"Jun": 6,
"Jul": 7,
"Aug": 8,
"Sep": 9,
"Oct": 10,
"Nov": 11,
"Dec": 12
}
weekday_map = {
"Sun": 1,
"Mon": 2,
"Tue": 3,
"Wed": 4,
"Thu": 5,
"Fri": 6,
"Sat": 7
}
if proc_data:
return {
"year": int(proc_data['year']),
'month_num': month_map[proc_data['month']],
"day": int(proc_data['day']),
"hour": int(proc_data['hour']),
"minute": int(proc_data['minute']),
"second": int(proc_data['second']),
"period": proc_data['period'] if 'period' in proc_data else None,
"month": proc_data['month'],
"weekday": proc_data['weekday'],
"weekday_num": weekday_map[proc_data['weekday']],
"timezone": proc_data['timezone']
}
else:
return {}
# no further processing
return proc_data
def parse(data, raw=False, quiet=False):
@ -152,34 +121,69 @@ def parse(data, raw=False, quiet=False):
raw_output = {}
if jc.utils.has_data(data):
data = data.replace(':', ' ')
split_data = data.split()
# date v8.32 uses a different format depending on locale, so need to support LANG=en_US.UTF-8
if len(split_data) == 9 and ('AM' in split_data or 'am' in split_data or 'PM' in split_data or 'pm' in split_data):
raw_output = {
"year": split_data[8],
"month": split_data[1],
"day": split_data[2],
"weekday": split_data[0],
"hour": split_data[3],
"minute": split_data[4],
"second": split_data[5],
"period": split_data[6],
"timezone": split_data[7]
}
else:
# standard LANG=C date output
raw_output = {
"year": split_data[7],
"month": split_data[1],
"day": split_data[2],
"weekday": split_data[0],
"hour": split_data[3],
"minute": split_data[4],
"second": split_data[5],
"timezone": split_data[6]
}
# find the timezone no matter where it is in the string
# from https://www.timeanddate.com/time/zones/
tz_abbr = ['A', 'ACDT', 'ACST', 'ACT', 'ACWST', 'ADT', 'AEDT', 'AEST', 'AET', 'AFT', 'AKDT', 'AKST', 'ALMT',
'AMST', 'AMT', 'ANAST', 'ANAT', 'AQTT', 'ART', 'AST', 'AT', 'AWDT', 'AWST', 'AZOST', 'AZOT',
'AZST', 'AZT', 'AoE', 'B', 'BNT', 'BOT', 'BRST', 'BRT', 'BST', 'BTT', 'C', 'CAST', 'CAT', 'CCT',
'CDT', 'CEST', 'CET', 'CHADT', 'CHAST', 'CHOST', 'CHOT', 'CHUT', 'CIDST', 'CIST', 'CKT', 'CLST',
'CLT', 'COT', 'CST', 'CT', 'CVT', 'CXT', 'ChST', 'D', 'DAVT', 'DDUT', 'E', 'EASST', 'EAST',
'EAT', 'ECT', 'EDT', 'EEST', 'EET', 'EGST', 'EGT', 'EST', 'ET', 'F', 'FET', 'FJST', 'FJT', 'FKST',
'FKT', 'FNT', 'G', 'GALT', 'GAMT', 'GET', 'GFT', 'GILT', 'GMT', 'GST', 'GYT', 'H', 'HDT', 'HKT',
'HOVST', 'HOVT', 'HST', 'I', 'ICT', 'IDT', 'IOT', 'IRDT', 'IRKST', 'IRKT', 'IRST', 'IST', 'JST',
'K', 'KGT', 'KOST', 'KRAST', 'KRAT', 'KST', 'KUYT', 'L', 'LHDT', 'LHST', 'LINT', 'M', 'MAGST',
'MAGT', 'MART', 'MAWT', 'MDT', 'MHT', 'MMT', 'MSD', 'MSK', 'MST', 'MT', 'MUT', 'MVT', 'MYT', 'N',
'NCT', 'NDT', 'NFDT', 'NFT', 'NOVST', 'NOVT', 'NPT', 'NRT', 'NST', 'NUT', 'NZDT', 'NZST', 'O',
'OMSST', 'OMST', 'ORAT', 'P', 'PDT', 'PET', 'PETST', 'PETT', 'PGT', 'PHOT', 'PHT', 'PKT', 'PMDT',
'PMST', 'PONT', 'PST', 'PT', 'PWT', 'PYST', 'PYT', 'Q', 'QYZT', 'R', 'RET', 'ROTT', 'S', 'SAKT',
'SAMT', 'SAST', 'SBT', 'SCT', 'SGT', 'SRET', 'SRT', 'SST', 'SYOT', 'T', 'TAHT', 'TFT', 'TJT', 'TKT',
'TLT', 'TMT', 'TOST', 'TOT', 'TRT', 'TVT', 'U', 'ULAST', 'ULAT', 'UYST', 'UYT', 'UZT', 'V', 'VET',
'VLAST', 'VLAT', 'VOST', 'VUT', 'W', 'WAKT', 'WARST', 'WAST', 'WAT', 'WEST', 'WET', 'WFT', 'WGST',
'WGT', 'WIB', 'WIT', 'WITA', 'WST', 'WT', 'X', 'Y', 'YAKST', 'YAKT', 'YAPT', 'YEKST', 'YEKT', 'Z',
'UTC', 'UTC-1200', 'UTC-1100', 'UTC-1000', 'UTC-0930', 'UTC-0900', 'UTC-0800', 'UTC-0700', 'UTC-0600',
'UTC-0500', 'UTC-0400', 'UTC-0300', 'UTC-0230', 'UTC-0200', 'UTC-0100', 'UTC+0000', 'UTC-0000',
'UTC+0100', 'UTC+0200', 'UTC+0300', 'UTC+0400', 'UTC+0430', 'UTC+0500', 'UTC+0530', 'UTC+0545',
'UTC+0600', 'UTC+0630', 'UTC+0700', 'UTC+0800', 'UTC+0845', 'UTC+0900', 'UTC+1000', 'UTC+1030',
'UTC+1100', 'UTC+1200', 'UTC+1300', 'UTC+1345', 'UTC+1400']
tz = None
for term in data.replace('(', '').replace(')', '').split():
if term in tz_abbr:
tz = term
dt = None
dt_utc = None
timestamp = jc.utils.timestamp(data)
if timestamp.naive:
dt = datetime.fromtimestamp(timestamp.naive)
if timestamp.utc:
dt_utc = datetime.fromtimestamp(timestamp.utc, timezone.utc)
if dt_utc:
dt = dt_utc
raw_output = {
'year': dt.year,
'month': dt.strftime('%b'),
'month_num': dt.month,
'day': dt.day,
'weekday': dt.strftime('%a'),
'weekday_num': dt.isoweekday(),
'hour': int(dt.strftime('%I')),
'hour_24': dt.hour,
'minute': dt.minute,
'second': dt.second,
'period': dt.strftime('%p'),
'timezone': tz,
'utc_offset': dt.strftime('%z') or None,
'day_of_year': int(dt.strftime('%j')),
'week_of_year': int(dt.strftime('%W')),
'iso': dt.isoformat(),
'epoch': timestamp.naive,
'epoch_utc': timestamp.utc,
'timezone_aware': True if timestamp.utc else False
}
if raw:
return raw_output

2
jc/parsers/df.py
View File

@ -83,7 +83,7 @@ import jc.parsers.universal
class info():
version = '1.5'
description = 'df command parser'
description = '`df` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

59
jc/parsers/dig.py
View File

@ -1,5 +1,9 @@
"""jc - JSON CLI output utility `dig` command output parser
The `when_epoch` calculated timestamp field is naive (i.e. based on the local time of the system the parser is run on)
The `when_epoch_utc` calculated timestamp field is timezone-aware and is only available if the timezone field is UTC.
Usage (cli):
$ dig example.com | jc --dig
@ -22,7 +26,7 @@ Examples:
$ dig cnn.com www.cnn.com @205.251.194.64 | jc --dig -p
[
{
"id": 34128,
"id": 52172,
"opcode": "QUERY",
"status": "NOERROR",
"flags": [
@ -44,38 +48,40 @@ Examples:
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 60,
"ttl": 27,
"data": "151.101.65.67"
},
{
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 60,
"data": "151.101.193.67"
"ttl": 27,
"data": "151.101.129.67"
},
{
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 60,
"ttl": 27,
"data": "151.101.1.67"
},
{
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 60,
"data": "151.101.129.67"
"ttl": 27,
"data": "151.101.193.67"
}
],
"query_time": 37,
"query_time": 38,
"server": "2600",
"when": "Tue Nov 12 07:14:42 PST 2019",
"rcvd": 100
"when": "Tue Mar 30 20:07:59 PDT 2021",
"rcvd": 100,
"when_epoch": 1617160079,
"when_epoch_utc": null
},
{
"id": 15273,
"id": 36292,
"opcode": "QUERY",
"status": "NOERROR",
"flags": [
@ -131,10 +137,12 @@ Examples:
"data": "ns-576.awsdns-08.net."
}
],
"query_time": 23,
"query_time": 27,
"server": "205.251.194.64#53(205.251.194.64)",
"when": "Tue Nov 12 07:14:42 PST 2019",
"rcvd": 212
"when": "Tue Mar 30 20:07:59 PDT 2021",
"rcvd": 212,
"when_epoch": 1617160079,
"when_epoch_utc": null
}
]
@ -260,7 +268,7 @@ Examples:
$ dig -x 1.1.1.1 | jc --dig -p
[
{
"id": 34898,
"id": 22191,
"opcode": "QUERY",
"status": "NOERROR",
"flags": [
@ -282,14 +290,16 @@ Examples:
"name": "1.1.1.1.in-addr.arpa.",
"class": "IN",
"type": "PTR",
"ttl": 952,
"ttl": 1800,
"data": "one.one.one.one."
}
],
"query_time": 103,
"query_time": 44,
"server": "2600",
"when": "Tue Nov 12 07:15:33 PST 2019",
"rcvd": 78
"when": "Tue Mar 30 20:10:34 PDT 2021",
"rcvd": 78,
"when_epoch": 1617160234,
"when_epoch_utc": null
}
]
@ -333,8 +343,8 @@ import jc.utils
class info():
version = '1.5'
description = 'dig command parser'
version = '1.6'
description = '`dig` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@ -405,6 +415,8 @@ def process(proc_data):
"query_time": integer, # in msec
"server": string,
"when": string,
"when_epoch": integer, # naive timestamp if when field is parsable, else null
"when_epoch_utc": integer, # timezone aware timestamp availabe for UTC, else null
"rcvd": integer
"size": string
}
@ -452,6 +464,11 @@ def process(proc_data):
except (ValueError):
entry['query_time'] = None
if 'when' in entry:
ts = jc.utils.timestamp(entry['when'])
entry['when_epoch'] = ts.naive
entry['when_epoch_utc'] = ts.utc
return proc_data

219
jc/parsers/dir.py Normal file
View File

@ -0,0 +1,219 @@
"""jc - JSON CLI output utility `dir` command output parser
Options supported:
- `/T timefield`
- `/O sortorder`
- `/C, /-C`
- `/S`
The `epoch` calculated timestamp field is naive (i.e. based on the local time of the system the parser is run on)
Usage (cli):
$ dir | jc --dir
or
$ jc dir
Usage (module):
import jc.parsers.dir
result = jc.parsers.dir.parse(dir_command_output)
Compatibility:
'win32'
Examples:
$ dir | jc --dir -p
[
{
"date": "03/24/2021",
"time": "03:15 PM",
"dir": true,
"size": null,
"filename": ".",
"parent": "C:\\Program Files\\Internet Explorer",
"epoch": 1616624100
},
{
"date": "03/24/2021",
"time": "03:15 PM",
"dir": true,
"size": null,
"filename": "..",
"parent": "C:\\Program Files\\Internet Explorer",
"epoch": 1616624100
},
{
"date": "12/07/2019",
"time": "02:49 AM",
"dir": true,
"size": null,
"filename": "en-US",
"parent": "C:\\Program Files\\Internet Explorer",
"epoch": 1575715740
},
{
"date": "12/07/2019",
"time": "02:09 AM",
"dir": false,
"size": 54784,
"filename": "ExtExport.exe",
"parent": "C:\\Program Files\\Internet Explorer",
"epoch": 1575713340
},
...
]
$ dir | jc --dir -p -r
[
{
"date": "03/24/2021",
"time": "03:15 PM",
"dir": true,
"size": null,
"filename": ".",
"parent": "C:\\Program Files\\Internet Explorer"
},
{
"date": "03/24/2021",
"time": "03:15 PM",
"dir": true,
"size": null,
"filename": "..",
"parent": "C:\\Program Files\\Internet Explorer"
},
{
"date": "12/07/2019",
"time": "02:49 AM",
"dir": true,
"size": null,
"filename": "en-US",
"parent": "C:\\Program Files\\Internet Explorer"
},
{
"date": "12/07/2019",
"time": "02:09 AM",
"dir": false,
"size": "54,784",
"filename": "ExtExport.exe",
"parent": "C:\\Program Files\\Internet Explorer"
},
...
]
"""
import re
import jc.utils
class info():
version = '1.0'
description = '`dir` command parser'
author = 'Rasheed Elsaleh'
author_email = 'rasheed@rebelliondefense.com'
# compatible options: win32
compatible = ['win32']
magic_commands = ['dir']
__version__ = info.version
def process(proc_data):
"""
Final processing to conform to the schema.
Parameters:
proc_data: (Dictionary of Lists) raw structured data to process
Returns:
List of Dictionaries. Structured data with the following schema:
[
{
"date": string,
"time": string,
"epoch": integer, # naive timestamp
"dir": boolean,
"size": integer,
"filename: string,
"parent": string
}
]
"""
for entry in proc_data:
# add timestamps
if 'date' in entry and 'time' in entry:
dt = entry['date'] + ' ' + entry['time']
timestamp = jc.utils.timestamp(dt)
entry['epoch'] = timestamp.naive
# add ints
int_list = ["size"]
for key in int_list:
if entry.get(key):
try:
key_int = int(entry[key].replace(",", ""))
except ValueError:
entry[key] = None
else:
entry[key] = key_int
return proc_data
def parse(data, raw=False, quiet=False):
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
if not quiet:
jc.utils.compatibility(__name__, info.compatible)
raw_output = []
if jc.utils.has_data(data):
for line in data.splitlines():
if line.startswith(" Directory of"):
parent_dir = line.lstrip(" Directory of ")
continue
# skip lines that don't start with a date
if not re.match(r'^\d{2}/\d{2}/\d{4}', line):
continue
output_line = {}
parsed_line = line.split()
output_line["date"] = parsed_line[0]
output_line["time"] = " ".join(parsed_line[1:3])
output_line.setdefault("dir", False)
output_line.setdefault("size", None)
if parsed_line[3] == "<DIR>":
output_line["dir"] = True
else:
output_line["size"] = parsed_line[3]
output_line["filename"] = " ".join(parsed_line[4:])
output_line["parent"] = parent_dir
raw_output.append(output_line)
if raw:
return raw_output
else:
return process(raw_output)

2
jc/parsers/dmidecode.py
View File

@ -112,7 +112,7 @@ import jc.utils
class info():
version = '1.1'
description = 'dmidecode command parser'
description = '`dmidecode` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'

248
jc/parsers/dpkg_l.py Normal file
View File

@ -0,0 +1,248 @@
"""jc - JSON CLI output utility `dpkg -l` command output parser
Set the `COLUMNS` environment variable to a large value to avoid field truncation. For example:
$ COLUMNS=500 dpkg -l | jc --dpkg-l
Usage (cli):
$ dpkg -l | jc --dpkg-l
or
$ jc dpkg -l
Usage (module):
import jc.parsers.dpkg
result = jc.parsers.dpkg.parse(dpkg_command_output)
Compatibility:
'linux'
Examples:
$ dpkg -l | jc --dpkg-l -p
[
{
"codes": "ii",
"name": "accountsservice",
"version": "0.6.45-1ubuntu1.3",
"architecture": "amd64",
"description": "query and manipulate user account information",
"desired": "install",
"status": "installed"
},
{
"codes": "rc",
"name": "acl",
"version": "2.2.52-3build1",
"architecture": "amd64",
"description": "Access control list utilities",
"desired": "remove",
"status": "config-files"
},
{
"codes": "uWR",
"name": "acpi",
"version": "1.7-1.1",
"architecture": "amd64",
"description": "displays information on ACPI devices",
"desired": "unknown",
"status": "trigger await",
"error": "reinstall required"
},
{
"codes": "rh",
"name": "acpid",
"version": "1:2.0.28-1ubuntu1",
"architecture": "amd64",
"description": "Advanced Configuration and Power Interface event daemon",
"desired": "remove",
"status": "half installed"
},
{
"codes": "pn",
"name": "adduser",
"version": "3.116ubuntu1",
"architecture": "all",
"description": "add and remove users and groups",
"desired": "purge",
"status": "not installed"
},
...
]
$ dpkg -l | jc --dpkg-l -p -r
[
{
"codes": "ii",
"name": "accountsservice",
"version": "0.6.45-1ubuntu1.3",
"architecture": "amd64",
"description": "query and manipulate user account information"
},
{
"codes": "rc",
"name": "acl",
"version": "2.2.52-3build1",
"architecture": "amd64",
"description": "Access control list utilities"
},
{
"codes": "uWR",
"name": "acpi",
"version": "1.7-1.1",
"architecture": "amd64",
"description": "displays information on ACPI devices"
},
{
"codes": "rh",
"name": "acpid",
"version": "1:2.0.28-1ubuntu1",
"architecture": "amd64",
"description": "Advanced Configuration and Power Interface event daemon"
},
{
"codes": "pn",
"name": "adduser",
"version": "3.116ubuntu1",
"architecture": "all",
"description": "add and remove users and groups"
},
...
]
"""
import jc.utils
import jc.parsers.universal
class info():
version = '1.0'
description = '`dpkg -l` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'
# compatible options: linux, darwin, cygwin, win32, aix, freebsd
compatible = ['linux']
magic_commands = ['dpkg -l']
__version__ = info.version
def process(proc_data):
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured data with the following schema:
[
{
"codes": string,
"name": string,
"version": string,
"architecture": string,
"description": string,
"desired": string,
"status": string,
"error": string
}
]
"""
for entry in proc_data:
if 'codes' in entry:
desired, status, *err = list(entry['codes'])
desired_map = {
'u': 'unknown',
'i': 'install',
'r': 'remove',
'p': 'purge',
'h': 'hold'
}
for key, value in desired_map.items():
if desired.lower() == key:
entry['desired'] = value
break
status_map = {
'n': 'not installed',
'i': 'installed',
'c': 'config-files',
'u': 'unpacked',
'f': 'failed config',
'h': 'half installed',
'w': 'trigger await',
't': 'trigger pending'
}
for key, value in status_map.items():
if status.lower() == key:
entry['status'] = value
break
if err:
err_map = {
'r': 'reinstall required'
}
for key, value in err_map.items():
if err[0].lower() == key:
entry['error'] = value
break
return proc_data
def parse(data, raw=False, quiet=False):
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
if not quiet:
jc.utils.compatibility(__name__, info.compatible)
working_list = []
raw_output = []
header_found = False
if jc.utils.has_data(data):
# clean up headers
for line in filter(None, data.splitlines()):
if 'Architecture' in line:
header_found = True
working_list.append(line.lower().replace('||/', 'codes'))
continue
if '=========' in line:
continue
if header_found:
working_list.append(line)
raw_output = jc.parsers.universal.simple_table_parse(working_list)
if raw:
return raw_output
else:
return process(raw_output)

2
jc/parsers/du.py
View File

@ -83,7 +83,7 @@ import jc.parsers.universal
class info():
version = '1.2'
description = 'du command parser'
description = '`du` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'

2
jc/parsers/env.py
View File

@ -64,7 +64,7 @@ import jc.utils
class info():
version = '1.2'
description = 'env command parser'
description = '`env` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/file.py
View File

@ -58,7 +58,7 @@ import jc.parsers.universal
class info():
version = '1.2'
description = 'file command parser'
description = '`file` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

215
jc/parsers/finger.py Normal file
View File

@ -0,0 +1,215 @@
"""jc - JSON CLI output utility `finger` command output parser
Supports `-s` output option. Does not support the `-l` detail option.
Usage (cli):
$ finger | jc --finger
or
$ jc finger
Usage (module):
import jc.parsers.finger
result = jc.parsers.finger.parse(finger_command_output)
Compatibility:
'linux', 'darwin', 'cygwin', freebsd'
Examples:
$ finger | jc --finger -p
[
{
"login": "jdoe",
"name": "John Doe",
"tty": "tty1",
"idle": "14d",
"login_time": "Mar 22 21:14",
"tty_writeable": false,
"idle_minutes": 0,
"idle_hours": 0,
"idle_days": 14,
"total_idle_minutes": 20160
},
{
"login": "jdoe",
"name": "John Doe",
"tty": "pts/0",
"idle": null,
"login_time": "Apr 5 15:33",
"details": "(192.168.1.22)",
"tty_writeable": true,
"idle_minutes": 0,
"idle_hours": 0,
"idle_days": 0,
"total_idle_minutes": 0
},
...
]
$ finger | jc --finger -p -r
[
{
"login": "jdoe",
"name": "John Doe",
"tty": "*tty1",
"idle": "14d",
"login_time": "Mar 22 21:14"
},
{
"login": "jdoe",
"name": "John Doe",
"tty": "pts/0",
"idle": null,
"login_time": "Apr 5 15:33",
"details": "(192.168.1.22)"
},
...
]
"""
import re
import jc.utils
import jc.parsers.universal
class info():
version = '1.0'
description = '`finger` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'
# compatible options: linux, darwin, cygwin, win32, aix, freebsd
compatible = ['linux', 'darwin', 'cygwin', 'freebsd']
magic_commands = ['finger']
__version__ = info.version
def process(proc_data):
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured data with the following schema:
[
{
"login": string,
"name": string,
"tty": string,
"idle": string, # null if empty
"login_time": string,
"details": string,
"tty_writeable": boolean,
"idle_minutes": integer,
"idle_hours": integer,
"idle_days": integer,
"total_idle_minutes": integer
}
]
"""
for entry in proc_data:
if 'tty' in entry:
entry['tty_writeable'] = True
if '*' in entry['tty']:
entry['tty'] = entry['tty'].replace('*', '')
entry['tty_writeable'] = False
if 'idle' in entry:
entry['idle_minutes'] = 0
entry['idle_hours'] = 0
entry['idle_days'] = 0
if entry['idle'] == '-':
entry['idle'] = None
if entry['idle'] and entry['idle'].isnumeric():
entry['idle_minutes'] = int(entry['idle'])
if entry['idle'] and ':' in entry['idle']:
entry['idle_hours'] = int(entry['idle'].split(':')[0])
entry['idle_minutes'] = int(entry['idle'].split(':')[1])
if entry['idle'] and 'd' in entry['idle']:
entry['idle_days'] = int(entry['idle'].replace('d', ''))
entry['total_idle_minutes'] = (entry['idle_days'] * 1440) + \
(entry['idle_hours'] * 60) + \
entry['idle_minutes']
if 'details' in entry:
if not entry['details']:
del entry['details']
return proc_data
def parse(data, raw=False, quiet=False):
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
if not quiet:
jc.utils.compatibility(__name__, info.compatible)
raw_output = []
if jc.utils.has_data(data):
# Finger output is an abomination that is nearly unparsable. But there is a way:
# First find the location of the last character of 'Idle' in the table and cut
# all lines at that spot. Data before that spot can use the unviversal.sparse_table_parse function.
# All data after that spot can be run through regex to find the login datetime and possibly
# other fields.
data_lines = list(filter(None, data.splitlines()))
sep_col = data_lines[0].find('Idle') + 4
first_half = []
second_half = []
for line in data_lines:
first_half.append(line[:sep_col])
second_half.append(line[sep_col:])
first_half[0] = first_half[0].lower()
# parse the first half
raw_output = jc.parsers.universal.sparse_table_parse(first_half)
# use regex to get login datetime and 'other' data
pattern = re.compile(r'([A-Z][a-z]{2}\s+\d{1,2}\s+)(\d\d:\d\d|\d{4})(\s?.+)?$')
# remove header row from list
second_half.pop(0)
for index, line in enumerate(second_half):
dt = re.search(pattern, line)
if dt:
if dt.group(1) and dt.group(2):
raw_output[index]['login_time'] = dt.group(1).strip() + ' ' + dt.group(2).strip()
if dt.group(3):
raw_output[index]['details'] = dt.group(3).strip()
if raw:
return raw_output
else:
return process(raw_output)

2
jc/parsers/foo.py
View File

@ -32,7 +32,7 @@ import jc.utils
class info():
version = '1.0'
description = 'foo command parser'
description = '`foo` command parser'
author = 'John Doe'
author_email = 'johndoe@gmail.com'
# details = 'enter any other details here'

2
jc/parsers/free.py
View File

@ -63,7 +63,7 @@ import jc.parsers.universal
class info():
version = '1.2'
description = 'free command parser'
description = '`free` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/fstab.py
View File

@ -76,7 +76,7 @@ import jc.utils
class info():
version = '1.3'
description = 'fstab file parser'
description = '`/etc/fstab` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/group.py
View File

@ -100,7 +100,7 @@ import jc.utils
class info():
version = '1.1'
description = '/etc/group file parser'
description = '`/etc/group` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'

2
jc/parsers/gshadow.py
View File

@ -66,7 +66,7 @@ import jc.utils
class info():
version = '1.1'
description = '/etc/gshadow file parser'
description = '`/etc/gshadow` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'

2
jc/parsers/hash.py
View File

@ -33,7 +33,7 @@ import jc.parsers.universal
class info():
version = '1.0'
description = 'hash command parser'
description = '`hash` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/hashsum.py
View File

@ -63,7 +63,7 @@ import jc.utils
class info():
version = '1.0'
description = 'hashsum command parser (md5sum, shasum, etc.)'
description = 'hashsum command parser (`md5sum`, `shasum`, etc.)'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
details = 'Parses MD5 and SHA hash program output'

2
jc/parsers/hciconfig.py
View File

@ -270,7 +270,7 @@ import jc.utils
class info():
version = '1.0'
description = 'hciconfig command parser'
description = '`hciconfig` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'

2
jc/parsers/history.py
View File

@ -52,7 +52,7 @@ import jc.utils
class info():
version = '1.3'
description = 'history command parser'
description = '`history` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
details = 'Optimizations by https://github.com/philippeitis'

2
jc/parsers/hosts.py
View File

@ -67,7 +67,7 @@ import jc.utils
class info():
version = '1.2'
description = '/etc/hosts file parser'
description = '`/etc/hosts` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/id.py
View File

@ -80,7 +80,7 @@ import jc.utils
class info():
version = '1.1'
description = 'id command parser'
description = '`id` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'

2
jc/parsers/ifconfig.py
View File

@ -157,7 +157,7 @@ import jc.utils
class info():
version = '1.8'
description = 'ifconfig command parser'
description = '`ifconfig` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
details = 'Using ifconfig-parser from https://github.com/KnightWhoSayNi/ifconfig-parser'

2
jc/parsers/iptables.py
View File

@ -144,7 +144,7 @@ import jc.utils
class info():
version = '1.4'
description = 'iptables command parser'
description = '`iptables` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

14
jc/parsers/iw_scan.py
View File

@ -1,6 +1,6 @@
"""jc - JSON CLI output utility `iw dev <device> scan` command output parser
This parser is considered beta quality. Not all fields are parsed.
This parser is considered beta quality. Not all fields are parsed and there are not enough samples to test.
Usage (cli):
@ -115,7 +115,7 @@ import jc.utils
class info():
version = '0.5'
description = 'iw dev <device> scan command parser'
description = '`iw dev [device] scan` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
details = 'Enhancements by Philipp Schmitt (https://pschmitt.dev/)'
@ -149,7 +149,7 @@ def process(proc_data):
"""
# convert ints and floats for top-level keys
for item in proc_data:
for item in proc_data:
for key in item:
try:
item[key] = int(item[key])
@ -163,21 +163,22 @@ def process(proc_data):
new_list = []
for list_item in item[key]:
try:
new_list.append(int(list_item))
new_list.append(int(list_item))
except (Exception):
try:
new_list.append(float(list_item))
new_list.append(float(list_item))
except (Exception):
pass
item[key] = new_list
return proc_data
def post_parse(data):
# remove empty items
cleandata = []
for ssid in data:
ssid = { k : v for k, v in ssid.items() if v}
ssid = {k: v for k, v in ssid.items() if v}
cleandata.append(ssid)
# remove asterisks from begining of values
@ -277,6 +278,7 @@ def post_parse(data):
return process(cleandata)
def parse(data, raw=False, quiet=False):
"""
Main text parsing function

2
jc/parsers/jobs.py
View File

@ -87,7 +87,7 @@ import jc.utils
class info():
version = '1.2'
description = 'jobs command parser'
description = '`jobs` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

23
jc/parsers/last.py
View File

@ -1,6 +1,8 @@
"""jc - JSON CLI output utility `last` and `lastb` command output parser
Supports -w and -F options.
Supports `-w` and `-F` options.
Calculated epoch time fields are naive (i.e. based on the local time of the system the parser is run on) since there is no timezone information in the `last` command output.
Usage (cli):
@ -85,13 +87,12 @@ Examples:
"""
import re
from datetime import datetime
import jc.utils
class info():
version = '1.4'
description = 'last and lastb command parser'
version = '1.5'
description = '`last` and `lastb` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
details = 'Enhancements by https://github.com/zerolagtime'
@ -124,8 +125,8 @@ def process(proc_data):
"login": string,
"logout": string,
"duration": string,
"login_epoch": integer, # available with last -F option
"logout_epoch": integer, # available with last -F option
"login_epoch": integer, # (naive) available with last -F option
"logout_epoch": integer, # (naive) available with last -F option
"duration_seconds": integer # available with last -F option
}
]
@ -152,11 +153,13 @@ def process(proc_data):
if 'logout' in entry and entry['logout'] == 'gone_-_no_logout':
entry['logout'] = 'gone - no logout'
if 'login' in entry and re.match(r'.*\d\d:\d\d:\d\d \d\d\d\d.*',entry['login']):
entry['login_epoch'] = int(datetime.strptime(entry['login'], '%a %b %d %H:%M:%S %Y').strftime('%s'))
if 'login' in entry and re.match(r'.*\d\d:\d\d:\d\d \d\d\d\d.*', entry['login']):
timestamp = jc.utils.timestamp(entry['login'])
entry['login_epoch'] = timestamp.naive
if 'logout' in entry and re.match(r'.*\d\d:\d\d:\d\d \d\d\d\d.*',entry['logout']):
entry['logout_epoch'] = int(datetime.strptime(entry['logout'], '%a %b %d %H:%M:%S %Y').strftime('%s'))
if 'logout' in entry and re.match(r'.*\d\d:\d\d:\d\d \d\d\d\d.*', entry['logout']):
timestamp = jc.utils.timestamp(entry['logout'])
entry['logout_epoch'] = timestamp.naive
if 'login_epoch' in entry and 'logout_epoch' in entry:
entry['duration_seconds'] = int(entry['logout_epoch']) - int(entry['login_epoch'])

38
jc/parsers/ls.py
View File

@ -1,11 +1,15 @@
"""jc - JSON CLI output utility `ls` and `vdir` command output parser
Options supported:
- `lbaR`
- `lbaR1`
- `--time-style=full-iso`
- `-h`: File sizes will be available in text form with `-r` but larger file sizes with human readable suffixes will be converted to `Null` in the default view since the parser attempts to convert this field to an integer.
Note: The `-l` or `-b` option of `ls` should be used to correctly parse filenames that include newline characters. Since `ls` does not encode newlines in filenames when outputting to a pipe it will cause `jc` to see multiple files instead of a single file if `-l` or `-b` is not used. Alternatively, `vdir` can be used, which is the same as running `ls -lb`.
Note: The `-1`, `-l`, or `-b` option of `ls` should be used to correctly parse filenames that include newline characters. Since `ls` does not encode newlines in filenames when outputting to a pipe it will cause `jc` to see multiple files instead of a single file if `-1`, `-l`, or `-b` is not used. Alternatively, `vdir` can be used, which is the same as running `ls -lb`.
The `epoch` calculated timestamp field is naive (i.e. based on the local time of the system the parser is run on)
The `epoch_utc` calculated timestamp field is timezone-aware and is only available if the timezone field is UTC.
Usage (cli):
@ -153,8 +157,8 @@ import jc.utils
class info():
version = '1.6'
description = 'ls command parser'
version = '1.7'
description = '`ls` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@ -180,18 +184,19 @@ def process(proc_data):
[
{
"filename": string,
"flags": string,
"links": integer,
"parent": string,
"owner": string,
"group": string,
"size": integer,
"date": string
"filename": string,
"flags": string,
"links": integer,
"parent": string,
"owner": string,
"group": string,
"size": integer,
"date": string,
"epoch": integer, # naive timestamp if date field exists and can be converted
"epoch_utc": integer # timezone aware timestamp if date field is in UTC and can be converted
}
]
"""
for entry in proc_data:
int_list = ['links', 'size']
for key in int_list:
@ -202,6 +207,13 @@ def process(proc_data):
except (ValueError):
entry[key] = None
if 'date' in entry:
# to speed up processing only try to convert the date if it's not the default format
if not re.match(r'[a-zA-Z]{3}\s{1,2}\d{1,2}\s{1,2}[0-9:]{4,5}', entry['date']):
ts = jc.utils.timestamp(entry['date'])
entry['epoch'] = ts.naive
entry['epoch_utc'] = ts.utc
return proc_data

2
jc/parsers/lsblk.py
View File

@ -226,7 +226,7 @@ import jc.parsers.universal
class info():
version = '1.5'
description = 'lsblk command parser'
description = '`lsblk` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/lsmod.py
View File

@ -117,7 +117,7 @@ import jc.parsers.universal
class info():
version = '1.3'
description = 'lsmod command parser'
description = '`lsmod` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/lsof.py
View File

@ -107,7 +107,7 @@ import jc.parsers.universal
class info():
version = '1.2'
description = 'lsof command parser'
description = '`lsof` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/mount.py
View File

@ -66,7 +66,7 @@ import jc.utils
class info():
version = '1.5'
description = 'mount command parser'
description = '`mount` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/netstat.py
View File

@ -256,7 +256,7 @@ Examples:
class info():
version = '1.8'
description = 'netstat command parser'
description = '`netstat` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/ntpq.py
View File

@ -193,7 +193,7 @@ import jc.parsers.universal
class info():
version = '1.3'
description = 'ntpq -p command parser'
description = '`ntpq -p` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/passwd.py
View File

@ -84,7 +84,7 @@ import jc.utils
class info():
version = '1.1'
description = '/etc/passwd file parser'
description = '`/etc/passwd` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'

2
jc/parsers/ping.py
View File

@ -120,7 +120,7 @@ import jc.utils
class info():
version = '1.2'
description = 'ping command parser'
description = '`ping` and `ping6` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/pip_list.py
View File

@ -42,7 +42,7 @@ import jc.parsers.universal
class info():
version = '1.3'
description = 'pip list command parser'
description = '`pip list` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/pip_show.py
View File

@ -52,7 +52,7 @@ import jc.utils
class info():
version = '1.1'
description = 'pip show command parser'
description = '`pip show` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/ps.py
View File

@ -187,7 +187,7 @@ import jc.parsers.universal
class info():
version = '1.3'
description = 'ps command parser'
description = '`ps` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/route.py
View File

@ -94,7 +94,7 @@ import jc.parsers.universal
class info():
version = '1.4'
description = 'route command parser'
description = '`route` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

258
jc/parsers/rpm_qi.py Normal file
View File

@ -0,0 +1,258 @@
"""jc - JSON CLI output utility `rpm -qi` command output parser
Works with `rpm -qi [package]` or `rpm -qia`.
The `build_epoch` calculated timestamp field is naive (i.e. based on the local time of the system the parser is run on)
The `build_epoch_utc` calculated timestamp field is timezone-aware and is only available if the timezone field is UTC.
Usage (cli):
$ rpm -qia | jc --rpm_qi
or
$ jc rpm -qia
Usage (module):
import jc.parsers.rpm_qi
result = jc.parsers.rpm_qi.parse(rpm_qi_command_output)
Compatibility:
'linux'
Examples:
$ rpm -qia | jc --rpm_qi -p
[
{
"name": "make",
"epoch": 1,
"version": "3.82",
"release": "24.el7",
"architecture": "x86_64",
"install_date": "Wed 16 Oct 2019 09:21:42 AM PDT",
"group": "Development/Tools",
"size": 1160660,
"license": "GPLv2+",
"signature": "RSA/SHA256, Thu 22 Aug 2019 02:34:59 PM PDT, Key ID 24c6a8a7f4a80eb5",
"source_rpm": "make-3.82-24.el7.src.rpm",
"build_date": "Thu 08 Aug 2019 05:47:25 PM PDT",
"build_host": "x86-01.bsys.centos.org",
"relocations": "(not relocatable)",
"packager": "CentOS BuildSystem <http://bugs.centos.org>",
"vendor": "CentOS",
"url": "http://www.gnu.org/software/make/",
"summary": "A GNU tool which simplifies the build process for users",
"description": "A GNU tool for controlling the generation of executables and other non-source...",
"build_epoch": 1565311645,
"build_epoch_utc": null
},
{
"name": "kbd-legacy",
"version": "1.15.5",
"release": "15.el7",
"architecture": "noarch",
"install_date": "Thu 15 Aug 2019 10:53:08 AM PDT",
"group": "System Environment/Base",
"size": 503608,
"license": "GPLv2+",
"signature": "RSA/SHA256, Mon 12 Nov 2018 07:17:49 AM PST, Key ID 24c6a8a7f4a80eb5",
"source_rpm": "kbd-1.15.5-15.el7.src.rpm",
"build_date": "Tue 30 Oct 2018 03:40:00 PM PDT",
"build_host": "x86-01.bsys.centos.org",
"relocations": "(not relocatable)",
"packager": "CentOS BuildSystem <http://bugs.centos.org>",
"vendor": "CentOS",
"url": "http://ftp.altlinux.org/pub/people/legion/kbd",
"summary": "Legacy data for kbd package",
"description": "The kbd-legacy package contains original keymaps for kbd package. Please note...",
"build_epoch": 1540939200,
"build_epoch_utc": null
},
...
]
$ rpm -qia | jc --rpm_qi -p -r
[
{
"name": "make",
"epoch": "1",
"version": "3.82",
"release": "24.el7",
"architecture": "x86_64",
"install_date": "Wed 16 Oct 2019 09:21:42 AM PDT",
"group": "Development/Tools",
"size": "1160660",
"license": "GPLv2+",
"signature": "RSA/SHA256, Thu 22 Aug 2019 02:34:59 PM PDT, Key ID 24c6a8a7f4a80eb5",
"source_rpm": "make-3.82-24.el7.src.rpm",
"build_date": "Thu 08 Aug 2019 05:47:25 PM PDT",
"build_host": "x86-01.bsys.centos.org",
"relocations": "(not relocatable)",
"packager": "CentOS BuildSystem <http://bugs.centos.org>",
"vendor": "CentOS",
"url": "http://www.gnu.org/software/make/",
"summary": "A GNU tool which simplifies the build process for users",
"description": "A GNU tool for controlling the generation of executables and other..."
},
{
"name": "kbd-legacy",
"version": "1.15.5",
"release": "15.el7",
"architecture": "noarch",
"install_date": "Thu 15 Aug 2019 10:53:08 AM PDT",
"group": "System Environment/Base",
"size": "503608",
"license": "GPLv2+",
"signature": "RSA/SHA256, Mon 12 Nov 2018 07:17:49 AM PST, Key ID 24c6a8a7f4a80eb5",
"source_rpm": "kbd-1.15.5-15.el7.src.rpm",
"build_date": "Tue 30 Oct 2018 03:40:00 PM PDT",
"build_host": "x86-01.bsys.centos.org",
"relocations": "(not relocatable)",
"packager": "CentOS BuildSystem <http://bugs.centos.org>",
"vendor": "CentOS",
"url": "http://ftp.altlinux.org/pub/people/legion/kbd",
"summary": "Legacy data for kbd package",
"description": "The kbd-legacy package contains original keymaps for kbd package..."
},
...
]
"""
import jc.utils
class info():
version = '1.0'
description = '`rpm -qi` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'
# compatible options: linux, darwin, cygwin, win32, aix, freebsd
compatible = ['linux']
magic_commands = ['rpm -qi', 'rpm -qia', 'rpm -qai']
__version__ = info.version
def process(proc_data):
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured data with the following schema:
[
{
"name": string,
"epoch": integer,
"version": string,
"release": string,
"architecture": string,
"install_date": string,
"group": string,
"size": integer,
"license": string,
"signature": string,
"source_rpm": string,
"build_date": string,
"build_epoch": integer, # naive timestamp
"build_epoch_utc": integer, # Aware timestamp if timezone is UTC
"build_host": string,
"relocations": string,
"packager": string,
"vendor": string,
"url": string,
"summary": string,
"description": string
}
]
"""
for entry in proc_data:
int_list = ['epoch', 'size']
for key in int_list:
if key in entry:
try:
entry[key] = int(entry[key])
except (ValueError):
entry[key] = None
if 'build_date' in entry:
timestamp = jc.utils.timestamp(entry['build_date'])
entry['build_epoch'] = timestamp.naive
entry['build_epoch_utc'] = timestamp.utc
return proc_data
def parse(data, raw=False, quiet=False):
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
if not quiet:
jc.utils.compatibility(__name__, info.compatible)
raw_output = []
entry_obj = {}
last_entry = None
this_entry = None
desc_entry = False
description = []
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
split_line = line.split(': ', maxsplit=1)
if split_line[0].startswith('Name') and len(split_line) == 2:
this_entry = split_line[1].strip()
if this_entry != last_entry:
if entry_obj:
if description:
entry_obj['description'] = ' '.join(description)
raw_output.append(entry_obj)
entry_obj = {}
last_entry = this_entry
desc_entry = False
if len(split_line) == 2:
entry_obj[split_line[0].strip().lower().replace(' ', '_')] = split_line[1].strip()
if line.startswith('Description :'):
desc_entry = True
description = []
continue
if desc_entry:
description.append(line)
if entry_obj:
if description:
entry_obj['description'] = ' '.join(description)
raw_output.append(entry_obj)
if raw:
return raw_output
else:
return process(raw_output)

2
jc/parsers/shadow.py
View File

@ -90,7 +90,7 @@ import jc.utils
class info():
version = '1.1'
description = '/etc/shadow file parser'
description = '`/etc/shadow` file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'

2
jc/parsers/ss.py
View File

@ -261,7 +261,7 @@ import jc.utils
class info():
version = '1.2'
description = 'ss command parser'
description = '`ss` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

89
jc/parsers/stat.py
View File

@ -1,5 +1,9 @@
"""jc - JSON CLI output utility `stat` command output parser
The `xxx_epoch` calculated timestamp fields are naive (i.e. based on the local time of the system the parser is run on)
The `xxx_epoch_utc` calculated timestamp fields are timezone-aware and are only available if the timezone field is UTC.
Usage (cli):
$ stat * | jc --stat
@ -39,7 +43,15 @@ Examples:
"access_time": "2019-11-14 08:18:03.509681766 +0000",
"modify_time": "2019-06-06 22:28:15.000000000 +0000",
"change_time": "2019-08-12 17:21:29.521945390 +0000",
"birth_time": null
"birth_time": null,
"access_time_epoch": 1573748283,
"access_time_epoch_utc": 1573719483,
"modify_time_epoch": 1559885295,
"modify_time_epoch_utc": 1559860095,
"change_time_epoch": 1565655689,
"change_time_epoch_utc": 1565630489,
"birth_time_epoch": null,
"birth_time_epoch_utc": null
},
{
"file": "/bin/btrfs",
@ -59,7 +71,15 @@ Examples:
"access_time": "2019-11-14 08:18:28.990834276 +0000",
"modify_time": "2018-03-12 23:04:27.000000000 +0000",
"change_time": "2019-08-12 17:21:29.545944399 +0000",
"birth_time": null
"birth_time": null,
"access_time_epoch": 1573748308,
"access_time_epoch_utc": 1573719508,
"modify_time_epoch": 1520921067,
"modify_time_epoch_utc": 1520895867,
"change_time_epoch": 1565655689,
"change_time_epoch_utc": 1565630489,
"birth_time_epoch": null,
"birth_time_epoch_utc": null
},
...
]
@ -106,7 +126,7 @@ Examples:
"change_time": "2019-08-12 17:21:29.545944399 +0000",
"birth_time": null
},
..
...
]
"""
import shlex
@ -114,8 +134,8 @@ import jc.utils
class info():
version = '1.5'
description = 'stat command parser'
version = '1.6'
description = '`stat` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@ -141,29 +161,37 @@ def process(proc_data):
[
{
"file": string,
"link_to" string,
"size": integer,
"blocks": integer,
"io_blocks": integer,
"type": string,
"device": string,
"inode": integer,
"links": integer,
"access": string,
"flags": string,
"uid": integer,
"user": string,
"gid": integer,
"group": string,
"access_time": string, # - = null
"modify_time": string, # - = null
"change_time": string, # - = null
"birth_time": string, # - = null
"unix_device": integer,
"rdev": integer,
"block_size": integer,
"unix_flags": string
"file": string,
"link_to" string,
"size": integer,
"blocks": integer,
"io_blocks": integer,
"type": string,
"device": string,
"inode": integer,
"links": integer,
"access": string,
"flags": string,
"uid": integer,
"user": string,
"gid": integer,
"group": string,
"access_time": string, # - = null
"access_time_epoch": integer, # naive timestamp
"access_time_epoch_utc": integer, # timezone-aware timestamp
"modify_time": string, # - = null
"modify_time_epoch": integer, # naive timestamp
"modify_time_epoch_utc": integer, # timezone-aware timestamp
"change_time": string, # - = null
"change_time_epoch": integer, # naive timestamp
"change_time_epoch_utc": integer, # timezone-aware timestamp
"birth_time": string, # - = null
"birth_time_epoch": integer, # naive timestamp
"birth_time_epoch_utc": integer, # timezone-aware timestamp
"unix_device": integer,
"rdev": integer,
"block_size": integer,
"unix_flags": string
}
]
"""
@ -177,13 +205,16 @@ def process(proc_data):
except (ValueError):
entry[key] = None
# turn - into null for time fields
# turn - into null for time fields and add calculated timestamp fields
for entry in proc_data:
null_list = ['access_time', 'modify_time', 'change_time', 'birth_time']
for key in null_list:
if key in entry:
if entry[key] == '-':
entry[key] = None
ts = jc.utils.timestamp(entry[key])
entry[key + '_epoch'] = ts.naive
entry[key + '_epoch_utc'] = ts.utc
return proc_data

2
jc/parsers/sysctl.py
View File

@ -50,7 +50,7 @@ import jc.utils
class info():
version = '1.0'
description = 'sysctl command parser'
description = '`sysctl` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'

2
jc/parsers/systemctl.py
View File

@ -50,7 +50,7 @@ import jc.utils
class info():
version = '1.3'
description = 'systemctl command parser'
description = '`systemctl` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/systemctl_lj.py
View File

@ -69,7 +69,7 @@ import jc.utils
class info():
version = '1.3'
description = 'systemctl list-jobs command parser'
description = '`systemctl list-jobs` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/systemctl_ls.py
View File

@ -44,7 +44,7 @@ import jc.utils
class info():
version = '1.3'
description = 'systemctl list-sockets command parser'
description = '`systemctl list-sockets` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/systemctl_luf.py
View File

@ -41,7 +41,7 @@ import jc.utils
class info():
version = '1.3'
description = 'systemctl list-unit-files command parser'
description = '`systemctl list-unit-files` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

333
jc/parsers/time.py Normal file
View File

@ -0,0 +1,333 @@
"""jc - JSON CLI output utility `/usr/bin/time` command output parser
Output from `/usr/bin/time` is sent to `STDERR`, so the `-o` option can be used to redirect the output to a file that can be read by `jc`.
Alternatively, the output from `/usr/bin/time` can be redirected to `STDOUT` so `jc` can receive it.
Note: `/usr/bin/time` is similar but different from the Bash builtin `time` command.
Usage (cli):
$ /usr/bin/time -o timefile.out sleep 2.5; cat timefile.out | jc --time -p
Usage (module):
import jc.parsers.time
result = jc.parsers.time.parse(time_command_output)
Compatibility:
'linux', 'darwin', 'cygwin', 'aix', 'freebsd'
Examples:
$ /usr/bin/time --verbose -o timefile.out sleep 2.5; cat timefile.out | jc --time -p
{
"command_being_timed": "sleep 2.5",
"user_time": 0.0,
"system_time": 0.0,
"cpu_percent": 0,
"elapsed_time": "0:02.50",
"average_shared_text_size": 0,
"average_unshared_data_size": 0,
"average_stack_size": 0,
"average_total_size": 0,
"maximum_resident_set_size": 2084,
"average_resident_set_size": 0,
"major_pagefaults": 0,
"minor_pagefaults": 72,
"voluntary_context_switches": 2,
"involuntary_context_switches": 1,
"swaps": 0,
"block_input_operations": 0,
"block_output_operations": 0,
"messages_sent": 0,
"messages_received": 0,
"signals_delivered": 0,
"page_size": 4096,
"exit_status": 0,
"elapsed_time_hours": 0,
"elapsed_time_minutes": 0,
"elapsed_time_seconds": 2,
"elapsed_time_centiseconds": 50,
"elapsed_time_total_seconds": 2.5
}
$ /usr/bin/time --verbose -o timefile.out sleep 2.5; cat timefile.out | jc --time -p -r
{
"command_being_timed": "\"sleep 2.5\"",
"user_time": "0.00",
"system_time": "0.00",
"cpu_percent": "0",
"elapsed_time": "0:02.50",
"average_shared_text_size": "0",
"average_unshared_data_size": "0",
"average_stack_size": "0",
"average_total_size": "0",
"maximum_resident_set_size": "2084",
"average_resident_set_size": "0",
"major_pagefaults": "0",
"minor_pagefaults": "72",
"voluntary_context_switches": "2",
"involuntary_context_switches": "0",
"swaps": "0",
"block_input_operations": "0",
"block_output_operations": "0",
"messages_sent": "0",
"messages_received": "0",
"signals_delivered": "0",
"page_size": "4096",
"exit_status": "0"
}
"""
import jc.utils
class info():
version = '1.0'
description = '`/usr/bin/time` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'
# compatible options: linux, darwin, cygwin, win32, aix, freebsd
compatible = ['linux', 'darwin', 'cygwin', 'aix', 'freebsd']
__version__ = info.version
def process(proc_data):
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
Dictionary. Structured data with the following schema:
Source: https://www.freebsd.org/cgi/man.cgi?query=getrusage
https://man7.org/linux/man-pages/man1/time.1.html
{
"real_time": float,
"user_time": float,
"system_time": float,
"elapsed_time": string,
"elapsed_time_hours": integer,
"elapsed_time_minutes": integer,
"elapsed_time_seconds": integer,
"elapsed_time_centiseconds": integer,
"elapsed_time_total_seconds": float,
"cpu_percent": integer, # null if ?
"average_shared_text_size": integer,
"average_unshared_data_size": integer,
"average_unshared_stack_size": integer,
"average_shared_memory_size": integer,
"maximum_resident_set_size": integer,
"block_input_operations": integer, # aka File system inputs
"block_output_operations": integer, # aka File system outputs
"major_pagefaults": integer,
"minor_pagefaults": integer,
"swaps": integer,
"page_reclaims": integer,
"page_faults": integer,
"messages_sent": integer,
"messages_received": integer,
"signals_received": integer,
"voluntary_context_switches": integer,
"involuntary_context_switches": integer
"command_being_timed": string,
"average_stack_size": integer,
"average_total_size": integer,
"average_resident_set_size": integer,
"signals_delivered": integer,
"page_size": integer,
"exit_status": integer
}
"""
if 'command_being_timed' in proc_data:
proc_data['command_being_timed'] = proc_data['command_being_timed'][1:-1]
if 'elapsed_time' in proc_data:
proc_data['elapsed_time'] = proc_data['elapsed_time'].replace('.', ':')
*hours, minutes, seconds, centiseconds = proc_data['elapsed_time'].split(':')
proc_data['elapsed_time'] = proc_data['elapsed_time'][::-1].replace(':', '.', 1)[::-1]
if hours:
proc_data['elapsed_time_hours'] = int(hours[0])
else:
proc_data['elapsed_time_hours'] = 0
proc_data['elapsed_time_minutes'] = int(minutes)
proc_data['elapsed_time_seconds'] = int(seconds)
proc_data['elapsed_time_centiseconds'] = int(centiseconds)
proc_data['elapsed_time_total_seconds'] = (proc_data['elapsed_time_hours'] * 3600) + \
(proc_data['elapsed_time_minutes'] * 60) + \
(proc_data['elapsed_time_seconds']) + \
(proc_data['elapsed_time_centiseconds'] / 100)
int_list = ['elapsed_time_hours', 'elapsed_time_minutes', 'elapsed_time_seconds', 'elapsed_time_microseconds',
'cpu_percent', 'average_shared_text_size', 'average_unshared_data_size', 'average_unshared_stack_size',
'average_shared_memory_size', 'maximum_resident_set_size', 'block_input_operations',
'block_output_operations', 'major_pagefaults', 'minor_pagefaults', 'swaps', 'page_reclaims',
'page_faults', 'messages_sent', 'messages_received', 'signals_received', 'voluntary_context_switches',
'involuntary_context_switches', 'average_stack_size', 'average_total_size', 'average_resident_set_size',
'signals_delivered', 'page_size', 'exit_status']
for key in int_list:
if key in proc_data:
try:
proc_data[key] = int(proc_data[key])
except (ValueError, TypeError):
proc_data[key] = None
float_list = ['real_time', 'user_time', 'system_time']
for key in float_list:
if key in proc_data:
try:
proc_data[key] = float(proc_data[key])
except (ValueError):
proc_data[key] = None
return proc_data
def parse(data, raw=False, quiet=False):
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
"""
if not quiet:
jc.utils.compatibility(__name__, info.compatible)
raw_output = {}
if jc.utils.has_data(data):
time_type = None # linux_brief, linux_long, bsd_brief, bsd_long, posix
for line in filter(None, data.splitlines()):
# linux default style:
# 0.00user 0.00system 0:03.00elapsed 0%CPU (0avgtext+0avgdata 2148maxresident)k
# 0inputs+0outputs (0major+71minor)pagefaults 0swaps
if time_type != 'linux_brief' and 'elapsed' in line:
time_type = 'linux_brief'
# BSD/OSX default style:
# 0.00 real 0.00 user 0.00 sys
elif time_type != 'bsd_brief' and ' user ' in line:
time_type = 'bsd_brief'
elif time_type != 'linux_long' and 'Command' in line:
time_type = 'linux_long'
elif time_type != 'bsd_long' and 'maximum resident set size' in line:
time_type = 'bsd_long'
# POSIX compliant output:
# real 0.00
# user 0.00
# sys 0.00
elif time_type != 'posix' and line.startswith('real '):
time_type = 'posix'
# start parsing lines
if time_type == 'linux_brief':
if 'elapsed' in line:
line_num = 0
else:
line_num = 1
new_line = line.replace('+', ' ').replace('(', ' ').replace(')', ' ')\
.replace('user', ' ').replace('system', ' ').replace('elapsed', ' ')\
.replace('elapsed', ' ').replace('%CPU', ' ').replace('avgtext', ' ')\
.replace('avgdata', ' ').replace('maxresident', ' ').replace('inputs', ' ')\
.replace('outputs', ' ').replace('major', ' ').replace('minor', ' ')\
.replace('pagefaults', ' ').replace('swaps', ' ').replace('k', ' ')
linux_brief_line = new_line.split()
if line_num == 0:
raw_output['user_time'] = linux_brief_line[0]
raw_output['system_time'] = linux_brief_line[1]
raw_output['elapsed_time'] = linux_brief_line[2]
raw_output['cpu_percent'] = None if linux_brief_line[3] == '?' else linux_brief_line[3]
raw_output['average_shared_text'] = linux_brief_line[4]
raw_output['average_unshared_data_size'] = linux_brief_line[5]
raw_output['maximum_resident_set_size'] = linux_brief_line[6]
else:
raw_output['block_input_operations'] = linux_brief_line[0]
raw_output['block_output_operations'] = linux_brief_line[1]
raw_output['major_pagefaults'] = linux_brief_line[2]
raw_output['minor_pagefaults'] = linux_brief_line[3]
raw_output['swaps'] = linux_brief_line[4]
if time_type == 'posix':
posix_line = line.split()
if 'real' in line:
raw_output['real_time'] = posix_line[1]
if 'user' in line:
raw_output['user_time'] = posix_line[1]
if 'sys' in line:
raw_output['system_time'] = posix_line[1]
if time_type == 'bsd_brief':
bsd_brief_line = line.split()
raw_output['real_time'] = bsd_brief_line[0]
raw_output['user_time'] = bsd_brief_line[2]
raw_output['system_time'] = bsd_brief_line[4]
if time_type == 'bsd_long':
bsd_long_line = line.split(maxsplit=1)
key = bsd_long_line[1].replace(' ', '_')
# fixup some key names
if key == 'average_shared_text':
key = 'average_shared_text_size'
value = bsd_long_line[0]
raw_output[key] = value
if time_type == 'linux_long':
# cleanup key names: (h:mm:ss or m:ss)
# line = line.replace('h:mm:ss', '', 1).replace('m:ss', '')
linux_long_line = line.split(': ', maxsplit=1)
key = linux_long_line[0].strip().lower().replace(' ', '_').replace('(', '').replace(')', '')\
.replace('/', '_').replace(':', '_').replace('_kbytes', '')\
.replace('_seconds', '').replace('socket_', '').replace('_bytes', '')
# fixup some key names
if key == 'file_system_inputs':
key = 'block_input_operations'
if key == 'file_system_outputs':
key = 'block_output_operations'
if key == 'percent_of_cpu_this_job_got':
key = 'cpu_percent'
if key == 'elapsed_wall_clock_time_h_mm_ss_or_m_ss':
key = 'elapsed_time'
if key == 'major_requiring_i_o_page_faults':
key = 'major_pagefaults'
if key == 'minor_reclaiming_a_frame_page_faults':
key = 'minor_pagefaults'
value = linux_long_line[1].replace('%', '')
raw_output[key] = value
if raw:
return raw_output
else:
return process(raw_output)

14
jc/parsers/timedatectl.py
View File

@ -1,5 +1,7 @@
"""jc - JSON CLI output utility `timedatectl` command output parser
The `epoch_utc` calculated timestamp field is timezone-aware and is only available if the universal_time field is available.
Usage (cli):
$ timedatectl | jc --timedatectl
@ -28,7 +30,8 @@ Examples:
"ntp_enabled": true,
"ntp_synchronized": true,
"rtc_in_local_tz": false,
"dst_active": true
"dst_active": true,
"epoch_utc": 1583888001
}
$ timedatectl | jc --timedatectl -p -r
@ -47,8 +50,8 @@ import jc.utils
class info():
version = '1.1'
description = 'timedatectl status command parser'
version = '1.2'
description = '`timedatectl status` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'
@ -76,6 +79,7 @@ def process(proc_data):
{
"local_time": string,
"universal_time": string,
"epoch_utc": integer, # timezone-aware timestamp
"rtc_time": string,
"time_zone": string,
"ntp_enabled": boolean,
@ -96,6 +100,10 @@ def process(proc_data):
except (ValueError):
proc_data[key] = None
if 'universal_time' in proc_data:
ts = jc.utils.timestamp(proc_data['universal_time'])
proc_data['epoch_utc'] = ts.utc
return proc_data

2
jc/parsers/tracepath.py
View File

@ -118,7 +118,7 @@ import jc.utils
class info():
version = '1.0'
description = 'tracepath command parser'
description = '`tracepath` and `tracepath6` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/traceroute.py
View File

@ -102,7 +102,7 @@ import jc.utils
class info():
version = '1.1'
description = 'traceroute command parser'
description = '`traceroute` and `traceroute6` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
details = 'Using the trparse library by Luis Benitez at https://github.com/lbenitez000/trparse'

2
jc/parsers/uname.py
View File

@ -38,7 +38,7 @@ import jc.utils
class info():
version = '1.4'
description = 'uname -a command parser'
description = '`uname -a` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

4
jc/parsers/universal.py
View File

@ -22,7 +22,7 @@ def simple_table_parse(data):
Returns:
dictionary raw structured data
List of Dictionaries raw structured data
"""
headers = [h for h in ' '.join(data[0].strip().split()).split() if h]
raw_data = map(lambda s: s.strip().split(None, len(headers) - 1), data[1:])
@ -52,7 +52,7 @@ def sparse_table_parse(data, delim='\u2063'):
Returns:
dictionary raw structured data
List of Dictionaries raw structured data
"""
output = []
header_text = data.pop(0)

417
jc/parsers/upower.py Normal file
View File

@ -0,0 +1,417 @@
"""jc - JSON CLI output utility `upower` command output parser
The `updated_epoch` calculated timestamp field is naive (i.e. based on the local time of the system the parser is run on)
The `updated_epoch_utc` calculated timestamp field is timezone-aware and is only available if the timezone field is UTC.
Usage (cli):
$ upower -d | jc --upower
or
$ jc upower -d
Usage (module):
import jc.parsers.upower
result = jc.parsers.upower.parse(upower_command_output)
Compatibility:
'linux'
Examples:
$ upower -i /org/freedesktop/UPower/devices/battery | jc --upower -p
[
{
"native_path": "/sys/devices/LNXSYSTM:00/device:00/PNP0C0A:00/power_supply/BAT0",
"vendor": "NOTEBOOK",
"model": "BAT",
"serial": "0001",
"power_supply": true,
"updated": "Thu 11 Mar 2021 06:28:08 PM UTC",
"has_history": true,
"has_statistics": true,
"detail": {
"type": "battery",
"present": true,
"rechargeable": true,
"state": "charging",
"energy": 22.3998,
"energy_empty": 0.0,
"energy_full": 52.6473,
"energy_full_design": 62.16,
"energy_rate": 31.6905,
"voltage": 12.191,
"time_to_full": 57.3,
"percentage": 42.5469,
"capacity": 84.6964,
"technology": "lithium-ion",
"energy_unit": "Wh",
"energy_empty_unit": "Wh",
"energy_full_unit": "Wh",
"energy_full_design_unit": "Wh",
"energy_rate_unit": "W",
"voltage_unit": "V",
"time_to_full_unit": "minutes"
},
"history_charge": [
{
"time": 1328809335,
"percent_charged": 42.547,
"status": "charging"
},
{
"time": 1328809305,
"percent_charged": 42.02,
"status": "charging"
}
],
"history_rate": [
{
"time": 1328809335,
"percent_charged": 31.691,
"status": "charging"
}
],
"updated_seconds_ago": 441975,
"updated_epoch": 1615516088,
"updated_epoch_utc": 1615487288
}
]
$ upower -i /org/freedesktop/UPower/devices/battery | jc --upower -p -r
[
{
"native_path": "/sys/devices/LNXSYSTM:00/device:00/PNP0C0A:00/power_supply/BAT0",
"vendor": "NOTEBOOK",
"model": "BAT",
"serial": "0001",
"power_supply": "yes",
"updated": "Thu 11 Mar 2021 06:28:08 PM UTC (441975 seconds ago)",
"has_history": "yes",
"has_statistics": "yes",
"detail": {
"type": "battery",
"present": "yes",
"rechargeable": "yes",
"state": "charging",
"energy": "22.3998 Wh",
"energy_empty": "0 Wh",
"energy_full": "52.6473 Wh",
"energy_full_design": "62.16 Wh",
"energy_rate": "31.6905 W",
"voltage": "12.191 V",
"time_to_full": "57.3 minutes",
"percentage": "42.5469%",
"capacity": "84.6964%",
"technology": "lithium-ion"
},
"history_charge": [
{
"time": "1328809335",
"percent_charged": "42.547",
"status": "charging"
},
{
"time": "1328809305",
"percent_charged": "42.020",
"status": "charging"
}
],
"history_rate": [
{
"time": "1328809335",
"percent_charged": "31.691",
"status": "charging"
}
]
}
]
"""
import jc.utils
class info():
version = '1.0'
description = '`upower` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'
# compatible options: linux, darwin, cygwin, win32, aix, freebsd
compatible = ['linux']
magic_commands = ['upower']
__version__ = info.version
def process(proc_data):
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured data with the following schema:
[
{
"type": string,
"device_name": string,
"native_path": string,
"power_supply": boolean,
"updated": string,
"updated_epoch": integer, # null if date-time conversion fails
"updated_epoch_utc": integer, # null if date-time conversion fails
"updated_seconds_ago": integer,
"has_history": boolean,
"has_statistics": boolean,
"detail": {
"type": string,
"warning_level": string, # null if none
"online": boolean,
"icon_name": string
"present": boolean,
"rechargeable": boolean,
"state": string,
"energy": float,
"energy_unit": string,
"energy_empty": float,
"energy_empty_unit": string,
"energy_full": float,
"energy_full_unit": string,
"energy_full_design": float,
"energy_full_design_unit": string,
"energy_rate": float,
"energy_rate_unit": string,
"voltage": float,
"voltage_unit": string,
"time_to_full": float,
"time_to_full_unit": string,
"percentage": float,
"capacity": float,
"technology": string
},
"history_charge": [
{
"time": integer,
"percent_charged": float,
"status": string
}
],
"history_rate":[
{
"time": integer,
"percent_charged": float,
"status": string
}
],
"daemon_version": string,
"on_battery": boolean,
"lid_is_closed": boolean,
"lid_is_present": boolean,
"critical_action": string
}
]
"""
for entry in proc_data:
# time conversions
if 'updated' in entry:
updated_list = entry['updated'].replace('(', '').replace(')', '').split()
entry['updated'] = ' '.join(updated_list[:-3])
entry['updated_seconds_ago'] = int(updated_list[-3])
if entry['updated']:
ts = jc.utils.timestamp(entry['updated'])
entry['updated_epoch'] = ts.naive
entry['updated_epoch_utc'] = ts.utc
# top level boolean conversions
bool_list = ['power_supply', 'has_history', 'has_statistics', 'on_battery', 'lid_is_closed', 'lid_is_present']
for key in entry:
if key in bool_list:
if entry[key].lower() == 'yes':
entry[key] = True
else:
entry[key] = False
# detail level boolean conversions
bool_list = ['online', 'present', 'rechargeable']
if 'detail' in entry:
for key in entry['detail']:
if key in bool_list:
if entry['detail'][key].lower() == 'yes':
entry['detail'][key] = True
else:
entry['detail'][key] = False
# detail level convert warning to null if value is none
if 'detail' in entry:
if 'warning_level' in entry['detail']:
if entry['detail']['warning_level'] == 'none':
entry['detail']['warning_level'] = None
# detail level convert energy readings to float and add unit keys
if 'detail' in entry:
add_items = []
for key, value in entry['detail'].items():
if value and isinstance(value, str):
if len(value.split()) == 2 and value.split()[0].replace('.', '').isnumeric():
entry['detail'][key] = float(value.split()[0])
add_items.append({
key + '_unit': value.split()[1]
})
if add_items:
for item in add_items:
for key, value in item.items():
entry['detail'][key] = value
# detail level fix percentages
if 'detail' in entry:
for key, value in entry['detail'].items():
if value and isinstance(value, str):
if value[-1] == '%':
entry['detail'][key] = float(value[:-1])
# detail level fix quoted values
if 'detail' in entry:
for key, value in entry['detail'].items():
if value and isinstance(value, str):
if value.startswith("'") and value.endswith("'"):
entry['detail'][key] = value[1:-1]
# history_charge and history_rate level convert floats and ints
histories = []
if 'history_charge' in entry:
histories.append('history_charge')
if 'history_rate' in entry:
histories.append('history_rate')
if histories:
for history_obj_list in histories:
new_history_list = []
for history_obj in entry[history_obj_list]:
new_history_obj = {}
for key, value in history_obj.items():
if key == 'time':
new_history_obj[key] = int(value)
elif key == 'percent_charged':
new_history_obj[key] = float(value)
else:
new_history_obj[key] = value
new_history_list.append(new_history_obj)
entry[history_obj_list] = new_history_list
return proc_data
def parse(data, raw=False, quiet=False):
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
if not quiet:
jc.utils.compatibility(__name__, info.compatible)
raw_output = []
device_obj = {}
device_name = None
history_key = ''
history_list = []
history_list_obj = {}
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
if line.startswith('Device:') or line.startswith('Daemon:'):
if device_obj:
raw_output.append(device_obj)
device_obj = {}
if line.startswith('Device:'):
device_name = line.split(':', maxsplit=1)[1].strip()
device_obj = {
'type': 'Device',
"device_name": device_name
}
elif line.startswith('Daemon:'):
device_obj = {
'type': 'Daemon'
}
continue
# history detail lines
if line.startswith(' ') and ':' not in line:
line_list = line.strip().split()
history_list_obj = {
'time': line_list[0],
'percent_charged': line_list[1],
'status': line_list[2]
}
history_list.append(history_list_obj)
device_obj[history_key] = history_list
continue
# general detail lines
if line.startswith(' ') and ':' in line:
key = line.split(':', maxsplit=1)[0].strip().lower().replace('-', '_').replace(' ', '_').replace('(', '').replace(')', '')
val = line.split(':', maxsplit=1)[1].strip()
device_obj['detail'][key] = val
continue
# history detail lines are a special case of detail lines
# set the history detail key
if line.startswith(' History (charge):') or line.startswith(' History (rate):'):
if line.startswith(' History (charge):'):
history_key = 'history_charge'
elif line.startswith(' History (rate):'):
history_key = 'history_rate'
device_obj[history_key] = {}
history_list = []
continue
# top level lines
if line.startswith(' ') and ':' in line:
key = line.split(':', maxsplit=1)[0].strip().lower().replace('-', '_').replace(' ', '_').replace('(', '').replace(')', '')
val = line.split(':', maxsplit=1)[1].strip()
device_obj[key] = val
continue
# set the general detail object
if line.startswith(' ') and ':' not in line:
detail_type = line.strip()
device_obj['detail'] = {
'type': detail_type
}
continue
if device_obj:
raw_output.append(device_obj)
if raw:
return raw_output
else:
return process(raw_output)

120
jc/parsers/uptime.py
View File

@ -21,30 +21,37 @@ Example:
$ uptime | jc --uptime -p
{
"time": "11:30:44",
"uptime": "1 day, 21:17",
"users": 1,
"load_1m": 0.01,
"load_5m": 0.04,
"load_15m": 0.05
"time": "11:35",
"uptime": "3 days, 4:03",
"users": 5,
"load_1m": 1.88,
"load_5m": 2.0,
"load_15m": 1.94,
"time_hour": 11,
"time_minute": 35,
"time_second": null,
"uptime_days": 3,
"uptime_hours": 4,
"uptime_minutes": 3,
"uptime_total_seconds": 273780
}
$ uptime | jc --uptime -p -r
{
"time": "11:31:09",
"uptime": "1 day, 21:17",
"users": "1",
"load_1m": "0.00",
"load_5m": "0.04",
"load_15m": "0.05"
"time": "11:36",
"uptime": "3 days, 4:04",
"users": "5",
"load_1m": "1.88",
"load_5m": "1.99",
"load_15m": "1.94"
}
"""
import jc.utils
class info():
version = '1.2'
description = 'uptime command parser'
version = '1.3'
description = '`uptime` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@ -69,14 +76,61 @@ def process(proc_data):
Dictionary. Structured data with the following schema:
{
"time": string,
"uptime": string,
"users": integer,
"load_1m": float,
"load_5m": float,
"load_15m": float
"time": string,
"time_hour": integer,
"time_minute": integer,
"time_second": integer, # null if not displayed
"uptime": string,
"uptime_days": integer,
"uptime_hours": integer,
"uptime_minutes": integer,
"uptime_total_seconds": integer,
"users": integer,
"load_1m": float,
"load_5m": float,
"load_15m": float
}
"""
if 'time' in proc_data:
time_list = proc_data['time'].split(':')
proc_data['time_hour'] = int(time_list[0])
proc_data['time_minute'] = int(time_list[1])
if len(time_list) == 3:
proc_data['time_second'] = int(time_list[2])
else:
proc_data['time_second'] = None
# parse the uptime field. Here are the variations:
# 0 min
# 3 mins
# 3 days, 2:54
# 2 days, 19:32
# 1 day, 29 min
# 16:59
if 'uptime' in proc_data:
uptime_days = 0
uptime_hours = 0
uptime_minutes = 0
uptime_total_seconds = 0
if 'min' in proc_data['uptime']:
uptime_minutes = int(proc_data['uptime'].split()[-2])
if ':' in proc_data['uptime']:
uptime_hours = int(proc_data['uptime'].split()[-1].split(':')[-2])
uptime_minutes = int(proc_data['uptime'].split(':')[-1])
if 'day' in proc_data['uptime']:
uptime_days = int(proc_data['uptime'].split()[0])
proc_data['uptime_days'] = uptime_days
proc_data['uptime_hours'] = uptime_hours
proc_data['uptime_minutes'] = uptime_minutes
uptime_total_seconds = (uptime_days * 86400) + (uptime_hours * 3600) + (uptime_minutes * 60)
proc_data['uptime_total_seconds'] = uptime_total_seconds
# integer conversions
int_list = ['users']
for key in int_list:
if key in proc_data:
@ -86,6 +140,7 @@ def process(proc_data):
except (ValueError):
proc_data[key] = None
# float conversions
float_list = ['load_1m', 'load_5m', 'load_15m']
for key in float_list:
if key in proc_data:
@ -119,25 +174,14 @@ def parse(data, raw=False, quiet=False):
cleandata = data.splitlines()
if jc.utils.has_data(data):
time, _, *uptime, users, _, _, _, load_1m, load_5m, load_15m = cleandata[0].split()
parsed_line = cleandata[0].split()
# allow space for odd times
while len(parsed_line) < 20:
parsed_line.insert(2, ' ')
# find first part of time
for i, word in enumerate(parsed_line[2:]):
if word != ' ':
marker = i + 2
break
raw_output['time'] = parsed_line[0]
raw_output['uptime'] = ' '.join(parsed_line[marker:13]).lstrip().rstrip(',')
raw_output['users'] = parsed_line[13]
raw_output['load_1m'] = parsed_line[17].rstrip(',')
raw_output['load_5m'] = parsed_line[18].rstrip(',')
raw_output['load_15m'] = parsed_line[19]
raw_output['time'] = time
raw_output['uptime'] = ' '.join(uptime).rstrip(',')
raw_output['users'] = users
raw_output['load_1m'] = load_1m.rstrip(',')
raw_output['load_5m'] = load_5m.rstrip(',')
raw_output['load_15m'] = load_15m
if raw:
return raw_output

2
jc/parsers/w.py
View File

@ -93,7 +93,7 @@ import jc.utils
class info():
version = '1.3'
description = 'w command parser'
description = '`w` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

2
jc/parsers/wc.py
View File

@ -47,7 +47,7 @@ import jc.utils
class info():
version = '1.0'
description = 'wc command parser'
description = '`wc` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

26
jc/parsers/who.py
View File

@ -2,6 +2,8 @@
Accepts any of the following who options (or no options): `-aTH`
The `epoch` calculated timestamp field is naive (i.e. based on the local time of the system the parser is run on)
Usage (cli):
$ who | jc --who
@ -26,7 +28,8 @@ Examples:
{
"event": "reboot",
"time": "Feb 7 23:31",
"pid": 1
"pid": 1,
"epoch": null
},
{
"user": "joeuser",
@ -34,7 +37,8 @@ Examples:
"tty": "console",
"time": "Feb 7 23:32",
"idle": "old",
"pid": 105
"pid": 105,
"epoch": null
},
{
"user": "joeuser",
@ -43,7 +47,8 @@ Examples:
"time": "Feb 13 16:44",
"idle": ".",
"pid": 51217,
"comment": "term=0 exit=0"
"comment": "term=0 exit=0",
"epoch": null
},
{
"user": "joeuser",
@ -51,7 +56,8 @@ Examples:
"tty": "ttys003",
"time": "Feb 28 08:59",
"idle": "01:36",
"pid": 41402
"pid": 41402,
"epoch": null
},
{
"user": "joeuser",
@ -60,7 +66,8 @@ Examples:
"time": "Mar 1 16:35",
"idle": ".",
"pid": 15679,
"from": "192.168.1.5"
"from": "192.168.1.5",
"epoch": null
}
]
@ -112,8 +119,8 @@ import jc.utils
class info():
version = '1.1'
description = 'who command parser'
version = '1.2'
description = '`who` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
# details = 'enter any other details here'
@ -145,6 +152,7 @@ def process(proc_data):
"writeable_tty": string,
"tty": string,
"time": string,
"epoch": integer, # naive timestamp. null if time cannot be converted
"idle": string,
"pid": integer,
"from": string,
@ -162,6 +170,10 @@ def process(proc_data):
except (ValueError):
entry[key] = None
if 'time' in entry:
ts = jc.utils.timestamp(entry['time'])
entry['epoch'] = ts.naive
return proc_data

10
jc/parsers/xml.py
View File

@ -59,12 +59,18 @@ Examples:
...
}
"""
import sys
import jc.utils
import xmltodict
# check if xml library is installed and fail gracefully if it is not
try:
import xmltodict
except Exception:
jc.utils.error_message('The xmltodict library is not installed.')
sys.exit(1)
class info():
version = '1.2'
version = '1.3'
description = 'XML file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

10
jc/parsers/yaml.py
View File

@ -71,12 +71,18 @@ Examples:
}
]
"""
import sys
import jc.utils
from ruamel.yaml import YAML
# check if yaml library is installed and fail gracefully if it is not
try:
from ruamel.yaml import YAML
except Exception:
jc.utils.error_message('The ruamel.yaml library is not installed.')
sys.exit(1)
class info():
version = '1.2'
version = '1.3'
description = 'YAML file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

180
jc/utils.py
View File

@ -1,6 +1,8 @@
"""jc - JSON CLI output utility utils"""
import textwrap
import sys
import re
import locale
from datetime import datetime, timezone
def warning_message(message):
@ -13,13 +15,11 @@ def warning_message(message):
Returns:
no return, just prints output to STDERR
None - just prints output to STDERR
"""
error_string = f'''
jc: Warning - {message}
'''
print(textwrap.dedent(error_string), file=sys.stderr)
error_string = f'jc: Warning - {message}'
print(error_string, file=sys.stderr)
def error_message(message):
@ -32,13 +32,11 @@ def error_message(message):
Returns:
no return, just prints output to STDERR
None - just prints output to STDERR
"""
error_string = f'''
jc: Error - {message}
'''
print(textwrap.dedent(error_string), file=sys.stderr)
error_string = f'jc: Error - {message}'
print(error_string, file=sys.stderr)
def compatibility(mod_name, compatible):
@ -54,7 +52,7 @@ def compatibility(mod_name, compatible):
Returns:
no return, just prints output to STDERR
None - just prints output to STDERR
"""
platform_found = False
@ -83,3 +81,161 @@ def has_data(data):
Boolean True if input string (data) contains non-whitespace characters, otherwise False
"""
return True if data and not data.isspace() else False
class timestamp:
"""
Input a date-time text string of several formats and convert to a naive or timezone-aware epoch timestamp in UTC
Parameters:
datetime_string: (str) a string representation of a date-time in several supported formats
Attributes:
string (str) the input datetime string
format (int) the format rule that was used to decode the datetime string
naive (int) timestamp based on locally configured timezone. None if conversion fails
utc (int) aware timestamp only if UTC timezone detected in datetime string. None if conversion fails
"""
def __init__(self, datetime_string):
self.string = datetime_string
dt = self._parse()
self.format = dt['format']
self.naive = dt['timestamp_naive']
self.utc = dt['timestamp_utc']
def __repr__(self):
return f'timestamp(string="{self.string}", format={self.format}, naive={self.naive}, utc={self.utc})'
def _parse(self):
"""
Input a date-time text string of several formats and convert to a naive or timezone-aware epoch timestamp in UTC
Parameters:
data: (string) a string representation of a date-time in several supported formats
Returns:
Dictionary A Dictionary of the following format:
{
"format": integer, # for debugging purposes. None if conversion fails
"timestamp_naive": integer, # timestamp based on locally configured timezone. None if conversion fails
"timestamp_utc": integer # aware timestamp only if UTC timezone detected. None if conversion fails
}
The format integer denotes which date_time format conversion succeeded.
The timestamp_naive integer is the converted date-time string to a naive epoch timestamp.
The timestamp_utc integer is the converted date-time string to an aware epoch timestamp
in the UTC timezone. If an aware conversion cannot be performed (e.g. the UTC timezone
is not found in the date-time string), then this field will be None.
If the conversion completely fails, all fields will be None.
"""
data = self.string or ''
normalized_datetime = ''
utc_tz = False
dt = None
dt_utc = None
timestamp_naive = None
timestamp_utc = None
timestamp_obj = {
'format': None,
'timestamp_naive': None,
'timestamp_utc': None
}
utc_tz = False
if 'UTC' in data:
utc_tz = True
if 'UTC+' in data or 'UTC-' in data:
if 'UTC+0000' in data or 'UTC-0000' in data:
utc_tz = True
else:
utc_tz = False
elif '+0000' in data or '-0000' in data:
utc_tz = True
formats = [
{'id': 1000, 'format': '%a %b %d %H:%M:%S %Y', 'locale': None}, # manual C locale format conversion: Tue Mar 23 16:12:11 2021 or Tue Mar 23 16:12:11 IST 2021
{'id': 1500, 'format': '%Y-%m-%d %H:%M', 'locale': None}, # en_US.UTF-8 local format (found in who cli output): 2021-03-23 00:14
{'id': 1600, 'format': '%m/%d/%Y %I:%M %p', 'locale': None}, # Windows english format (found in dir cli output): 12/07/2019 02:09 AM
{'id': 2000, 'format': '%a %d %b %Y %I:%M:%S %p %Z', 'locale': None}, # en_US.UTF-8 local format (found in upower cli output): Tue 23 Mar 2021 04:12:11 PM UTC
{'id': 3000, 'format': '%a %d %b %Y %I:%M:%S %p', 'locale': None}, # en_US.UTF-8 local format with non-UTC tz (found in upower cli output): Tue 23 Mar 2021 04:12:11 PM IST
{'id': 4000, 'format': '%A %d %B %Y %I:%M:%S %p %Z', 'locale': None}, # European-style local format (found in upower cli output): Tuesday 01 October 2019 12:50:41 PM UTC
{'id': 5000, 'format': '%A %d %B %Y %I:%M:%S %p', 'locale': None}, # European-style local format with non-UTC tz (found in upower cli output): Tuesday 01 October 2019 12:50:41 PM IST
{'id': 6000, 'format': '%a %b %d %I:%M:%S %p %Z %Y', 'locale': None}, # en_US.UTF-8 format (found in date cli): Wed Mar 24 06:16:19 PM UTC 2021
{'id': 7000, 'format': '%a %b %d %H:%M:%S %Z %Y', 'locale': None}, # C locale format (found in date cli): Wed Mar 24 11:11:30 UTC 2021
{'id': 7100, 'format': '%b %d %H:%M:%S %Y', 'locale': None}, # C locale format (found in stat cli output - osx): # Mar 29 11:49:05 2021
{'id': 7200, 'format': '%Y-%m-%d %H:%M:%S.%f %z', 'locale': None}, # C locale format (found in stat cli output - linux): 2019-08-13 18:13:43.555604315 -0400
{'id': 7300, 'format': '%a %Y-%m-%d %H:%M:%S %Z', 'locale': None}, # C locale format (found in timedatectl cli output): # Wed 2020-03-11 00:53:21 UTC
# attempt locale changes last
{'id': 8000, 'format': '%a %d %b %Y %H:%M:%S %Z', 'locale': ''}, # current locale format (found in upower cli output): # mar. 23 mars 2021 23:12:11 UTC
{'id': 8100, 'format': '%a %d %b %Y %H:%M:%S', 'locale': ''}, # current locale format with non-UTC tz (found in upower cli output): # mar. 23 mars 2021 19:12:11 EDT
{'id': 8200, 'format': '%A %d %B %Y, %H:%M:%S UTC%z', 'locale': ''}, # fr_FR.utf8 locale format (found in date cli output): vendredi 26 mars 2021, 13:26:46 (UTC+0000)
{'id': 8300, 'format': '%A %d %B %Y, %H:%M:%S', 'locale': ''}, # fr_FR.utf8 locale format with non-UTC tz (found in date cli output): vendredi 26 mars 2021, 13:26:46 (UTC-0400)
{'id': 9000, 'format': '%c', 'locale': ''} # locally configured locale format conversion: Could be anything :) this is a last-gasp attempt
]
# from https://www.timeanddate.com/time/zones/
# only removed UTC timezone and added known non-UTC offsets
tz_abbr = ['A', 'ACDT', 'ACST', 'ACT', 'ACWST', 'ADT', 'AEDT', 'AEST', 'AET', 'AFT', 'AKDT', 'AKST', 'ALMT',
'AMST', 'AMT', 'ANAST', 'ANAT', 'AQTT', 'ART', 'AST', 'AT', 'AWDT', 'AWST', 'AZOST', 'AZOT',
'AZST', 'AZT', 'AoE', 'B', 'BNT', 'BOT', 'BRST', 'BRT', 'BST', 'BTT', 'C', 'CAST', 'CAT', 'CCT',
'CDT', 'CEST', 'CET', 'CHADT', 'CHAST', 'CHOST', 'CHOT', 'CHUT', 'CIDST', 'CIST', 'CKT', 'CLST',
'CLT', 'COT', 'CST', 'CT', 'CVT', 'CXT', 'ChST', 'D', 'DAVT', 'DDUT', 'E', 'EASST', 'EAST',
'EAT', 'ECT', 'EDT', 'EEST', 'EET', 'EGST', 'EGT', 'EST', 'ET', 'F', 'FET', 'FJST', 'FJT', 'FKST',
'FKT', 'FNT', 'G', 'GALT', 'GAMT', 'GET', 'GFT', 'GILT', 'GMT', 'GST', 'GYT', 'H', 'HDT', 'HKT',
'HOVST', 'HOVT', 'HST', 'I', 'ICT', 'IDT', 'IOT', 'IRDT', 'IRKST', 'IRKT', 'IRST', 'IST', 'JST',
'K', 'KGT', 'KOST', 'KRAST', 'KRAT', 'KST', 'KUYT', 'L', 'LHDT', 'LHST', 'LINT', 'M', 'MAGST',
'MAGT', 'MART', 'MAWT', 'MDT', 'MHT', 'MMT', 'MSD', 'MSK', 'MST', 'MT', 'MUT', 'MVT', 'MYT', 'N',
'NCT', 'NDT', 'NFDT', 'NFT', 'NOVST', 'NOVT', 'NPT', 'NRT', 'NST', 'NUT', 'NZDT', 'NZST', 'O',
'OMSST', 'OMST', 'ORAT', 'P', 'PDT', 'PET', 'PETST', 'PETT', 'PGT', 'PHOT', 'PHT', 'PKT', 'PMDT',
'PMST', 'PONT', 'PST', 'PT', 'PWT', 'PYST', 'PYT', 'Q', 'QYZT', 'R', 'RET', 'ROTT', 'S', 'SAKT',
'SAMT', 'SAST', 'SBT', 'SCT', 'SGT', 'SRET', 'SRT', 'SST', 'SYOT', 'T', 'TAHT', 'TFT', 'TJT', 'TKT',
'TLT', 'TMT', 'TOST', 'TOT', 'TRT', 'TVT', 'U', 'ULAST', 'ULAT', 'UYST', 'UYT', 'UZT', 'V', 'VET',
'VLAST', 'VLAT', 'VOST', 'VUT', 'W', 'WAKT', 'WARST', 'WAST', 'WAT', 'WEST', 'WET', 'WFT', 'WGST',
'WGT', 'WIB', 'WIT', 'WITA', 'WST', 'WT', 'X', 'Y', 'YAKST', 'YAKT', 'YAPT', 'YEKST', 'YEKT', 'Z',
'UTC-1200', 'UTC-1100', 'UTC-1000', 'UTC-0930', 'UTC-0900', 'UTC-0800', 'UTC-0700', 'UTC-0600',
'UTC-0500', 'UTC-0400', 'UTC-0300', 'UTC-0230', 'UTC-0200', 'UTC-0100', 'UTC+0100', 'UTC+0200',
'UTC+0300', 'UTC+0400', 'UTC+0430', 'UTC+0500', 'UTC+0530', 'UTC+0545', 'UTC+0600', 'UTC+0630',
'UTC+0700', 'UTC+0800', 'UTC+0845', 'UTC+0900', 'UTC+1000', 'UTC+1030', 'UTC+1100', 'UTC+1200',
'UTC+1300', 'UTC+1345', 'UTC+1400']
# normalize the timezone by taking out any timezone reference, except UTC
cleandata = data.replace('(', '').replace(')', '')
normalized_datetime_list = []
for term in cleandata.split():
if term not in tz_abbr:
normalized_datetime_list.append(term)
normalized_datetime = ' '.join(normalized_datetime_list)
# normalize further by converting any greater-than 6-digit subsecond to 6-digits
p = re.compile(r'(\W\d\d:\d\d:\d\d\.\d{6})\d+\W')
normalized_datetime = p.sub(r'\g<1> ', normalized_datetime)
for fmt in formats:
try:
locale.setlocale(locale.LC_TIME, fmt['locale'])
dt = datetime.strptime(normalized_datetime, fmt['format'])
timestamp_naive = int(dt.replace(tzinfo=None).timestamp())
timestamp_obj['format'] = fmt['id']
locale.setlocale(locale.LC_TIME, None)
break
except Exception:
locale.setlocale(locale.LC_TIME, None)
continue
if dt and utc_tz:
dt_utc = dt.replace(tzinfo=timezone.utc)
timestamp_utc = int(dt_utc.timestamp())
if timestamp_naive:
timestamp_obj['timestamp_naive'] = timestamp_naive
timestamp_obj['timestamp_utc'] = timestamp_utc
return timestamp_obj

294
man/jc.1
View File

@ -1,285 +1,393 @@
.TH jc 1 2021-01-05 1.14.3 "JSON CLI output utility"
.TH jc 1 2021-04-07 1.15.0 "JSON CLI output utility"
.SH NAME
jc \- JSONifies the output of many CLI tools and file-types
.SH SYNOPSIS
COMMAND | jc PARSER [OPTIONS]
or magic syntax:
or magic syntax:
jc [OPTIONS] COMMAND
jc [OPTIONS] COMMAND
.SH DESCRIPTION
jc JSONifies the output of many CLI tools and file-types for easier parsing in scripts. jc accepts piped input from STDIN and outputs a JSON representation of the previous command's output to STDOUT. Alternatively, the "magic" syntax can be used by prepending jc to the command to be converted. Options can be passed to jc immediately before the command is given. (Note: command aliases are not supported).
jc JSONifies the output of many CLI tools and file-types for easier parsing in scripts. jc accepts piped input from \fBSTDIN\fP and outputs a JSON representation of the previous command's output to \fBSTDOUT\fP. Alternatively, the "magic" syntax can be used by prepending jc to the command to be converted. Options can be passed to jc immediately before the command is given. (Note: command aliases are not supported).
.SH OPTIONS
.B
Parsers:
.RS
.TP
.B
\fB--acpi\fP
`acpi` command parser
.TP
.B
\fB--airport\fP
airport \fB-I\fP command parser
`airport -I` command parser
.TP
.B
\fB--airport-s\fP
airport \fB-s\fP command parser
`airport -s` command parser
.TP
.B
\fB--arp\fP
arp command parser
`arp` command parser
.TP
.B
\fB--blkid\fP
blkid command parser
`blkid` command parser
.TP
.B
\fB--cksum\fP
cksum and sum command parser
`cksum` and `sum` command parser
.TP
.B
\fB--crontab\fP
crontab command and file parser
`crontab` command and file parser
.TP
.B
\fB--crontab-u\fP
crontab file parser with user support
`crontab` file parser with user support
.TP
.B
\fB--csv\fP
CSV file parser
.TP
.B
\fB--date\fP
date command parser
`date` command parser
.TP
.B
\fB--df\fP
df command parser
`df` command parser
.TP
.B
\fB--dig\fP
dig command parser
`dig` command parser
.TP
.B
\fB--dir\fP
`dir` command parser
.TP
.B
\fB--dmidecode\fP
dmidecode command parser
`dmidecode` command parser
.TP
.B
\fB--dpkg-l\fP
`dpkg -l` command parser
.TP
.B
\fB--du\fP
du command parser
`du` command parser
.TP
.B
\fB--env\fP
env and printenv command parser
`env` command parser
.TP
.B
\fB--file\fP
file command parser
`file` command parser
.TP
.B
\fB--finger\fP
`finger` command parser
.TP
.B
\fB--free\fP
free command parser
`free` command parser
.TP
.B
\fB--fstab\fP
fstab file parser
`/etc/fstab` file parser
.TP
.B
\fB--group\fP
/etc/group file parser
`/etc/group` file parser
.TP
.B
\fB--gshadow\fP
/etc/gshadow file parser
`/etc/gshadow` file parser
.TP
.B
\fB--hash\fP
hash BASH builtin command parser
`hash` command parser
.TP
.B
\fB--hashsum\fP
md5, md5sum, shasum, sha1sum, sha224sum, sha256sum, sha384sum, and sha512sum command parser
hashsum command parser (`md5sum`, `shasum`, etc.)
.TP
.B
\fB--hciconfig\fP
hciconfig command parser
`hciconfig` command parser
.TP
.B
\fB--history\fP
history command parser
`history` command parser
.TP
.B
\fB--hosts\fP
/etc/hosts file parser
`/etc/hosts` file parser
.TP
.B
\fB--id\fP
id command parser
`id` command parser
.TP
.B
\fB--ifconfig\fP
ifconfig command parser
`ifconfig` command parser
.TP
.B
\fB--ini\fP
INI file parser
.TP
.B
\fB--iptables\fP
iptables command parser
`iptables` command parser
.TP
.B
\fB--iw-scan\fP
iw dev <device> scan command parser
`iw dev [device] scan` command parser
.TP
.B
\fB--jobs\fP
jobs command parser
`jobs` command parser
.TP
.B
\fB--kv\fP
Key/Value file parser
.TP
.B
\fB--last\fP
last and lastb command parser
`last` and `lastb` command parser
.TP
.B
\fB--ls\fP
ls and vdir command parser
`ls` command parser
.TP
.B
\fB--lsblk\fP
lsblk command parser
`lsblk` command parser
.TP
.B
\fB--lsmod\fP
lsmod command parser
`lsmod` command parser
.TP
.B
\fB--lsof\fP
lsof command parser
`lsof` command parser
.TP
.B
\fB--mount\fP
mount command parser
`mount` command parser
.TP
.B
\fB--netstat\fP
netstat command parser
`netstat` command parser
.TP
.B
\fB--ntpq\fP
ntpq \fB-p\fP command parser
`ntpq -p` command parser
.TP
.B
\fB--passwd\fP
/etc/passwd file parser
`/etc/passwd` file parser
.TP
.B
\fB--ping\fP
ping command parser
`ping` and `ping6` command parser
.TP
.B
\fB--pip-list\fP
pip list command parser
`pip list` command parser
.TP
.B
\fB--pip-show\fP
pip show command parser
`pip show` command parser
.TP
.B
\fB--ps\fP
ps command parser
`ps` command parser
.TP
.B
\fB--route\fP
route command parser
`route` command parser
.TP
.B
\fB--rpm_qi\fP
`rpm -qi` command parser
.TP
.B
\fB--shadow\fP
/etc/shadow file parser
`/etc/shadow` file parser
.TP
.B
\fB--ss\fP
ss command parser
`ss` command parser
.TP
.B
\fB--stat\fP
stat command parser
`stat` command parser
.TP
.B
\fB--sysctl\fP
sysctl command parser
`sysctl` command parser
.TP
.B
\fB--systemctl\fP
systemctl command parser
`systemctl` command parser
.TP
.B
\fB--systemctl-lj\fP
systemctl list-jobs command parser
`systemctl list-jobs` command parser
.TP
.B
\fB--systemctl-ls\fP
systemctl list-sockets command parser
`systemctl list-sockets` command parser
.TP
.B
\fB--systemctl-luf\fP
systemctl list-unit-files command parser
`systemctl list-unit-files` command parser
.TP
.B
\fB--time\fP
`/usr/bin/time` command parser
.TP
.B
\fB--timedatectl\fP
timedatectl status command parser
`timedatectl status` command parser
.TP
.B
\fB--tracepath\fP
tracepath command parser
`tracepath` and `tracepath6` command parser
.TP
.B
\fB--traceroute\fP
traceroute command parser
`traceroute` and `traceroute6` command parser
.TP
.B
\fB--uname\fP
uname \fB-a\fP command parser
`uname -a` command parser
.TP
.B
\fB--upower\fP
`upower` command parser
.TP
.B
\fB--uptime\fP
uptime command parser
`uptime` command parser
.TP
.B
\fB--w\fP
w command parser
`w` command parser
.TP
.B
\fB--wc\fP
wc command parser
`wc` command parser
.TP
.B
\fB--who\fP
who command parser
`who` command parser
.TP
.B
\fB--xml\fP
XML file parser
.TP
.B
\fB--yaml\fP
YAML file parser
.RE
.PP
.B
Options:
.RS
.TP
.B
\fB-a\fP
about jc
about jc (JSON output)
.TP
.B
\fB-d\fP
debug - show traceback (\fB-dd\fP for verbose traceback)
.TP
.B
\fB-h\fP
help
.TP
.B
\fB-m\fP
monochrome output
.TP
@ -294,12 +402,60 @@ quiet - suppress warnings
.B
\fB-r\fP
raw JSON output
.RE
.PP
Example:
ls \fB-al\fP | jc \fB--ls\fP \fB-p\fP
.TP
.B
\fB-v\fP
version information
.SH ENVIRONMENT
You can specify custom colors via the \fBJC_COLORS\fP environment variable. The \fBJC_COLORS\fP environment variable takes four comma separated string values in the following format:
JC_COLORS=<keyname_color>,<keyword_color>,<number_color>,<string_color>
Where colors are: \fBblack\fP, \fBred\fP, \fBgreen\fP, \fByellow\fP, \fBblue\fP, \fBmagenta\fP, \fBcyan\fP, \fBgray\fP, \fBbrightblack\fP, \fBbrightred\fP, \fBbrightgreen\fP, \fBbrightyellow\fP, \fBbrightblue\fP, \fBbrightmagenta\fP, \fBbrightcyan\fP, \fBwhite\fP, or \fBdefault\fP
For example, to set to the default colors:
.RS
.PP
JC_COLORS=blue,brightblack,magenta,green
or
JC_COLORS=default,default,default,default
.RE
.SH CUSTOM PARSERS
Custom local parser plugins may be placed in a \fBjc/jcparsers\fP folder in your local "App data directory":
.RS
- Linux/unix: \fB$HOME/.local/share/jc/jcparsers\fP
- macOS: \fB$HOME/Library/Application Support/jc/jcparsers\fP
- Windows: \fB$LOCALAPPDATA\\jc\\jc\\jcparsers\fP
.RE
Local parser plugins are standard python module files. Use the \fBjc/parsers/foo.py\fP parser as a template and simply place a \fB.py\fP file in the \fBjcparsers\fP subfolder.
Local plugin filenames must be valid python module names, therefore must consist entirely of alphanumerics and start with a letter. Local plugins may override default plugins.
Note: The application data directory follows the XDG Base Directory Specification
.SH EXAMPLE
ls \fB-al\fP | jc \fB--ls\fP \fB-p\fP
or using the magic syntax:
.PP
jc \fB-p\fP ls \fB-al\fP
.SH AUTHOR
Kelly Brazil (kellyjonbrazil@gmail.com)
https://github.com/kellyjonbrazil/jc
.SH COPYRIGHT
Copyright (c) 2019-2021 Kelly Brazil
License: MIT License

14
mangen.py Executable file
View File

@ -0,0 +1,14 @@
#!/usr/bin/env python3
# Genereate man page from jc metadata using jinja2 templates
from datetime import date
import jc.cli
from jinja2 import Environment, FileSystemLoader
file_loader = FileSystemLoader('templates')
env = Environment(loader=file_loader)
template = env.get_template('manpage_template')
output = template.render(today=date.today(),
jc=jc.cli.about_jc())
with open('man/jc.1', 'w') as f:
f.write(output)

12
readmegen.py Executable file
View File

@ -0,0 +1,12 @@
#!/usr/bin/env python3
# Genereate README.md from jc metadata using jinja2 templates
import jc.cli
from jinja2 import Environment, FileSystemLoader
file_loader = FileSystemLoader('templates')
env = Environment(loader=file_loader)
template = env.get_template('readme_template')
output = template.render(jc=jc.cli.about_jc())
with open('README.md', 'w') as f:
f.write(output)

6
runtests.sh
View File

@ -1,9 +1,3 @@
#!/bin/bash
python3 -m unittest -v
echo
echo "Running local-only tests:"
echo
python3 -m unittest tests.localtest_last -v

Some files were not shown because too many files have changed in this diff Show More