1
0
mirror of https://github.com/kellyjonbrazil/jc.git synced 2026-04-03 17:44:07 +02:00

Compare commits

..

194 Commits

Author SHA1 Message Date
Kelly Brazil
5023e5be4c Dev v1.23.3 (#426)
* make certificate search more robust to different line endings

* use license_files instead of license_file which is deprecated

* version bump

* parsing extra options -e, -o, -p

* fix for extra opts and different field length at option -[aeop]

* test integration for extra opts -e -o -p

* formatting and use ast.literal_eval instead of eval

* doc update

* doc update

* Add a parser to parse mounted encrypted veracrypt volumes (fixes #403)

* update compatibility warning message

* netstat windows parser

* tests

* Windows route parser

* tests

* id should be a string

* add veracrypt parser and docs

* formatting

* doc update

* lsattr parser

* Update test_lsattr.py

* changed keys to lowercase

* changed info

* support missing data for stat

* doc update

* doc update

* doc update

* ensure compatibility warning prints even with no data

* improve compatibility message

* add support for dig +nsid option

* New parser: srt (#415)

* srt parser

* changed the parser to support more complex cases

* doc updates

* Adding certificate request parser (#416)

* Adding certificate request parser

* Adding the CSR type for Windows-style CSR

---------

Co-authored-by: Stg22 <stephane.for.test@gmail.com>

* doc update

* add csr tests

* Last -x (#422)

* Refactored the parser

* last -x support

* doc update

* fix for ping on linux with missing hostname

* allow less strict email decoding with a warning.

* doc update

* use explicit ascii decode with backslashreplace

* doc update

* use jc warning function instead of print for warning message

* last -x shutdown fix (#423)

* inject quiet setting into asn1crypto library

* Parse appearance and modalias lines for mouse devices (fixes #419) (#425)

The bluetoothctl device parser is implemented so that it aborts the parsing
process immediately returning what it has collected so far. This is because
the parser should work in hydrid way to support outputs comming from bluetoothctl
devices and bluetoothctl info calls.

* doc update

* doc update

---------

Co-authored-by: gerd <gerd.augstein@gmail.com>
Co-authored-by: Jake Ob <iakopap@gmail.com>
Co-authored-by: Mevaser <mevaser.rotner@gmail.com>
Co-authored-by: M.R <69431152+YeahItsMeAgain@users.noreply.github.com>
Co-authored-by: Stg22 <46686290+Stg22@users.noreply.github.com>
Co-authored-by: Stg22 <stephane.for.test@gmail.com>
2023-06-21 15:48:23 -07:00
Kelly Brazil
5527d22459 Merge pull request #398 from kellyjonbrazil/dev
Dev v1.23.2
2023-04-30 10:40:21 -07:00
Kelly Brazil
9d567c2e70 python 3.6 fix 2023-04-30 10:13:00 -07:00
Kelly Brazil
39c03a15d5 doc update 2023-04-30 09:34:22 -07:00
Kelly Brazil
5e6d2562f9 add certbot command parser 2023-04-29 16:34:23 -07:00
Kelly Brazil
7009d5a014 fix for incorrect insertion and deletion parsing 2023-04-22 14:04:00 -07:00
Kelly Brazil
42f9ddabb9 change bluetoothctl compatibility 2023-04-21 14:22:23 -07:00
Kelly Brazil
30efb5afc0 fix for multiline description 2023-04-18 11:23:49 -07:00
Kelly Brazil
786dc76c09 fix for battery not charging message 2023-04-17 18:12:57 -07:00
Kelly Brazil
fc48874a5d pytest warning fixes 2023-04-17 15:15:40 -07:00
Kelly Brazil
67164e7b23 fix mypy issues 2023-04-16 18:50:40 -07:00
Kelly Brazil
3db9774ac6 doc update 2023-04-16 11:46:21 -07:00
Kelly Brazil
03aef93d9a Merge pull request #392 from tzeikob/bluetoothctl-parser
Add parser for the bluetoothctl utility
2023-04-16 11:39:54 -07:00
Jake Ob
9b7e3de3ed Fix broken unit tests for the bluetoothctl parser 2023-04-16 20:52:29 +03:00
Jake Ob
291b6b061a Parse numeric fields to their actual numeric value 2023-04-16 20:42:48 +03:00
Jake Ob
08496533e2 Process data only if data is given 2023-04-16 20:19:15 +03:00
Jake Ob
f8dceb5046 Fix minor typo in documentation 2023-04-16 20:15:24 +03:00
Jake Ob
240ed4047f Add parser for the bluetoothctl utility 2023-04-14 18:23:45 +03:00
Kelly Brazil
0bf6f7cd7c fix incorrect variable parsing when wildcard schedule 2023-04-02 13:18:16 -07:00
Kelly Brazil
313bd86e3e doc update 2023-04-02 12:57:56 -07:00
Kelly Brazil
56259d5605 fix for dashes in name 2023-04-02 12:57:50 -07:00
Kelly Brazil
8a4885c1fe Merge pull request #384 from six3six/patch-1
Fix iwconfig re_interface regex
2023-03-31 08:40:00 -07:00
Louis Desplanche
5b06f84917 Fix iwconfig re_interface regex
Small fix of re_interface in iwconfig.py.
When the network SSID contains a "-" character, the network interface was ignored.
2023-03-31 11:15:31 +02:00
Kelly Brazil
37a1428914 Merge pull request #381 from kellyjonbrazil/dev
Dev v1.23.1
2023-03-23 20:16:11 -04:00
Kelly Brazil
bff065daf3 doc update 2023-03-23 16:41:50 -07:00
Kelly Brazil
c60d899f31 ignore non-parser-plugin python files in plugin directory 2023-03-23 16:40:13 -07:00
Kelly Brazil
fb7c390506 add lsusb tests 2023-03-23 06:43:06 -07:00
Kelly Brazil
22afb69573 doc update 2023-03-22 16:47:45 -07:00
Kelly Brazil
be51304c9c add support for CDC MBIM and CDC MBIM Extended 2023-03-22 16:43:57 -07:00
Kelly Brazil
44f83d800f doc update 2023-03-19 12:49:25 -07:00
Kelly Brazil
bf07973d90 doc update 2023-03-19 12:39:28 -07:00
Kelly Brazil
11e94b686c mypy fix 2023-03-18 15:47:19 -07:00
Kelly Brazil
c68bf674a1 add py.typed file to wheel 2023-03-18 15:43:22 -07:00
Kelly Brazil
dbbc310082 Merge pull request #378 from jwilk-forks/proc-status-umask
Make umask optional in /proc/PID/status
2023-03-18 22:37:13 +00:00
Kelly Brazil
e861f4a597 Merge pull request #379 from jwilk-forks/spelling
Fix typo
2023-03-18 22:25:21 +00:00
Kelly Brazil
049e93707c doc update 2023-03-18 15:16:02 -07:00
Kelly Brazil
164294ecb7 add integer and float conversions 2023-03-18 15:14:59 -07:00
Jakub Wilk
22ef489795 fix typo 2023-03-18 11:07:15 +01:00
Jakub Wilk
8f9d650f6c make umask optional in /proc/PID/status
Zombie processes don't have the umask field available.
2023-03-18 10:28:05 +01:00
Kelly Brazil
7e134a63bd add support for the timesync-status option 2023-03-15 06:58:31 -07:00
Kelly Brazil
9b5c25cb5b fix for rtc configured as UTC 2023-03-13 19:47:38 -07:00
Kelly Brazil
894946b207 doc update 2023-03-05 10:07:23 -08:00
Kelly Brazil
92ad2068db version bump 2023-03-05 10:06:54 -08:00
Kelly Brazil
59662a1500 fix for lines that start with tab 2023-03-05 10:04:00 -08:00
Kelly Brazil
125b88a2ca Merge pull request #371 from kellyjonbrazil/master
sync to dev
2023-03-05 17:59:09 +00:00
Kelly Brazil
79fce8c769 doc update 2023-02-27 17:37:49 -08:00
Kelly Brazil
a4a53f8b3b Merge pull request #368 from kellyjonbrazil/dev
Dev v1.23.0
2023-02-27 15:13:57 -08:00
Kelly Brazil
743e1ae90f doc update 2023-02-27 15:01:00 -08:00
Kelly Brazil
dbcff80907 add zpool tests 2023-02-27 13:44:16 -08:00
Kelly Brazil
6e10965aed doc update 2023-02-21 17:19:17 -08:00
Kelly Brazil
0c8c4a9c53 formatting 2023-02-21 17:18:24 -08:00
Kelly Brazil
7632541a1e Merge pull request #365 from tzeikob/xrandr-edid-364
Modify the xrandr parser to extract display model info from EDID #364
2023-02-21 17:04:46 -08:00
Kelly Brazil
a213ad9a85 doc update 2023-02-21 15:50:09 -08:00
Kelly Brazil
15ac5a9004 fix for crontab files with only shortcut entries 2023-02-21 15:44:55 -08:00
Jake Ob
0658668eb0 Fix broken TypedDict compatibility with older versions of python
By missing to declare the Model as a generic dictionary type we cause
the script to fail because the TypedDict is not supported by older
versions of python (e.g. 3.6, 3.7).
2023-02-21 14:43:10 +02:00
Kelly Brazil
ccef69ac37 add ubuntu16 tests 2023-02-20 12:04:45 -08:00
Kelly Brazil
64676fda2e fix for older linux style output 2023-02-20 11:56:20 -08:00
Jake Ob
6ea2d776ae Add unit tests for xrandr --properties outputs 2023-02-17 16:25:41 +02:00
Jake Ob
98ced9616c Modify xrandr parser to extract model info from EDID (#364)
This change will make the xrandr parser to work also with outputs from
`xrandr --properties`, where the option `--properties` enables xrandr
to return the EDID data of each output device. The parser is expected
to remain backward compatible for regular xrandr outputs. The extra info
that will be added to every device object is:

```json
{
  "model_name": "str",
  "product_id": "str",
  "serial_number": "str"
}
```

Because the EDID data come encoded in a hexadecimal string, we have to
decode them by vendoring the pyedid (https://github.com/jojonas/pyedid) library.

Fixes: #364
2023-02-17 16:09:50 +02:00
Kelly Brazil
ae19183803 add test for is_current fix 2023-02-13 18:17:14 -08:00
Kelly Brazil
8536514baf add test for output with extra spaces that caused infinite loops 2023-02-13 18:09:10 -08:00
Kelly Brazil
81982a9f79 doc updagte 2023-02-12 18:30:31 -08:00
Kelly Brazil
1acbb2f096 Merge pull request #361 from tzeikob/reflect-360
Move the reflect value in its own key (xrandr) #360
2023-02-12 18:28:37 -08:00
Jake Ob
80fb4d40a5 Add unit test to assert devices in reflect mode 2023-02-09 21:22:34 +02:00
Kelly Brazil
c972dd1aac Merge pull request #362 from jpmat296/master
Improve accepted formats in proc_pid_smaps
2023-02-09 09:51:26 -08:00
Jean-Pierre Matsumoto
9dde65c25c Improve accepted formats in proc_pid_smaps 2023-02-07 14:57:15 +01:00
Jake Ob
7486b0c7cd Move the reflect value in its own key (xrandr)
Currently the reflect value of a device is included in the rotation
key as a combination of both rotation and reflection (e.g. normal X axis).
With this modification we move the reflection value to its own key in the
JSON document like so:

```json
{
  ...
  "rotation": "...",
  "reflection": "...,
  ...
}
```

Fixes: issue #360
2023-02-07 13:02:21 +02:00
Kelly Brazil
5c7a520a0b formatting 2023-02-05 13:04:32 -08:00
Kelly Brazil
193ddf71f8 fix typos 2023-02-05 09:52:59 -08:00
Kelly Brazil
c46fe9816c doc update 2023-02-05 09:49:32 -08:00
Kelly Brazil
aada5f0794 add int conversions 2023-02-04 19:04:43 -08:00
Kelly Brazil
5c7bf363a6 add schema and docs 2023-02-04 18:51:31 -08:00
Kelly Brazil
00b74be540 version bump 2023-02-04 17:37:56 -08:00
Kelly Brazil
96cb01f57a fix zpool-status for multi-line fields 2023-02-04 17:37:48 -08:00
Kelly Brazil
adf5f403ae regex fix for infinite loop? 2023-02-04 14:37:03 -08:00
Kelly Brazil
1c09c95c71 doc update 2023-02-04 11:26:41 -08:00
Kelly Brazil
64f442d743 fix for is_current? 2023-02-04 11:20:48 -08:00
Kelly Brazil
cd8d43446b add zpool-status parser 2023-02-02 15:57:15 -08:00
Kelly Brazil
7361eac1a4 formatting 2023-01-31 17:02:04 -08:00
Kelly Brazil
9d41f0a938 doc update 2023-01-31 17:00:42 -08:00
Kelly Brazil
00274c15df add process conversions 2023-01-31 16:52:56 -08:00
Kelly Brazil
d0b8a91f94 add zpool-iostat parser 2023-01-31 11:59:19 -08:00
Kelly Brazil
098e8dbef6 doc update 2023-01-31 11:59:06 -08:00
Kelly Brazil
ec29b8bbc6 add test for acpi fix for never fully discharge state 2023-01-31 09:51:23 -08:00
Kelly Brazil
1d8f83b8c6 add ver parser tests 2023-01-31 09:44:23 -08:00
Kelly Brazil
12c4419c6a doc update 2023-01-31 08:30:37 -08:00
Kelly Brazil
b134c53f33 add ssh_conf tests 2023-01-31 08:29:43 -08:00
Kelly Brazil
23ff19fdb5 add slicer tests 2023-01-31 06:48:37 -08:00
Kelly Brazil
7a43ba631b fix tests for slice info metadata 2023-01-30 08:59:54 -08:00
Kelly Brazil
2beb26f3e3 add slice info to Metadata output 2023-01-30 08:52:22 -08:00
Kelly Brazil
4b55f49e99 formatting 2023-01-30 08:11:21 -08:00
Kelly Brazil
c0239a771c clarify parser argument quote wrapping code 2023-01-27 16:28:55 -08:00
Kelly Brazil
bc4738e900 don't quote spaces in command arguments 2023-01-27 15:58:01 -08:00
Kelly Brazil
1c76caf59b formatting 2023-01-27 15:22:36 -08:00
Kelly Brazil
0679951d4a add <nobr> tag 2023-01-27 15:17:41 -08:00
Kelly Brazil
df7aee9e4b doc update 2023-01-27 14:47:08 -08:00
Kelly Brazil
a8d97a2521 add slice info to help and doc update 2023-01-27 11:41:21 -08:00
Kelly Brazil
9370b336d8 add slice to man page 2023-01-27 11:31:16 -08:00
Kelly Brazil
118f98222a change restapi link 2023-01-27 08:15:53 -08:00
Kelly Brazil
dc997821f4 ad jc-api demo link 2023-01-27 08:12:07 -08:00
Kelly Brazil
aef0fdb435 formatting 2023-01-27 08:02:07 -08:00
Kelly Brazil
21ee3c0e78 clarify slice usage 2023-01-27 08:01:32 -08:00
Kelly Brazil
d6c665f74b add both negative slice description 2023-01-26 17:06:07 -08:00
Kelly Brazil
79305a50d0 formatting 2023-01-26 16:59:15 -08:00
Kelly Brazil
e758aa41ef formatting 2023-01-26 16:49:27 -08:00
Kelly Brazil
910cb34b09 formatting 2023-01-26 16:49:09 -08:00
Kelly Brazil
976fea4eb2 Formatting 2023-01-26 16:46:54 -08:00
Kelly Brazil
81447ec9e6 formatting 2023-01-26 16:44:26 -08:00
Kelly Brazil
61b8e9f7b5 doc update 2023-01-26 16:24:32 -08:00
Kelly Brazil
8e86a04448 add slicing info 2023-01-26 16:18:54 -08:00
Kelly Brazil
f23715a783 formatting 2023-01-26 08:53:50 -08:00
Kelly Brazil
fa693a8bb1 formatting 2023-01-26 08:53:06 -08:00
Kelly Brazil
103168c25b add integer conversions 2023-01-26 08:48:59 -08:00
Kelly Brazil
ae7ed4eec5 doc update 2023-01-26 08:48:46 -08:00
Kelly Brazil
caa516db72 formatting 2023-01-25 16:59:49 -08:00
Kelly Brazil
f05c3d6113 docstring update 2023-01-25 16:57:15 -08:00
Kelly Brazil
5d872b1535 docstring update: add see also section. 2023-01-25 16:30:20 -08:00
Kelly Brazil
1559cd9f5c fixup patch numbers 2023-01-25 16:13:33 -08:00
Kelly Brazil
a5a87c7da1 add ver parser 2023-01-25 15:58:27 -08:00
Kelly Brazil
0e6cec62c1 remove unneeded slicer syntax check 2023-01-23 16:53:06 -08:00
Kelly Brazil
3bb1d89165 fix multiple slicer warnings 2023-01-23 16:50:41 -08:00
Kelly Brazil
f5f5f102fb full functioning slicer feature 2023-01-23 16:13:17 -08:00
Kelly Brazil
0c82fe7e4d add slicer functionality 2023-01-23 13:57:48 -08:00
Kelly Brazil
13ffe8a84d doc update 2023-01-22 12:44:16 -08:00
Kelly Brazil
4c6eebaa33 doc update 2023-01-22 10:37:27 -08:00
Kelly Brazil
f8fbb2dce2 use derivative of sshd_conf parser instead of paramiko 2023-01-22 10:36:29 -08:00
Kelly Brazil
dbfe682674 fix for never fully discharging state 2023-01-22 08:57:50 -08:00
Kelly Brazil
2a148d44a1 add host to objects 2023-01-21 17:00:22 -08:00
Kelly Brazil
0648d2e9e3 don't interpolate local host values 2023-01-21 16:37:27 -08:00
Kelly Brazil
ac9128fa0c add ssh-conf parser 2023-01-21 13:10:45 -08:00
Kelly Brazil
4d7a872670 version bump 2023-01-21 10:56:45 -08:00
Kelly Brazil
07f716b254 Merge pull request #355 from kellyjonbrazil/master
sync to dev
2023-01-21 10:53:03 -08:00
Kelly Brazil
b0d6a7307c Merge pull request #353 from kellyjonbrazil/dev
Dev v1.22.5
2023-01-11 14:39:13 -08:00
Kelly Brazil
53ad793ff8 doc update 2023-01-11 11:17:47 -08:00
Kelly Brazil
ea4332d8e4 add ini-dup tests 2023-01-10 16:06:05 -08:00
Kelly Brazil
5c5ff9324f update multi-line value behavior 2023-01-09 08:31:21 -08:00
Kelly Brazil
2ab6352fdb doc update 2023-01-08 19:54:32 -08:00
Kelly Brazil
56dcbaf40c doc update 2023-01-08 19:53:21 -08:00
Kelly Brazil
029b5abcac add ini-dup parser 2023-01-08 19:46:13 -08:00
Kelly Brazil
32521ac91a doc update 2023-01-08 10:55:48 -08:00
Kelly Brazil
bf88407902 remove special DEFAULT section handling 2023-01-08 10:55:44 -08:00
Kelly Brazil
63cf47db63 doc update 2023-01-06 13:37:34 -08:00
Kelly Brazil
9970866b3b Merge pull request #350 from davemq/arp_aix
AIX ARP support
2023-01-06 13:34:44 -08:00
Dave Marquardt
d61941b276 Merge branch 'dev' into arp_aix 2023-01-06 15:14:39 -06:00
Dave Marquardt
081fdb8026 Updated AIX ARP test comments 2023-01-06 15:13:03 -06:00
Dave Marquardt
f64dfbf79d Update comments about AIX ARP 2023-01-06 15:10:02 -06:00
Dave Marquardt
5fb73f4ad5 Added AIX support for ARP, along with AIX ARP test support 2023-01-06 14:53:50 -06:00
Dave Marquardt
1c16d32420 Added test data for AIX ARP 2023-01-06 14:51:09 -06:00
Kelly Brazil
e367e0d714 doc update 2023-01-06 09:28:13 -08:00
Kelly Brazil
4046649e32 toml tests and doc update 2023-01-06 09:11:00 -08:00
Kelly Brazil
77fcfe439c remove unused list 2023-01-06 08:50:03 -08:00
Kelly Brazil
71d1355419 formatting 2023-01-06 08:36:48 -08:00
Kelly Brazil
d1b270f336 formatting 2023-01-06 08:26:42 -08:00
Dave Marquardt
9190a08332 More AIX permanent ARP entry fixes 2023-01-06 10:25:04 -06:00
Dave Marquardt
3fae50e305 Handle permanent ARP entries on AIX 2023-01-06 10:21:32 -06:00
Dave Marquardt
3582497ed4 Fix interface parsing for incomplete ARP entry on AIX 2023-01-06 10:16:13 -06:00
Dave Marquardt
19a67daabf rewrite check for incomplete ARP entries 2023-01-06 10:12:50 -06:00
Dave Marquardt
02a7e5fd8a Updated ARP incomplete handling for AIX 2023-01-06 10:08:33 -06:00
Dave Marquardt
df72b16022 Start to fix AIX ARP support 2023-01-06 10:00:58 -06:00
Kelly Brazil
61bdc13810 doc update 2023-01-05 17:05:03 -08:00
Kelly Brazil
1350a34316 beautify process function 2023-01-05 16:59:22 -08:00
Kelly Brazil
7cfe68b96a beautify remove_quotes function 2023-01-05 16:55:53 -08:00
Kelly Brazil
7a93a61f54 move unsectioned k/v's to the top level 2023-01-05 15:44:21 -08:00
Kelly Brazil
362977e598 comment unused objects 2023-01-05 14:26:01 -08:00
Kelly Brazil
5cac897beb zipinfo fix for paths with spaces 2023-01-05 12:00:20 -08:00
Kelly Brazil
cad94cc6b3 formatting and doc update 2023-01-05 11:46:14 -08:00
Kelly Brazil
fe4e429e85 Merge pull request #349 from davemq/mount_aix
Add AIX mount support
2023-01-05 11:39:42 -08:00
Kelly Brazil
d83b10e2d5 doc update 2023-01-05 11:27:49 -08:00
Kelly Brazil
bd2a757fcd more refactoring 2023-01-05 11:27:00 -08:00
Dave Marquardt
358b69a4cb Merge branch 'dev' into mount_aix 2023-01-05 11:06:00 -06:00
Dave Marquardt
08821a1454 Removed unneeded imports from mount parser 2023-01-05 10:57:23 -06:00
Kelly Brazil
90277c1d87 formatting 2023-01-05 08:54:40 -08:00
Dave Marquardt
a51e702f77 Added AIX mount test 2023-01-05 10:53:02 -06:00
Dave Marquardt
466e37128b Merge branch 'dev' into mount_aix 2023-01-05 10:32:47 -06:00
Dave Marquardt
f1ea61388f Added AIX mount support 2023-01-05 10:28:01 -06:00
Kelly Brazil
72c11fda3a more refactoring 2023-01-05 08:00:18 -08:00
Kelly Brazil
d6645983ef more refactoring 2023-01-04 20:54:31 -08:00
Kelly Brazil
01e911ecdb remove more type annotations 2023-01-04 19:17:29 -08:00
Kelly Brazil
12c0b3d889 remove more type annotations 2023-01-04 19:13:12 -08:00
Kelly Brazil
c1075e40b7 remove more type hinting for python 3.6 2023-01-04 19:09:41 -08:00
Kelly Brazil
fabe3f01b7 remove type annotations for python 3.6 compatibility 2023-01-04 19:00:49 -08:00
Kelly Brazil
3a2ff61899 fix lsusb for extra hub port info and add videocontrol and videostreaming sections 2023-01-04 18:27:51 -08:00
Kelly Brazil
aeff94d272 add toml parser 2023-01-04 18:26:13 -08:00
Kelly Brazil
c94f5d83fa fix kv and ini parsers to only remove 1 quote from beginning and end 2023-01-03 13:26:52 -08:00
Kelly Brazil
b0756fa0f9 doc update 2023-01-03 13:05:51 -08:00
Kelly Brazil
6c85abd57b doc update 2023-01-03 13:04:13 -08:00
Kelly Brazil
7659ae94bd formatting 2023-01-03 12:59:10 -08:00
Kelly Brazil
e306e81e43 formatting 2023-01-03 12:57:04 -08:00
Kelly Brazil
6d5768b26b simplify code and doc update 2023-01-03 12:55:38 -08:00
Kelly Brazil
cee9f8bf32 remove duplicate code. add tests 2023-01-03 12:44:12 -08:00
Kelly Brazil
b8e2678c01 doc update 2023-01-03 12:30:23 -08:00
Kelly Brazil
b0fe96ed03 doc update 2023-01-03 12:28:26 -08:00
Kelly Brazil
bb65ec380b separate kv and ini parsers. add fake top-level section in ini files that are missing it. 2023-01-03 12:11:05 -08:00
Kelly Brazil
ae6248227b version bump 2023-01-03 11:53:57 -08:00
296 changed files with 21546 additions and 1083 deletions

View File

@@ -1,5 +1,70 @@
jc changelog
20230621 v1.23.3
- Add `lsattr` command parser
- Add `srt` file parser
- Add `veracrypt` command parser
- Add X509 Certificate Request file parser
- Enhance X509 Certificate parser to allow non-compliant email addresses with a warning
- Enhance `dig` command parser to support the `+nsid` option
- Enhance `last` and `lastb` command parser to support the `-x` option
- Enhance `route` command parser to add Windows support
- Enhnace `netstat` command parser to add Windows support
- Enhance `ss` command parser to support extended options
- Enhance the compatibility warning message
- Fix `bluetoothctl` command parser for some mouse devices
- Fix `certbot` command parser to be more robust with different line endings
- Fix `ping` command parsers for output with missing hostname
- Fix `stat` command parser for older versions that may not contain all fields
- Fix deprecated option in `setup.cfg`
20230429 v1.23.2
- Add `bluetoothctl` command parser
- Add `certbot` command parser for `certificates` and `show_account` options
- Fix `acpi` command parser for "Not charging" battery status lines
- Fix `iwconfig` command parser for SSIDs with dashes in the name
- Fix `crontab` command parsers for incorrect variable parsing in some cases
- Fix `git-log` and `git-log-s` command parsers for incorrect insertion/deletion parsing
- Fix `ufw-appinfo` command parser for parsing errors on multiline description fields
- Fix pytest warnings
20230323 v1.23.1
- Fix `zpool-status` command parser for lines that start with tab
- Fix `timedatectl` command parser when RTC set to local
- Fix to ensure `py.typed` file is included in the package wheel
- Fix `lsusb` command parser to support CDC MBIM and CDC MBIM Extended fields
- Add support for the `timesync-status` for the `timedatectl` command parser
- Fix to ignore non-parser-plugins in the parser plugin directory
20230227 v1.23.0
- Add input slicing as a `jc` command-line option
- Add `ssh` configuration file parser
- Add `ver` Version string parser
- Add `zpool iostat` command parser
- Add `zpool status` command parser
- Fix `acpi` command parser for "will never fully discharge" battery state
- Fix `crontab` and `crontab-u` command and file parsers for cases where only
shortcut schedule items exist
- Fix `ifconfig` command parser for older-style linux output
- Fix `xrandr` command parser for proper `is_current` output
- Fix `xrandr` command parser for infinite loop with some device configurations
- Add `reflection` key to `xrandr` parser schema
- Add display model info from EDID to `xrandr` parser
- Add `MPX-specific VMA` support for VM Flags in `/proc/<pid>/smaps` parser
20230111 v1.22.5
- Add TOML file parser
- Add INI with duplicate key support file parser
- Add AIX support for the `arp` command parser
- Add AIX support for the `mount` command parser
- Fix `lsusb` command parser when extra hub port status information is output
- Refactor `lsusb` command parser for more code reuse
- Fix INI file parser to include top-level values with no section header
- Fix INI file parser to not specially handle the [DEFAULT] section
- Fix INI file and Key/Value parsers to only remove one quotation mark from the
beginning and end of values.
- Update copyright dates
20221230 v1.22.4
- Add `iwconfig` command parser
- Add NeXTSTEP format support to the PLIST file parser

View File

@@ -1636,21 +1636,21 @@ cat example.ini | jc --ini -p
```
```json
{
"bitbucket.org": {
"ServeraLiveInterval": "45",
"DEFAULT": {
"ServerAliveInterval": "45",
"Compression": "yes",
"CompressionLevel": "9",
"ForwardX11": "yes",
"ForwardX11": "yes"
},
"bitbucket.org": {
"User": "hg"
},
"topsecret.server.com": {
"ServeraLiveInterval": "45",
"Compression": "yes",
"CompressionLevel": "9",
"ForwardX11": "no",
"Port": "50022"
"Port": "50022",
"ForwardX11": "no"
}
}
```
### iostat
```bash
@@ -4551,6 +4551,57 @@ cat entrust.pem | jc --x509-cert -p
}
]
```
### X.509 PEM and DER certificate request files
```bash
cat myserver.csr | jc --x509-csr -p
```
```json
[
{
"certification_request_info": {
"version": "v1",
"subject": {
"common_name": "myserver.for.example"
},
"subject_pk_info": {
"algorithm": {
"algorithm": "ec",
"parameters": "secp256r1"
},
"public_key": "04:40:33:c0:91:8f:e9:46:ea:d0:dc:d0:f9:63:2c:a4:35:1f:0f:54:c8:a9:9b:e3:9e:d4:f3:64:b8:60:cc:7f:39:75:dd:a7:61:31:02:7c:9e:89:c6:db:45:15:f2:5f:b0:65:29:0b:42:d2:6e:c2:ea:a6:23:bd:fc:65:e5:7d:4e"
},
"attributes": [
{
"type": "extension_request",
"values": [
[
{
"extn_id": "extended_key_usage",
"critical": false,
"extn_value": [
"server_auth"
]
},
{
"extn_id": "subject_alt_name",
"critical": false,
"extn_value": [
"myserver.for.example"
]
}
]
]
}
]
},
"signature_algorithm": {
"algorithm": "sha384_ecdsa",
"parameters": null
},
"signature": "30:45:02:20:77:ac:5b:51:bf:c5:f5:43:02:52:ae:66:8a:fe:95:98:98:98:a9:45:34:31:08:ff:2c:cc:92:d9:1c:70:28:74:02:21:00:97:79:7b:e7:45:18:76:cf:d7:3b:79:34:56:d2:69:b5:73:41:9b:8a:b7:ad:ec:80:23:c1:2f:64:da:e5:28:19"
}
]
```
### XML files
```bash
cat cd_catalog.xml

396
README.md
View File

@@ -3,13 +3,15 @@
> Check out the `jc` Python [package documentation](https://github.com/kellyjonbrazil/jc/tree/master/docs) for developers
> Try the `jc` [web demo](https://jc-web.onrender.com/)
> Try the `jc` [web demo](https://jc-web.onrender.com/) and [REST API](https://github.com/kellyjonbrazil/jc-restapi)
> JC is [now available](https://galaxy.ansible.com/community/general) as an
> `jc` is [now available](https://galaxy.ansible.com/community/general) as an
Ansible filter plugin in the `community.general` collection. See this
[blog post](https://blog.kellybrazil.com/2020/08/30/parsing-command-output-in-ansible-with-jc/)
for an example.
> Looking for something like `jc` but lower-level? Check out [regex2json](https://gitlab.com/tozd/regex2json).
# JC
JSON Convert
@@ -44,8 +46,8 @@ $ jc dig example.com | jq -r '.[].answer[].data'
93.184.216.34
```
`jc` can also be used as a python library. In this case the output will be
a python dictionary, a list of dictionaries, or even a
`jc` can also be used as a python library. In this case the returned value
will be a python dictionary, a list of dictionaries, or even a
[lazy iterable of dictionaries](#using-streaming-parsers-as-python-modules)
instead of JSON:
```python
@@ -133,9 +135,9 @@ on Github.
`jc` accepts piped input from `STDIN` and outputs a JSON representation of the
previous command's output to `STDOUT`.
```bash
COMMAND | jc [OPTIONS] PARSER
cat FILE | jc [OPTIONS] PARSER
echo STRING | jc [OPTIONS] PARSER
COMMAND | jc [SLICE] [OPTIONS] PARSER
cat FILE | jc [SLICE] [OPTIONS] PARSER
echo STRING | jc [SLICE] [OPTIONS] PARSER
```
Alternatively, the "magic" syntax can be used by prepending `jc` to the command
@@ -143,8 +145,8 @@ to be converted or in front of the absolute path for Proc files. Options can be
passed to `jc` immediately before the command or Proc file path is given.
(Note: command aliases and shell builtins are not supported)
```bash
jc [OPTIONS] COMMAND
jc [OPTIONS] /proc/<path-to-procfile>
jc [SLICE] [OPTIONS] COMMAND
jc [SLICE] [OPTIONS] /proc/<path-to-procfile>
```
The JSON output can be compact (default) or pretty formatted with the `-p`
@@ -154,141 +156,153 @@ option.
| Argument | Command or Filetype | Documentation |
|-------------------|---------------------------------------------------------|----------------------------------------------------------------------------|
| ` --acpi` | `acpi` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/acpi) |
| ` --airport` | `airport -I` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/airport) |
| ` --airport-s` | `airport -s` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/airport_s) |
| ` --arp` | `arp` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/arp) |
| ` --asciitable` | ASCII and Unicode table parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/asciitable) |
| ` --asciitable-m` | multi-line ASCII and Unicode table parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/asciitable_m) |
| ` --blkid` | `blkid` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/blkid) |
| ` --cbt` | `cbt` (Google Bigtable) command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/cbt) |
| ` --cef` | CEF string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/cef) |
| ` --cef-s` | CEF string streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/cef_s) |
| ` --chage` | `chage --list` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/chage) |
| ` --cksum` | `cksum` and `sum` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/cksum) |
| ` --clf` | Common and Combined Log Format file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/clf) |
| ` --clf-s` | Common and Combined Log Format file streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/clf_s) |
| ` --crontab` | `crontab` command and file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab) |
| ` --crontab-u` | `crontab` file parser with user support | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab_u) |
| ` --csv` | CSV file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/csv) |
| ` --csv-s` | CSV file streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/csv_s) |
| ` --date` | `date` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/date) |
| ` --datetime-iso` | ISO 8601 Datetime string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/datetime_iso) |
| ` --df` | `df` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/df) |
| ` --dig` | `dig` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/dig) |
| ` --dir` | `dir` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/dir) |
| ` --dmidecode` | `dmidecode` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/dmidecode) |
| ` --dpkg-l` | `dpkg -l` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/dpkg_l) |
| ` --du` | `du` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/du) |
| `--acpi` | `acpi` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/acpi) |
| `--airport` | `airport -I` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/airport) |
| `--airport-s` | `airport -s` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/airport_s) |
| `--arp` | `arp` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/arp) |
| `--asciitable` | ASCII and Unicode table parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/asciitable) |
| `--asciitable-m` | multi-line ASCII and Unicode table parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/asciitable_m) |
| `--blkid` | `blkid` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/blkid) |
| `--bluetoothctl` | `bluetoothctl` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/bluetoothctl) |
| `--cbt` | `cbt` (Google Bigtable) command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/cbt) |
| `--cef` | CEF string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/cef) |
| `--cef-s` | CEF string streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/cef_s) |
| `--certbot` | `certbot` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/certbot) |
| `--chage` | `chage --list` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/chage) |
| `--cksum` | `cksum` and `sum` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/cksum) |
| `--clf` | Common and Combined Log Format file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/clf) |
| `--clf-s` | Common and Combined Log Format file streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/clf_s) |
| `--crontab` | `crontab` command and file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab) |
| `--crontab-u` | `crontab` file parser with user support | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab_u) |
| `--csv` | CSV file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/csv) |
| `--csv-s` | CSV file streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/csv_s) |
| `--date` | `date` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/date) |
| `--datetime-iso` | ISO 8601 Datetime string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/datetime_iso) |
| `--df` | `df` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/df) |
| `--dig` | `dig` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/dig) |
| `--dir` | `dir` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/dir) |
| `--dmidecode` | `dmidecode` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/dmidecode) |
| `--dpkg-l` | `dpkg -l` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/dpkg_l) |
| `--du` | `du` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/du) |
| `--email-address` | Email Address string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/email_address) |
| ` --env` | `env` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/env) |
| ` --file` | `file` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/file) |
| ` --findmnt` | `findmnt` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/findmnt) |
| ` --finger` | `finger` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/finger) |
| ` --free` | `free` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/free) |
| ` --fstab` | `/etc/fstab` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/fstab) |
| ` --git-log` | `git log` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/git_log) |
| ` --git-log-s` | `git log` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/git_log_s) |
| `--env` | `env` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/env) |
| `--file` | `file` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/file) |
| `--findmnt` | `findmnt` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/findmnt) |
| `--finger` | `finger` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/finger) |
| `--free` | `free` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/free) |
| `--fstab` | `/etc/fstab` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/fstab) |
| `--git-log` | `git log` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/git_log) |
| `--git-log-s` | `git log` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/git_log_s) |
| `--git-ls-remote` | `git ls-remote` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/git_ls_remote) |
| ` --gpg` | `gpg --with-colons` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/gpg) |
| ` --group` | `/etc/group` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/group) |
| ` --gshadow` | `/etc/gshadow` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/gshadow) |
| ` --hash` | `hash` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/hash) |
| ` --hashsum` | hashsum command parser (`md5sum`, `shasum`, etc.) | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/hashsum) |
| ` --hciconfig` | `hciconfig` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/hciconfig) |
| ` --history` | `history` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/history) |
| ` --hosts` | `/etc/hosts` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/hosts) |
| ` --id` | `id` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/id) |
| ` --ifconfig` | `ifconfig` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ifconfig) |
| ` --ini` | INI file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ini) |
| ` --iostat` | `iostat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/iostat) |
| ` --iostat-s` | `iostat` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/iostat_s) |
| ` --ip-address` | IPv4 and IPv6 Address string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ip_address) |
| ` --iptables` | `iptables` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/iptables) |
| ` --iw-scan` | `iw dev [device] scan` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/iw_scan) |
| ` --iwconfig` | `iwconfig` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/iwconfig) |
| ` --jar-manifest` | Java MANIFEST.MF file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/jar_manifest) |
| ` --jobs` | `jobs` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/jobs) |
| ` --jwt` | JWT string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/jwt) |
| ` --kv` | Key/Value file and string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/kv) |
| ` --last` | `last` and `lastb` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/last) |
| ` --ls` | `ls` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ls) |
| ` --ls-s` | `ls` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ls_s) |
| ` --lsblk` | `lsblk` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lsblk) |
| ` --lsmod` | `lsmod` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lsmod) |
| ` --lsof` | `lsof` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lsof) |
| ` --lspci` | `lspci -mmv` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lspci) |
| ` --lsusb` | `lsusb` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lsusb) |
| ` --m3u` | M3U and M3U8 file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/m3u) |
| ` --mdadm` | `mdadm` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/mdadm) |
| ` --mount` | `mount` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/mount) |
| ` --mpstat` | `mpstat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/mpstat) |
| ` --mpstat-s` | `mpstat` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/mpstat_s) |
| ` --netstat` | `netstat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/netstat) |
| ` --nmcli` | `nmcli` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/nmcli) |
| ` --ntpq` | `ntpq -p` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ntpq) |
| ` --openvpn` | openvpn-status.log file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/openvpn) |
| ` --os-prober` | `os-prober` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/os_prober) |
| ` --passwd` | `/etc/passwd` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/passwd) |
| ` --pci-ids` | `pci.ids` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pci_ids) |
| ` --pgpass` | PostgreSQL password file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pgpass) |
| ` --pidstat` | `pidstat -H` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pidstat) |
| ` --pidstat-s` | `pidstat -H` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pidstat_s) |
| ` --ping` | `ping` and `ping6` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ping) |
| ` --ping-s` | `ping` and `ping6` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ping_s) |
| ` --pip-list` | `pip list` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_list) |
| ` --pip-show` | `pip show` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_show) |
| ` --plist` | PLIST file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/plist) |
| ` --postconf` | `postconf -M` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/postconf) |
| ` --proc` | `/proc/` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/proc) |
| ` --ps` | `ps` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ps) |
| ` --route` | `route` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/route) |
| ` --rpm-qi` | `rpm -qi` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/rpm_qi) |
| ` --rsync` | `rsync` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/rsync) |
| ` --rsync-s` | `rsync` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/rsync_s) |
| ` --semver` | Semantic Version string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/semver) |
| ` --sfdisk` | `sfdisk` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/sfdisk) |
| ` --shadow` | `/etc/shadow` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/shadow) |
| ` --ss` | `ss` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ss) |
| ` --sshd-conf` | sshd config file and `sshd -T` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/sshd_conf) |
| ` --stat` | `stat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/stat) |
| ` --stat-s` | `stat` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/stat_s) |
| ` --sysctl` | `sysctl` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/sysctl) |
| ` --syslog` | Syslog RFC 5424 string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/syslog) |
| ` --syslog-s` | Syslog RFC 5424 string streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/syslog_s) |
| ` --syslog-bsd` | Syslog RFC 3164 string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/syslog_bsd) |
| ` --syslog-bsd-s` | Syslog RFC 3164 string streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/syslog_bsd_s) |
| ` --systemctl` | `systemctl` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl) |
| ` --systemctl-lj` | `systemctl list-jobs` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_lj) |
| ` --systemctl-ls` | `systemctl list-sockets` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_ls) |
| `--gpg` | `gpg --with-colons` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/gpg) |
| `--group` | `/etc/group` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/group) |
| `--gshadow` | `/etc/gshadow` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/gshadow) |
| `--hash` | `hash` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/hash) |
| `--hashsum` | hashsum command parser (`md5sum`, `shasum`, etc.) | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/hashsum) |
| `--hciconfig` | `hciconfig` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/hciconfig) |
| `--history` | `history` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/history) |
| `--hosts` | `/etc/hosts` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/hosts) |
| `--id` | `id` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/id) |
| `--ifconfig` | `ifconfig` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ifconfig) |
| `--ini` | INI file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ini) |
| `--ini-dup` | INI with duplicate key file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ini_dup) |
| `--iostat` | `iostat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/iostat) |
| `--iostat-s` | `iostat` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/iostat_s) |
| `--ip-address` | IPv4 and IPv6 Address string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ip_address) |
| `--iptables` | `iptables` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/iptables) |
| `--iw-scan` | `iw dev [device] scan` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/iw_scan) |
| `--iwconfig` | `iwconfig` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/iwconfig) |
| `--jar-manifest` | Java MANIFEST.MF file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/jar_manifest) |
| `--jobs` | `jobs` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/jobs) |
| `--jwt` | JWT string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/jwt) |
| `--kv` | Key/Value file and string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/kv) |
| `--last` | `last` and `lastb` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/last) |
| `--ls` | `ls` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ls) |
| `--ls-s` | `ls` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ls_s) |
| `--lsattr` | `lsattr` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lsattr) |
| `--lsblk` | `lsblk` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lsblk) |
| `--lsmod` | `lsmod` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lsmod) |
| `--lsof` | `lsof` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lsof) |
| `--lspci` | `lspci -mmv` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lspci) |
| `--lsusb` | `lsusb` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lsusb) |
| `--m3u` | M3U and M3U8 file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/m3u) |
| `--mdadm` | `mdadm` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/mdadm) |
| `--mount` | `mount` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/mount) |
| `--mpstat` | `mpstat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/mpstat) |
| `--mpstat-s` | `mpstat` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/mpstat_s) |
| `--netstat` | `netstat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/netstat) |
| `--nmcli` | `nmcli` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/nmcli) |
| `--ntpq` | `ntpq -p` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ntpq) |
| `--openvpn` | openvpn-status.log file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/openvpn) |
| `--os-prober` | `os-prober` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/os_prober) |
| `--passwd` | `/etc/passwd` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/passwd) |
| `--pci-ids` | `pci.ids` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pci_ids) |
| `--pgpass` | PostgreSQL password file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pgpass) |
| `--pidstat` | `pidstat -H` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pidstat) |
| `--pidstat-s` | `pidstat -H` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pidstat_s) |
| `--ping` | `ping` and `ping6` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ping) |
| `--ping-s` | `ping` and `ping6` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ping_s) |
| `--pip-list` | `pip list` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_list) |
| `--pip-show` | `pip show` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_show) |
| `--plist` | PLIST file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/plist) |
| `--postconf` | `postconf -M` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/postconf) |
| `--proc` | `/proc/` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/proc) |
| `--ps` | `ps` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ps) |
| `--route` | `route` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/route) |
| `--rpm-qi` | `rpm -qi` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/rpm_qi) |
| `--rsync` | `rsync` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/rsync) |
| `--rsync-s` | `rsync` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/rsync_s) |
| `--semver` | Semantic Version string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/semver) |
| `--sfdisk` | `sfdisk` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/sfdisk) |
| `--shadow` | `/etc/shadow` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/shadow) |
| `--srt` | SRT file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/srt) |
| `--ss` | `ss` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ss) |
| `--ssh-conf` | `ssh` config file and `ssh -G` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ssh_conf) |
| `--sshd-conf` | `sshd` config file and `sshd -T` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/sshd_conf) |
| `--stat` | `stat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/stat) |
| `--stat-s` | `stat` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/stat_s) |
| `--sysctl` | `sysctl` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/sysctl) |
| `--syslog` | Syslog RFC 5424 string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/syslog) |
| `--syslog-s` | Syslog RFC 5424 string streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/syslog_s) |
| `--syslog-bsd` | Syslog RFC 3164 string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/syslog_bsd) |
| `--syslog-bsd-s` | Syslog RFC 3164 string streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/syslog_bsd_s) |
| `--systemctl` | `systemctl` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl) |
| `--systemctl-lj` | `systemctl list-jobs` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_lj) |
| `--systemctl-ls` | `systemctl list-sockets` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_ls) |
| `--systemctl-luf` | `systemctl list-unit-files` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_luf) |
| ` --systeminfo` | `systeminfo` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/systeminfo) |
| ` --time` | `/usr/bin/time` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/time) |
| ` --timedatectl` | `timedatectl status` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/timedatectl) |
| ` --timestamp` | Unix Epoch Timestamp string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/timestamp) |
| ` --top` | `top -b` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/top) |
| ` --top-s` | `top -b` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/top_s) |
| ` --tracepath` | `tracepath` and `tracepath6` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/tracepath) |
| ` --traceroute` | `traceroute` and `traceroute6` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/traceroute) |
| ` --udevadm` | `udevadm info` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/udevadm) |
| ` --ufw` | `ufw status` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ufw) |
| ` --ufw-appinfo` | `ufw app info [application]` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ufw_appinfo) |
| ` --uname` | `uname -a` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/uname) |
| `--systeminfo` | `systeminfo` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/systeminfo) |
| `--time` | `/usr/bin/time` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/time) |
| `--timedatectl` | `timedatectl status` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/timedatectl) |
| `--timestamp` | Unix Epoch Timestamp string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/timestamp) |
| `--toml` | TOML file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/toml) |
| `--top` | `top -b` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/top) |
| `--top-s` | `top -b` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/top_s) |
| `--tracepath` | `tracepath` and `tracepath6` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/tracepath) |
| `--traceroute` | `traceroute` and `traceroute6` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/traceroute) |
| `--udevadm` | `udevadm info` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/udevadm) |
| `--ufw` | `ufw status` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ufw) |
| `--ufw-appinfo` | `ufw app info [application]` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ufw_appinfo) |
| `--uname` | `uname -a` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/uname) |
| `--update-alt-gs` | `update-alternatives --get-selections` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/update_alt_gs) |
| ` --update-alt-q` | `update-alternatives --query` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/update_alt_q) |
| ` --upower` | `upower` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/upower) |
| ` --uptime` | `uptime` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/uptime) |
| ` --url` | URL string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/url) |
| ` --vmstat` | `vmstat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/vmstat) |
| ` --vmstat-s` | `vmstat` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/vmstat_s) |
| ` --w` | `w` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/w) |
| ` --wc` | `wc` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/wc) |
| ` --who` | `who` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/who) |
| ` --x509-cert` | X.509 PEM and DER certificate file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/x509_cert) |
| ` --xml` | XML file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/xml) |
| ` --xrandr` | `xrandr` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/xrandr) |
| ` --yaml` | YAML file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/yaml) |
| ` --zipinfo` | `zipinfo` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/zipinfo) |
| `--update-alt-q` | `update-alternatives --query` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/update_alt_q) |
| `--upower` | `upower` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/upower) |
| `--uptime` | `uptime` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/uptime) |
| `--url` | URL string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/url) |
| `--ver` | Version string parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ver) |
| `--veracrypt` | `veracrypt` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/veracrypt) |
| `--vmstat` | `vmstat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/vmstat) |
| `--vmstat-s` | `vmstat` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/vmstat_s) |
| `--w` | `w` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/w) |
| `--wc` | `wc` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/wc) |
| `--who` | `who` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/who) |
| `--x509-cert` | X.509 PEM and DER certificate file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/x509_cert) |
| `--x509-csr` | X.509 PEM and DER certificate request file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/x509_csr) |
| `--xml` | XML file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/xml) |
| `--xrandr` | `xrandr` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/xrandr) |
| `--yaml` | YAML file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/yaml) |
| `--zipinfo` | `zipinfo` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/zipinfo) |
| `--zpool-iostat` | `zpool iostat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/zpool_iostat) |
| `--zpool-status` | `zpool status` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/zpool_status) |
### Options
@@ -309,6 +323,54 @@ option.
| `-B` | `--bash-comp` | Generate Bash shell completion script ([more info](https://github.com/kellyjonbrazil/jc/wiki/Shell-Completions)) |
| `-Z` | `--zsh-comp` | Generate Zsh shell completion script ([more info](https://github.com/kellyjonbrazil/jc/wiki/Shell-Completions)) |
### Slice
Line slicing is supported using the `START:STOP` syntax similar to Python
slicing. This allows you to skip lines at the beginning and/or end of the
`STDIN` input you would like `jc` to convert.
`START` and `STOP` can be positive or negative integers or blank and allow
you to specify how many lines to skip and how many lines to process.
Positive and blank slices are the most memory efficient. Any negative
integers in the slice will use more memory.
For example, to skip the first and last line of the following text, you
could express the slice in a couple ways:
```bash
$ cat table.txt
### We want to skip this header ###
col1 col2
foo 1
bar 2
### We want to skip this footer ###
$ cat table.txt | jc 1:-1 --asciitable
[{"col1":"foo","col2":"1"},{"col1":"bar","col2":"2"}]
$ cat table.txt | jc 1:4 --asciitable
[{"col1":"foo","col2":"1"},{"col1":"bar","col2":"2"}]
```
In this example `1:-1` and `1:4` line slices provide the same output.
When using positive integers the index location of `STOP` is non-inclusive.
Positive slices count from the first line of the input toward the end
starting at `0` as the first line. Negative slices count from the last line
toward the beginning starting at `-1` as the last line. This is also the way
[Python's slicing](https://stackoverflow.com/questions/509211/understanding-slicing)
feature works.
Here is a breakdown of line slice options:
| Slice Notation | Input Lines Processed |
|----------------|--------------------------------------------------------------|
| `START:STOP` | lines `START` through `STOP - 1` |
| `START:` | lines `START` through the rest of the output |
| `:STOP` | lines from the beginning through `STOP - 1` |
| `-START:STOP` | `START` lines from the end through `STOP - 1` |
| `START:-STOP` | lines `START` through `STOP` lines from the end |
| `-START:-STOP` | `START` lines from the end through `STOP` lines from the end |
| `-START:` | `START` lines from the end through the rest of the output |
| `:-STOP` | lines from the beginning through `STOP` lines from the end |
| `:` | all lines |
### Exit Codes
Any fatal errors within `jc` will generate an exit code of `100`, otherwise the
exit code will be `0`.
@@ -479,20 +541,22 @@ for item in result:
print(item["filename"])
```
### Custom Parsers
Custom local parser plugins may be placed in a `jc/jcparsers` folder in your
local **"App data directory"**:
### Parser Plugins
Parser plugins may be placed in a `jc/jcparsers` folder in your local
**"App data directory"**:
- Linux/unix: `$HOME/.local/share/jc/jcparsers`
- macOS: `$HOME/Library/Application Support/jc/jcparsers`
- Windows: `$LOCALAPPDATA\jc\jc\jcparsers`
Local parser plugins are standard python module files. Use the
Parser plugins are standard python module files. Use the
[`jc/parsers/foo.py`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py)
or [`jc/parsers/foo_s.py (streaming)`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo_s.py)
parser as a template and simply place a `.py` file in the `jcparsers` subfolder.
Any dependencies can be placed in the `jc` folder above `jcparsers` and can
be imported in the parser code.
Local plugin filenames must be valid python module names and therefore must
Parser plugin filenames must be valid python module names and therefore must
start with a letter and consist entirely of alphanumerics and underscores.
Local plugins may override default parsers.
@@ -550,7 +614,7 @@ they are run on an unsupported platform. To see all parser information,
including compatibility, run `jc -ap`.
You may still use a parser on an unsupported platform - for example, you may
want to parse a file with linux `lsof` output on an macOS or Windows laptop. In
want to parse a file with linux `lsof` output on a macOS or Windows laptop. In
that case you can suppress the warning message with the `-q` cli option or the
`quiet=True` function parameter in `parse()`:
@@ -756,37 +820,31 @@ ifconfig | jc -p --ifconfig # or: jc -p ifconfig
cat example.ini
```
```
[DEFAULT]
ServerAliveInterval = 45
Compression = yes
CompressionLevel = 9
ForwardX11 = yes
foo = fiz
bar = buz
[bitbucket.org]
User = hg
[section1]
fruit = apple
color = blue
[topsecret.server.com]
Port = 50022
ForwardX11 = no
[section2]
fruit = pear
color = green
```
```bash
cat example.ini | jc -p --ini
```
```json
{
"bitbucket.org": {
"ServeraLiveInterval": "45",
"Compression": "yes",
"CompressionLevel": "9",
"ForwardX11": "yes",
"User": "hg"
"foo": "fiz",
"bar": "buz",
"section1": {
"fruit": "apple",
"color": "blue"
},
"topsecret.server.com": {
"ServeraLiveInterval": "45",
"Compression": "yes",
"CompressionLevel": "9",
"ForwardX11": "no",
"Port": "50022"
"section2": {
"fruit": "pear",
"color": "green"
}
}
```
@@ -1242,4 +1300,4 @@ cat istio.yaml | jc -p --yaml
]
```
© 2019-2022 Kelly Brazil
© 2019-2023 Kelly Brazil

View File

@@ -3,8 +3,8 @@ _jc()
local cur prev words cword jc_commands jc_parsers jc_options \
jc_about_options jc_about_mod_options jc_help_options jc_special_options
jc_commands=(acpi airport arp blkid cbt chage cksum crontab date df dig dmidecode dpkg du env file findmnt finger free git gpg hciconfig id ifconfig iostat iptables iw iwconfig jobs last lastb ls lsblk lsmod lsof lspci lsusb md5 md5sum mdadm mount mpstat netstat nmcli ntpq os-prober pidstat ping ping6 pip pip3 postconf printenv ps route rpm rsync sfdisk sha1sum sha224sum sha256sum sha384sum sha512sum shasum ss sshd stat sum sysctl systemctl systeminfo timedatectl top tracepath tracepath6 traceroute traceroute6 udevadm ufw uname update-alternatives upower uptime vdir vmstat w wc who xrandr zipinfo)
jc_parsers=(--acpi --airport --airport-s --arp --asciitable --asciitable-m --blkid --cbt --cef --cef-s --chage --cksum --clf --clf-s --crontab --crontab-u --csv --csv-s --date --datetime-iso --df --dig --dir --dmidecode --dpkg-l --du --email-address --env --file --findmnt --finger --free --fstab --git-log --git-log-s --git-ls-remote --gpg --group --gshadow --hash --hashsum --hciconfig --history --hosts --id --ifconfig --ini --iostat --iostat-s --ip-address --iptables --iw-scan --iwconfig --jar-manifest --jobs --jwt --kv --last --ls --ls-s --lsblk --lsmod --lsof --lspci --lsusb --m3u --mdadm --mount --mpstat --mpstat-s --netstat --nmcli --ntpq --openvpn --os-prober --passwd --pci-ids --pgpass --pidstat --pidstat-s --ping --ping-s --pip-list --pip-show --plist --postconf --proc --proc-buddyinfo --proc-consoles --proc-cpuinfo --proc-crypto --proc-devices --proc-diskstats --proc-filesystems --proc-interrupts --proc-iomem --proc-ioports --proc-loadavg --proc-locks --proc-meminfo --proc-modules --proc-mtrr --proc-pagetypeinfo --proc-partitions --proc-slabinfo --proc-softirqs --proc-stat --proc-swaps --proc-uptime --proc-version --proc-vmallocinfo --proc-vmstat --proc-zoneinfo --proc-driver-rtc --proc-net-arp --proc-net-dev --proc-net-dev-mcast --proc-net-if-inet6 --proc-net-igmp --proc-net-igmp6 --proc-net-ipv6-route --proc-net-netlink --proc-net-netstat --proc-net-packet --proc-net-protocols --proc-net-route --proc-net-unix --proc-pid-fdinfo --proc-pid-io --proc-pid-maps --proc-pid-mountinfo --proc-pid-numa-maps --proc-pid-smaps --proc-pid-stat --proc-pid-statm --proc-pid-status --ps --route --rpm-qi --rsync --rsync-s --semver --sfdisk --shadow --ss --sshd-conf --stat --stat-s --sysctl --syslog --syslog-s --syslog-bsd --syslog-bsd-s --systemctl --systemctl-lj --systemctl-ls --systemctl-luf --systeminfo --time --timedatectl --timestamp --top --top-s --tracepath --traceroute --udevadm --ufw --ufw-appinfo --uname --update-alt-gs --update-alt-q --upower --uptime --url --vmstat --vmstat-s --w --wc --who --x509-cert --xml --xrandr --yaml --zipinfo)
jc_commands=(acpi airport arp blkid bluetoothctl cbt certbot chage cksum crontab date df dig dmidecode dpkg du env file findmnt finger free git gpg hciconfig id ifconfig iostat iptables iw iwconfig jobs last lastb ls lsattr lsblk lsmod lsof lspci lsusb md5 md5sum mdadm mount mpstat netstat nmcli ntpq os-prober pidstat ping ping6 pip pip3 postconf printenv ps route rpm rsync sfdisk sha1sum sha224sum sha256sum sha384sum sha512sum shasum ss ssh sshd stat sum sysctl systemctl systeminfo timedatectl top tracepath tracepath6 traceroute traceroute6 udevadm ufw uname update-alternatives upower uptime vdir veracrypt vmstat w wc who xrandr zipinfo zpool)
jc_parsers=(--acpi --airport --airport-s --arp --asciitable --asciitable-m --blkid --bluetoothctl --cbt --cef --cef-s --certbot --chage --cksum --clf --clf-s --crontab --crontab-u --csv --csv-s --date --datetime-iso --df --dig --dir --dmidecode --dpkg-l --du --email-address --env --file --findmnt --finger --free --fstab --git-log --git-log-s --git-ls-remote --gpg --group --gshadow --hash --hashsum --hciconfig --history --hosts --id --ifconfig --ini --ini-dup --iostat --iostat-s --ip-address --iptables --iw-scan --iwconfig --jar-manifest --jobs --jwt --kv --last --ls --ls-s --lsattr --lsblk --lsmod --lsof --lspci --lsusb --m3u --mdadm --mount --mpstat --mpstat-s --netstat --nmcli --ntpq --openvpn --os-prober --passwd --pci-ids --pgpass --pidstat --pidstat-s --ping --ping-s --pip-list --pip-show --plist --postconf --proc --proc-buddyinfo --proc-consoles --proc-cpuinfo --proc-crypto --proc-devices --proc-diskstats --proc-filesystems --proc-interrupts --proc-iomem --proc-ioports --proc-loadavg --proc-locks --proc-meminfo --proc-modules --proc-mtrr --proc-pagetypeinfo --proc-partitions --proc-slabinfo --proc-softirqs --proc-stat --proc-swaps --proc-uptime --proc-version --proc-vmallocinfo --proc-vmstat --proc-zoneinfo --proc-driver-rtc --proc-net-arp --proc-net-dev --proc-net-dev-mcast --proc-net-if-inet6 --proc-net-igmp --proc-net-igmp6 --proc-net-ipv6-route --proc-net-netlink --proc-net-netstat --proc-net-packet --proc-net-protocols --proc-net-route --proc-net-unix --proc-pid-fdinfo --proc-pid-io --proc-pid-maps --proc-pid-mountinfo --proc-pid-numa-maps --proc-pid-smaps --proc-pid-stat --proc-pid-statm --proc-pid-status --ps --route --rpm-qi --rsync --rsync-s --semver --sfdisk --shadow --srt --ss --ssh-conf --sshd-conf --stat --stat-s --sysctl --syslog --syslog-s --syslog-bsd --syslog-bsd-s --systemctl --systemctl-lj --systemctl-ls --systemctl-luf --systeminfo --time --timedatectl --timestamp --toml --top --top-s --tracepath --traceroute --udevadm --ufw --ufw-appinfo --uname --update-alt-gs --update-alt-q --upower --uptime --url --ver --veracrypt --vmstat --vmstat-s --w --wc --who --x509-cert --x509-csr --xml --xrandr --yaml --zipinfo --zpool-iostat --zpool-status)
jc_options=(--force-color -C --debug -d --monochrome -m --meta-out -M --pretty -p --quiet -q --raw -r --unbuffer -u --yaml-out -y)
jc_about_options=(--about -a)
jc_about_mod_options=(--pretty -p --yaml-out -y --monochrome -m --force-color -C)

View File

@@ -9,13 +9,15 @@ _jc() {
jc_help_options jc_help_options_describe \
jc_special_options jc_special_options_describe
jc_commands=(acpi airport arp blkid cbt chage cksum crontab date df dig dmidecode dpkg du env file findmnt finger free git gpg hciconfig id ifconfig iostat iptables iw iwconfig jobs last lastb ls lsblk lsmod lsof lspci lsusb md5 md5sum mdadm mount mpstat netstat nmcli ntpq os-prober pidstat ping ping6 pip pip3 postconf printenv ps route rpm rsync sfdisk sha1sum sha224sum sha256sum sha384sum sha512sum shasum ss sshd stat sum sysctl systemctl systeminfo timedatectl top tracepath tracepath6 traceroute traceroute6 udevadm ufw uname update-alternatives upower uptime vdir vmstat w wc who xrandr zipinfo)
jc_commands=(acpi airport arp blkid bluetoothctl cbt certbot chage cksum crontab date df dig dmidecode dpkg du env file findmnt finger free git gpg hciconfig id ifconfig iostat iptables iw iwconfig jobs last lastb ls lsattr lsblk lsmod lsof lspci lsusb md5 md5sum mdadm mount mpstat netstat nmcli ntpq os-prober pidstat ping ping6 pip pip3 postconf printenv ps route rpm rsync sfdisk sha1sum sha224sum sha256sum sha384sum sha512sum shasum ss ssh sshd stat sum sysctl systemctl systeminfo timedatectl top tracepath tracepath6 traceroute traceroute6 udevadm ufw uname update-alternatives upower uptime vdir veracrypt vmstat w wc who xrandr zipinfo zpool)
jc_commands_describe=(
'acpi:run "acpi" command with magic syntax.'
'airport:run "airport" command with magic syntax.'
'arp:run "arp" command with magic syntax.'
'blkid:run "blkid" command with magic syntax.'
'bluetoothctl:run "bluetoothctl" command with magic syntax.'
'cbt:run "cbt" command with magic syntax.'
'certbot:run "certbot" command with magic syntax.'
'chage:run "chage" command with magic syntax.'
'cksum:run "cksum" command with magic syntax.'
'crontab:run "crontab" command with magic syntax.'
@@ -43,6 +45,7 @@ _jc() {
'last:run "last" command with magic syntax.'
'lastb:run "lastb" command with magic syntax.'
'ls:run "ls" command with magic syntax.'
'lsattr:run "lsattr" command with magic syntax.'
'lsblk:run "lsblk" command with magic syntax.'
'lsmod:run "lsmod" command with magic syntax.'
'lsof:run "lsof" command with magic syntax.'
@@ -76,6 +79,7 @@ _jc() {
'sha512sum:run "sha512sum" command with magic syntax.'
'shasum:run "shasum" command with magic syntax.'
'ss:run "ss" command with magic syntax.'
'ssh:run "ssh" command with magic syntax.'
'sshd:run "sshd" command with magic syntax.'
'stat:run "stat" command with magic syntax.'
'sum:run "sum" command with magic syntax.'
@@ -95,14 +99,16 @@ _jc() {
'upower:run "upower" command with magic syntax.'
'uptime:run "uptime" command with magic syntax.'
'vdir:run "vdir" command with magic syntax.'
'veracrypt:run "veracrypt" command with magic syntax.'
'vmstat:run "vmstat" command with magic syntax.'
'w:run "w" command with magic syntax.'
'wc:run "wc" command with magic syntax.'
'who:run "who" command with magic syntax.'
'xrandr:run "xrandr" command with magic syntax.'
'zipinfo:run "zipinfo" command with magic syntax.'
'zpool:run "zpool" command with magic syntax.'
)
jc_parsers=(--acpi --airport --airport-s --arp --asciitable --asciitable-m --blkid --cbt --cef --cef-s --chage --cksum --clf --clf-s --crontab --crontab-u --csv --csv-s --date --datetime-iso --df --dig --dir --dmidecode --dpkg-l --du --email-address --env --file --findmnt --finger --free --fstab --git-log --git-log-s --git-ls-remote --gpg --group --gshadow --hash --hashsum --hciconfig --history --hosts --id --ifconfig --ini --iostat --iostat-s --ip-address --iptables --iw-scan --iwconfig --jar-manifest --jobs --jwt --kv --last --ls --ls-s --lsblk --lsmod --lsof --lspci --lsusb --m3u --mdadm --mount --mpstat --mpstat-s --netstat --nmcli --ntpq --openvpn --os-prober --passwd --pci-ids --pgpass --pidstat --pidstat-s --ping --ping-s --pip-list --pip-show --plist --postconf --proc --proc-buddyinfo --proc-consoles --proc-cpuinfo --proc-crypto --proc-devices --proc-diskstats --proc-filesystems --proc-interrupts --proc-iomem --proc-ioports --proc-loadavg --proc-locks --proc-meminfo --proc-modules --proc-mtrr --proc-pagetypeinfo --proc-partitions --proc-slabinfo --proc-softirqs --proc-stat --proc-swaps --proc-uptime --proc-version --proc-vmallocinfo --proc-vmstat --proc-zoneinfo --proc-driver-rtc --proc-net-arp --proc-net-dev --proc-net-dev-mcast --proc-net-if-inet6 --proc-net-igmp --proc-net-igmp6 --proc-net-ipv6-route --proc-net-netlink --proc-net-netstat --proc-net-packet --proc-net-protocols --proc-net-route --proc-net-unix --proc-pid-fdinfo --proc-pid-io --proc-pid-maps --proc-pid-mountinfo --proc-pid-numa-maps --proc-pid-smaps --proc-pid-stat --proc-pid-statm --proc-pid-status --ps --route --rpm-qi --rsync --rsync-s --semver --sfdisk --shadow --ss --sshd-conf --stat --stat-s --sysctl --syslog --syslog-s --syslog-bsd --syslog-bsd-s --systemctl --systemctl-lj --systemctl-ls --systemctl-luf --systeminfo --time --timedatectl --timestamp --top --top-s --tracepath --traceroute --udevadm --ufw --ufw-appinfo --uname --update-alt-gs --update-alt-q --upower --uptime --url --vmstat --vmstat-s --w --wc --who --x509-cert --xml --xrandr --yaml --zipinfo)
jc_parsers=(--acpi --airport --airport-s --arp --asciitable --asciitable-m --blkid --bluetoothctl --cbt --cef --cef-s --certbot --chage --cksum --clf --clf-s --crontab --crontab-u --csv --csv-s --date --datetime-iso --df --dig --dir --dmidecode --dpkg-l --du --email-address --env --file --findmnt --finger --free --fstab --git-log --git-log-s --git-ls-remote --gpg --group --gshadow --hash --hashsum --hciconfig --history --hosts --id --ifconfig --ini --ini-dup --iostat --iostat-s --ip-address --iptables --iw-scan --iwconfig --jar-manifest --jobs --jwt --kv --last --ls --ls-s --lsattr --lsblk --lsmod --lsof --lspci --lsusb --m3u --mdadm --mount --mpstat --mpstat-s --netstat --nmcli --ntpq --openvpn --os-prober --passwd --pci-ids --pgpass --pidstat --pidstat-s --ping --ping-s --pip-list --pip-show --plist --postconf --proc --proc-buddyinfo --proc-consoles --proc-cpuinfo --proc-crypto --proc-devices --proc-diskstats --proc-filesystems --proc-interrupts --proc-iomem --proc-ioports --proc-loadavg --proc-locks --proc-meminfo --proc-modules --proc-mtrr --proc-pagetypeinfo --proc-partitions --proc-slabinfo --proc-softirqs --proc-stat --proc-swaps --proc-uptime --proc-version --proc-vmallocinfo --proc-vmstat --proc-zoneinfo --proc-driver-rtc --proc-net-arp --proc-net-dev --proc-net-dev-mcast --proc-net-if-inet6 --proc-net-igmp --proc-net-igmp6 --proc-net-ipv6-route --proc-net-netlink --proc-net-netstat --proc-net-packet --proc-net-protocols --proc-net-route --proc-net-unix --proc-pid-fdinfo --proc-pid-io --proc-pid-maps --proc-pid-mountinfo --proc-pid-numa-maps --proc-pid-smaps --proc-pid-stat --proc-pid-statm --proc-pid-status --ps --route --rpm-qi --rsync --rsync-s --semver --sfdisk --shadow --srt --ss --ssh-conf --sshd-conf --stat --stat-s --sysctl --syslog --syslog-s --syslog-bsd --syslog-bsd-s --systemctl --systemctl-lj --systemctl-ls --systemctl-luf --systeminfo --time --timedatectl --timestamp --toml --top --top-s --tracepath --traceroute --udevadm --ufw --ufw-appinfo --uname --update-alt-gs --update-alt-q --upower --uptime --url --ver --veracrypt --vmstat --vmstat-s --w --wc --who --x509-cert --x509-csr --xml --xrandr --yaml --zipinfo --zpool-iostat --zpool-status)
jc_parsers_describe=(
'--acpi:`acpi` command parser'
'--airport:`airport -I` command parser'
@@ -111,9 +117,11 @@ _jc() {
'--asciitable:ASCII and Unicode table parser'
'--asciitable-m:multi-line ASCII and Unicode table parser'
'--blkid:`blkid` command parser'
'--bluetoothctl:`bluetoothctl` command parser'
'--cbt:`cbt` (Google Bigtable) command parser'
'--cef:CEF string parser'
'--cef-s:CEF string streaming parser'
'--certbot:`certbot` command parser'
'--chage:`chage --list` command parser'
'--cksum:`cksum` and `sum` command parser'
'--clf:Common and Combined Log Format file parser'
@@ -151,6 +159,7 @@ _jc() {
'--id:`id` command parser'
'--ifconfig:`ifconfig` command parser'
'--ini:INI file parser'
'--ini-dup:INI with duplicate key file parser'
'--iostat:`iostat` command parser'
'--iostat-s:`iostat` command streaming parser'
'--ip-address:IPv4 and IPv6 Address string parser'
@@ -164,6 +173,7 @@ _jc() {
'--last:`last` and `lastb` command parser'
'--ls:`ls` command parser'
'--ls-s:`ls` command streaming parser'
'--lsattr:`lsattr` command parser'
'--lsblk:`lsblk` command parser'
'--lsmod:`lsmod` command parser'
'--lsof:`lsof` command parser'
@@ -248,8 +258,10 @@ _jc() {
'--semver:Semantic Version string parser'
'--sfdisk:`sfdisk` command parser'
'--shadow:`/etc/shadow` file parser'
'--srt:SRT file parser'
'--ss:`ss` command parser'
'--sshd-conf:sshd config file and `sshd -T` command parser'
'--ssh-conf:`ssh` config file and `ssh -G` command parser'
'--sshd-conf:`sshd` config file and `sshd -T` command parser'
'--stat:`stat` command parser'
'--stat-s:`stat` command streaming parser'
'--sysctl:`sysctl` command parser'
@@ -265,6 +277,7 @@ _jc() {
'--time:`/usr/bin/time` command parser'
'--timedatectl:`timedatectl status` command parser'
'--timestamp:Unix Epoch Timestamp string parser'
'--toml:TOML file parser'
'--top:`top -b` command parser'
'--top-s:`top -b` command streaming parser'
'--tracepath:`tracepath` and `tracepath6` command parser'
@@ -278,16 +291,21 @@ _jc() {
'--upower:`upower` command parser'
'--uptime:`uptime` command parser'
'--url:URL string parser'
'--ver:Version string parser'
'--veracrypt:`veracrypt` command parser'
'--vmstat:`vmstat` command parser'
'--vmstat-s:`vmstat` command streaming parser'
'--w:`w` command parser'
'--wc:`wc` command parser'
'--who:`who` command parser'
'--x509-cert:X.509 PEM and DER certificate file parser'
'--x509-csr:X.509 PEM and DER certificate request file parser'
'--xml:XML file parser'
'--xrandr:`xrandr` command parser'
'--yaml:YAML file parser'
'--zipinfo:`zipinfo` command parser'
'--zpool-iostat:`zpool iostat` command parser'
'--zpool-status:`zpool status` command parser'
)
jc_options=(--force-color -C --debug -d --monochrome -m --meta-out -M --pretty -p --quiet -q --raw -r --unbuffer -u --yaml-out -y)
jc_options_describe=(

View File

@@ -26,7 +26,7 @@ def parse(
data: Union[str, bytes, Iterable[str]],
quiet: bool = False,
raw: bool = False,
ignore_exceptions: bool = None,
ignore_exceptions: Optional[bool] = None,
**kwargs
) -> Union[JSONDictType, List[JSONDictType], Iterator[JSONDictType]]
```

View File

@@ -250,4 +250,4 @@ Returns:
### Parser Information
Compatibility: linux
Version 1.4 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.6 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -140,4 +140,4 @@ Returns:
### Parser Information
Compatibility: linux, aix, freebsd, darwin
Version 1.11 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.12 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -0,0 +1,133 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.bluetoothctl"></a>
# jc.parsers.bluetoothctl
jc - JSON Convert `bluetoothctl` command output parser
Supports the following `bluetoothctl` subcommands:
- `bluetoothctl list`
- `bluetoothctl show`
- `bluetoothctl show <ctrl>`
- `bluetoothctl devices`
- `bluetoothctl info <dev>`
Usage (cli):
$ bluetoothctl info <dev> | jc --bluetoothctl
or
$ jc bluetoothctl info <dev>
Usage (module):
import jc
result = jc.parse('bluetoothctl', bluetoothctl_command_output)
Schema:
Because bluetoothctl is handling two main entities, controllers and devices,
the schema is shared between them. Most of the fields are common between
a controller and a device but there might be fields corresponding to one entity.
Controller:
[
{
"name": string,
"is_default": boolean,
"is_public": boolean,
"is_random": boolean,
"address": string,
"alias": string,
"class": string,
"powered": string,
"discoverable": string,
"discoverable_timeout": string,
"pairable": string,
"modalias": string,
"discovering": string,
"uuids": array
}
]
Device:
[
{
"name": string,
"is_public": boolean,
"is_random": boolean,
"address": string,
"alias": string,
"appearance": string,
"class": string,
"icon": string,
"paired": string,
"bonded": string,
"trusted": string,
"blocked": string,
"connected": string,
"legacy_pairing": string,
"rssi": int,
"txpower": int,
"uuids": array,
"modalias": string
}
]
Examples:
$ bluetoothctl info EB:06:EF:62:B3:19 | jc --bluetoothctl -p
[
{
"address": "22:06:33:62:B3:19",
"is_public": true,
"name": "TaoTronics TT-BH336",
"alias": "TaoTronics TT-BH336",
"class": "0x00240455",
"icon": "audio-headset",
"paired": "no",
"bonded": "no",
"trusted": "no",
"blocked": "no",
"connected": "no",
"legacy_pairing": "no",
"uuids": [
"Advanced Audio Distribu.. (0000120d-0000-1000-8000-00805f9b34fb)",
"Audio Sink (0000130b-0000-1000-8000-00805f9b34fb)",
"A/V Remote Control (0000140e-0000-1000-8000-00805f9b34fb)",
"A/V Remote Control Cont.. (0000150f-0000-1000-8000-00805f9b34fb)",
"Handsfree (0000161e-0000-1000-8000-00805f9b34fb)",
"Headset (00001708-0000-1000-8000-00805f9b34fb)",
"Headset HS (00001831-0000-1000-8000-00805f9b34fb)"
],
"rssi": -52,
"txpower": 4
}
]
<a id="jc.parsers.bluetoothctl.parse"></a>
### parse
```python
def parse(data: str,
raw: bool = False,
quiet: bool = False) -> List[JSONDictType]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.1 by Jake Ob (iakopap at gmail.com)

161
docs/parsers/certbot.md Normal file
View File

@@ -0,0 +1,161 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.certbot"></a>
# jc.parsers.certbot
jc - JSON Convert `certbot` command output parser
Supports the following `certbot` commands:
- `certbot show_account`
- `certbot certificates`
Verbose options are not supported.
Usage (cli):
$ certbot show_account | jc --certbot
$ certbot certificates | jc --certbot
or
$ jc certbot show_account
$ jc certbot certificates
Usage (module):
import jc
result = jc.parse('certbot', certbot_command_output)
Schema:
{
"certificates": [
{
"name": string,
"serial_number": string,
"key_type": string,
"domains": [
string
],
"expiration_date": string,
"expiration_date_epoch": integer,
"expiration_date_epoch_utc": integer,
"expiration_date_iso": string,
"validity": string,
"certificate_path": string,
"private_key_path": string
}
],
"account": {
"server": string,
"url": string,
"email": string
}
}
Examples:
$ certbot certificates | jc --certbot -p
{
"certificates": [
{
"name": "example.com",
"serial_number": "3f7axxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"key_type": "RSA",
"domains": [
"example.com",
"www.example.com"
],
"expiration_date": "2023-05-11 01:33:10+00:00",
"validity": "63 days",
"certificate_path": "/etc/letsencrypt/live/example.com/chain.pem",
"private_key_path": "/etc/letsencrypt/live/example.com/priv.pem",
"expiration_date_epoch": 1683793990,
"expiration_date_epoch_utc": 1683768790,
"expiration_date_iso": "2023-05-11T01:33:10+00:00"
},
{
"name": "example.org",
"serial_number": "3bcyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy",
"key_type": "RSA",
"domains": [
"example.org",
"www.example.org"
],
"expiration_date": "2023-06-12 01:35:30+00:00",
"validity": "63 days",
"certificate_path": "/etc/letsencrypt/live/example.org/chain.pem",
"private_key_path": "/etc/letsencrypt/live/example.org/key.pem",
"expiration_date_epoch": 1686558930,
"expiration_date_epoch_utc": 1686533730,
"expiration_date_iso": "2023-06-12T01:35:30+00:00"
}
]
}
$ certbot certificates | jc --certbot -p -r
{
"certificates": [
{
"name": "example.com",
"serial_number": "3f7axxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"key_type": "RSA",
"domains": [
"example.com",
"www.example.com"
],
"expiration_date": "2023-05-11 01:33:10+00:00",
"validity": "63 days",
"certificate_path": "/etc/letsencrypt/live/example.com/chain.pem",
"private_key_path": "/etc/letsencrypt/live/example.com/priv.pem"
},
{
"name": "example.org",
"serial_number": "3bcyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy",
"key_type": "RSA",
"domains": [
"example.org",
"www.example.org"
],
"expiration_date": "2023-06-12 01:35:30+00:00",
"validity": "63 days",
"certificate_path": "/etc/letsencrypt/live/example.org/chain.pem",
"private_key_path": "/etc/letsencrypt/live/example.org/key.pem"
}
]
}
$ certbot show_account | jc --certbot -p
{
"account": {
"server": "https://acme-staging-v02.api.letsencrypt.org/directory",
"url": "https://acme-staging-v02.api.letsencrypt.org/acme/acct/123",
"email": "some@example.com"
}
}
<a id="jc.parsers.certbot.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> JSONDictType
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.1 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -196,4 +196,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, aix, freebsd
Version 1.6 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.8 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -193,4 +193,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, aix, freebsd
Version 1.7 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.9 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -9,6 +9,7 @@ Options supported:
- `+noall +answer` options are supported in cases where only the answer
information is desired.
- `+axfr` option is supported on its own
- `+nsid` option is supported
The `when_epoch` calculated timestamp field is naive. (i.e. based on the
local time of the system the parser is run on)
@@ -345,4 +346,4 @@ Returns:
### Parser Information
Compatibility: linux, aix, freebsd, darwin, win32, cygwin
Version 2.4 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 2.5 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -172,4 +172,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.3 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.4 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -108,4 +108,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.3 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.4 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -240,4 +240,4 @@ Returns:
### Parser Information
Compatibility: linux, aix, freebsd, darwin
Version 2.2 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 2.3 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -3,13 +3,15 @@
# jc.parsers.ini
jc - JSON Convert `INI` file parser
jc - JSON Convert INI file parser
Parses standard `INI` files and files containing simple key/value pairs.
Parses standard INI files.
- Delimiter can be `=` or `:`. Missing values are supported.
- Comment prefix can be `#` or `;`. Comments must be on their own line.
- If duplicate keys are found, only the last value will be used.
- If any section names have the same name as a top-level key, the top-level
key will be overwritten by the section data.
> Note: Values starting and ending with double or single quotation marks
> will have the marks removed. If you would like to keep the quotation
@@ -27,45 +29,47 @@ Usage (module):
Schema:
ini or key/value document converted to a dictionary - see the configparser
INI document converted to a dictionary - see the python configparser
standard library documentation for more details.
{
"key1": string,
"key2": string
"<key1>": string,
"<key2>": string,
"<section1>": {
"<key1>": string,
"<key2>": string
},
"<section2>": {
"<key1>": string,
"<key2>": string
}
}
Examples:
$ cat example.ini
[DEFAULT]
ServerAliveInterval = 45
Compression = yes
CompressionLevel = 9
ForwardX11 = yes
foo = fiz
bar = buz
[bitbucket.org]
User = hg
[section1]
fruit = apple
color = blue
[topsecret.server.com]
Port = 50022
ForwardX11 = no
[section2]
fruit = pear
color = green
$ cat example.ini | jc --ini -p
{
"bitbucket.org": {
"ServerAliveInterval": "45",
"Compression": "yes",
"CompressionLevel": "9",
"ForwardX11": "yes",
"User": "hg"
"foo": "fiz",
"bar": "buz",
"section1": {
"fruit": "apple",
"color": "blue"
},
"topsecret.server.com": {
"ServerAliveInterval": "45",
"Compression": "yes",
"CompressionLevel": "9",
"ForwardX11": "no",
"Port": "50022"
"section2": {
"fruit": "pear",
"color": "green"
}
}
@@ -87,9 +91,9 @@ Parameters:
Returns:
Dictionary representing the ini file
Dictionary representing the INI file.
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.8 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 2.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

121
docs/parsers/ini_dup.md Normal file
View File

@@ -0,0 +1,121 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.ini_dup"></a>
# jc.parsers.ini\_dup
jc - JSON Convert INI with duplicate key file parser
Parses standard INI files and preserves duplicate values. All values are
contained in lists/arrays.
- Delimiter can be `=` or `:`. Missing values are supported.
- Comment prefix can be `#` or `;`. Comments must be on their own line.
- If any section names have the same name as a top-level key, the top-level
key will be overwritten by the section data.
- If multi-line values are used, each line will be a separate item in the
value list. Blank lines in multi-line values are not supported.
> Note: Values starting and ending with double or single quotation marks
> will have the marks removed. If you would like to keep the quotation
> marks, use the `-r` command-line argument or the `raw=True` argument in
> `parse()`.
Usage (cli):
$ cat foo.ini | jc --ini
Usage (module):
import jc
result = jc.parse('ini', ini_file_output)
Schema:
INI document converted to a dictionary - see the python configparser
standard library documentation for more details.
{
"<key1>": [
string
],
"<key2>": [
string
],
"<section1>": {
"<key1>": [
string
],
"<key2>": [
string
]
}
}
Examples:
$ cat example.ini
foo = fiz
bar = buz
[section1]
fruit = apple
color = blue
color = red
[section2]
fruit = pear
fruit = peach
color = green
$ cat example.ini | jc --ini -p
{
"foo": [
"fiz"
],
"bar": [
"buz"
],
"section1": {
"fruit": [
"apple"
],
"color": [
"blue",
"red"
]
},
"section2": {
"fruit": [
"pear",
"peach"
],
"color": [
"green"
]
}
}
<a id="jc.parsers.ini_dup.parse"></a>
### parse
```python
def parse(data, raw=False, quiet=False)
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary representing the INI file.
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -51,7 +51,6 @@ Schema:
}
]
Examples:
$ iwconfig | jc --iwconfig -p
@@ -109,4 +108,4 @@ Returns:
### Parser Information
Compatibility: linux
Version 1.0 by Thomas Vincent (vrince@gmail.com)
Version 1.1 by Thomas Vincent (vrince@gmail.com)

View File

@@ -26,8 +26,8 @@ Usage (module):
Schema:
key/value document converted to a dictionary - see the configparser standard
library documentation for more details.
Key/Value document converted to a dictionary - see the python configparser
standard library documentation for more details.
{
"key1": string,
@@ -41,6 +41,7 @@ Examples:
name = John Doe
address=555 California Drive
age: 34
; comments can include # or ;
# delimiter can be = or :
# quoted values have quotation marks stripped by default
@@ -65,8 +66,6 @@ def parse(data, raw=False, quiet=False)
Main text parsing function
Note: this is just a wrapper for jc.parsers.ini
Parameters:
data: (string) text data to parse
@@ -75,9 +74,9 @@ Parameters:
Returns:
Dictionary representing the key/value file
Dictionary representing a Key/Value pair document.
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.2 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 2.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -5,7 +5,7 @@
jc - JSON Convert `last` and `lastb` command output parser
Supports `-w` and `-F` options.
Supports `-w`, `-F`, and `-x` options.
Calculated epoch time fields are naive (i.e. based on the local time of the
system the parser is run on) since there is no timezone information in the
@@ -127,4 +127,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, aix, freebsd
Version 1.8 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.9 by Kelly Brazil (kellyjonbrazil@gmail.com)

89
docs/parsers/lsattr.md Normal file
View File

@@ -0,0 +1,89 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.lsattr"></a>
# jc.parsers.lsattr
jc - JSON Convert `lsattr` command output parser
Usage (cli):
$ lsattr | jc --lsattr
or
$ jc lsattr
Usage (module):
import jc
result = jc.parse('lsattr', lsattr_command_output)
Schema:
Information from https://github.com/mirror/busybox/blob/2d4a3d9e6c1493a9520b907e07a41aca90cdfd94/e2fsprogs/e2fs_lib.c#L40
used to define field names
[
{
"file": string,
"compressed_file": Optional[boolean],
"compressed_dirty_file": Optional[boolean],
"compression_raw_access": Optional[boolean],
"secure_deletion": Optional[boolean],
"undelete": Optional[boolean],
"synchronous_updates": Optional[boolean],
"synchronous_directory_updates": Optional[boolean],
"immutable": Optional[boolean],
"append_only": Optional[boolean],
"no_dump": Optional[boolean],
"no_atime": Optional[boolean],
"compression_requested": Optional[boolean],
"encrypted": Optional[boolean],
"journaled_data": Optional[boolean],
"indexed_directory": Optional[boolean],
"no_tailmerging": Optional[boolean],
"top_of_directory_hierarchies": Optional[boolean],
"extents": Optional[boolean],
"no_cow": Optional[boolean],
"casefold": Optional[boolean],
"inline_data": Optional[boolean],
"project_hierarchy": Optional[boolean],
"verity": Optional[boolean],
}
]
Examples:
$ sudo lsattr /etc/passwd | jc --lsattr
[
{
"file": "/etc/passwd",
"extents": true
}
]
<a id="jc.parsers.lsattr.parse"></a>
### parse
```python
def parse(data: str,
raw: bool = False,
quiet: bool = False) -> List[JSONDictType]
```
Main text parsing function
Parameters:
data: (string) text data to parse
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Mark Rotner (rotner.mr@gmail.com)

View File

@@ -102,6 +102,46 @@ Schema:
]
}
},
"cdc_mbim": {
"<item>": {
"value": string,
"description": string,
"attributes": [
string
]
}
},
"cdc_mbim_extended": {
"<item>": {
"value": string,
"description": string,
"attributes": [
string
]
}
},
"videocontrol_descriptors": [
{
"<item>": {
"value": string,
"description": string,
"attributes": [
string
]
}
}
],
"videostreaming_descriptors": [
{
"<item>": {
"value": string,
"description": string,
"attributes": [
string
]
}
}
],
"endpoint_descriptors": [
{
"<item>": {
@@ -290,4 +330,4 @@ Returns:
### Parser Information
Compatibility: linux
Version 1.2 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.4 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -25,7 +25,7 @@ Schema:
"filesystem": string,
"mount_point": string,
"type": string,
"access": [
"options": [
string
]
}
@@ -39,7 +39,7 @@ Example:
"filesystem": "sysfs",
"mount_point": "/sys",
"type": "sysfs",
"access": [
"options": [
"rw",
"nosuid",
"nodev",
@@ -51,7 +51,7 @@ Example:
"filesystem": "proc",
"mount_point": "/proc",
"type": "proc",
"access": [
"options": [
"rw",
"nosuid",
"nodev",
@@ -63,7 +63,7 @@ Example:
"filesystem": "udev",
"mount_point": "/dev",
"type": "devtmpfs",
"access": [
"options": [
"rw",
"nosuid",
"relatime",
@@ -96,6 +96,6 @@ Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux, darwin, freebsd
Compatibility: linux, darwin, freebsd, aix
Version 1.7 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.8 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -376,6 +376,6 @@ Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux, darwin, freebsd
Compatibility: linux, darwin, freebsd, win32
Version 1.13 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.14 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -185,4 +185,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, freebsd
Version 1.8 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.9 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -106,4 +106,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, freebsd
Version 1.2 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.3 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -114,7 +114,8 @@ Examples:
"mw",
"me",
"dw",
"sd"
"sd",
"mp"
],
"VmFlags_pretty": [
"readable",

View File

@@ -22,6 +22,13 @@ Schema:
[
{
"interfaces": [
{
"id": string,
"mac": string,
"name": string,
}
]
"destination": string,
"gateway": string,
"genmask": string,
@@ -129,6 +136,6 @@ Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Compatibility: linux, win32
Version 1.8 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.9 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -7,6 +7,8 @@ jc - JSON Convert Semantic Version string parser
This parser conforms to the specification at https://semver.org/
See Also: `ver` parser.
Usage (cli):
$ echo 1.2.3-rc.1+44837 | jc --semver

136
docs/parsers/srt.md Normal file
View File

@@ -0,0 +1,136 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.srt"></a>
# jc.parsers.srt
jc - JSON Convert `SRT` file parser
Usage (cli):
$ cat foo.srt | jc --srt
Usage (module):
import jc
result = jc.parse('srt', srt_file_output)
Schema:
[
{
"index": int,
"start": {
"hours": int,
"minutes": int,
"seconds": int,
"milliseconds": int,
"timestamp": string
},
"end": {
"hours": int,
"minutes": int,
"seconds": int,
"milliseconds": int,
"timestamp": string
},
"content": string
}
]
Examples:
$ cat attack_of_the_clones.srt
1
00:02:16,612 --> 00:02:19,376
Senator, we're making
our final approach into Coruscant.
2
00:02:19,482 --> 00:02:21,609
Very good, Lieutenant.
...
$ cat attack_of_the_clones.srt | jc --srt
[
{
"index": 1,
"start": {
"hours": 0,
"minutes": 2,
"seconds": 16,
"milliseconds": 612,
"timestamp": "00:02:16,612"
},
"end": {
"hours": 0,
"minutes": 2,
"seconds": 19,
"milliseconds": 376,
"timestamp": "00:02:19,376"
},
"content": "Senator, we're making\nour final approach into Coruscant."
},
{
"index": 2,
"start": {
"hours": 0,
"minutes": 2,
"seconds": 19,
"milliseconds": 482,
"timestamp": "00:02:19,482"
},
"end": {
"hours": 0,
"minutes": 2,
"seconds": 21,
"milliseconds": 609,
"timestamp": "00:02:21,609"
},
"content": "Very good, Lieutenant."
},
...
]
<a id="jc.parsers.srt.parse_timestamp"></a>
### parse\_timestamp
```python
def parse_timestamp(timestamp: str) -> Dict
```
timestamp: "hours:minutes:seconds,milliseconds" --->
{
"hours": "hours",
"minutes": "minutes",
"seconds": "seconds",
"milliseconds": "milliseconds",
"timestamp": "hours:minutes:seconds,milliseconds"
}
<a id="jc.parsers.srt.parse"></a>
### parse
```python
def parse(data: str,
raw: bool = False,
quiet: bool = False) -> List[JSONDictType]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.0 by Mark Rotner (rotner.mr@gmail.com)

View File

@@ -5,9 +5,6 @@
jc - JSON Convert `ss` command output parser
Extended information options like `-e` and `-p` are not supported and may
cause parsing irregularities.
Usage (cli):
$ ss | jc --ss
@@ -28,21 +25,29 @@ field names
[
{
"netid": string,
"state": string,
"recv_q": integer,
"send_q": integer,
"local_address": string,
"local_port": string,
"local_port_num": integer,
"peer_address": string,
"peer_port": string,
"peer_port_num": integer,
"interface": string,
"link_layer" string,
"channel": string,
"path": string,
"pid": integer
"netid": string,
"state": string,
"recv_q": integer,
"send_q": integer,
"local_address": string,
"local_port": string,
"local_port_num": integer,
"peer_address": string,
"peer_port": string,
"peer_port_num": integer,
"interface": string,
"link_layer" string,
"channel": string,
"path": string,
"pid": integer,
"opts": {
"process_id": {
"<process_id>": {
"user": string,
"file_descriptor": string
}
}
}
}
]
@@ -303,4 +308,4 @@ Returns:
### Parser Information
Compatibility: linux
Version 1.6 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.7 by Kelly Brazil (kellyjonbrazil@gmail.com)

548
docs/parsers/ssh_conf.md Normal file
View File

@@ -0,0 +1,548 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.ssh_conf"></a>
# jc.parsers.ssh\_conf
jc - JSON Convert `ssh` configuration file and `ssh -G` command output parser
This parser will work with `ssh` configuration files or the output of
`ssh -G`. Any `Match` blocks in the `ssh` configuration file will be
ignored.
Usage (cli):
$ ssh -G hostname | jc --ssh-conf
or
$ jc ssh -G hostname
or
$ cat ~/.ssh/config | jc --ssh-conf
Usage (module):
import jc
result = jc.parse('ssh_conf', ssh_conf_output)
Schema:
[
{
"host": string,
"host_list": [
string
],
"addkeystoagent": string,
"addressfamily": string,
"batchmode": string,
"bindaddress": string,
"bindinterface": string,
"canonicaldomains": [
string
],
"canonicalizefallbacklocal": string,
"canonicalizehostname": string,
"canonicalizemaxdots": integer,
"canonicalizepermittedcnames": [
string
],
"casignaturealgorithms": [
string
],
"certificatefile": [
string
],
"checkhostip": string,
"ciphers": [
string
],
"clearallforwardings": string,
"compression": string,
"connectionattempts": integer,
"connecttimeout": integer,
"controlmaster": string,
"controlpath": string,
"controlpersist": string,
"dynamicforward": string,
"enableescapecommandline": string,
"enablesshkeysign": string,
"escapechar": string,
"exitonforwardfailure": string,
"fingerprinthash": string,
"forkafterauthentication": string,
"forwardagent": string,
"forwardx11": string,
"forwardx11timeout": integer,
"forwardx11trusted": string,
"gatewayports": string,
"globalknownhostsfile": [
string
],
"gssapiauthentication": string,
"gssapidelegatecredentials": string,
"hashknownhosts": string,
"hostbasedacceptedalgorithms": [
string
],
"hostbasedauthentication": string,
"hostkeyalgorithms": [
string
],
"hostkeyalias": string,
"hostname": string,
"identitiesonly": string,
"identityagent": string,
"identityfile": [
string
],
"ignoreunknown": string,
"include": [
string
],
"ipqos": [
string
],
"kbdinteractiveauthentication": string,
"kbdinteractivedevices": [
string
],
"kexalgorithms": [
string
],
"kexalgorithms_strategy": string,
"knownhostscommand": string,
"localcommand": string,
"localforward": [
string
],
"loglevel": string,
"logverbose": [
string
],
"macs": [
string
],
"macs_strategy": string,
"nohostauthenticationforlocalhost": string,
"numberofpasswordprompts": integer,
"passwordauthentication": string,
"permitlocalcommand": string,
"permitremoteopen": [
string
],
"pkcs11provider": string,
"port": integer,
"preferredauthentications": [
string
],
"protocol": integer,
"proxycommand": string,
"proxyjump": [
string
],
"proxyusefdpass": string,
"pubkeyacceptedalgorithms": [
string
],
"pubkeyacceptedalgorithms_strategy": string,
"pubkeyauthentication": string,
"rekeylimit": string,
"remotecommand": string,
"remoteforward": string,
"requesttty": string,
"requiredrsasize": integer,
"revokedhostkeys": string,
"securitykeyprovider": string,
"sendenv": [
string
],
"serveralivecountmax": integer,
"serveraliveinterval": integer,
"sessiontype": string,
"setenv": [
string
],
"stdinnull": string,
"streamlocalbindmask": string,
"streamlocalbindunlink": string,
"stricthostkeychecking": string,
"syslogfacility": string,
"tcpkeepalive": string,
"tunnel": string,
"tunneldevice": string,
"updatehostkeys": string,
"user": string,
"userknownhostsfile": [
string
],
"verifyhostkeydns": string,
"visualhostkey": string,
"xauthlocation": string
}
]
Examples:
$ ssh -G - | jc --ssh-conf -p
[
{
"user": "foo",
"hostname": "-",
"port": 22,
"addressfamily": "any",
"batchmode": "no",
"canonicalizefallbacklocal": "yes",
"canonicalizehostname": "false",
"checkhostip": "no",
"compression": "no",
"controlmaster": "false",
"enablesshkeysign": "no",
"clearallforwardings": "no",
"exitonforwardfailure": "no",
"fingerprinthash": "SHA256",
"forwardx11": "no",
"forwardx11trusted": "no",
"gatewayports": "no",
"gssapiauthentication": "no",
"gssapidelegatecredentials": "no",
"hashknownhosts": "no",
"hostbasedauthentication": "no",
"identitiesonly": "no",
"kbdinteractiveauthentication": "yes",
"nohostauthenticationforlocalhost": "no",
"passwordauthentication": "yes",
"permitlocalcommand": "no",
"proxyusefdpass": "no",
"pubkeyauthentication": "true",
"requesttty": "auto",
"sessiontype": "default",
"stdinnull": "no",
"forkafterauthentication": "no",
"streamlocalbindunlink": "no",
"stricthostkeychecking": "ask",
"tcpkeepalive": "yes",
"tunnel": "false",
"verifyhostkeydns": "false",
"visualhostkey": "no",
"updatehostkeys": "true",
"applemultipath": "no",
"canonicalizemaxdots": 1,
"connectionattempts": 1,
"forwardx11timeout": 1200,
"numberofpasswordprompts": 3,
"serveralivecountmax": 3,
"serveraliveinterval": 0,
"ciphers": [
"chacha20-poly1305@openssh.com",
"aes128-ctr",
"aes192-ctr",
"aes256-ctr",
"aes128-gcm@openssh.com",
"aes256-gcm@openssh.com"
],
"hostkeyalgorithms": [
"ssh-ed25519-cert-v01@openssh.com",
"ecdsa-sha2-nistp256-cert-v01@openssh.com",
"ecdsa-sha2-nistp384-cert-v01@openssh.com",
"ecdsa-sha2-nistp521-cert-v01@openssh.com",
"rsa-sha2-512-cert-v01@openssh.com",
"rsa-sha2-256-cert-v01@openssh.com",
"ssh-ed25519",
"ecdsa-sha2-nistp256",
"ecdsa-sha2-nistp384",
"ecdsa-sha2-nistp521",
"rsa-sha2-512",
"rsa-sha2-256"
],
"hostbasedacceptedalgorithms": [
"ssh-ed25519-cert-v01@openssh.com",
"ecdsa-sha2-nistp256-cert-v01@openssh.com",
"ecdsa-sha2-nistp384-cert-v01@openssh.com",
"ecdsa-sha2-nistp521-cert-v01@openssh.com",
"rsa-sha2-512-cert-v01@openssh.com",
"rsa-sha2-256-cert-v01@openssh.com",
"ssh-ed25519",
"ecdsa-sha2-nistp256",
"ecdsa-sha2-nistp384",
"ecdsa-sha2-nistp521",
"rsa-sha2-512",
"rsa-sha2-256"
],
"kexalgorithms": [
"sntrup761x25519-sha512@openssh.com",
"curve25519-sha256",
"curve25519-sha256@libssh.org",
"ecdh-sha2-nistp256",
"ecdh-sha2-nistp384",
"ecdh-sha2-nistp521",
"diffie-hellman-group-exchange-sha256",
"diffie-hellman-group16-sha512",
"diffie-hellman-group18-sha512",
"diffie-hellman-group14-sha256"
],
"casignaturealgorithms": [
"ssh-ed25519",
"ecdsa-sha2-nistp256",
"ecdsa-sha2-nistp384",
"ecdsa-sha2-nistp521",
"rsa-sha2-512",
"rsa-sha2-256"
],
"loglevel": "INFO",
"macs": [
"umac-64-etm@openssh.com",
"umac-128-etm@openssh.com",
"hmac-sha2-256-etm@openssh.com",
"hmac-sha2-512-etm@openssh.com",
"hmac-sha1-etm@openssh.com",
"umac-64@openssh.com",
"umac-128@openssh.com",
"hmac-sha2-256",
"hmac-sha2-512",
"hmac-sha1"
],
"securitykeyprovider": "$SSH_SK_PROVIDER",
"pubkeyacceptedalgorithms": [
"ssh-ed25519-cert-v01@openssh.com",
"ecdsa-sha2-nistp256-cert-v01@openssh.com",
"ecdsa-sha2-nistp384-cert-v01@openssh.com",
"ecdsa-sha2-nistp521-cert-v01@openssh.com",
"rsa-sha2-512-cert-v01@openssh.com",
"rsa-sha2-256-cert-v01@openssh.com",
"ssh-ed25519",
"ecdsa-sha2-nistp256",
"ecdsa-sha2-nistp384",
"ecdsa-sha2-nistp521",
"rsa-sha2-512",
"rsa-sha2-256"
],
"xauthlocation": "/usr/X11R6/bin/xauth",
"identityfile": [
"~/.ssh/id_rsa",
"~/.ssh/id_ecdsa",
"~/.ssh/id_ecdsa_sk",
"~/.ssh/id_ed25519",
"~/.ssh/id_ed25519_sk",
"~/.ssh/id_xmss",
"~/.ssh/id_dsa"
],
"canonicaldomains": [
"none"
],
"globalknownhostsfile": [
"/etc/ssh/ssh_known_hosts",
"/etc/ssh/ssh_known_hosts2"
],
"userknownhostsfile": [
"/Users/foo/.ssh/known_hosts",
"/Users/foo/.ssh/known_hosts2"
],
"sendenv": [
"LANG",
"LC_*"
],
"logverbose": [
"none"
],
"permitremoteopen": [
"any"
],
"addkeystoagent": "false",
"forwardagent": "no",
"connecttimeout": null,
"tunneldevice": "any:any",
"canonicalizepermittedcnames": [
"none"
],
"controlpersist": "no",
"escapechar": "~",
"ipqos": [
"af21",
"cs1"
],
"rekeylimit": "0 0",
"streamlocalbindmask": "0177",
"syslogfacility": "USER"
}
]
$ cat ~/.ssh/config | jc --ssh-conf -p
[
{
"host": "server1",
"host_list": [
"server1"
],
"hostname": "server1.cyberciti.biz",
"user": "nixcraft",
"port": 4242,
"identityfile": [
"/nfs/shared/users/nixcraft/keys/server1/id_rsa"
]
},
{
"host": "nas01",
"host_list": [
"nas01"
],
"hostname": "192.168.1.100",
"user": "root",
"identityfile": [
"~/.ssh/nas01.key"
]
},
{
"host": "aws.apache",
"host_list": [
"aws.apache"
],
"hostname": "1.2.3.4",
"user": "wwwdata",
"identityfile": [
"~/.ssh/aws.apache.key"
]
},
{
"host": "uk.gw.lan uk.lan",
"host_list": [
"uk.gw.lan",
"uk.lan"
],
"hostname": "192.168.0.251",
"user": "nixcraft",
"proxycommand": "ssh nixcraft@gateway.uk.cyberciti.biz nc %h %p 2> /dev/null"
},
{
"host": "proxyus",
"host_list": [
"proxyus"
],
"hostname": "vps1.cyberciti.biz",
"user": "breakfree",
"identityfile": [
"~/.ssh/vps1.cyberciti.biz.key"
],
"localforward": [
"3128 127.0.0.1:3128"
]
},
{
"host": "*",
"host_list": [
"*"
],
"forwardagent": "no",
"forwardx11": "no",
"forwardx11trusted": "yes",
"user": "nixcraft",
"port": 22,
"protocol": 2,
"serveraliveinterval": 60,
"serveralivecountmax": 30
}
]
$ cat ~/.ssh/config | jc --ssh-conf -p -r
[
{
"host": "server1",
"host_list": [
"server1"
],
"hostname": "server1.cyberciti.biz",
"user": "nixcraft",
"port": "4242",
"identityfile": [
"/nfs/shared/users/nixcraft/keys/server1/id_rsa"
]
},
{
"host": "nas01",
"host_list": [
"nas01"
],
"hostname": "192.168.1.100",
"user": "root",
"identityfile": [
"~/.ssh/nas01.key"
]
},
{
"host": "aws.apache",
"host_list": [
"aws.apache"
],
"hostname": "1.2.3.4",
"user": "wwwdata",
"identityfile": [
"~/.ssh/aws.apache.key"
]
},
{
"host": "uk.gw.lan uk.lan",
"host_list": [
"uk.gw.lan",
"uk.lan"
],
"hostname": "192.168.0.251",
"user": "nixcraft",
"proxycommand": "ssh nixcraft@gateway.uk.cyberciti.biz nc %h %p 2> /dev/null"
},
{
"host": "proxyus",
"host_list": [
"proxyus"
],
"hostname": "vps1.cyberciti.biz",
"user": "breakfree",
"identityfile": [
"~/.ssh/vps1.cyberciti.biz.key"
],
"localforward": [
"3128 127.0.0.1:3128"
]
},
{
"host": "*",
"host_list": [
"*"
],
"forwardagent": "no",
"forwardx11": "no",
"forwardx11trusted": "yes",
"user": "nixcraft",
"port": "22",
"protocol": "2",
"serveraliveinterval": "60",
"serveralivecountmax": "30"
}
]
<a id="jc.parsers.ssh_conf.parse"></a>
### parse
```python
def parse(data: str,
raw: bool = False,
quiet: bool = False) -> List[JSONDictType]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux, darwin, freebsd
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -3,7 +3,7 @@
# jc.parsers.sshd\_conf
jc - JSON Convert sshd configuration file and `sshd -T` command output parser
jc - JSON Convert `sshd` configuration file and `sshd -T` command output parser
This parser will work with `sshd` configuration files or the output of
`sshd -T`. Any `Match` blocks in the `sshd` configuration file will be
@@ -504,4 +504,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, freebsd
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.1 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -193,4 +193,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, freebsd
Version 1.12 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.13 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -5,6 +5,8 @@
jc - JSON Convert `timedatectl` command output parser
Also supports the `timesync-status` option.
The `epoch_utc` calculated timestamp field is timezone-aware and is only
available if the `universal_time` field is available.
@@ -34,7 +36,24 @@ Schema:
"system_clock_synchronized": boolean,
"systemd-timesyncd.service_active": boolean,
"rtc_in_local_tz": boolean,
"dst_active": boolean
"dst_active": boolean,
"server": string,
"poll_interval": string,
"leap": string,
"version": integer,
"stratum": integer,
"reference": string,
"precision": string,
"root_distance": string,
"offset": float,
"offset_unit": string,
"delay": float,
"delay_unit": string,
"jitter": float,
"jitter_unit": string,
"packet_count": integer,
"frequency": float,
"frequency_unit": string
}
Examples:
@@ -87,4 +106,4 @@ Returns:
### Parser Information
Compatibility: linux
Version 1.7 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.8 by Kelly Brazil (kellyjonbrazil@gmail.com)

81
docs/parsers/toml.md Normal file
View File

@@ -0,0 +1,81 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.toml"></a>
# jc.parsers.toml
jc - JSON Convert TOML file parser
Usage (cli):
$ cat file.toml | jc --toml
Usage (module):
import jc
result = jc.parse('toml', toml_file_output)
Schema:
TOML Document converted to a Dictionary.
See https://toml.io/en/ for details.
{
"key1": string/int/float/boolean/null/array/object,
"key2": string/int/float/boolean/null/array/object
}
Examples:
$ cat file.toml
title = "TOML Example"
[owner]
name = "Tom Preston-Werner"
dob = 1979-05-27T07:32:00-08:00
[database]
enabled = true
ports = [ 8000, 8001, 8002 ]
$ cat file.toml | jc --toml -p
{
"title": "TOML Example",
"owner": {
"name": "Tom Preston-Werner",
"dob": 296667120,
"dob_iso": "1979-05-27T07:32:00-08:00"
},
"database": {
"enabled": true,
"ports": [
8000,
8001,
8002
]
}
}
<a id="jc.parsers.toml.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> JSONDictType
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -161,4 +161,4 @@ Returns:
### Parser Information
Compatibility: linux
Version 1.2 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.3 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -42,8 +42,7 @@ Parameters:
underscore '_'. You should also ensure headers are
lowercase by using .lower().
Also, ensure there are no blank lines (list items)
in the data.
Also, ensure there are no blank rows in the data.
Returns:

112
docs/parsers/ver.md Normal file
View File

@@ -0,0 +1,112 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.ver"></a>
# jc.parsers.ver
jc - JSON Convert Version string output parser
Best-effort attempt to parse various styles of version numbers. This parser
is based off of the version parser included in the CPython distutils
libary.
If the version string conforms to some de facto-standard versioning rules
followed by many developers a `strict` key will be present in the output
with a value of `true` along with the named parsed components.
All other version strings will have a `strict` value of `false` and a
`components` key will contain a list of detected parts of the version
string.
See Also: `semver` parser.
Usage (cli):
$ echo 1.2a1 | jc --ver
Usage (module):
import jc
result = jc.parse('ver', version_string_output)
Schema:
{
"major": integer,
"minor": integer,
"patch": integer,
"prerelease": string,
"prerelease_num": integer,
"components": [
integer/string
],
"strict": boolean
}
Examples:
$ echo 1.2a1 | jc --ver -p
{
"major": 1,
"minor": 2,
"patch": 0,
"prerelease": "a",
"prerelease_num": 1,
"strict": true
}
$ echo 1.2a1 | jc --ver -p -r
{
"major": "1",
"minor": "2",
"patch": "0",
"prerelease": "a",
"prerelease_num": "1",
"strict": true
}
$ echo 1.2beta3 | jc --ver -p
{
"components": [
1,
2,
"beta",
3
],
"strict": false
}
$ echo 1.2beta3 | jc --ver -p -r
{
"components": [
"1",
"2",
"beta",
"3"
],
"strict": false
}
<a id="jc.parsers.ver.parse"></a>
### parse
```python
def parse(data: str, raw: bool = False, quiet: bool = False) -> JSONDictType
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

108
docs/parsers/veracrypt.md Normal file
View File

@@ -0,0 +1,108 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.veracrypt"></a>
# jc.parsers.veracrypt
jc - JSON Convert `veracrypt` command output parser
Supports the following `veracrypt` subcommands:
- `veracrypt --text --list`
- `veracrypt --text --list --verbose`
- `veracrypt --text --volume-properties <volume>`
Usage (cli):
$ veracrypt --text --list | jc --veracrypt
or
$ jc veracrypt --text --list
Usage (module):
import jc
result = jc.parse('veracrypt', veracrypt_command_output)
Schema:
Volume:
[
{
"slot": integer,
"path": string,
"device": string,
"mountpoint": string,
"size": string,
"type": string,
"readonly": string,
"hidden_protected": string,
"encryption_algo": string,
"pk_size": string,
"sk_size": string,
"block_size": string,
"mode": string,
"prf": string,
"format_version": integer,
"backup_header": string
}
]
Examples:
$ veracrypt --text --list | jc --veracrypt -p
[
{
"slot": 1,
"path": "/dev/sdb1",
"device": "/dev/mapper/veracrypt1",
"mountpoint": "/home/bob/mount/encrypt/sdb1"
}
]
$ veracrypt --text --list --verbose | jc --veracrypt -p
[
{
"slot": 1,
"path": "/dev/sdb1",
"device": "/dev/mapper/veracrypt1",
"mountpoint": "/home/bob/mount/encrypt/sdb1",
"size": "522 MiB",
"type": "Normal",
"readonly": "No",
"hidden_protected": "No",
"encryption_algo": "AES",
"pk_size": "256 bits",
"sk_size": "256 bits",
"block_size": "128 bits",
"mode": "XTS",
"prf": "HMAC-SHA-512",
"format_version": 2,
"backup_header": "Yes"
}
]
<a id="jc.parsers.veracrypt.parse"></a>
### parse
```python
def parse(data: str,
raw: bool = False,
quiet: bool = False) -> List[JSONDictType]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux
Version 1.0 by Jake Ob (iakopap at gmail.com)

View File

@@ -433,4 +433,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.1 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.2 by Kelly Brazil (kellyjonbrazil@gmail.com)

282
docs/parsers/x509_csr.md Normal file
View File

@@ -0,0 +1,282 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.x509_csr"></a>
# jc.parsers.x509\_csr
jc - JSON Convert X.509 Certificate Request format file parser
This parser will convert DER and PEM encoded X.509 certificate request files.
Usage (cli):
$ cat certificateRequest.pem | jc --x509-csr
Usage (module):
import jc
result = jc.parse('x509_csr', x509_csr_file_output)
Schema:
[
{
"certification_request_info": {
"version": string,
"serial_number": string, # [0]
"serial_number_str": string,
"signature": {
"algorithm": string,
"parameters": string/null,
},
"issuer": {
"country_name": string,
"state_or_province_name" string,
"locality_name": string,
"organization_name": array/string,
"organizational_unit_name": array/string,
"common_name": string,
"email_address": string,
"serial_number": string, # [0]
"serial_number_str": string
},
"validity": {
"not_before": integer, # [1]
"not_after": integer, # [1]
"not_before_iso": string,
"not_after_iso": string
},
"subject": {
"country_name": string,
"state_or_province_name": string,
"locality_name": string,
"organization_name": array/string,
"organizational_unit_name": array/string,
"common_name": string,
"email_address": string,
"serial_number": string, # [0]
"serial_number_str": string
},
"subject_public_key_info": {
"algorithm": {
"algorithm": string,
"parameters": string/null,
},
"public_key": {
"modulus": string, # [0]
"public_exponent": integer
}
},
"issuer_unique_id": string/null,
"subject_unique_id": string/null,
"extensions": [
{
"extn_id": string,
"critical": boolean,
"extn_value": array/object/string/integer # [2]
}
]
},
"signature_algorithm": {
"algorithm": string,
"parameters": string/null
},
"signature_value": string # [0]
}
]
[0] in colon-delimited hex notation
[1] time-zone-aware (UTC) epoch timestamp
[2] See below for well-known Extension schemas:
Basic Constraints:
{
"extn_id": "basic_constraints",
"critical": boolean,
"extn_value": {
"ca": boolean,
"path_len_constraint": string/null
}
}
Key Usage:
{
"extn_id": "key_usage",
"critical": boolean,
"extn_value": [
string
]
}
Key Identifier:
{
"extn_id": "key_identifier",
"critical": boolean,
"extn_value": string # [0]
}
Authority Key Identifier:
{
"extn_id": "authority_key_identifier",
"critical": boolean,
"extn_value": {
"key_identifier": string, # [0]
"authority_cert_issuer": string/null,
"authority_cert_serial_number": string/null
}
}
Subject Alternative Name:
{
"extn_id": "subject_alt_name",
"critical": boolean,
"extn_value": [
string
]
}
Certificate Policies:
{
"extn_id": "certificate_policies",
"critical": boolean,
"extn_value": [
{
"policy_identifier": string,
"policy_qualifiers": [ array or null
{
"policy_qualifier_id": string,
"qualifier": string
}
]
}
]
}
Signed Certificate Timestamp List:
{
"extn_id": "signed_certificate_timestamp_list",
"critical": boolean,
"extn_value": string # [0]
}
Examples:
$ cat server.csr| jc --x509-csr -p
[
{
"certification_request_info": {
"version": "v1",
"subject": {
"common_name": "myserver.for.example"
},
"subject_pk_info": {
"algorithm": {
"algorithm": "ec",
"parameters": "secp256r1"
},
"public_key": "04:40:33:c0:91:8f:e9:46:ea:d0:dc:d0:f9:63:2..."
},
"attributes": [
{
"type": "extension_request",
"values": [
[
{
"extn_id": "extended_key_usage",
"critical": false,
"extn_value": [
"server_auth"
]
},
{
"extn_id": "subject_alt_name",
"critical": false,
"extn_value": [
"myserver.for.example"
]
}
]
]
}
]
},
"signature_algorithm": {
"algorithm": "sha384_ecdsa",
"parameters": null
},
"signature": "30:45:02:20:77:ac:5b:51:bf:c5:f5:43:02:52:ae:66:..."
}
]
$ openssl req -in server.csr | jc --x509-csr -p
[
{
"certification_request_info": {
"version": "v1",
"subject": {
"common_name": "myserver.for.example"
},
"subject_pk_info": {
"algorithm": {
"algorithm": "ec",
"parameters": "secp256r1"
},
"public_key": "04:40:33:c0:91:8f:e9:46:ea:d0:dc:d0:f9:63:2..."
},
"attributes": [
{
"type": "extension_request",
"values": [
[
{
"extn_id": "extended_key_usage",
"critical": false,
"extn_value": [
"server_auth"
]
},
{
"extn_id": "subject_alt_name",
"critical": false,
"extn_value": [
"myserver.for.example"
]
}
]
]
}
]
},
"signature_algorithm": {
"algorithm": "sha384_ecdsa",
"parameters": null
},
"signature": "30:45:02:20:77:ac:5b:51:bf:c5:f5:43:02:52:ae:66:..."
}
]
<a id="jc.parsers.x509_csr.parse"></a>
### parse
```python
def parse(data: Union[str, bytes],
raw: bool = False,
quiet: bool = False) -> List[Dict]
```
Main text parsing function
Parameters:
data: (string or bytes) text or binary data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -8,6 +8,7 @@ jc - JSON Convert `xrandr` command output parser
Usage (cli):
$ xrandr | jc --xrandr
$ xrandr --properties | jc --xrandr
or
@@ -49,13 +50,17 @@ Schema:
"is_connected": boolean,
"is_primary": boolean,
"device_name": string,
"model_name": string,
"product_id" string,
"serial_number": string,
"resolution_width": integer,
"resolution_height": integer,
"offset_width": integer,
"offset_height": integer,
"dimension_width": integer,
"dimension_height": integer,
"rotation": string
"rotation": string,
"reflection": string
}
],
"unassociated_devices": [
@@ -132,7 +137,71 @@ Examples:
"offset_height": 0,
"dimension_width": 310,
"dimension_height": 170,
"rotation": "normal"
"rotation": "normal",
"reflection": "normal"
}
}
],
"unassociated_devices": []
}
$ xrandr --properties | jc --xrandr -p
{
"screens": [
{
"screen_number": 0,
"minimum_width": 8,
"minimum_height": 8,
"current_width": 1920,
"current_height": 1080,
"maximum_width": 32767,
"maximum_height": 32767,
"associated_device": {
"associated_modes": [
{
"resolution_width": 1920,
"resolution_height": 1080,
"is_high_resolution": false,
"frequencies": [
{
"frequency": 60.03,
"is_current": true,
"is_preferred": true
},
{
"frequency": 59.93,
"is_current": false,
"is_preferred": false
}
]
},
{
"resolution_width": 1680,
"resolution_height": 1050,
"is_high_resolution": false,
"frequencies": [
{
"frequency": 59.88,
"is_current": false,
"is_preferred": false
}
]
}
],
"is_connected": true,
"is_primary": true,
"device_name": "eDP1",
"model_name": "ASUS VW193S",
"product_id": "54297",
"serial_number": "78L8021107",
"resolution_width": 1920,
"resolution_height": 1080,
"offset_width": 0,
"offset_height": 0,
"dimension_width": 310,
"dimension_height": 170,
"rotation": "normal",
"reflection": "normal"
}
}
],
@@ -162,4 +231,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, cygwin, aix, freebsd
Version 1.1 by Kevin Lyter (lyter_git at sent.com)
Version 1.2 by Kevin Lyter (lyter_git at sent.com)

View File

@@ -18,8 +18,8 @@ Usage (module):
Schema:
YAML Document converted to a Dictionary
See https://pypi.org/project/ruamel.yaml for details
YAML Document converted to a Dictionary.
See https://pypi.org/project/ruamel.yaml for details.
[
{
@@ -30,7 +30,7 @@ Schema:
Examples:
$ cat istio-mtls-permissive.yaml
$ cat file.yaml
apiVersion: "authentication.istio.io/v1alpha1"
kind: "Policy"
metadata:
@@ -51,7 +51,7 @@ Examples:
tls:
mode: ISTIO_MUTUAL
$ cat istio-mtls-permissive.yaml | jc --yaml -p
$ cat file.yaml | jc --yaml -p
[
{
"apiVersion": "authentication.istio.io/v1alpha1",

View File

@@ -99,4 +99,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin
Version 1.1 by Matt J (https://github.com/listuser)
Version 1.2 by Matt J (https://github.com/listuser)

View File

@@ -0,0 +1,125 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.zpool_iostat"></a>
# jc.parsers.zpool\_iostat
jc - JSON Convert `zpool iostat` command output parser
Supports with or without the `-v` flag.
Usage (cli):
$ zpool iostat | jc --zpool-iostat
or
$ jc zpool iostat
Usage (module):
import jc
result = jc.parse('zpool_iostat', zpool_iostat_command_output)
Schema:
[
{
"pool": string,
"parent": string,
"cap_alloc": float,
"cap_alloc_unit": string,
"cap_free": float,
"cap_free_unit": string,
"ops_read": integer,
"ops_write": integer,
"bw_read": float,
"bw_read_unit": string,
"bw_write": float,
"bw_write_unit": string
}
]
Examples:
$ zpool iostat -v | jc --zpool-iostat -p
[
{
"pool": "zhgstera6",
"cap_alloc": 2.89,
"cap_free": 2.2,
"ops_read": 0,
"ops_write": 2,
"bw_read": 349.0,
"bw_write": 448.0,
"cap_alloc_unit": "T",
"cap_free_unit": "T",
"bw_read_unit": "K",
"bw_write_unit": "K"
},
{
"pool": "726060ALE614-K8JAPRGN:10",
"parent": "zhgstera6",
"cap_alloc": 2.89,
"cap_free": 2.2,
"ops_read": 0,
"ops_write": 2,
"bw_read": 349.0,
"bw_write": 448.0,
"cap_alloc_unit": "T",
"cap_free_unit": "T",
"bw_read_unit": "K",
"bw_write_unit": "K"
},
...
]
$ zpool iostat -v | jc --zpool-iostat -p -r
[
{
"pool": "zhgstera6",
"cap_alloc": "2.89T",
"cap_free": "2.20T",
"ops_read": "0",
"ops_write": "2",
"bw_read": "349K",
"bw_write": "448K"
},
{
"pool": "726060ALE614-K8JAPRGN:10",
"parent": "zhgstera6",
"cap_alloc": "2.89T",
"cap_free": "2.20T",
"ops_read": "0",
"ops_write": "2",
"bw_read": "349K",
"bw_write": "448K"
},
...
]
<a id="jc.parsers.zpool_iostat.parse"></a>
### parse
```python
def parse(data: str,
raw: bool = False,
quiet: bool = False) -> List[JSONDictType]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux, darwin, freebsd
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -0,0 +1,163 @@
[Home](https://kellyjonbrazil.github.io/jc/)
<a id="jc.parsers.zpool_status"></a>
# jc.parsers.zpool\_status
jc - JSON Convert `zpool status` command output parser
Works with or without the `-v` option.
Usage (cli):
$ zpool status | jc --zpool-status
or
$ jc zpool status
Usage (module):
import jc
result = jc.parse('zpool_status', zpool_status_command_output)
Schema:
[
{
"pool": string,
"state": string,
"status": string,
"action": string,
"see": string,
"scan": string,
"scrub": string,
"config": [
{
"name": string,
"state": string,
"read": integer,
"write": integer,
"checksum": integer,
"errors": string,
}
],
"errors": string
}
]
Examples:
$ zpool status -v | jc --zpool-status -p
[
{
"pool": "tank",
"state": "DEGRADED",
"status": "One or more devices could not be opened. Suffic...",
"action": "Attach the missing device and online it using 'zpool...",
"see": "http://www.sun.com/msg/ZFS-8000-2Q",
"scrub": "none requested",
"config": [
{
"name": "tank",
"state": "DEGRADED",
"read": 0,
"write": 0,
"checksum": 0
},
{
"name": "mirror-0",
"state": "DEGRADED",
"read": 0,
"write": 0,
"checksum": 0
},
{
"name": "c1t0d0",
"state": "ONLINE",
"read": 0,
"write": 0,
"checksum": 0
},
{
"name": "c1t1d0",
"state": "UNAVAIL",
"read": 0,
"write": 0,
"checksum": 0,
"errors": "cannot open"
}
],
"errors": "No known data errors"
}
]
$ zpool status -v | jc --zpool-status -p -r
[
{
"pool": "tank",
"state": "DEGRADED",
"status": "One or more devices could not be opened. Sufficient...",
"action": "Attach the missing device and online it using 'zpool...",
"see": "http://www.sun.com/msg/ZFS-8000-2Q",
"scrub": "none requested",
"config": [
{
"name": "tank",
"state": "DEGRADED",
"read": "0",
"write": "0",
"checksum": "0"
},
{
"name": "mirror-0",
"state": "DEGRADED",
"read": "0",
"write": "0",
"checksum": "0"
},
{
"name": "c1t0d0",
"state": "ONLINE",
"read": "0",
"write": "0",
"checksum": "0"
},
{
"name": "c1t1d0",
"state": "UNAVAIL",
"read": "0",
"write": "0",
"checksum": "0",
"errors": "cannot open"
}
],
"errors": "No known data errors"
}
]
<a id="jc.parsers.zpool_status.parse"></a>
### parse
```python
def parse(data: str,
raw: bool = False,
quiet: bool = False) -> List[JSONDictType]
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
### Parser Information
Compatibility: linux, darwin, freebsd
Version 1.1 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -124,6 +124,14 @@ Get a list of streaming parser module names to be used in
`parse()`, `parser_info()`, and `get_help()`. This list is a subset of
`parser_mod_list()`.
"""
from .lib import (__version__, parse, parser_mod_list, plugin_parser_mod_list,
standard_parser_mod_list, streaming_parser_mod_list,
parser_info, all_parser_info, get_help)
from .lib import (
__version__ as __version__,
parse as parse,
parser_mod_list as parser_mod_list,
plugin_parser_mod_list as plugin_parser_mod_list,
standard_parser_mod_list as standard_parser_mod_list,
streaming_parser_mod_list as streaming_parser_mod_list,
parser_info as parser_info,
all_parser_info as all_parser_info,
get_help as get_help
)

212
jc/cli.py
View File

@@ -5,11 +5,13 @@ JC cli module
import io
import sys
import os
import re
from itertools import islice
from datetime import datetime, timezone
import textwrap
import shlex
import subprocess
from typing import List, Dict, Union, Optional, TextIO
from typing import List, Dict, Iterable, Union, Optional, TextIO
from types import ModuleType
from .lib import (
__version__, parser_info, all_parser_info, parsers, _get_parser, _parser_is_streaming,
@@ -40,6 +42,10 @@ except Exception:
JC_CLEAN_EXIT: int = 0
JC_ERROR_EXIT: int = 100
MAX_EXIT: int = 255
SLICER_PATTERN: str = r'-?[0-9]*\:-?[0-9]*$'
SLICER_RE = re.compile(SLICER_PATTERN)
NEWLINES_PATTERN: str = r'(\r\n|\r|\n)'
NEWLINES_RE = re.compile(NEWLINES_PATTERN)
class info():
@@ -48,7 +54,7 @@ class info():
author: str = 'Kelly Brazil'
author_email: str = 'kellyjonbrazil@gmail.com'
website: str = 'https://github.com/kellyjonbrazil/jc'
copyright: str = '© 2019-2022 Kelly Brazil'
copyright: str = '© 2019-2023 Kelly Brazil'
license: str = 'MIT License'
@@ -69,11 +75,11 @@ class JcCli():
'help_me', 'pretty', 'quiet', 'ignore_exceptions', 'raw', 'meta_out', 'unbuffer',
'version_info', 'yaml_output', 'bash_comp', 'zsh_comp', 'magic_found_parser',
'magic_options', 'magic_run_command', 'magic_run_command_str', 'magic_stdout',
'magic_stderr', 'magic_returncode'
'magic_stderr', 'magic_returncode', 'slice_str', 'slice_start', 'slice_end'
)
def __init__(self) -> None:
self.data_in: Optional[Union[str, bytes, TextIO]] = None
self.data_in: Optional[Union[str, bytes, TextIO, Iterable[str]]] = None
self.data_out: Optional[Union[List[JSONDictType], JSONDictType]] = None
self.options: List[str] = []
self.args: List[str] = []
@@ -89,6 +95,11 @@ class JcCli():
self.json_indent: Optional[int] = None
self.run_timestamp: Optional[datetime] = None
# slicer
self.slice_str: str = ''
self.slice_start: Optional[int] = None
self.slice_end: Optional[int] = None
# cli options
self.about: bool = False
self.debug: bool = False
@@ -204,14 +215,14 @@ class JcCli():
category_text: str = ''
padding_char: str = ' '
all_parsers = all_parser_info(show_hidden=True, show_deprecated=False)
generic = [{'arg': x['argument'], 'desc': x['description']} for x in all_parsers if 'generic' in x['tags']]
standard = [{'arg': x['argument'], 'desc': x['description']} for x in all_parsers if 'standard' in x['tags']]
command = [{'arg': x['argument'], 'desc': x['description']} for x in all_parsers if 'command' in x['tags']]
generic = [{'arg': x['argument'], 'desc': x['description']} for x in all_parsers if 'generic' in x.get('tags', [])]
standard = [{'arg': x['argument'], 'desc': x['description']} for x in all_parsers if 'standard' in x.get('tags', [])]
command = [{'arg': x['argument'], 'desc': x['description']} for x in all_parsers if 'command' in x.get('tags', [])]
file_str_bin = [
{'arg': x['argument'], 'desc': x['description']} for x in all_parsers
if 'file' in x['tags'] or
'string' in x['tags'] or
'binary' in x['tags']
if 'file' in x.get('tags', []) or
'string' in x.get('tags', []) or
'binary' in x.get('tags', [])
]
streaming = [{'arg': x['argument'], 'desc': x['description']} for x in all_parsers if x.get('streaming')]
categories: Dict = {
@@ -432,6 +443,17 @@ class JcCli():
self.magic_options = []
return
# slicer found
if ':' in arg:
if SLICER_RE.match(arg):
self.slice_str = arg
args_given.pop(0)
continue
else:
utils.warning_message(['Invalid slice syntax.'])
args_given.pop(0)
continue
# option found - populate option list
if arg.startswith('-'):
self.magic_options.extend(args_given.pop(0)[1:])
@@ -574,59 +596,6 @@ class JcCli():
utils.error_message(['Missing piped data. Use "jc -h" for help.'])
self.exit_error()
def streaming_parse_and_print(self) -> None:
"""only supports UTF-8 string data for now"""
self.data_in = sys.stdin
if self.parser_module:
result = self.parser_module.parse(
self.data_in,
raw=self.raw,
quiet=self.quiet,
ignore_exceptions=self.ignore_exceptions
)
for line in result:
self.data_out = line
if self.meta_out:
self.run_timestamp = datetime.now(timezone.utc)
self.add_metadata_to_output()
self.safe_print_out()
def standard_parse_and_print(self) -> None:
"""supports binary and UTF-8 string data"""
self.data_in = self.magic_stdout or sys.stdin.buffer.read()
# convert to UTF-8, if possible. Otherwise, leave as bytes
try:
if isinstance(self.data_in, bytes):
self.data_in = self.data_in.decode('utf-8')
except UnicodeDecodeError:
pass
if self.parser_module:
self.data_out = self.parser_module.parse(
self.data_in,
raw=self.raw,
quiet=self.quiet
)
if self.meta_out:
self.run_timestamp = datetime.now(timezone.utc)
self.add_metadata_to_output()
self.safe_print_out()
def exit_clean(self) -> None:
exit_code: int = self.magic_returncode + JC_CLEAN_EXIT
exit_code = min(exit_code, MAX_EXIT)
sys.exit(exit_code)
def exit_error(self) -> None:
exit_code: int = self.magic_returncode + JC_ERROR_EXIT
exit_code = min(exit_code, MAX_EXIT)
sys.exit(exit_code)
def add_metadata_to_output(self) -> None:
"""
This function mutates data_out in place. If the _jc_meta field
@@ -641,7 +610,9 @@ class JcCli():
if self.run_timestamp:
meta_obj: JSONDictType = {
'parser': self.parser_name,
'timestamp': self.run_timestamp.timestamp()
'timestamp': self.run_timestamp.timestamp(),
'slice_start': self.slice_start,
'slice_end': self.slice_end
}
if self.magic_run_command:
@@ -669,6 +640,116 @@ class JcCli():
utils.error_message(['Parser returned an unsupported object type.'])
self.exit_error()
@staticmethod
def lazy_splitlines(text: str) -> Iterable[str]:
start = 0
for m in NEWLINES_RE.finditer(text):
begin, end = m.span()
if begin != start:
yield text[start:begin]
start = end
if text[start:]:
yield text[start:]
def slicer(self) -> None:
"""Slice input data lazily, if possible. Updates self.data_in"""
if self.slice_str:
slice_start_str, slice_end_str = self.slice_str.split(':', maxsplit=1)
if slice_start_str:
self.slice_start = int(slice_start_str)
if slice_end_str:
self.slice_end = int(slice_end_str)
if not self.slice_start is None or not self.slice_end is None:
# standard parsers UTF-8 input
if isinstance(self.data_in, str):
data_in_iter = self.lazy_splitlines(self.data_in)
# positive slices
if (self.slice_start is None or self.slice_start >= 0) \
and (self.slice_end is None or self.slice_end >= 0):
self.data_in = '\n'.join(islice(data_in_iter, self.slice_start, self.slice_end))
# negative slices found (non-lazy, uses more memory)
else:
self.data_in = '\n'.join(list(data_in_iter)[self.slice_start:self.slice_end])
# standard parsers bytes input
elif isinstance(self.data_in, bytes):
utils.warning_message(['Cannot slice bytes data.'])
# streaming parsers UTF-8 input
else:
# positive slices
if (self.slice_start is None or self.slice_start >= 0) \
and (self.slice_end is None or self.slice_end >= 0) \
and self.data_in:
self.data_in = islice(self.data_in, self.slice_start, self.slice_end)
# negative slices found (non-lazy, uses more memory)
elif self.data_in:
self.data_in = list(self.data_in)[self.slice_start:self.slice_end]
def streaming_parse_and_print(self) -> None:
"""only supports UTF-8 string data for now"""
self.data_in = sys.stdin
self.slicer()
if self.parser_module:
result = self.parser_module.parse(
self.data_in,
raw=self.raw,
quiet=self.quiet,
ignore_exceptions=self.ignore_exceptions
)
for line in result:
self.data_out = line
if self.meta_out:
self.run_timestamp = datetime.now(timezone.utc)
self.add_metadata_to_output()
self.safe_print_out()
def standard_parse_and_print(self) -> None:
"""supports binary and UTF-8 string data"""
self.data_in = self.magic_stdout or sys.stdin.buffer.read()
# convert to UTF-8, if possible. Otherwise, leave as bytes
try:
if isinstance(self.data_in, bytes):
self.data_in = self.data_in.decode('utf-8')
except UnicodeDecodeError:
pass
self.slicer()
if self.parser_module:
self.data_out = self.parser_module.parse(
self.data_in,
raw=self.raw,
quiet=self.quiet
)
if self.meta_out:
self.run_timestamp = datetime.now(timezone.utc)
self.add_metadata_to_output()
self.safe_print_out()
def exit_clean(self) -> None:
exit_code: int = self.magic_returncode + JC_CLEAN_EXIT
exit_code = min(exit_code, MAX_EXIT)
sys.exit(exit_code)
def exit_error(self) -> None:
exit_code: int = self.magic_returncode + JC_ERROR_EXIT
exit_code = min(exit_code, MAX_EXIT)
sys.exit(exit_code)
def _run(self) -> None:
# enable colors for Windows cmd.exe terminal
if sys.platform.startswith('win32'):
@@ -684,6 +765,9 @@ class JcCli():
# find options if magic_parser did not find a command
if not self.magic_found_parser:
for opt in self.args:
if SLICER_RE.match(opt):
self.slice_str = opt
if opt in long_options_map:
self.options.extend(long_options_map[opt][0])

View File

@@ -63,17 +63,17 @@ Usage:
Standard syntax:
COMMAND | jc [OPTIONS] PARSER
COMMAND | jc [SLICE] [OPTIONS] PARSER
cat FILE | jc [OPTIONS] PARSER
cat FILE | jc [SLICE] [OPTIONS] PARSER
echo STRING | jc [OPTIONS] PARSER
echo STRING | jc [SLICE] [OPTIONS] PARSER
Magic syntax:
jc [OPTIONS] COMMAND
jc [SLICE] [OPTIONS] COMMAND
jc [OPTIONS] /proc/<path-to-procfile>
jc [SLICE] [OPTIONS] /proc/<path-to-procfile>
Parsers:
'''
@@ -88,6 +88,9 @@ Examples:
$ jc --pretty dig www.google.com
$ jc --pretty /proc/meminfo
Line Slicing:
$ cat file.csv | jc :101 --csv # parse first 100 lines
Parser Documentation:
$ jc --help --dig

View File

@@ -3,13 +3,13 @@ import sys
import os
import re
import importlib
from typing import List, Iterable, Union, Iterator
from typing import List, Iterable, Optional, Union, Iterator
from types import ModuleType
from .jc_types import ParserInfoType, JSONDictType
from jc import appdirs
__version__ = '1.22.4'
__version__ = '1.23.3'
parsers: List[str] = [
'acpi',
@@ -19,9 +19,11 @@ parsers: List[str] = [
'asciitable',
'asciitable-m',
'blkid',
'bluetoothctl',
'cbt',
'cef',
'cef-s',
'certbot',
'chage',
'cksum',
'clf',
@@ -59,6 +61,7 @@ parsers: List[str] = [
'id',
'ifconfig',
'ini',
'ini-dup',
'iostat',
'iostat-s',
'ip-address',
@@ -73,6 +76,7 @@ parsers: List[str] = [
'last',
'ls',
'ls-s',
'lsattr',
'lsblk',
'lsmod',
'lsof',
@@ -157,7 +161,9 @@ parsers: List[str] = [
'semver',
'sfdisk',
'shadow',
'srt',
'ss',
'ssh-conf',
'sshd-conf',
'stat',
'stat-s',
@@ -174,6 +180,7 @@ parsers: List[str] = [
'time',
'timedatectl',
'timestamp',
'toml',
'top',
'top-s',
'tracepath',
@@ -187,16 +194,21 @@ parsers: List[str] = [
'upower',
'uptime',
'url',
'ver',
'veracrypt',
'vmstat',
'vmstat-s',
'w',
'wc',
'who',
'x509-cert',
'x509-csr',
'xml',
'xrandr',
'yaml',
'zipinfo'
'zipinfo',
'zpool-iostat',
'zpool-status'
]
def _cliname_to_modname(parser_cli_name: str) -> str:
@@ -207,6 +219,19 @@ def _modname_to_cliname(parser_mod_name: str) -> str:
"""Return module's cli name (underscores converted to dashes)"""
return parser_mod_name.replace('_', '-')
def _is_valid_parser_plugin(name: str, local_parsers_dir: str) -> bool:
if re.match(r'\w+\.py$', name) and os.path.isfile(os.path.join(local_parsers_dir, name)):
try:
parser_mod_name = _cliname_to_modname(name)[0:-3]
modpath = 'jcparsers.'
plugin = importlib.import_module(f'{modpath}{parser_mod_name}')
if hasattr(plugin, 'info') and hasattr(plugin, 'parse'):
del plugin
return True
except Exception:
return False
return False
# Create the local_parsers list. This is a list of custom or
# override parsers from <user_data_dir>/jc/jcparsers/*.py.
# Once this list is created, extend the parsers list with it.
@@ -216,7 +241,7 @@ local_parsers_dir = os.path.join(data_dir, 'jcparsers')
if os.path.isdir(local_parsers_dir):
sys.path.append(data_dir)
for name in os.listdir(local_parsers_dir):
if re.match(r'\w+\.py$', name) and os.path.isfile(os.path.join(local_parsers_dir, name)):
if _is_valid_parser_plugin(name, local_parsers_dir):
plugin_name = name[0:-3]
local_parsers.append(_modname_to_cliname(plugin_name))
if plugin_name not in parsers:
@@ -277,7 +302,7 @@ def parse(
data: Union[str, bytes, Iterable[str]],
quiet: bool = False,
raw: bool = False,
ignore_exceptions: bool = None,
ignore_exceptions: Optional[bool] = None,
**kwargs
) -> Union[JSONDictType, List[JSONDictType], Iterator[JSONDictType]]:
"""

View File

@@ -227,7 +227,7 @@ import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.4'
version = '1.6'
description = '`acpi` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -333,10 +333,19 @@ def parse(data, raw=False, quiet=False):
if obj_type == 'Battery':
output_line['type'] = obj_type
output_line['id'] = obj_id
if 'Charging' in line or 'Discharging' in line or 'Full' in line:
if 'Not charging' in line:
output_line['state'] = 'Not charging'
output_line['charge_percent'] = line.split()[-1].rstrip('%,')
if 'Charging' in line \
or 'Discharging' in line \
or 'Full' in line:
output_line['state'] = line.split()[2][:-1]
output_line['charge_percent'] = line.split()[3].rstrip('%,')
if 'rate information unavailable' not in line:
if 'will never fully discharge' in line:
pass
elif 'rate information unavailable' not in line:
if 'Charging' in line:
output_line['until_charged'] = line.split()[4]
if 'Discharging' in line:

View File

@@ -119,7 +119,7 @@ import jc.parsers.universal
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.11'
version = '1.12'
description = '`arp` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -222,14 +222,26 @@ def parse(
else:
for line in cleandata:
splitline = line.split()
if '<incomplete>' not in splitline:
# Ignore AIX bucket information
if 'bucket:' in splitline[0]:
continue
elif 'There' in splitline[0] and 'are' in splitline[1]:
continue
# AIX uses (incomplete)
elif '<incomplete>' not in splitline and '(incomplete)' not in splitline:
output_line = {
'name': splitline[0],
'address': splitline[1].lstrip('(').rstrip(')'),
'hwtype': splitline[4].lstrip('[').rstrip(']'),
'hwaddress': splitline[3],
'iface': splitline[6],
}
# Handle permanence and ignore interface in AIX
if 'permanent' in splitline:
output_line['permanent'] = True
elif 'in' not in splitline[6]: # AIX doesn't show interface
output_line['iface'] = splitline[6]
else:
output_line = {
@@ -237,8 +249,10 @@ def parse(
'address': splitline[1].lstrip('(').rstrip(')'),
'hwtype': None,
'hwaddress': None,
'iface': splitline[5],
}
# AIX doesn't show interface
if len(splitline) >= 5:
output_line['iface'] = splitline[5]
raw_output.append(output_line)

View File

@@ -0,0 +1 @@
quiet = False

View File

@@ -251,7 +251,18 @@ class EmailAddress(IA5String):
self._unicode = contents.decode('cp1252')
else:
mailbox, hostname = contents.rsplit(b'@', 1)
self._unicode = mailbox.decode('cp1252') + '@' + hostname.decode('idna')
# fix to allow incorrectly encoded email addresses to succeed with warning
try:
self._unicode = mailbox.decode('cp1252') + '@' + hostname.decode('idna')
except UnicodeDecodeError:
ascii_mailbox = mailbox.decode('ascii', errors='backslashreplace')
ascii_hostname = hostname.decode('ascii', errors='backslashreplace')
from jc.utils import warning_message
import jc.parsers.asn1crypto.jc_global as jc_global
if not jc_global.quiet:
warning_message([f'Invalid email address found: {ascii_mailbox}@{ascii_hostname}'])
self._unicode = ascii_mailbox + '@' + ascii_hostname
return self._unicode
def __ne__(self, other):

421
jc/parsers/bluetoothctl.py Normal file
View File

@@ -0,0 +1,421 @@
"""jc - JSON Convert `bluetoothctl` command output parser
Supports the following `bluetoothctl` subcommands:
- `bluetoothctl list`
- `bluetoothctl show`
- `bluetoothctl show <ctrl>`
- `bluetoothctl devices`
- `bluetoothctl info <dev>`
Usage (cli):
$ bluetoothctl info <dev> | jc --bluetoothctl
or
$ jc bluetoothctl info <dev>
Usage (module):
import jc
result = jc.parse('bluetoothctl', bluetoothctl_command_output)
Schema:
Because bluetoothctl is handling two main entities, controllers and devices,
the schema is shared between them. Most of the fields are common between
a controller and a device but there might be fields corresponding to one entity.
Controller:
[
{
"name": string,
"is_default": boolean,
"is_public": boolean,
"is_random": boolean,
"address": string,
"alias": string,
"class": string,
"powered": string,
"discoverable": string,
"discoverable_timeout": string,
"pairable": string,
"modalias": string,
"discovering": string,
"uuids": array
}
]
Device:
[
{
"name": string,
"is_public": boolean,
"is_random": boolean,
"address": string,
"alias": string,
"appearance": string,
"class": string,
"icon": string,
"paired": string,
"bonded": string,
"trusted": string,
"blocked": string,
"connected": string,
"legacy_pairing": string,
"rssi": int,
"txpower": int,
"uuids": array,
"modalias": string
}
]
Examples:
$ bluetoothctl info EB:06:EF:62:B3:19 | jc --bluetoothctl -p
[
{
"address": "22:06:33:62:B3:19",
"is_public": true,
"name": "TaoTronics TT-BH336",
"alias": "TaoTronics TT-BH336",
"class": "0x00240455",
"icon": "audio-headset",
"paired": "no",
"bonded": "no",
"trusted": "no",
"blocked": "no",
"connected": "no",
"legacy_pairing": "no",
"uuids": [
"Advanced Audio Distribu.. (0000120d-0000-1000-8000-00805f9b34fb)",
"Audio Sink (0000130b-0000-1000-8000-00805f9b34fb)",
"A/V Remote Control (0000140e-0000-1000-8000-00805f9b34fb)",
"A/V Remote Control Cont.. (0000150f-0000-1000-8000-00805f9b34fb)",
"Handsfree (0000161e-0000-1000-8000-00805f9b34fb)",
"Headset (00001708-0000-1000-8000-00805f9b34fb)",
"Headset HS (00001831-0000-1000-8000-00805f9b34fb)"
],
"rssi": -52,
"txpower": 4
}
]
"""
import re
from typing import List, Dict, Optional, Any
from jc.jc_types import JSONDictType
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.1'
description = '`bluetoothctl` command parser'
author = 'Jake Ob'
author_email = 'iakopap at gmail.com'
compatible = ['linux']
magic_commands = ['bluetoothctl']
tags = ['command']
__version__ = info.version
try:
from typing import TypedDict
Controller = TypedDict(
"Controller",
{
"name": str,
"is_default": bool,
"is_public": bool,
"is_random": bool,
"address": str,
"alias": str,
"class": str,
"powered": str,
"discoverable": str,
"discoverable_timeout": str,
"pairable": str,
"modalias": str,
"discovering": str,
"uuids": List[str],
},
)
Device = TypedDict(
"Device",
{
"name": str,
"is_public": bool,
"is_random": bool,
"address": str,
"alias": str,
"appearance": str,
"class": str,
"icon": str,
"paired": str,
"bonded": str,
"trusted": str,
"blocked": str,
"connected": str,
"legacy_pairing": str,
"rssi": int,
"txpower": int,
"uuids": List[str],
"modalias": str
},
)
except ImportError:
Controller = Dict[str, Any] # type: ignore
Device = Dict[str, Any] # type: ignore
_controller_head_pattern = r"Controller (?P<address>([0-9A-F]{2}:){5}[0-9A-F]{2}) (?P<name>.+)"
_controller_line_pattern = (
r"(\s*Name:\s*(?P<name>.+)"
+ r"|\s*Alias:\s*(?P<alias>.+)"
+ r"|\s*Class:\s*(?P<class>.+)"
+ r"|\s*Powered:\s*(?P<powered>.+)"
+ r"|\s*Discoverable:\s*(?P<discoverable>.+)"
+ r"|\s*DiscoverableTimeout:\s*(?P<discoverable_timeout>.+)"
+ r"|\s*Pairable:\s*(?P<pairable>.+)"
+ r"|\s*Modalias:\s*(?P<modalias>.+)"
+ r"|\s*Discovering:\s*(?P<discovering>.+)"
+ r"|\s*UUID:\s*(?P<uuid>.+))"
)
def _parse_controller(next_lines: List[str]) -> Optional[Controller]:
next_line = next_lines.pop()
result = re.match(_controller_head_pattern, next_line)
if not result:
next_lines.append(next_line)
return None
matches = result.groupdict()
name = matches["name"]
if name.endswith("not available"):
return None
controller: Controller = {
"name": '',
"is_default": False,
"is_public": False,
"is_random": False,
"address": matches["address"],
"alias": '',
"class": '',
"powered": '',
"discoverable": '',
"discoverable_timeout": '',
"pairable": '',
"modalias": '',
"discovering": '',
"uuids": [],
}
if name.endswith("[default]"):
controller["is_default"] = True
name = name.replace("[default]", "")
elif name.endswith("(public)"):
controller["is_public"] = True
name = name.replace("(public)", "")
elif name.endswith("(random)"):
controller["is_random"] = True
name = name.replace("(random)", "")
controller["name"] = name.strip()
while next_lines:
next_line = next_lines.pop()
result = re.match(_controller_line_pattern, next_line)
if not result:
next_lines.append(next_line)
return controller
matches = result.groupdict()
if matches["name"]:
controller["name"] = matches["name"]
elif matches["alias"]:
controller["alias"] = matches["alias"]
elif matches["class"]:
controller["class"] = matches["class"]
elif matches["powered"]:
controller["powered"] = matches["powered"]
elif matches["discoverable"]:
controller["discoverable"] = matches["discoverable"]
elif matches["discoverable_timeout"]:
controller["discoverable_timeout"] = matches["discoverable_timeout"]
elif matches["pairable"]:
controller["pairable"] = matches["pairable"]
elif matches["modalias"]:
controller["modalias"] = matches["modalias"]
elif matches["discovering"]:
controller["discovering"] = matches["discovering"]
elif matches["uuid"]:
if not "uuids" in controller:
controller["uuids"] = []
controller["uuids"].append(matches["uuid"])
return controller
_device_head_pattern = r"Device (?P<address>([0-9A-F]{2}:){5}[0-9A-F]{2}) (?P<name>.+)"
_device_line_pattern = (
r"(\s*Name:\s*(?P<name>.+)"
+ r"|\s*Alias:\s*(?P<alias>.+)"
+ r"|\s*Appearance:\s*(?P<appearance>.+)"
+ r"|\s*Class:\s*(?P<class>.+)"
+ r"|\s*Icon:\s*(?P<icon>.+)"
+ r"|\s*Paired:\s*(?P<paired>.+)"
+ r"|\s*Bonded:\s*(?P<bonded>.+)"
+ r"|\s*Trusted:\s*(?P<trusted>.+)"
+ r"|\s*Blocked:\s*(?P<blocked>.+)"
+ r"|\s*Connected:\s*(?P<connected>.+)"
+ r"|\s*LegacyPairing:\s*(?P<legacy_pairing>.+)"
+ r"|\s*Modalias:\s*(?P<modalias>.+)"
+ r"|\s*RSSI:\s*(?P<rssi>.+)"
+ r"|\s*TxPower:\s*(?P<txpower>.+)"
+ r"|\s*UUID:\s*(?P<uuid>.+))"
)
def _parse_device(next_lines: List[str], quiet: bool) -> Optional[Device]:
next_line = next_lines.pop()
result = re.match(_device_head_pattern, next_line)
if not result:
next_lines.append(next_line)
return None
matches = result.groupdict()
name = matches["name"]
if name.endswith("not available"):
return None
device: Device = {
"name": '',
"is_public": False,
"is_random": False,
"address": matches["address"],
"alias": '',
"appearance": '',
"class": '',
"icon": '',
"paired": '',
"bonded": '',
"trusted": '',
"blocked": '',
"connected": '',
"legacy_pairing": '',
"rssi": 0,
"txpower": 0,
"uuids": [],
"modalias": ''
}
if name.endswith("(public)"):
device["is_public"] = True
name = name.replace("(public)", "")
elif name.endswith("(random)"):
device["is_random"] = True
name = name.replace("(random)", "")
device["name"] = name.strip()
while next_lines:
next_line = next_lines.pop()
result = re.match(_device_line_pattern, next_line)
if not result:
next_lines.append(next_line)
return device
matches = result.groupdict()
if matches["name"]:
device["name"] = matches["name"]
elif matches["alias"]:
device["alias"] = matches["alias"]
elif matches["appearance"]:
device["appearance"] = matches["appearance"]
elif matches["class"]:
device["class"] = matches["class"]
elif matches["icon"]:
device["icon"] = matches["icon"]
elif matches["paired"]:
device["paired"] = matches["paired"]
elif matches["bonded"]:
device["bonded"] = matches["bonded"]
elif matches["trusted"]:
device["trusted"] = matches["trusted"]
elif matches["blocked"]:
device["blocked"] = matches["blocked"]
elif matches["connected"]:
device["connected"] = matches["connected"]
elif matches["legacy_pairing"]:
device["legacy_pairing"] = matches["legacy_pairing"]
elif matches["rssi"]:
rssi = matches["rssi"]
try:
device["rssi"] = int(rssi)
except ValueError:
if not quiet:
jc.utils.warning_message([f"{next_line} : rssi - {rssi} is not int-able"])
elif matches["txpower"]:
txpower = matches["txpower"]
try:
device["txpower"] = int(txpower)
except ValueError:
if not quiet:
jc.utils.warning_message([f"{next_line} : txpower - {txpower} is not int-able"])
elif matches["uuid"]:
if not "uuids" in device:
device["uuids"] = []
device["uuids"].append(matches["uuid"])
elif matches["modalias"]:
device["modalias"] = matches["modalias"]
return device
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[JSONDictType]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
result: List = []
if jc.utils.has_data(data):
linedata = data.splitlines()
linedata.reverse()
while linedata:
element = None
if data.startswith("Controller"):
element = _parse_controller(linedata)
elif data.startswith("Device"):
element = _parse_device(linedata, quiet) # type: ignore
if element:
result.append(element)
else:
break
return result

263
jc/parsers/certbot.py Normal file
View File

@@ -0,0 +1,263 @@
"""jc - JSON Convert `certbot` command output parser
Supports the following `certbot` commands:
- `certbot show_account`
- `certbot certificates`
Verbose options are not supported.
Usage (cli):
$ certbot show_account | jc --certbot
$ certbot certificates | jc --certbot
or
$ jc certbot show_account
$ jc certbot certificates
Usage (module):
import jc
result = jc.parse('certbot', certbot_command_output)
Schema:
{
"certificates": [
{
"name": string,
"serial_number": string,
"key_type": string,
"domains": [
string
],
"expiration_date": string,
"expiration_date_epoch": integer,
"expiration_date_epoch_utc": integer,
"expiration_date_iso": string,
"validity": string,
"certificate_path": string,
"private_key_path": string
}
],
"account": {
"server": string,
"url": string,
"email": string
}
}
Examples:
$ certbot certificates | jc --certbot -p
{
"certificates": [
{
"name": "example.com",
"serial_number": "3f7axxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"key_type": "RSA",
"domains": [
"example.com",
"www.example.com"
],
"expiration_date": "2023-05-11 01:33:10+00:00",
"validity": "63 days",
"certificate_path": "/etc/letsencrypt/live/example.com/chain.pem",
"private_key_path": "/etc/letsencrypt/live/example.com/priv.pem",
"expiration_date_epoch": 1683793990,
"expiration_date_epoch_utc": 1683768790,
"expiration_date_iso": "2023-05-11T01:33:10+00:00"
},
{
"name": "example.org",
"serial_number": "3bcyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy",
"key_type": "RSA",
"domains": [
"example.org",
"www.example.org"
],
"expiration_date": "2023-06-12 01:35:30+00:00",
"validity": "63 days",
"certificate_path": "/etc/letsencrypt/live/example.org/chain.pem",
"private_key_path": "/etc/letsencrypt/live/example.org/key.pem",
"expiration_date_epoch": 1686558930,
"expiration_date_epoch_utc": 1686533730,
"expiration_date_iso": "2023-06-12T01:35:30+00:00"
}
]
}
$ certbot certificates | jc --certbot -p -r
{
"certificates": [
{
"name": "example.com",
"serial_number": "3f7axxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"key_type": "RSA",
"domains": [
"example.com",
"www.example.com"
],
"expiration_date": "2023-05-11 01:33:10+00:00",
"validity": "63 days",
"certificate_path": "/etc/letsencrypt/live/example.com/chain.pem",
"private_key_path": "/etc/letsencrypt/live/example.com/priv.pem"
},
{
"name": "example.org",
"serial_number": "3bcyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy",
"key_type": "RSA",
"domains": [
"example.org",
"www.example.org"
],
"expiration_date": "2023-06-12 01:35:30+00:00",
"validity": "63 days",
"certificate_path": "/etc/letsencrypt/live/example.org/chain.pem",
"private_key_path": "/etc/letsencrypt/live/example.org/key.pem"
}
]
}
$ certbot show_account | jc --certbot -p
{
"account": {
"server": "https://acme-staging-v02.api.letsencrypt.org/directory",
"url": "https://acme-staging-v02.api.letsencrypt.org/acme/acct/123",
"email": "some@example.com"
}
}
"""
import re
from typing import List, Dict
from jc.jc_types import JSONDictType
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.1'
description = '`certbot` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux', 'darwin', 'cygwin', 'win32', 'aix', 'freebsd']
tags = ['command']
magic_commands = ['certbot']
__version__ = info.version
def _process(proc_data: JSONDictType) -> JSONDictType:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
Dictionary. Structured to conform to the schema.
"""
if 'certificates' in proc_data:
for cert in proc_data['certificates']:
if 'expiration_date' in cert:
dt = jc.utils.timestamp(cert['expiration_date'], format_hint=(1760,))
cert['expiration_date_epoch'] = dt.naive
cert['expiration_date_epoch_utc'] = dt.utc
cert['expiration_date_iso'] = dt.iso
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> JSONDictType:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: Dict = {}
cert_list: List = []
cert_dict: Dict = {}
acct_dict: Dict = {}
cmd_option = ''
if jc.utils.has_data(data):
cert_pattern = re.compile(r'^Found the following certs:$', re.MULTILINE)
if re.search(cert_pattern, data):
cmd_option = 'certificates'
else:
cmd_option = 'account'
for line in filter(None, data.splitlines()):
if cmd_option == 'certificates':
if line.startswith(' Certificate Name:'):
if cert_dict:
cert_list.append(cert_dict)
cert_dict = {}
cert_dict['name'] = line.split()[-1]
if line.startswith(' Serial Number:'):
cert_dict['serial_number'] = line.split()[-1]
if line.startswith(' Key Type:'):
cert_dict['key_type'] = line.split(': ', maxsplit=1)[1]
if line.startswith(' Domains:'):
splitline = line.split(': ', maxsplit=1)[1]
cert_dict['domains'] = splitline.split()
if line.startswith(' Expiry Date:'):
splitline = line.split(': ', maxsplit=1)[1]
cert_datetime = splitline.split('(')[0]
validity = splitline.split('(')[1]
cert_dict['expiration_date'] = cert_datetime.strip()
cert_dict['validity'] = validity[:-1].replace('VALID: ', '')
if line.startswith(' Certificate Path:'):
cert_dict['certificate_path'] = line.split(': ', maxsplit=1)[1]
if line.startswith(' Private Key Path:'):
cert_dict['private_key_path'] = line.split(': ', maxsplit=1)[1]
if cmd_option == 'account':
if line.startswith('Account details for server'):
acct_dict['server'] = line.split()[-1][:-1]
if line.startswith(' Account URL:'):
acct_dict['url'] = line.split()[-1]
if line.startswith(' Email contact:'):
acct_dict['email'] = line.split()[-1]
if acct_dict:
raw_output['account'] = acct_dict
if cert_dict:
cert_list.append(cert_dict)
if cert_list:
raw_output['certificates'] = cert_list
return raw_output if raw else _process(raw_output)

View File

@@ -174,7 +174,7 @@ import jc.parsers.universal
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.6'
version = '1.8'
description = '`crontab` command and file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -245,7 +245,11 @@ def parse(data, raw=False, quiet=False):
# Pop any variable assignment lines
cron_var = []
for i, line in reversed(list(enumerate(cleandata))):
if '=' in line and not line.strip()[0].isdigit() and not line.strip()[0] == '@':
if '=' in line \
and not line.strip()[0].isdigit() \
and not line.strip()[0] == '@' \
and not line.strip()[0] == '*':
var_line = cleandata.pop(i)
var_name = var_line.split('=', maxsplit=1)[0].strip()
var_value = var_line.split('=', maxsplit=1)[1].strip()
@@ -273,6 +277,9 @@ def parse(data, raw=False, quiet=False):
raw_output['schedule'] = cron_list
# Add shortcut entries back in
if 'schedule' not in raw_output:
raw_output['schedule'] = []
for item in shortcut_list:
raw_output['schedule'].append(item)

View File

@@ -171,7 +171,7 @@ import jc.parsers.universal
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.7'
version = '1.9'
description = '`crontab` file parser with user support'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -241,7 +241,11 @@ def parse(data, raw=False, quiet=False):
# Pop any variable assignment lines
cron_var = []
for i, line in reversed(list(enumerate(cleandata))):
if '=' in line and not line.strip()[0].isdigit() and not line.strip()[0] == '@':
if '=' in line \
and not line.strip()[0].isdigit() \
and not line.strip()[0] == '@' \
and not line.strip()[0] == '*':
var_line = cleandata.pop(i)
var_name = var_line.split('=', maxsplit=1)[0].strip()
var_value = var_line.split('=', maxsplit=1)[1].strip()
@@ -271,6 +275,9 @@ def parse(data, raw=False, quiet=False):
raw_output['schedule'] = cron_list
# Add shortcut entries back in
if 'schedule' not in raw_output:
raw_output['schedule'] = []
for item in shortcut_list:
raw_output['schedule'].append(item)

View File

@@ -4,6 +4,7 @@ Options supported:
- `+noall +answer` options are supported in cases where only the answer
information is desired.
- `+axfr` option is supported on its own
- `+nsid` option is supported
The `when_epoch` calculated timestamp field is naive. (i.e. based on the
local time of the system the parser is run on)
@@ -322,7 +323,7 @@ import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '2.4'
version = '2.5'
description = '`dig` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -427,6 +428,7 @@ def _parse_flags_line(flagsline):
def _parse_opt_pseudosection(optline):
# ;; OPT PSEUDOSECTION:
# ; EDNS: version: 0, flags:; udp: 4096
# ; NSID: 67 70 64 6e 73 2d 73 66 6f ("gpdns-sfo")
# ; COOKIE: 1cbc06703eaef210
if optline.startswith('; EDNS:'):
optline_list = optline.replace(',', ' ').split(';')
@@ -443,11 +445,18 @@ def _parse_opt_pseudosection(optline):
}
}
elif optline.startswith('; COOKIE:'):
if optline.startswith('; COOKIE:'):
return {
'cookie': optline.split()[2]
}
if optline.startswith('; NSID:'):
return {
'nsid': optline.split('("')[-1].rstrip('")')
}
return {}
def _parse_question(question):
# ;www.cnn.com. IN A

View File

@@ -149,11 +149,11 @@ from typing import List, Dict
import jc.utils
hash_pattern = re.compile(r'(?:[0-9]|[a-f]){40}')
changes_pattern = re.compile(r'\s(?P<files>\d+)\s+(files? changed),\s+(?P<insertions>\d+)\s(insertions?\(\+\))?(,\s+)?(?P<deletions>\d+)?(\s+deletions?\(\-\))?')
changes_pattern = re.compile(r'\s(?P<files>\d+)\s+(files? changed)(?:,\s+(?P<insertions>\d+)\s+(insertions?\(\+\)))?(?:,\s+(?P<deletions>\d+)\s+(deletions?\(\-\)))?')
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.3'
version = '1.4'
description = '`git log` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

View File

@@ -83,12 +83,12 @@ from jc.exceptions import ParseError
hash_pattern = re.compile(r'(?:[0-9]|[a-f]){40}')
changes_pattern = re.compile(r'\s(?P<files>\d+)\s+(files? changed),\s+(?P<insertions>\d+)\s(insertions?\(\+\))?(,\s+)?(?P<deletions>\d+)?(\s+deletions?\(\-\))?')
changes_pattern = re.compile(r'\s(?P<files>\d+)\s+(files? changed)(?:,\s+(?P<insertions>\d+)\s+(insertions?\(\+\)))?(?:,\s+(?P<deletions>\d+)\s+(deletions?\(\-\)))?')
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.3'
version = '1.4'
description = '`git log` command streaming parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'

View File

@@ -219,7 +219,7 @@ import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '2.2'
version = '2.3'
description = '`ifconfig` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -425,18 +425,18 @@ def parse(
# Linux syntax
re_linux_interface = re.compile(r'''
(?P<name>[a-zA-Z0-9:._-]+)\s+
Link encap:(?P<type>\S+\s?\S+)
Link\sencap:(?P<type>\S+\s?\S+)
(\s+HWaddr\s+\b(?P<mac_addr>[0-9A-Fa-f:?]+))?
''', re.IGNORECASE | re.VERBOSE
)
re_linux_ipv4 = re.compile(r'''
inet addr:(?P<address>(?:[0-9]{1,3}\.){3}[0-9]{1,3})(\s+
inet\saddr:(?P<address>(?:[0-9]{1,3}\.){3}[0-9]{1,3})(\s+
Bcast:(?P<broadcast>(?:[0-9]{1,3}\.){3}[0-9]{1,3}))?\s+
Mask:(?P<mask>(?:[0-9]{1,3}\.){3}[0-9]{1,3})
''', re.IGNORECASE | re.VERBOSE
)
re_linux_ipv6 = re.compile(r'''
inet6 addr:\s+(?P<address>\S+)/
inet6\saddr:\s+(?P<address>\S+)/
(?P<mask>[0-9]+)\s+
Scope:(?P<scope>Link|Host)
''', re.IGNORECASE | re.VERBOSE
@@ -448,7 +448,7 @@ def parse(
''', re.IGNORECASE | re.VERBOSE
)
re_linux_rx = re.compile(r'''
RX packets:(?P<rx_packets>[0-9]+)\s+
RX\spackets:(?P<rx_packets>[0-9]+)\s+
errors:(?P<rx_errors>[0-9]+)\s+
dropped:(?P<rx_dropped>[0-9]+)\s+
overruns:(?P<rx_overruns>[0-9]+)\s+
@@ -456,7 +456,7 @@ def parse(
''', re.IGNORECASE | re.VERBOSE
)
re_linux_tx = re.compile(r'''
TX packets:(?P<tx_packets>[0-9]+)\s+
TX\spackets:(?P<tx_packets>[0-9]+)\s+
errors:(?P<tx_errors>[0-9]+)\s+
dropped:(?P<tx_dropped>[0-9]+)\s+
overruns:(?P<tx_overruns>[0-9]+)\s+
@@ -464,8 +464,8 @@ def parse(
''', re.IGNORECASE | re.VERBOSE
)
re_linux_bytes = re.compile(r'''
\W+RX bytes:(?P<rx_bytes>\d+)\s+\(.*\)\s+
TX bytes:(?P<tx_bytes>\d+)\s+\(.*\)
\W+RX\sbytes:(?P<rx_bytes>\d+)\s+\(.*\)\s+
TX\sbytes:(?P<tx_bytes>\d+)\s+\(.*\)
''', re.IGNORECASE | re.VERBOSE
)
re_linux_tx_stats = re.compile(r'''

View File

@@ -1,10 +1,12 @@
"""jc - JSON Convert `INI` file parser
"""jc - JSON Convert INI file parser
Parses standard `INI` files and files containing simple key/value pairs.
Parses standard INI files.
- Delimiter can be `=` or `:`. Missing values are supported.
- Comment prefix can be `#` or `;`. Comments must be on their own line.
- If duplicate keys are found, only the last value will be used.
- If any section names have the same name as a top-level key, the top-level
key will be overwritten by the section data.
> Note: Values starting and ending with double or single quotation marks
> will have the marks removed. If you would like to keep the quotation
@@ -22,59 +24,62 @@ Usage (module):
Schema:
ini or key/value document converted to a dictionary - see the configparser
INI document converted to a dictionary - see the python configparser
standard library documentation for more details.
{
"key1": string,
"key2": string
"<key1>": string,
"<key2>": string,
"<section1>": {
"<key1>": string,
"<key2>": string
},
"<section2>": {
"<key1>": string,
"<key2>": string
}
}
Examples:
$ cat example.ini
[DEFAULT]
ServerAliveInterval = 45
Compression = yes
CompressionLevel = 9
ForwardX11 = yes
foo = fiz
bar = buz
[bitbucket.org]
User = hg
[section1]
fruit = apple
color = blue
[topsecret.server.com]
Port = 50022
ForwardX11 = no
[section2]
fruit = pear
color = green
$ cat example.ini | jc --ini -p
{
"bitbucket.org": {
"ServerAliveInterval": "45",
"Compression": "yes",
"CompressionLevel": "9",
"ForwardX11": "yes",
"User": "hg"
"foo": "fiz",
"bar": "buz",
"section1": {
"fruit": "apple",
"color": "blue"
},
"topsecret.server.com": {
"ServerAliveInterval": "45",
"Compression": "yes",
"CompressionLevel": "9",
"ForwardX11": "no",
"Port": "50022"
"section2": {
"fruit": "pear",
"color": "green"
}
}
"""
import jc.utils
import configparser
import uuid
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.8'
version = '2.0'
description = 'INI file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
details = 'Using configparser from the standard library'
details = 'Using configparser from the python standard library'
compatible = ['linux', 'darwin', 'cygwin', 'win32', 'aix', 'freebsd']
tags = ['standard', 'file', 'string']
@@ -82,6 +87,19 @@ class info():
__version__ = info.version
def _remove_quotes(value):
if value is None:
value = ''
elif value.startswith('"') and value.endswith('"'):
value = value[1:-1]
elif value.startswith("'") and value.endswith("'"):
value = value[1:-1]
return value
def _process(proc_data):
"""
Final processing to conform to the schema.
@@ -92,32 +110,16 @@ def _process(proc_data):
Returns:
Dictionary representing an ini or simple key/value pair document.
Dictionary representing the INI file.
"""
# remove quotation marks from beginning and end of values
for heading in proc_data:
# standard ini files with headers
if isinstance(proc_data[heading], dict):
for key, value in proc_data[heading].items():
if value is not None and value.startswith('"') and value.endswith('"'):
proc_data[heading][key] = value.lstrip('"').rstrip('"')
for k, v in proc_data.items():
if isinstance(v, dict):
for key, value in v.items():
v[key] = _remove_quotes(value)
continue
elif value is not None and value.startswith("'") and value.endswith("'"):
proc_data[heading][key] = value.lstrip("'").rstrip("'")
elif value is None:
proc_data[heading][key] = ''
# simple key/value files with no headers
else:
if proc_data[heading] is not None and proc_data[heading].startswith('"') and proc_data[heading].endswith('"'):
proc_data[heading] = proc_data[heading].lstrip('"').rstrip('"')
elif proc_data[heading] is not None and proc_data[heading].startswith("'") and proc_data[heading].endswith("'"):
proc_data[heading] = proc_data[heading].lstrip("'").rstrip("'")
elif proc_data[heading] is None:
proc_data[heading] = ''
proc_data[k] = _remove_quotes(v)
return proc_data
@@ -134,7 +136,7 @@ def parse(data, raw=False, quiet=False):
Returns:
Dictionary representing the ini file
Dictionary representing the INI file.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
@@ -143,25 +145,36 @@ def parse(data, raw=False, quiet=False):
if jc.utils.has_data(data):
ini = configparser.ConfigParser(allow_no_value=True,
interpolation=None,
strict=False)
ini_parser = configparser.ConfigParser(
allow_no_value=True,
interpolation=None,
default_section=None,
strict=False
)
# don't convert keys to lower-case:
ini.optionxform = lambda option: option
ini_parser.optionxform = lambda option: option
try:
ini.read_string(data)
raw_output = {s: dict(ini.items(s)) for s in ini.sections()}
ini_parser.read_string(data)
raw_output = {s: dict(ini_parser.items(s)) for s in ini_parser.sections()}
except configparser.MissingSectionHeaderError:
data = '[data]\n' + data
ini.read_string(data)
output_dict = {s: dict(ini.items(s)) for s in ini.sections()}
for key, value in output_dict['data'].items():
raw_output[key] = value
# find a top-level section name that will not collide with any existing ones
while True:
my_uuid = str(uuid.uuid4())
if my_uuid not in data:
break
data = f'[{my_uuid}]\n' + data
ini_parser.read_string(data)
temp_dict = {s: dict(ini_parser.items(s)) for s in ini_parser.sections()}
# move items under fake top-level sections to the root
raw_output = temp_dict.pop(my_uuid)
# get the rest of the sections
raw_output.update(temp_dict)
return raw_output if raw else _process(raw_output)
if raw:
return raw_output
else:
return _process(raw_output)

225
jc/parsers/ini_dup.py Normal file
View File

@@ -0,0 +1,225 @@
"""jc - JSON Convert INI with duplicate key file parser
Parses standard INI files and preserves duplicate values. All values are
contained in lists/arrays.
- Delimiter can be `=` or `:`. Missing values are supported.
- Comment prefix can be `#` or `;`. Comments must be on their own line.
- If any section names have the same name as a top-level key, the top-level
key will be overwritten by the section data.
- If multi-line values are used, each line will be a separate item in the
value list. Blank lines in multi-line values are not supported.
> Note: Values starting and ending with double or single quotation marks
> will have the marks removed. If you would like to keep the quotation
> marks, use the `-r` command-line argument or the `raw=True` argument in
> `parse()`.
Usage (cli):
$ cat foo.ini | jc --ini
Usage (module):
import jc
result = jc.parse('ini', ini_file_output)
Schema:
INI document converted to a dictionary - see the python configparser
standard library documentation for more details.
{
"<key1>": [
string
],
"<key2>": [
string
],
"<section1>": {
"<key1>": [
string
],
"<key2>": [
string
]
}
}
Examples:
$ cat example.ini
foo = fiz
bar = buz
[section1]
fruit = apple
color = blue
color = red
[section2]
fruit = pear
fruit = peach
color = green
$ cat example.ini | jc --ini -p
{
"foo": [
"fiz"
],
"bar": [
"buz"
],
"section1": {
"fruit": [
"apple"
],
"color": [
"blue",
"red"
]
},
"section2": {
"fruit": [
"pear",
"peach"
],
"color": [
"green"
]
}
}
"""
import jc.utils
import configparser
import uuid
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = 'INI with duplicate key file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
details = 'Using configparser from the python standard library'
compatible = ['linux', 'darwin', 'cygwin', 'win32', 'aix', 'freebsd']
tags = ['standard', 'file', 'string']
__version__ = info.version
class MultiDict(dict):
# https://stackoverflow.com/a/38286559/12303989
def __setitem__(self, key, value):
if key in self:
if isinstance(value, list):
self[key].extend(value)
elif isinstance(value, str):
if len(self[key])>1:
return
else:
super().__setitem__(key, value)
def _remove_quotes(value):
if value is None:
value = ''
elif value.startswith('"') and value.endswith('"'):
value = value[1:-1]
elif value.startswith("'") and value.endswith("'"):
value = value[1:-1]
return value
def _process(proc_data):
"""
Final processing to conform to the schema.
Parameters:
proc_data: (Dictionary) raw structured data to process
Returns:
Dictionary representing the INI file.
"""
# remove quotation marks from beginning and end of values
for k, v in proc_data.items():
if isinstance(v, dict):
for key, value in v.items():
if isinstance(value, list):
v[key] = [_remove_quotes(x) for x in value]
else:
v[key] = _remove_quotes(value)
continue
elif isinstance(v, list):
proc_data[k] = [_remove_quotes(x) for x in v]
else:
proc_data[k] = _remove_quotes(v)
return proc_data
def parse(data, raw=False, quiet=False):
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary representing the INI file.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output = {}
if jc.utils.has_data(data):
ini_parser = configparser.ConfigParser(
dict_type = MultiDict,
allow_no_value=True,
interpolation=None,
default_section=None,
empty_lines_in_values=False,
strict=False
)
# don't convert keys to lower-case:
ini_parser.optionxform = lambda option: option
try:
ini_parser.read_string(data)
raw_output = {s: dict(ini_parser.items(s)) for s in ini_parser.sections()}
except configparser.MissingSectionHeaderError:
# find a top-level section name that will not collide with any existing ones
while True:
my_uuid = str(uuid.uuid4())
if my_uuid not in data:
break
data = f'[{my_uuid}]\n' + data
ini_parser.read_string(data)
temp_dict = {s: dict(ini_parser.items(s)) for s in ini_parser.sections()}
# move items under fake top-level sections to the root
raw_output = temp_dict.pop(my_uuid)
# get the rest of the sections
raw_output.update(temp_dict)
return raw_output if raw else _process(raw_output)

View File

@@ -46,7 +46,6 @@ Schema:
}
]
Examples:
$ iwconfig | jc --iwconfig -p
@@ -78,7 +77,6 @@ Examples:
"missed_beacon": 0
}
]
"""
import re
from typing import List, Dict
@@ -87,7 +85,7 @@ import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
version = '1.1'
description = '`iwconfig` command parser'
author = 'Thomas Vincent'
author_email = 'vrince@gmail.com'
@@ -148,7 +146,7 @@ def parse(
raw_output: List[Dict] = []
re_interface = re.compile(r'^(?P<name>[a-zA-Z0-9:._-]+)\s+(?P<protocol>([a-zA-Z0-9]+\s)*[a-zA-Z0-9.]+)\s+ESSID:\"(?P<essid>[a-zA-Z0-9:._\s]+)\"')
re_interface = re.compile(r'^(?P<name>[a-zA-Z0-9:._\-]+)\s+(?P<protocol>([a-zA-Z0-9]+\s)*[a-zA-Z0-9.]+)\s+ESSID:\"(?P<essid>[a-zA-Z0-9:._\s\-]+)\"')
re_mode = re.compile(r'Mode:(?P<mode>\w+)')
re_frequency = re.compile(r'Frequency:(?P<frequency>[0-9.]+)\s(?P<frequency_unit>\w+)')
re_access_point = re.compile(r'Access Point:\s*(?P<access_point>[0-9A-F:]+)')

View File

@@ -21,8 +21,8 @@ Usage (module):
Schema:
key/value document converted to a dictionary - see the configparser standard
library documentation for more details.
Key/Value document converted to a dictionary - see the python configparser
standard library documentation for more details.
{
"key1": string,
@@ -36,6 +36,7 @@ Examples:
name = John Doe
address=555 California Drive
age: 34
; comments can include # or ;
# delimiter can be = or :
# quoted values have quotation marks stripped by default
@@ -50,15 +51,17 @@ Examples:
"occupation": "Engineer"
}
"""
import jc.utils
import configparser
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.2'
version = '2.0'
description = 'Key/Value file and string parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
details = 'This is a wrapper for the INI parser'
details = 'Using configparser from the python standard library'
compatible = ['linux', 'darwin', 'cygwin', 'win32', 'aix', 'freebsd']
tags = ['generic', 'file', 'string']
@@ -66,12 +69,36 @@ class info():
__version__ = info.version
def _process(proc_data):
"""
Final processing to conform to the schema.
Parameters:
proc_data: (Dictionary) raw structured data to process
Returns:
Dictionary representing a Key/Value pair document.
"""
# remove quotation marks from beginning and end of values
for key in proc_data:
if proc_data[key] is None:
proc_data[key] = ''
elif proc_data[key].startswith('"') and proc_data[key].endswith('"'):
proc_data[key] = proc_data[key][1:-1]
elif proc_data[key].startswith("'") and proc_data[key].endswith("'"):
proc_data[key] = proc_data[key][1:-1]
return proc_data
def parse(data, raw=False, quiet=False):
"""
Main text parsing function
Note: this is just a wrapper for jc.parsers.ini
Parameters:
data: (string) text data to parse
@@ -80,7 +107,30 @@ def parse(data, raw=False, quiet=False):
Returns:
Dictionary representing the key/value file
Dictionary representing a Key/Value pair document.
"""
import jc.parsers.ini
return jc.parsers.ini.parse(data, raw=raw, quiet=quiet)
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output = {}
if jc.utils.has_data(data):
kv_parser = configparser.ConfigParser(
allow_no_value=True,
interpolation=None,
default_section=None,
strict=False
)
# don't convert keys to lower-case:
kv_parser.optionxform = lambda option: option
data = '[data]\n' + data
kv_parser.read_string(data)
output_dict = {s: dict(kv_parser.items(s)) for s in kv_parser.sections()}
for key, value in output_dict['data'].items():
raw_output[key] = value
return raw_output if raw else _process(raw_output)

View File

@@ -1,6 +1,6 @@
"""jc - JSON Convert `last` and `lastb` command output parser
Supports `-w` and `-F` options.
Supports `-w`, `-F`, and `-x` options.
Calculated epoch time fields are naive (i.e. based on the local time of the
system the parser is run on) since there is no timezone information in the
@@ -103,10 +103,15 @@ Examples:
import re
import jc.utils
DATE_RE = re.compile(r'[MTWFS][ouerha][nedritnu] [JFMASOND][aepuco][nbrynlgptvc]')
LAST_F_DATE_RE = re.compile(r'\d\d:\d\d:\d\d \d\d\d\d')
LOGIN_LOGOUT_EPOCH_RE = re.compile(r'.*\d\d:\d\d:\d\d \d\d\d\d.*')
LOGOUT_IGNORED_EVENTS = ['down', 'crash']
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.8'
version = '1.9'
description = '`last` and `lastb` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -138,9 +143,6 @@ def _process(proc_data):
if 'tty' in entry and entry['tty'] == '~':
entry['tty'] = None
if 'tty' in entry and entry['tty'] == 'system_boot':
entry['tty'] = 'system boot'
if 'hostname' in entry and entry['hostname'] == '-':
entry['hostname'] = None
@@ -153,11 +155,11 @@ def _process(proc_data):
if 'logout' in entry and entry['logout'] == 'gone_-_no_logout':
entry['logout'] = 'gone - no logout'
if 'login' in entry and re.match(r'.*\d\d:\d\d:\d\d \d\d\d\d.*', entry['login']):
if 'login' in entry and LOGIN_LOGOUT_EPOCH_RE.match(entry['login']):
timestamp = jc.utils.timestamp(entry['login'])
entry['login_epoch'] = timestamp.naive
if 'logout' in entry and re.match(r'.*\d\d:\d\d:\d\d \d\d\d\d.*', entry['logout']):
if 'logout' in entry and LOGIN_LOGOUT_EPOCH_RE.match(entry['logout']):
timestamp = jc.utils.timestamp(entry['logout'])
entry['logout_epoch'] = timestamp.naive
@@ -194,66 +196,71 @@ def parse(data, raw=False, quiet=False):
# Clear any blank lines
cleandata = list(filter(None, data.splitlines()))
if jc.utils.has_data(data):
if not jc.utils.has_data(data):
return []
for entry in cleandata:
output_line = {}
for entry in cleandata:
output_line = {}
if (entry.startswith('wtmp begins ') or
entry.startswith('btmp begins ') or
entry.startswith('utx.log begins ')):
if any(
entry.startswith(f'{prefix} begins ')
for prefix in ['wtmp', 'btmp', 'utx.log']
):
continue
continue
entry = entry.replace('boot time', 'boot_time')
entry = entry.replace(' still logged in', '- still_logged_in')
entry = entry.replace(' gone - no logout', '- gone_-_no_logout')
entry = entry.replace('system boot', 'system_boot')
entry = entry.replace('boot time', 'boot_time')
entry = entry.replace(' still logged in', '- still_logged_in')
entry = entry.replace(' gone - no logout', '- gone_-_no_logout')
linedata = entry.split()
linedata = entry.split()
if re.match(r'[MTWFS][ouerha][nedritnu] [JFMASOND][aepuco][nbrynlgptvc]', ' '.join(linedata[2:4])):
linedata.insert(2, '-')
# Adding "-" before the date part.
if DATE_RE.match(' '.join(linedata[2:4])):
linedata.insert(2, '-')
# freebsd fix
if linedata[0] == 'boot_time':
linedata.insert(1, '-')
linedata.insert(1, '~')
# freebsd fix
if linedata[0] == 'boot_time':
linedata.insert(1, '-')
linedata.insert(1, '~')
output_line['user'] = linedata[0]
output_line['tty'] = linedata[1]
output_line['hostname'] = linedata[2]
output_line['user'] = linedata[0]
# last -F support
if re.match(r'\d\d:\d\d:\d\d \d\d\d\d', ' '.join(linedata[6:8])):
output_line['login'] = ' '.join(linedata[3:8])
# Fix for last -x (runlevel).
if output_line['user'] == 'runlevel' and linedata[1] == '(to':
linedata[1] += f' {linedata.pop(2)} {linedata.pop(2)}'
elif output_line['user'] in ['reboot', 'shutdown'] and linedata[1] == 'system': # system down\system boot
linedata[1] += f' {linedata.pop(2)}'
if len(linedata) > 9 and linedata[9] != 'crash' and linedata[9] != 'down':
output_line['tty'] = linedata[1]
output_line['hostname'] = linedata[2]
# last -F support
if LAST_F_DATE_RE.match(' '.join(linedata[6:8])):
output_line['login'] = ' '.join(linedata[3:8])
if len(linedata) > 9:
if linedata[9] not in LOGOUT_IGNORED_EVENTS:
output_line['logout'] = ' '.join(linedata[9:14])
if len(linedata) > 9 and (linedata[9] == 'crash' or linedata[9] == 'down'):
else:
output_line['logout'] = linedata[9]
# add more items to the list to line up duration
linedata.insert(10, '-')
linedata.insert(10, '-')
linedata.insert(10, '-')
linedata.insert(10, '-')
for _ in range(4):
linedata.insert(10, '-')
if len(linedata) > 14:
output_line['duration'] = linedata[14].replace('(', '').replace(')', '')
if len(linedata) > 14:
output_line['duration'] = linedata[14].replace('(', '').replace(')', '')
else: # normal last support
output_line['login'] = ' '.join(linedata[3:7])
# normal last support
else:
output_line['login'] = ' '.join(linedata[3:7])
if len(linedata) > 8:
output_line['logout'] = linedata[8]
if len(linedata) > 8:
output_line['logout'] = linedata[8]
if len(linedata) > 9:
output_line['duration'] = linedata[9].replace('(', '').replace(')', '')
if len(linedata) > 9:
output_line['duration'] = linedata[9].replace('(', '').replace(')', '')
raw_output.append(output_line)
raw_output.append(output_line)
if raw:
return raw_output
else:
return _process(raw_output)
return _process(raw_output)

162
jc/parsers/lsattr.py Normal file
View File

@@ -0,0 +1,162 @@
"""jc - JSON Convert `lsattr` command output parser
Usage (cli):
$ lsattr | jc --lsattr
or
$ jc lsattr
Usage (module):
import jc
result = jc.parse('lsattr', lsattr_command_output)
Schema:
Information from https://github.com/mirror/busybox/blob/2d4a3d9e6c1493a9520b907e07a41aca90cdfd94/e2fsprogs/e2fs_lib.c#L40
used to define field names
[
{
"file": string,
"compressed_file": Optional[boolean],
"compressed_dirty_file": Optional[boolean],
"compression_raw_access": Optional[boolean],
"secure_deletion": Optional[boolean],
"undelete": Optional[boolean],
"synchronous_updates": Optional[boolean],
"synchronous_directory_updates": Optional[boolean],
"immutable": Optional[boolean],
"append_only": Optional[boolean],
"no_dump": Optional[boolean],
"no_atime": Optional[boolean],
"compression_requested": Optional[boolean],
"encrypted": Optional[boolean],
"journaled_data": Optional[boolean],
"indexed_directory": Optional[boolean],
"no_tailmerging": Optional[boolean],
"top_of_directory_hierarchies": Optional[boolean],
"extents": Optional[boolean],
"no_cow": Optional[boolean],
"casefold": Optional[boolean],
"inline_data": Optional[boolean],
"project_hierarchy": Optional[boolean],
"verity": Optional[boolean],
}
]
Examples:
$ sudo lsattr /etc/passwd | jc --lsattr
[
{
"file": "/etc/passwd",
"extents": true
}
]
"""
from typing import List, Dict
from jc.jc_types import JSONDictType
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`lsattr` command parser'
author = 'Mark Rotner'
author_email = 'rotner.mr@gmail.com'
compatible = ['linux']
magic_commands = ['lsattr']
tags = ['command']
__version__ = info.version
ERROR_PREFIX = "lsattr:"
# https://github.com/mirror/busybox/blob/2d4a3d9e6c1493a9520b907e07a41aca90cdfd94/e2fsprogs/e2fs_lib.c#L40
# https://github.com/landley/toybox/blob/f1682dc79fd75f64042b5438918fe5a507977e1c/toys/other/lsattr.c#L97
ATTRIBUTES = {
"B": "compressed_file",
"Z": "compressed_dirty_file",
"X": "compression_raw_access",
"s": "secure_deletion",
"u": "undelete",
"S": "synchronous_updates",
"D": "synchronous_directory_updates",
"i": "immutable",
"a": "append_only",
"d": "no_dump",
"A": "no_atime",
"c": "compression_requested",
"E": "encrypted",
"j": "journaled_data",
"I": "indexed_directory",
"t": "no_tailmerging",
"T": "top_of_directory_hierarchies",
"e": "extents",
"C": "no_cow",
"F": "casefold",
"N": "inline_data",
"P": "project_hierarchy",
"V": "verity",
}
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[JSONDictType]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
output: List = []
cleandata = list(filter(None, data.splitlines()))
if not jc.utils.has_data(data):
return output
for line in cleandata:
# -R flag returns the output in the format:
# Folder:
# attributes file_in_folder
if line.endswith(':'):
continue
# lsattr: Operation not supported ....
if line.startswith(ERROR_PREFIX):
continue
line_output: Dict = {}
# attributes file
# --------------e----- /etc/passwd
attributes, file = line.split()
line_output['file'] = file
for attribute in list(attributes):
attribute_key = ATTRIBUTES.get(attribute)
if attribute_key:
line_output[attribute_key] = True
if line_output:
output.append(line_output)
return output

View File

@@ -97,6 +97,46 @@ Schema:
]
}
},
"cdc_mbim": {
"<item>": {
"value": string,
"description": string,
"attributes": [
string
]
}
},
"cdc_mbim_extended": {
"<item>": {
"value": string,
"description": string,
"attributes": [
string
]
}
},
"videocontrol_descriptors": [
{
"<item>": {
"value": string,
"description": string,
"attributes": [
string
]
}
}
],
"videostreaming_descriptors": [
{
"<item>": {
"value": string,
"description": string,
"attributes": [
string
]
}
}
],
"endpoint_descriptors": [
{
"<item>": {
@@ -269,7 +309,7 @@ from jc.exceptions import ParseError
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.2'
version = '1.4'
description = '`lsusb` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -307,6 +347,145 @@ class _NestedDict(dict):
return self.setdefault(key, _NestedDict())
class _root_obj:
def __init__(self, name):
self.name = name
self.list = []
def _entries_for_this_bus_exist(self, bus_idx):
"""Returns true if there are object entries for the corresponding bus index"""
for item in self.list:
keyname = tuple(item.keys())[0]
if '_state' in item[keyname] and item[keyname]['_state']['bus_idx'] == bus_idx:
return True
return False
def _update_output(self, bus_idx, output_line):
"""modifies output_line dictionary for the corresponding bus index.
output_line is the self.output_line attribute from the _lsusb object."""
for item in self.list:
keyname = tuple(item.keys())[0]
if '_state' in item[keyname] and item[keyname]['_state']['bus_idx'] == bus_idx:
# is this a top level value or an attribute?
if item[keyname]['_state']['attribute_value']:
last_item = item[keyname]['_state']['last_item']
if 'attributes' not in output_line[f'{self.name}'][last_item]:
output_line[f'{self.name}'][last_item]['attributes'] = []
this_attribute = f'{keyname} {item[keyname].get("value", "")} {item[keyname].get("description", "")}'.strip()
output_line[f'{self.name}'][last_item]['attributes'].append(this_attribute)
continue
output_line[f'{self.name}'].update(item)
del output_line[f'{self.name}'][keyname]['_state']
class _descriptor_obj:
def __init__(self, name):
self.name = name
self.list = []
def _entries_for_this_bus_and_interface_idx_exist(self, bus_idx, iface_idx):
"""Returns true if there are object entries for the corresponding bus index
and interface index"""
for item in self.list:
keyname = tuple(item.keys())[0]
if '_state' in item[keyname] and item[keyname]['_state']['bus_idx'] == bus_idx \
and item[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
return True
return False
def _update_output(self, bus_idx, iface_idx, output_line):
"""modifies output_line dictionary for the corresponding bus index and
interface index. output_line is the i_desc_obj object."""
for item in self.list:
keyname = tuple(item.keys())[0]
if '_state' in item[keyname] and item[keyname]['_state']['bus_idx'] == bus_idx \
and item[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
# is this a top level value or an attribute?
if item[keyname]['_state']['attribute_value']:
last_item = item[keyname]['_state']['last_item']
if 'attributes' not in output_line[f'{self.name}'][last_item]:
output_line[f'{self.name}'][last_item]['attributes'] = []
this_attribute = f'{keyname} {item[keyname].get("value", "")} {item[keyname].get("description", "")}'.strip()
output_line[f'{self.name}'][last_item]['attributes'].append(this_attribute)
continue
output_line[f'{self.name}'].update(item)
del output_line[f'{self.name}'][keyname]['_state']
class _descriptor_list:
def __init__(self, name):
self.name = name
self.list = []
def _entries_for_this_bus_and_interface_idx_exist(self, bus_idx, iface_idx):
"""Returns true if there are object entries for the corresponding bus index
and interface index"""
for item in self.list:
keyname = tuple(item.keys())[0]
if '_state' in item[keyname] and item[keyname]['_state']['bus_idx'] == bus_idx \
and item[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
return True
return False
def _get_objects_list(self, bus_idx, iface_idx):
"""Returns a list of descriptor object dictionaries for the corresponding
bus index and interface index"""
object_collection = []
# find max number of items in this object that match the bus_idx and iface_idx
num_of_items = -1
for item in self.list:
keyname = tuple(item.keys())[0]
if '_state' in item[keyname] and item[keyname]['_state']['bus_idx'] == bus_idx \
and item[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
num_of_items = item[keyname]['_state'][f'{self.name}_idx']
# create and return the collection of objects that match the bus_idx and iface_idx
if num_of_items > -1:
for obj_idx in range(num_of_items + 1):
this_object = {}
for item in self.list:
keyname = tuple(item.keys())[0]
if '_state' in item[keyname] and item[keyname]['_state']['bus_idx'] == bus_idx \
and item[keyname]['_state']['interface_descriptor_idx'] == iface_idx \
and item[keyname]['_state'][f'{self.name}_idx'] == obj_idx:
# is this a top level value or an attribute?
if item[keyname]['_state']['attribute_value']:
last_item = item[keyname]['_state']['last_item']
if 'attributes' not in this_object[last_item]:
this_object[last_item]['attributes'] = []
this_attribute = f'{keyname} {item[keyname].get("value", "")} {item[keyname].get("description", "")}'.strip()
this_object[last_item]['attributes'].append(this_attribute)
continue
this_object.update(item)
del item[keyname]['_state']
object_collection.append(this_object)
return object_collection
class _LsUsb():
def __init__(self):
self.raw_output = []
@@ -314,27 +493,39 @@ class _LsUsb():
self.section = ''
self.old_section = ''
# section_header is formatted with the correct spacing to be used with
# jc.parsers.universal.sparse_table_parse(). Pad end of string to be at least len of 25
# this value changes for different sections (e.g. videocontrol & videostreaming)
self.normal_section_header = 'key val description'
self.larger_section_header = 'key val description'
self.bus_idx = -1
self.interface_descriptor_idx = -1
self.endpoint_descriptor_idx = -1
self.videocontrol_interface_descriptor_idx = -1
self.videostreaming_interface_descriptor_idx = -1
self.last_item = ''
self.last_indent = 0
self.attribute_value = False
self.bus_list = []
self.device_descriptor_list = []
self.configuration_descriptor_list = []
self.interface_association_list = []
self.device_descriptor = _root_obj('device_descriptor')
self.configuration_descriptor = _root_obj('configuration_descriptor')
self.interface_association = _root_obj('interface_association')
self.interface_descriptor_list = []
self.interface_descriptor_attribute_list = []
self.cdc_header_list = []
self.cdc_call_management_list = []
self.cdc_acm_list = []
self.cdc_union_list = []
self.endpoint_descriptor_list = []
self.hid_device_descriptor_list = []
self.report_descriptors_list = []
self.hub_descriptor_list = []
self.cdc_header = _descriptor_obj('cdc_header')
self.cdc_call_management = _descriptor_obj('cdc_call_management')
self.cdc_acm = _descriptor_obj('cdc_acm')
self.cdc_union = _descriptor_obj('cdc_union')
self.cdc_mbim = _descriptor_obj('cdc_mbim')
self.cdc_mbim_extended = _descriptor_obj('cdc_mbim_extended')
self.endpoint_descriptors = _descriptor_list('endpoint_descriptor')
self.videocontrol_interface_descriptors = _descriptor_list('videocontrol_interface_descriptor')
self.videostreaming_interface_descriptors = _descriptor_list('videostreaming_interface_descriptor')
self.hid_device_descriptor = _descriptor_obj('hid_device_descriptor')
# self.report_descriptors_list = [] # not implemented
self.hub_descriptor = _root_obj('hub_descriptor')
self.hub_port_status_list = []
self.device_qualifier_list = []
self.device_status_list = []
@@ -355,14 +546,22 @@ class _LsUsb():
# determine whether this is a top-level value item or lower-level attribute
if indent > self.last_indent and self.old_section == self.section:
self.attribute_value = True
elif indent == self.last_indent and self.attribute_value and self.old_section == self.section:
elif indent == self.last_indent and self.attribute_value \
and self.old_section == self.section:
self.attribute_value = True
else:
self.attribute_value = False
# section_header is formatted with the correct spacing to be used with
# jc.parsers.universal.sparse_table_parse(). Pad end of string to be at least len of 25
section_header = 'key val description'
section_header = self.normal_section_header
if self.section == 'videocontrol_interface_descriptor' \
or self.section == 'videostreaming_interface_descriptor' \
or self.section == 'cdc_mbim_extended':
section_header = self.larger_section_header
temp_obj = [section_header, line.strip() + (' ' * 25)]
temp_obj = sparse_table_parse(temp_obj)
@@ -377,7 +576,9 @@ class _LsUsb():
'last_item': self.last_item,
'bus_idx': self.bus_idx,
'interface_descriptor_idx': self.interface_descriptor_idx,
'endpoint_descriptor_idx': self.endpoint_descriptor_idx
'endpoint_descriptor_idx': self.endpoint_descriptor_idx,
'videocontrol_interface_descriptor_idx': self.videocontrol_interface_descriptor_idx,
'videostreaming_interface_descriptor_idx': self.videostreaming_interface_descriptor_idx
}
}
}
@@ -429,12 +630,15 @@ class _LsUsb():
self.attribute_value = False
return True
# bus information is on the same line so need to extract data immediately and set indexes
# bus information is on the same line so need to extract data
# immediately and set indexes
if line.startswith('Bus '):
self.section = 'bus'
self.bus_idx += 1
self.interface_descriptor_idx = -1
self.endpoint_descriptor_idx = -1
self.videocontrol_interface_descriptor_idx = -1
self.videostreaming_interface_descriptor_idx = -1
self.attribute_value = False
line_split = line.strip().split(maxsplit=6)
self.bus_list.append(
@@ -442,7 +646,8 @@ class _LsUsb():
'bus': line_split[1],
'device': line_split[3][:-1],
'id': line_split[5],
'description': (line_split[6:7] or [None])[0], # way to get a list item or None
# way to get a list item or None
'description': (line_split[6:7] or [None])[0],
'_state': {
'bus_idx': self.bus_idx
}
@@ -450,22 +655,36 @@ class _LsUsb():
)
return True
# This section is a list, so need to update indexes
# These sections are lists, so need to update indexes
if line.startswith(' Interface Descriptor:'):
self.section = 'interface_descriptor'
self.interface_descriptor_idx += 1
self.endpoint_descriptor_idx = -1
self.videocontrol_interface_descriptor_idx = -1
self.videostreaming_interface_descriptor_idx = -1
self.attribute_value = False
return True
# This section is a list, so need to update the index
if line.startswith(' Endpoint Descriptor:'):
self.section = 'endpoint_descriptor'
self.endpoint_descriptor_idx += 1
self.attribute_value = False
return True
# some device status information is displayed on the initial line so need to extract immediately
if line.startswith(' VideoControl Interface Descriptor:'):
self.section = 'videocontrol_interface_descriptor'
self.videocontrol_interface_descriptor_idx += 1
self.attribute_value = False
return True
if line.startswith(' VideoStreaming Interface Descriptor:'):
self.section = 'videostreaming_interface_descriptor'
self.videostreaming_interface_descriptor_idx += 1
self.attribute_value = False
return True
# some device status information is displayed on the initial line so
# need to extract immediately
if line.startswith('Device Status:'):
self.section = 'device_status'
self.attribute_value = False
@@ -491,6 +710,8 @@ class _LsUsb():
' CDC Union:': 'cdc_union',
' HID Device Descriptor:': 'hid_device_descriptor',
' Report Descriptors:': 'report_descriptors',
' CDC MBIM:': 'cdc_mbim',
' CDC MBIM Extended:': 'cdc_mbim_extended',
'Hub Descriptor:': 'hub_descriptor',
' Hub Port Status:': 'hub_port_status',
'Device Qualifier (for other device speed):': 'device_qualifier',
@@ -507,18 +728,22 @@ class _LsUsb():
def _populate_lists(self, line):
section_list_map = {
'device_descriptor': self.device_descriptor_list,
'configuration_descriptor': self.configuration_descriptor_list,
'interface_association': self.interface_association_list,
'device_descriptor': self.device_descriptor.list,
'configuration_descriptor': self.configuration_descriptor.list,
'interface_association': self.interface_association.list,
'interface_descriptor': self.interface_descriptor_list,
'cdc_header': self.cdc_header_list,
'cdc_call_management': self.cdc_call_management_list,
'cdc_acm': self.cdc_acm_list,
'cdc_union': self.cdc_union_list,
'hid_device_descriptor': self.hid_device_descriptor_list,
'report_descriptors': self.report_descriptors_list,
'endpoint_descriptor': self.endpoint_descriptor_list,
'hub_descriptor': self.hub_descriptor_list,
'cdc_header': self.cdc_header.list,
'cdc_call_management': self.cdc_call_management.list,
'cdc_acm': self.cdc_acm.list,
'cdc_union': self.cdc_union.list,
'cdc_mbim': self.cdc_mbim.list,
'cdc_mbim_extended': self.cdc_mbim_extended.list,
'hid_device_descriptor': self.hid_device_descriptor.list,
# 'report_descriptors': self.report_descriptors_list, # not implemented
'videocontrol_interface_descriptor': self.videocontrol_interface_descriptors.list,
'videostreaming_interface_descriptor': self.videostreaming_interface_descriptors.list,
'endpoint_descriptor': self.endpoint_descriptors.list,
'hub_descriptor': self.hub_descriptor.list,
'device_qualifier': self.device_qualifier_list
}
@@ -528,7 +753,9 @@ class _LsUsb():
return True
# special handling of these sections
if line.startswith(' ') and self.section == 'hub_port_status':
if line.startswith(' ') and not line.startswith(' ') \
and self.section == 'hub_port_status':
self.hub_port_status_list.append(self._add_hub_port_status_attributes(line))
return True
@@ -547,10 +774,16 @@ class _LsUsb():
['device_descriptor']['configuration_descriptor']['interface_association'] = {}
['device_descriptor']['configuration_descriptor']['interface_descriptors'] = []
['device_descriptor']['configuration_descriptor']['interface_descriptors'][0] = {}
['device_descriptor']['configuration_descriptor']['interface_descriptors'][0]['videocontrol_interface_descriptors'] = []
['device_descriptor']['configuration_descriptor']['interface_descriptors'][0]['videocontrol_interface_descriptors'][0] = {}
['device_descriptor']['configuration_descriptor']['interface_descriptors'][0]['videostreaming_interface_descriptors'] = []
['device_descriptor']['configuration_descriptor']['interface_descriptors'][0]['videostreaming_interface_descriptors'][0] = {}
['device_descriptor']['configuration_descriptor']['interface_descriptors'][0]['cdc_header'] = {}
['device_descriptor']['configuration_descriptor']['interface_descriptors'][0]['cdc_call_management'] = {}
['device_descriptor']['configuration_descriptor']['interface_descriptors'][0]['cdc_acm'] = {}
['device_descriptor']['configuration_descriptor']['interface_descriptors'][0]['cdc_union'] = {}
['device_descriptor']['configuration_descriptor']['interface_descriptors'][0]['cdc_mbim'] = {}
['device_descriptor']['configuration_descriptor']['interface_descriptors'][0]['cdc_mbim_extended'] = {}
['device_descriptor']['configuration_descriptor']['interface_descriptors'][0]['hid_device_descriptor'] = {}
['device_descriptor']['configuration_descriptor']['interface_descriptors'][0]['endpoint_descriptors'] = []
['device_descriptor']['configuration_descriptor']['interface_descriptors'][0]['endpoint_descriptors'][0] = {}
@@ -568,81 +801,56 @@ class _LsUsb():
del item['_state']
self.output_line.update(item)
for dd in self.device_descriptor_list:
keyname = tuple(dd.keys())[0]
if '_state' in dd[keyname] and dd[keyname]['_state']['bus_idx'] == idx:
# add initial root-level keys
if self.device_descriptor._entries_for_this_bus_exist(idx):
self.device_descriptor._update_output(idx, self.output_line)
# is this a top level value or an attribute?
if dd[keyname]['_state']['attribute_value']:
last_item = dd[keyname]['_state']['last_item']
if 'attributes' not in self.output_line['device_descriptor'][last_item]:
self.output_line['device_descriptor'][last_item]['attributes'] = []
if self.configuration_descriptor._entries_for_this_bus_exist(idx):
self.configuration_descriptor._update_output(
idx, self.output_line['device_descriptor']
)
this_attribute = f'{keyname} {dd[keyname].get("value", "")} {dd[keyname].get("description", "")}'.strip()
self.output_line['device_descriptor'][last_item]['attributes'].append(this_attribute)
continue
if self.interface_association._entries_for_this_bus_exist(idx):
self.interface_association._update_output(
idx, self.output_line['device_descriptor']['configuration_descriptor']
)
self.output_line['device_descriptor'].update(dd)
del self.output_line['device_descriptor'][keyname]['_state']
for cd in self.configuration_descriptor_list:
keyname = tuple(cd.keys())[0]
if '_state' in cd[keyname] and cd[keyname]['_state']['bus_idx'] == idx:
# is this a top level value or an attribute?
if cd[keyname]['_state']['attribute_value']:
last_item = cd[keyname]['_state']['last_item']
if 'attributes' not in self.output_line['device_descriptor']['configuration_descriptor'][last_item]:
self.output_line['device_descriptor']['configuration_descriptor'][last_item]['attributes'] = []
this_attribute = f'{keyname} {cd[keyname].get("value", "")} {cd[keyname].get("description", "")}'.strip()
self.output_line['device_descriptor']['configuration_descriptor'][last_item]['attributes'].append(this_attribute)
continue
self.output_line['device_descriptor']['configuration_descriptor'].update(cd)
del self.output_line['device_descriptor']['configuration_descriptor'][keyname]['_state']
for ia in self.interface_association_list:
keyname = tuple(ia.keys())[0]
if '_state' in ia[keyname] and ia[keyname]['_state']['bus_idx'] == idx:
# is this a top level value or an attribute?
if ia[keyname]['_state']['attribute_value']:
last_item = ia[keyname]['_state']['last_item']
if 'attributes' not in self.output_line['device_descriptor']['configuration_descriptor']['interface_association'][last_item]:
self.output_line['device_descriptor']['configuration_descriptor']['interface_association'][last_item]['attributes'] = []
this_attribute = f'{keyname} {ia[keyname].get("value", "")} {ia[keyname].get("description", "")}'.strip()
self.output_line['device_descriptor']['configuration_descriptor']['interface_association'][last_item]['attributes'].append(this_attribute)
continue
self.output_line['device_descriptor']['configuration_descriptor']['interface_association'].update(ia)
del self.output_line['device_descriptor']['configuration_descriptor']['interface_association'][keyname]['_state']
# add interface_descriptor key if it doesn't exist and there are entries for this bus
# add interface_descriptor key if it doesn't exist and there
# are entries for this bus
for iface_attrs in self.interface_descriptor_list:
keyname = tuple(iface_attrs.keys())[0]
if '_state' in iface_attrs[keyname] and iface_attrs[keyname]['_state']['bus_idx'] == idx:
if '_state' in iface_attrs[keyname] \
and iface_attrs[keyname]['_state']['bus_idx'] == idx:
self.output_line['device_descriptor']['configuration_descriptor']['interface_descriptors'] = []
# find max index for this bus idx, then iterate over that range
i_desc_iters = -1
for iface_attrs in self.interface_descriptor_list:
keyname = tuple(iface_attrs.keys())[0]
if '_state' in iface_attrs[keyname] and iface_attrs[keyname]['_state']['bus_idx'] == idx:
if '_state' in iface_attrs[keyname] \
and iface_attrs[keyname]['_state']['bus_idx'] == idx:
i_desc_iters = iface_attrs[keyname]['_state']['interface_descriptor_idx']
# create the interface descriptor object
if i_desc_iters > -1:
for iface_idx in range(i_desc_iters + 1):
i_desc_obj = _NestedDict()
for iface_attrs in self.interface_descriptor_list:
keyname = tuple(iface_attrs.keys())[0]
if '_state' in iface_attrs[keyname] and iface_attrs[keyname]['_state']['bus_idx'] == idx and iface_attrs[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
if '_state' in iface_attrs[keyname] \
and iface_attrs[keyname]['_state']['bus_idx'] == idx \
and iface_attrs[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
# is this a top level value or an attribute?
if iface_attrs[keyname]['_state']['attribute_value']:
last_item = iface_attrs[keyname]['_state']['last_item']
if 'attributes' not in i_desc_obj[last_item]:
i_desc_obj[last_item]['attributes'] = []
@@ -653,168 +861,76 @@ class _LsUsb():
del iface_attrs[keyname]['_state']
i_desc_obj.update(iface_attrs)
# add other nodes to the object (cdc_header, endpoint descriptors, etc.)
for ch in self.cdc_header_list:
keyname = tuple(ch.keys())[0]
if '_state' in ch[keyname] and ch[keyname]['_state']['bus_idx'] == idx and ch[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
# add the rest of the interface descriptor keys to the object
if self.cdc_header._entries_for_this_bus_and_interface_idx_exist(idx, iface_idx):
self.cdc_header._update_output(idx, iface_idx, i_desc_obj)
# is this a top level value or an attribute?
if ch[keyname]['_state']['attribute_value']:
last_item = ch[keyname]['_state']['last_item']
if 'attributes' not in i_desc_obj['cdc_header'][last_item]:
i_desc_obj['cdc_header'][last_item]['attributes'] = []
if self.cdc_call_management._entries_for_this_bus_and_interface_idx_exist(idx, iface_idx):
self.cdc_call_management._update_output(idx, iface_idx, i_desc_obj)
this_attribute = f'{keyname} {ch[keyname].get("value", "")} {ch[keyname].get("description", "")}'.strip()
i_desc_obj['cdc_header'][last_item]['attributes'].append(this_attribute)
continue
if self.cdc_acm._entries_for_this_bus_and_interface_idx_exist(idx, iface_idx):
self.cdc_acm._update_output(idx, iface_idx, i_desc_obj)
i_desc_obj['cdc_header'].update(ch)
del i_desc_obj['cdc_header'][keyname]['_state']
if self.cdc_union._entries_for_this_bus_and_interface_idx_exist(idx, iface_idx):
self.cdc_union._update_output(idx, iface_idx, i_desc_obj)
for ccm in self.cdc_call_management_list:
keyname = tuple(ccm.keys())[0]
if '_state' in ccm[keyname] and ccm[keyname]['_state']['bus_idx'] == idx and ccm[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
if self.cdc_mbim._entries_for_this_bus_and_interface_idx_exist(idx, iface_idx):
self.cdc_mbim._update_output(idx, iface_idx, i_desc_obj)
# is this a top level value or an attribute?
if ccm[keyname]['_state']['attribute_value']:
last_item = ccm[keyname]['_state']['last_item']
if 'attributes' not in i_desc_obj['cdc_call_management'][last_item]:
i_desc_obj['cdc_call_management'][last_item]['attributes'] = []
if self.cdc_mbim_extended._entries_for_this_bus_and_interface_idx_exist(idx, iface_idx):
self.cdc_mbim_extended._update_output(idx, iface_idx, i_desc_obj)
this_attribute = f'{keyname} {ccm[keyname].get("value", "")} {ccm[keyname].get("description", "")}'.strip()
i_desc_obj['cdc_call_management'][last_item]['attributes'].append(this_attribute)
continue
if self.hid_device_descriptor._entries_for_this_bus_and_interface_idx_exist(idx, iface_idx):
self.hid_device_descriptor._update_output(idx, iface_idx, i_desc_obj)
i_desc_obj['cdc_call_management'].update(ccm)
del i_desc_obj['cdc_call_management'][keyname]['_state']
# Not Implemented: Report Descriptors (need more samples)
# for rd in self.report_descriptors_list:
# keyname = tuple(rd.keys())[0]
# if '_state' in rd[keyname] and rd[keyname]['_state']['bus_idx'] == idx and rd[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
# i_desc_obj['hid_device_descriptor']['report_descriptors'].update(rd)
# del i_desc_obj['hid_device_descriptor']['report_descriptors'][keyname]['_state']
for ca in self.cdc_acm_list:
keyname = tuple(ca.keys())[0]
if '_state' in ca[keyname] and ca[keyname]['_state']['bus_idx'] == idx and ca[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
if self.videocontrol_interface_descriptors._entries_for_this_bus_and_interface_idx_exist(idx, iface_idx):
i_desc_obj['videocontrol_interface_descriptors'] = []
i_desc_obj['videocontrol_interface_descriptors'].extend(
self.videocontrol_interface_descriptors._get_objects_list(idx, iface_idx)
)
# is this a top level value or an attribute?
if ca[keyname]['_state']['attribute_value']:
last_item = ca[keyname]['_state']['last_item']
if 'attributes' not in i_desc_obj['cdc_acm'][last_item]:
i_desc_obj['cdc_acm'][last_item]['attributes'] = []
if self.videostreaming_interface_descriptors._entries_for_this_bus_and_interface_idx_exist(idx, iface_idx):
i_desc_obj['videostreaming_interface_descriptors'] = []
i_desc_obj['videostreaming_interface_descriptors'].extend(
self.videostreaming_interface_descriptors._get_objects_list(idx, iface_idx)
)
this_attribute = f'{keyname} {ca[keyname].get("value", "")} {ca[keyname].get("description", "")}'.strip()
i_desc_obj['cdc_acm'][last_item]['attributes'].append(this_attribute)
continue
i_desc_obj['cdc_acm'].update(ca)
del i_desc_obj['cdc_acm'][keyname]['_state']
for cu in self.cdc_union_list:
keyname = tuple(cu.keys())[0]
if '_state' in cu[keyname] and cu[keyname]['_state']['bus_idx'] == idx and cu[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
# is this a top level value or an attribute?
if cu[keyname]['_state']['attribute_value']:
last_item = cu[keyname]['_state']['last_item']
if 'attributes' not in i_desc_obj['cdc_union'][last_item]:
i_desc_obj['cdc_union'][last_item]['attributes'] = []
this_attribute = f'{keyname} {cu[keyname].get("value", "")} {cu[keyname].get("description", "")}'.strip()
i_desc_obj['cdc_union'][last_item]['attributes'].append(this_attribute)
continue
i_desc_obj['cdc_union'].update(cu)
del i_desc_obj['cdc_union'][keyname]['_state']
for hidd in self.hid_device_descriptor_list:
keyname = tuple(hidd.keys())[0]
if '_state' in hidd[keyname] and hidd[keyname]['_state']['bus_idx'] == idx and hidd[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
# is this a top level value or an attribute?
if hidd[keyname]['_state']['attribute_value']:
last_item = hidd[keyname]['_state']['last_item']
if 'attributes' not in i_desc_obj['hid_device_descriptor'][last_item]:
i_desc_obj['hid_device_descriptor'][last_item]['attributes'] = []
this_attribute = f'{keyname} {hidd[keyname].get("value", "")} {hidd[keyname].get("description", "")}'.strip()
i_desc_obj['hid_device_descriptor'][last_item]['attributes'].append(this_attribute)
continue
i_desc_obj['hid_device_descriptor'].update(hidd)
del i_desc_obj['hid_device_descriptor'][keyname]['_state']
# Not Implemented: Report Descriptors (need more samples)
# for rd in self.report_descriptors_list:
# keyname = tuple(rd.keys())[0]
# if '_state' in rd[keyname] and rd[keyname]['_state']['bus_idx'] == idx and rd[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
# i_desc_obj['hid_device_descriptor']['report_descriptors'].update(rd)
# del i_desc_obj['hid_device_descriptor']['report_descriptors'][keyname]['_state']
# add endpoint_descriptor key if it doesn't exist and there are entries for this interface_descriptor
for endpoint_attrs in self.endpoint_descriptor_list:
keyname = tuple(endpoint_attrs.keys())[0]
if '_state' in endpoint_attrs[keyname] and endpoint_attrs[keyname]['_state']['bus_idx'] == idx and endpoint_attrs[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
i_desc_obj['endpoint_descriptors'] = []
# find max index for this endpoint_descriptor idx, then iterate over that range
e_desc_iters = -1
for endpoint_attrs in self.endpoint_descriptor_list:
keyname = tuple(endpoint_attrs.keys())[0]
if '_state' in endpoint_attrs[keyname] and endpoint_attrs[keyname]['_state']['bus_idx'] == idx and endpoint_attrs[keyname]['_state']['interface_descriptor_idx'] == iface_idx:
e_desc_iters = endpoint_attrs[keyname]['_state']['endpoint_descriptor_idx']
# create the endpoint descriptor object
if e_desc_iters > -1:
for endpoint_idx in range(e_desc_iters + 1):
e_desc_obj = {}
for endpoint_attrs in self.endpoint_descriptor_list:
keyname = tuple(endpoint_attrs.keys())[0]
if '_state' in endpoint_attrs[keyname] and endpoint_attrs[keyname]['_state']['bus_idx'] == idx and endpoint_attrs[keyname]['_state']['interface_descriptor_idx'] == iface_idx and endpoint_attrs[keyname]['_state']['endpoint_descriptor_idx'] == endpoint_idx:
# is this a top level value or an attribute?
if endpoint_attrs[keyname]['_state']['attribute_value']:
last_item = endpoint_attrs[keyname]['_state']['last_item']
if 'attributes' not in e_desc_obj[last_item]:
e_desc_obj[last_item]['attributes'] = []
this_attribute = f'{keyname} {endpoint_attrs[keyname].get("value", "")} {endpoint_attrs[keyname].get("description", "")}'.strip()
e_desc_obj[last_item]['attributes'].append(this_attribute)
continue
e_desc_obj.update(endpoint_attrs)
del endpoint_attrs[keyname]['_state']
i_desc_obj['endpoint_descriptors'].append(e_desc_obj)
if self.endpoint_descriptors._entries_for_this_bus_and_interface_idx_exist(idx, iface_idx):
i_desc_obj['endpoint_descriptors'] = []
i_desc_obj['endpoint_descriptors'].extend(
self.endpoint_descriptors._get_objects_list(idx, iface_idx)
)
# add the object to the list of interface descriptors
self.output_line['device_descriptor']['configuration_descriptor']['interface_descriptors'].append(i_desc_obj)
for hd in self.hub_descriptor_list:
keyname = tuple(hd.keys())[0]
if '_state' in hd[keyname] and hd[keyname]['_state']['bus_idx'] == idx:
# is this a top level value or an attribute?
if hd[keyname]['_state']['attribute_value']:
last_item = hd[keyname]['_state']['last_item']
if 'attributes' not in self.output_line['hub_descriptor'][last_item]:
self.output_line['hub_descriptor'][last_item]['attributes'] = []
this_attribute = f'{keyname} {hd[keyname].get("value", "")} {hd[keyname].get("description", "")}'.strip()
self.output_line['hub_descriptor'][last_item]['attributes'].append(this_attribute)
continue
self.output_line['hub_descriptor'].update(hd)
del self.output_line['hub_descriptor'][keyname]['_state']
# add final root-level keys
if self.hub_descriptor._entries_for_this_bus_exist(idx):
self.hub_descriptor._update_output(idx, self.output_line)
for hps in self.hub_port_status_list:
keyname = tuple(hps.keys())[0]
if '_state' in hps[keyname] and hps[keyname]['_state']['bus_idx'] == idx:
self.output_line['hub_descriptor']['hub_port_status'].update(hps)
del self.output_line['hub_descriptor']['hub_port_status'][keyname]['_state']
for dq in self.device_qualifier_list:
keyname = tuple(dq.keys())[0]
if '_state' in dq[keyname] and dq[keyname]['_state']['bus_idx'] == idx:
self.output_line['device_qualifier'].update(dq)
del self.output_line['device_qualifier'][keyname]['_state']
for ds in self.device_status_list:
if '_state' in ds and ds['_state']['bus_idx'] == idx:
self.output_line['device_status'].update(ds)
del self.output_line['device_status']['_state']
@@ -840,6 +956,10 @@ def parse(data, raw=False, quiet=False):
lsusb = _LsUsb()
if jc.utils.has_data(data):
# fix known too-long field names
data = data.replace('bmNetworkCapabilities', 'bmNetworkCapabilit ')
for line in data.splitlines():
# only -v option or no options are supported
if line.startswith('/'):

View File

@@ -20,7 +20,7 @@ Schema:
"filesystem": string,
"mount_point": string,
"type": string,
"access": [
"options": [
string
]
}
@@ -34,7 +34,7 @@ Example:
"filesystem": "sysfs",
"mount_point": "/sys",
"type": "sysfs",
"access": [
"options": [
"rw",
"nosuid",
"nodev",
@@ -46,7 +46,7 @@ Example:
"filesystem": "proc",
"mount_point": "/proc",
"type": "proc",
"access": [
"options": [
"rw",
"nosuid",
"nodev",
@@ -58,7 +58,7 @@ Example:
"filesystem": "udev",
"mount_point": "/dev",
"type": "devtmpfs",
"access": [
"options": [
"rw",
"nosuid",
"relatime",
@@ -75,11 +75,11 @@ import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.7'
version = '1.8'
description = '`mount` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux', 'darwin', 'freebsd']
compatible = ['linux', 'darwin', 'freebsd', 'aix']
magic_commands = ['mount']
tags = ['command']
@@ -138,10 +138,38 @@ def _linux_parse(data):
output_line['filesystem'] = parsed_line[0]
output_line['mount_point'] = parsed_line[2]
output_line['type'] = parsed_line[4]
output_line['options'] = parsed_line[5].lstrip('(').rstrip(')').split(',')
options = parsed_line[5].lstrip('(').rstrip(')').split(',')
output.append(output_line)
output_line['options'] = options
return output
def _aix_parse(data):
output = []
# AIX mount command starts with these headers:
# node mounted mounted over vfs date options
# -------- --------------- --------------- ------ ------------ ---------------
# Remove them
data.pop(0)
data.pop(0)
for entry in data:
output_line = {}
parsed_line = entry.split()
# AIX mount entries have the remote node as the zeroth element. If the
# mount is local, the zeroth element is the filesystem instead. We can
# detect this by the lenth of the list. For local mounts, length is 7,
# and for remote mounts, the length is 8. In the remote case, pop off
# the zeroth element. Then parsed_line has a consistent format.
if len(parsed_line) == 8:
parsed_line.pop(0)
output_line['filesystem'] = parsed_line[0]
output_line['mount_point'] = parsed_line[1]
output_line['type'] = parsed_line[2]
output_line['options'] = parsed_line[6].lstrip('(').rstrip(')').split(',')
output.append(output_line)
@@ -171,9 +199,12 @@ def parse(data, raw=False, quiet=False):
if jc.utils.has_data(data):
# check for OSX output
# check for OSX and AIX output
if ' type ' not in cleandata[0]:
raw_output = _osx_parse(cleandata)
if 'node' in cleandata[0]:
raw_output = _aix_parse(cleandata)
else:
raw_output = _osx_parse(cleandata)
else:
raw_output = _linux_parse(cleandata)

View File

@@ -355,17 +355,18 @@ import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.13'
version = '1.14'
description = '`netstat` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux', 'darwin', 'freebsd']
compatible = ['linux', 'darwin', 'freebsd', 'win32']
magic_commands = ['netstat']
tags = ['command']
__version__ = info.version
WINDOWS_NETSTAT_HEADER = "Active Connections"
def _process(proc_data):
"""
@@ -450,9 +451,10 @@ def parse(data, raw=False, quiet=False):
import jc.parsers.netstat_freebsd_osx
raw_output = jc.parsers.netstat_freebsd_osx.parse(cleandata)
# use linux parser
else:
elif cleandata[0] == WINDOWS_NETSTAT_HEADER: # use windows parser.
import jc.parsers.netstat_windows
raw_output = jc.parsers.netstat_windows.parse(cleandata)
else: # use linux parser.
import jc.parsers.netstat_linux
raw_output = jc.parsers.netstat_linux.parse(cleandata)

View File

@@ -0,0 +1,75 @@
"""
jc - JSON Convert Windows `netstat` command output parser
"""
from typing import Dict, List
POSSIBLE_PROTOCOLS = ("TCP", "UDP", "TCPv6", "UDPv6")
def normalize_headers(headers: str):
"""
Normalizes the headers to match the jc netstat parser style
(local_address -> local_address, local_port...).
"""
headers = headers.lower().strip()
headers = headers.replace("local address", "local_address")
headers = headers.replace("foreign address", "foreign_address")
return headers.split()
def parse(cleandata: List[str]):
"""
Main text parsing function for Windows netstat
Parameters:
cleandata: (string) text data to parse
Returns:
List of Dictionaries. Raw structured data.
"""
raw_output = []
cleandata.pop(0) # Removing the "Active Connections" header.
headers = normalize_headers(cleandata.pop(0))
for line in cleandata:
line = line.strip()
if not line.startswith(POSSIBLE_PROTOCOLS): # -b.
line_data = raw_output.pop(len(raw_output) - 1)
line_data['program_name'] = line
raw_output.append(line_data)
continue
line_data = line.split()
line_data: Dict[str, str] = dict(zip(headers, line_data))
for key in list(line_data.keys()):
if key == "local_address":
local_address, local_port = line_data[key].rsplit(
":", maxsplit=1)
line_data["local_address"] = local_address
line_data["local_port"] = local_port
continue
if key == "foreign_address":
foreign_address, foreign_port = line_data[key].rsplit(
":", maxsplit=1)
line_data["foreign_address"] = foreign_address
line_data["foreign_port"] = foreign_port
continue
# There is no state in UDP, so the data after the "state" header will leak.
if key == "proto" and "state" in headers and line_data["proto"] == "UDP":
next_header = headers.index("state") + 1
if len(headers) > next_header:
next_header = headers[next_header]
line_data[next_header] = line_data["state"]
line_data["state"] = ''
raw_output.append(line_data)
return raw_output

View File

@@ -96,14 +96,14 @@ class PBParser(object):
prefix = self.data[0:6]
for case in Switch(prefix):
if case('bplist'):
self.file_type = 'binary'
import biplist
parsed_plist = biplist.readPlist(self.file_path)
# self.file_type = 'binary'
# import biplist
# parsed_plist = biplist.readPlist(self.file_path)
break
if case('<?xml '):
self.file_type = 'xml'
import plistlib
parsed_plist = plistlib.readPlist(self.file_path)
# self.file_type = 'xml'
# import plistlib
# parsed_plist = plistlib.readPlist(self.file_path)
break
if case():
self.file_type = 'ascii'
@@ -111,7 +111,7 @@ class PBParser(object):
if self.data[0:2] == '//':
# this is to try to see if we can locate the desired string encoding of the file
import re
result = re.search('^// !\$\*(.+?)\*\$!', self.data) # pylint: disable=anomalous-backslash-in-string
result = re.search(r'^// !\$\*(.+?)\*\$!', self.data) # pylint: disable=anomalous-backslash-in-string
if result:
self.string_encoding = result.group(1)
#now return the parse

View File

@@ -57,7 +57,7 @@ def KeySorter(obj1, obj2):
result = StringCmp(str(obj1), str(obj2))
return result
class pbRoot(collections.MutableMapping):
class pbRoot(collections.abc.MutableMapping):
def __init__(self, *args, **kwargs):
self.store = dict()

View File

@@ -164,7 +164,7 @@ import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.8'
version = '1.9'
description = '`ping` and `ping6` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -284,6 +284,8 @@ def _linux_parse(data):
if ipv4 and linedata[0][5] not in string.digits:
hostname = True
# fixup for missing hostname
linedata[0] = linedata[0][:5] + 'nohost' + linedata[0][5:]
elif ipv4 and linedata[0][5] in string.digits:
hostname = False
elif not ipv4 and ' (' in linedata[0]:
@@ -314,7 +316,8 @@ def _linux_parse(data):
if line.startswith('---'):
footer = True
raw_output['destination'] = line.split()[1]
if line[4] != ' ': # fixup for missing hostname
raw_output['destination'] = line.split()[1]
continue
if footer:

View File

@@ -85,7 +85,7 @@ from jc.exceptions import ParseError
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.2'
version = '1.3'
description = '`ping` and `ping6` command streaming parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -340,6 +340,8 @@ def _linux_parse(line, s):
if s.ipv4 and line[5] not in string.digits:
s.hostname = True
# fixup for missing hostname
line = line[:5] + 'nohost' + line[5:]
elif s.ipv4 and line[5] in string.digits:
s.hostname = False
elif not s.ipv4 and ' (' in line:

View File

@@ -204,7 +204,7 @@ def parse(
pid_smaps_p = re.compile(r'^[0-9a-f]{12}-[0-9a-f]{12} [rwxsp\-]{4} [0-9a-f]{8} [0-9a-f]{2}:[0-9a-f]{2} \d+ [^\n]+\nSize:\s+\d+ \S\S\n')
pid_stat_p = re.compile(r'^\d+ \(.+\) \S \d+ \d+ \d+ \d+ -?\d+ (?:\d+ ){43}\d+$', re.DOTALL)
pid_statm_p = re.compile(r'^\d+ \d+ \d+\s\d+\s\d+\s\d+\s\d+$')
pid_status_p = re.compile(r'^Name:\t.+\nUmask:\t\d+\nState:\t.+\nTgid:\t\d+\n')
pid_status_p = re.compile(r'^Name:\t.+\n(?:Umask:\t\d+\n)?State:\t.+\nTgid:\t\d+\n')
# scsi_device_info = re.compile(r"^'\w+' '.+' 0x\d+")
# scsi_scsi_p = re.compile(r'^Attached devices:\nHost: \w+ ')

View File

@@ -109,7 +109,8 @@ Examples:
"mw",
"me",
"dw",
"sd"
"sd",
"mp"
],
"VmFlags_pretty": [
"readable",
@@ -211,6 +212,7 @@ def _process(proc_data: List[Dict]) -> List[Dict]:
'mw': 'may write',
'me': 'may execute',
'ms': 'may share',
'mp': 'MPX-specific VMA',
'gd': 'stack segment growns down',
'pf': 'pure PFN range',
'dw': 'disabled write to the mapped file',
@@ -274,10 +276,10 @@ def parse(
if jc.utils.has_data(data):
map_line = re.compile(r'''
^(?P<start>[0-9a-f]{12,16})-
(?P<end>[0-9a-f]{12,16})\s
^(?P<start>[0-9a-f]{8,16})-
(?P<end>[0-9a-f]{8,16})\s
(?P<perms>[rwxsp\-]{4})\s
(?P<offset>[0-9a-f]{8})\s
(?P<offset>[0-9a-f]{8,9})\s
(?P<maj>[0-9a-f]{2}):
(?P<min>[0-9a-f]{2})\s
(?P<inode>\d+)\s+

18
jc/parsers/pyedid/LICENSE Normal file
View File

@@ -0,0 +1,18 @@
Copyright 2019-2020 Jonas Lieb, Davydov Denis
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the "Software"), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of
the Software, and to permit persons to whom the Software is furnished to do so,
subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS
FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR
COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

View File

171
jc/parsers/pyedid/edid.py Executable file
View File

@@ -0,0 +1,171 @@
"""
Edid module
"""
import struct
from collections import namedtuple
from typing import ByteString
__all__ = ["Edid"]
class Edid:
"""Edid class
Raises:
`ValueError`: if invalid edid data
"""
_STRUCT_FORMAT = (
"<" # little-endian
"8s" # constant header (8 bytes)
"H" # manufacturer id (2 bytes)
"H" # product id (2 bytes)
"I" # serial number (4 bytes)
"B" # manufactoring week (1 byte)
"B" # manufactoring year (1 byte)
"B" # edid version (1 byte)
"B" # edid revision (1 byte)
"B" # video input type (1 byte)
"B" # horizontal size in cm (1 byte)
"B" # vertical size in cm (1 byte)
"B" # display gamma (1 byte)
"B" # supported features (1 byte)
"10s" # color characteristics (10 bytes)
"H" # supported timings (2 bytes)
"B" # reserved timing (1 byte)
"16s" # EDID supported timings (16 bytes)
"18s" # detailed timing block 1 (18 bytes)
"18s" # detailed timing block 2 (18 bytes)
"18s" # detailed timing block 3 (18 bytes)
"18s" # detailed timing block 4 (18 bytes)
"B" # extension flag (1 byte)
"B"
) # checksum (1 byte)
_TIMINGS = {
0: (1280, 1024, 75.0),
1: (1024, 768, 75.0),
2: (1024, 768, 70.0),
3: (1024, 768, 60.0),
4: (1024, 768, 87.0),
5: (832, 624, 75.0),
6: (800, 600, 75.0),
7: (800, 600, 72.0),
8: (800, 600, 60.0),
9: (800, 600, 56.0),
10: (640, 480, 75.0),
11: (640, 480, 72.0),
12: (640, 480, 67.0),
13: (640, 480, 60.0),
14: (720, 400, 88.0),
15: (720, 400, 70.0),
}
_ASPECT_RATIOS = {
0b00: (16, 10),
0b01: ( 4, 3),
0b10: ( 5, 4),
0b11: (16, 9),
}
_RawEdid = namedtuple("RawEdid",
("header",
"manu_id",
"prod_id",
"serial_no",
"manu_week",
"manu_year",
"edid_version",
"edid_revision",
"input_type",
"width",
"height",
"gamma",
"features",
"color",
"timings_supported",
"timings_reserved",
"timings_edid",
"timing_1",
"timing_2",
"timing_3",
"timing_4",
"extension",
"checksum")
)
def __init__(self, edid: ByteString):
self._parse_edid(edid)
def _parse_edid(self, edid: ByteString):
"""Convert edid byte string to edid object"""
if struct.calcsize(self._STRUCT_FORMAT) != 128:
raise ValueError("Wrong edid size.")
if sum(map(int, edid)) % 256 != 0:
raise ValueError("Checksum mismatch.")
unpacked = struct.unpack(self._STRUCT_FORMAT, edid)
raw_edid = self._RawEdid(*unpacked)
if raw_edid.header != b'\x00\xff\xff\xff\xff\xff\xff\x00':
raise ValueError("Invalid header.")
self.raw = edid
self.manufacturer_id = raw_edid.manu_id
self.product = raw_edid.prod_id
self.year = raw_edid.manu_year + 1990
self.edid_version = "{:d}.{:d}".format(raw_edid.edid_version, raw_edid.edid_revision)
self.type = "digital" if (raw_edid.input_type & 0xFF) else "analog"
self.width = float(raw_edid.width)
self.height = float(raw_edid.height)
self.gamma = (raw_edid.gamma+100)/100
self.dpms_standby = bool(raw_edid.features & 0xFF)
self.dpms_suspend = bool(raw_edid.features & 0x7F)
self.dpms_activeoff = bool(raw_edid.features & 0x3F)
self.resolutions = []
for i in range(16):
bit = raw_edid.timings_supported & (1 << i)
if bit:
self.resolutions.append(self._TIMINGS[i])
for i in range(8):
bytes_data = raw_edid.timings_edid[2*i:2*i+2]
if bytes_data == b'\x01\x01':
continue
byte1, byte2 = bytes_data
x_res = 8*(int(byte1)+31)
aspect_ratio = self._ASPECT_RATIOS[(byte2>>6) & 0b11]
y_res = int(x_res * aspect_ratio[1]/aspect_ratio[0])
rate = (int(byte2) & 0b00111111) + 60.0
self.resolutions.append((x_res, y_res, rate))
self.name = None
self.serial = None
for timing_bytes in (raw_edid.timing_1, raw_edid.timing_2, raw_edid.timing_3, raw_edid.timing_4):
# "other" descriptor
if timing_bytes[0:2] == b'\x00\x00':
timing_type = timing_bytes[3]
if timing_type in (0xFF, 0xFE, 0xFC):
buffer = timing_bytes[5:]
buffer = buffer.partition(b"\x0a")[0]
text = buffer.decode("cp437")
if timing_type == 0xFF:
self.serial = text
elif timing_type == 0xFC:
self.name = text
if not self.serial:
self.serial = raw_edid.serial_no
def __repr__(self):
clsname = self.__class__.__name__
attributes = []
for name in dir(self):
if not name.startswith("_"):
value = getattr(self, name)
attributes.append("\t{}={}".format(name, value))
return "{}(\n{}\n)".format(clsname, ", \n".join(attributes))

View File

View File

@@ -0,0 +1,61 @@
"""
EDID helper
"""
from subprocess import CalledProcessError, check_output
from typing import ByteString, List
__all__ = ["EdidHelper"]
class EdidHelper:
"""Class for working with EDID data"""
@staticmethod
def hex2bytes(hex_data: str) -> ByteString:
"""Convert hex EDID string to bytes
Args:
hex_data (str): hex edid string
Returns:
ByteString: edid byte string
"""
# delete edid 1.3 additional block
if len(hex_data) > 256:
hex_data = hex_data[:256]
numbers = []
for i in range(0, len(hex_data), 2):
pair = hex_data[i : i + 2]
numbers.append(int(pair, 16))
return bytes(numbers)
@classmethod
def get_edids(cls) -> List[ByteString]:
"""Get edids from xrandr
Raises:
`RuntimeError`: if error with retrieving xrandr util data
Returns:
List[ByteString]: list with edids
"""
try:
output = check_output(["xrandr", "--verbose"])
except (CalledProcessError, FileNotFoundError) as err:
raise RuntimeError(
"Error retrieving xrandr util data: {}".format(err)
) from None
edids = []
lines = output.splitlines()
for i, line in enumerate(lines):
line = line.decode().strip()
if line.startswith("EDID:"):
selection = lines[i + 1 : i + 9]
selection = list(s.decode().strip() for s in selection)
selection = "".join(selection)
bytes_section = cls.hex2bytes(selection)
edids.append(bytes_section)
return edids

View File

@@ -0,0 +1,136 @@
"""
Module for working with PNP ID REGISTRY
"""
import csv
import string
from html.parser import HTMLParser
from urllib import request
__all__ = ["Registry"]
class WebPnpIdParser(HTMLParser):
"""Parser pnp id from https://uefi.org/PNP_ID_List
Examples:
p = WebPnpIdParser()
p.feed(html_data)
p.result
"""
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._find_table = False
self._find_row = False
# first -- company name, second -- pnp id, third -- approved date
self._last_field = []
# key -- pnp id, value -- tuple (company_name, approved_date)
self.result = {}
def handle_starttag(self, tag, attrs):
if tag == "tbody":
self._find_table = True
elif self._find_table and tag == "tr":
self._find_row = True
def handle_endtag(self, tag):
if tag == "tbody":
self._find_table = False
elif self._find_table and tag == "tr":
self._find_row = False
# add table row to result
self.result[self._last_field[1]] = (
self._last_field[0],
self._last_field[-1],
)
self._last_field.clear()
def handle_data(self, data):
# skip processing until table is found
if not self._find_table:
return
if self._find_row:
data = data.strip()
if data:
self._last_field.append(data)
def error(self, message):
super().close()
class Registry(dict):
"""Registry pnp id data dictionary
key -- pnp_id
value -- company name
"""
@classmethod
def from_web(cls, filter_by_id: str = None):
"""Get registry from https://uefi.org/PNP_ID_List
Args:
filter_by_id (str), optional: filter registry by id
Raises:
Returns:
"""
url = "https://uefi.org/PNP_ID_List"
if filter_by_id:
url += "?search={}".format(filter_by_id)
with request.urlopen(url) as req:
parse = WebPnpIdParser()
parse.feed(req.read().decode())
registry = cls()
for key, value in parse.result.items():
# skip invalid search value
if filter_by_id and key != filter_by_id:
continue
registry[key] = value[0]
return registry
@classmethod
def from_csv(cls, csv_path: str, filter_by_id: str = None):
"""Get registry by csv local file
Args:
csv_path (str): path to csv file
filter_by_id (str), optional: filter registry by id
Raises:
Returns:
"""
registry = cls()
with open(csv_path, "r") as file:
reader = csv.reader(file)
for line in reader:
# filter
if filter_by_id and filter_by_id != line[0]:
continue
registry[line[0]] = line[1]
return registry
def to_csv(self, csv_path: str):
"""Dump registry to csv file"""
with open(csv_path, "w") as csv_file:
writer = csv.writer(csv_file)
writer.writerows(self.items())
return self
def get_company_from_id(self, pnp_id: str) -> str:
"""Convert PNP id to company name"""
return self.get(pnp_id, "Unknown")
def get_company_from_raw(self, raw: int) -> str:
"""Convert raw edid value to company name"""
tmp = [(raw >> 10) & 31, (raw >> 5) & 31, raw & 31]
pnp_id = "".join(string.ascii_uppercase[n - 1] for n in tmp)
return self.get_company_from_id(pnp_id)

29
jc/parsers/pyedid/main.py Normal file
View File

@@ -0,0 +1,29 @@
"""
Entrypoint
"""
from pyedid.edid import Edid
from pyedid.helpers.edid_helper import EdidHelper
from pyedid.helpers.registry import Registry
def main():
"""Main func"""
edid_csv_cache = "/tmp/pyedid-database.csv"
try:
registry = Registry.from_csv(edid_csv_cache)
except FileNotFoundError:
print("Loading registry from web...")
registry = Registry.from_web()
print("Done!\n")
registry.to_csv(edid_csv_cache)
for raw in EdidHelper.get_edids():
edid = Edid(raw, registry)
print(edid)
if __name__ == "__main__":
main()

View File

@@ -17,6 +17,13 @@ Schema:
[
{
"interfaces": [
{
"id": string,
"mac": string,
"name": string,
}
]
"destination": string,
"gateway": string,
"genmask": string,
@@ -109,11 +116,11 @@ import jc.parsers.universal
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.8'
version = '1.9'
description = '`route` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux']
compatible = ['linux', 'win32']
magic_commands = ['route']
tags = ['command']
@@ -152,6 +159,14 @@ def _process(proc_data):
if key in int_list:
entry[key] = jc.utils.convert_to_int(entry[key])
if 'interfaces' in entry:
interfaces = []
for interface in entry["interfaces"]:
# 00 ff 58 60 5f 61 -> 00:ff:58:60:5f:61
interface['mac'] = interface['mac'].replace(' ', ':').replace('.', '')
interfaces.append(interface)
entry["interfaces"] = interfaces
# add flags_pretty
# Flag mapping from https://www.man7.org/linux/man-pages/man8/route.8.html
if 'flags' in entry:
@@ -165,6 +180,16 @@ def _process(proc_data):
return proc_data
def normalize_headers(headers: str):
# fixup header row for ipv6
if ' Next Hop ' in headers:
headers = headers.replace(' If', ' Iface')
headers = headers.replace(' Next Hop ', ' Next_Hop ')
headers = headers.replace(' Flag ', ' Flags ')
headers = headers.replace(' Met ', ' Metric ')
headers = headers.lower()
return headers
def parse(data, raw=False, quiet=False):
"""
@@ -180,24 +205,22 @@ def parse(data, raw=False, quiet=False):
List of Dictionaries. Raw or processed structured data.
"""
import jc.utils
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
cleandata = data.splitlines()[1:]
cleandata = data.splitlines()
raw_output = []
if jc.utils.has_data(data):
# fixup header row for ipv6
if ' Next Hop ' in cleandata[0]:
cleandata[0] = cleandata[0].replace(' If', ' Iface')
cleandata[0] = cleandata[0].replace(' Next Hop ', ' Next_Hop ')\
.replace(' Flag ', ' Flags ')\
.replace(' Met ', ' Metric ')
cleandata[0] = cleandata[0].lower()
raw_output = jc.parsers.universal.simple_table_parse(cleandata)
import jc.parsers.route_windows
if cleandata[0] in jc.parsers.route_windows.SEPERATORS:
raw_output = jc.parsers.route_windows.parse(cleandata)
else:
cleandata.pop(0) # Removing "Kernel IP routing table".
cleandata[0] = normalize_headers(cleandata[0])
import jc.parsers.universal
raw_output = jc.parsers.universal.simple_table_parse(cleandata)
if raw:
return raw_output

128
jc/parsers/route_windows.py Normal file
View File

@@ -0,0 +1,128 @@
"""
jc - JSON Convert Windows `route` command output parser
"""
import re
from typing import List
SEPERATORS = (
"===========================================================================",
" None"
)
# 22...00 50 56 c0 00 01 ......VMware Virtual Ethernet Adapter for VMnet1
# {"id": 22, "mac": "00 50 56 c0 00 01", "name": "VMware Virtual Ethernet Adapter for VMnet1"}
INTERFACE_REGEX = re.compile(
r"^(?P<id>\d+)\.{3}(?P<mac>.{17})[\s+\.]+(?P<name>[^\n\r]+)$"
)
ROUTE_TABLES = ("IPv4 Route Table", "IPv6 Route Table")
ROUTE_TYPES = ("Active Routes:", "Persistent Routes:")
def get_lines_until_seperator(iterator):
lines = []
for line in iterator:
if line in SEPERATORS:
break
lines.append(line)
return lines
def normalize_route_table(route_table: List[str]):
headers = route_table[0]
headers = headers.lower()
headers = headers.replace("network destination", "destination")
headers = headers.replace("if", "iface")
headers = headers.replace("interface", "iface")
headers = headers.replace("netmask", "genmask")
headers_count = len(headers.split())
previous_line_has_all_the_data = True
normalized_route_table = [headers]
for row in route_table[1:]:
row = row.strip()
has_all_the_data = len(row.split()) == headers_count
# If the number of columns doesn't match the number of headers in the current and previous line, concatenating them.
if not has_all_the_data and not previous_line_has_all_the_data:
previous_line = normalized_route_table.pop(
len(normalized_route_table) - 1)
row = f'{previous_line} {row}'
has_all_the_data = True
normalized_route_table.append(row.strip())
previous_line_has_all_the_data = has_all_the_data
return normalized_route_table
def parse(cleandata: List[str]):
"""
Main text parsing function for Windows route
Parameters:
cleandata: (string) text data to parse
Returns:
List of Dictionaries. Raw structured data.
"""
raw_output = []
data_iterator = iter(cleandata)
for line in data_iterator:
if not line:
continue
if line == "Interface List":
# Interface List
# 8...00 ff 58 60 5f 61 ......TAP-Windows Adapter V9
# 52...00 15 5d fd 0d 45 ......Hyper-V Virtual Ethernet Adapter
# ===========================================================================
interfaces = []
for interface_line in data_iterator:
interface_line = interface_line.strip()
if interface_line in SEPERATORS:
break
interface_match = INTERFACE_REGEX.search(interface_line)
if interface_match:
interfaces.append(interface_match.groupdict())
if interfaces:
raw_output.append({"interfaces": interfaces})
continue
full_route_table = []
if line in ROUTE_TABLES:
next(data_iterator) # Skipping the table title.
# Persistent Routes:
# Network Address Netmask Gateway Address Metric
# 157.0.0.0 255.0.0.0 157.55.80.1 3
# ===========================================================================
for route_line in data_iterator:
if route_line in ROUTE_TYPES:
import jc.parsers.universal
route_table = get_lines_until_seperator(
data_iterator
)
if not route_table:
continue
route_table = normalize_route_table(
route_table
)
full_route_table.extend(
jc.parsers.universal.simple_table_parse(
route_table
)
)
raw_output.extend(full_route_table)
return raw_output

View File

@@ -2,6 +2,8 @@
This parser conforms to the specification at https://semver.org/
See Also: `ver` parser.
Usage (cli):
$ echo 1.2.3-rc.1+44837 | jc --semver

281
jc/parsers/srt.py Normal file
View File

@@ -0,0 +1,281 @@
"""jc - JSON Convert `SRT` file parser
Usage (cli):
$ cat foo.srt | jc --srt
Usage (module):
import jc
result = jc.parse('srt', srt_file_output)
Schema:
[
{
"index": int,
"start": {
"hours": int,
"minutes": int,
"seconds": int,
"milliseconds": int,
"timestamp": string
},
"end": {
"hours": int,
"minutes": int,
"seconds": int,
"milliseconds": int,
"timestamp": string
},
"content": string
}
]
Examples:
$ cat attack_of_the_clones.srt
1
00:02:16,612 --> 00:02:19,376
Senator, we're making
our final approach into Coruscant.
2
00:02:19,482 --> 00:02:21,609
Very good, Lieutenant.
...
$ cat attack_of_the_clones.srt | jc --srt
[
{
"index": 1,
"start": {
"hours": 0,
"minutes": 2,
"seconds": 16,
"milliseconds": 612,
"timestamp": "00:02:16,612"
},
"end": {
"hours": 0,
"minutes": 2,
"seconds": 19,
"milliseconds": 376,
"timestamp": "00:02:19,376"
},
"content": "Senator, we're making\nour final approach into Coruscant."
},
{
"index": 2,
"start": {
"hours": 0,
"minutes": 2,
"seconds": 19,
"milliseconds": 482,
"timestamp": "00:02:19,482"
},
"end": {
"hours": 0,
"minutes": 2,
"seconds": 21,
"milliseconds": 609,
"timestamp": "00:02:21,609"
},
"content": "Very good, Lieutenant."
},
...
]
"""
import jc.utils
import re
from typing import List, Dict
from jc.jc_types import JSONDictType
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = 'SRT file parser'
author = 'Mark Rotner'
author_email = 'rotner.mr@gmail.com'
compatible = ['linux', 'darwin', 'cygwin', 'win32', 'aix', 'freebsd']
tags = ['standard', 'file', 'string']
__version__ = info.version
# Regex from https://github.com/cdown/srt/blob/434d0c1c9d5c26d5c3fb1ce979fc05b478e9253c/srt.py#LL16C1.
# The MIT License
# Copyright (c) 2014-present Christopher Down
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
# The format: (int)index\n(timestamp)start --> (timestamp)end\n(str)content\n.
# Example:
# 1
# 00:02:16,612 --> 00:02:19,376
# Senator, we're making our final approach into Coruscant.
# Start & End timestamp format: hours:minutes:seconds,millisecond.
# "." is not technically valid as a delimiter, but many editors create SRT
# files with this delimiter for whatever reason. Many editors and players
# accept it, so we do too.
RGX_TIMESTAMP_MAGNITUDE_DELIM = r"[,.:,.。:]"
RGX_POSITIVE_INT = r"[0-9]+"
RGX_POSITIVE_INT_OPTIONAL = r"[0-9]*"
RGX_TIMESTAMP = '{field}{separator}{field}{separator}{field}{separator}?{optional_field}'.format(
separator=RGX_TIMESTAMP_MAGNITUDE_DELIM,
field=RGX_POSITIVE_INT,
optional_field=RGX_POSITIVE_INT_OPTIONAL
)
RGX_INDEX = r"-?[0-9]+\.?[0-9]*" # int\float\negative.
RGX_CONTENT = r".*?" # Anything(except newline) but lazy.
RGX_NEWLINE = r"\r?\n" # Newline(CRLF\LF).
SRT_REGEX = re.compile(
r"\s*(?:({index})\s*{newline})?({ts}) *-[ -] *> *({ts}) ?(?:{newline}|\Z)({content})"
# Many sub editors don't add a blank line to the end, and many editors and
# players accept that. We allow it to be missing in input.
#
# We also allow subs that are missing a double blank newline. This often
# happens on subs which were first created as a mixed language subtitle,
# for example chs/eng, and then were stripped using naive methods (such as
# ed/sed) that don't understand newline preservation rules in SRT files.
#
# This means that when you are, say, only keeping chs, and the line only
# contains english, you end up with not only no content, but also all of
# the content lines are stripped instead of retaining a newline.
r"(?:{newline}|\Z)(?:{newline}|\Z|(?=(?:{index}\s*{newline}{ts})))"
# Some SRT blocks, while this is technically invalid, have blank lines
# inside the subtitle content. We look ahead a little to check that the
# next lines look like an index and a timestamp as a best-effort
# solution to work around these.
r"(?=(?:(?:{index}\s*{newline})?{ts}|\Z))".format(
index=RGX_INDEX,
ts=RGX_TIMESTAMP,
content=RGX_CONTENT,
newline=RGX_NEWLINE,
),
re.DOTALL,
)
TIMESTAMP_REGEX = re.compile(
'^({field}){separator}({field}){separator}({field}){separator}?({optional_field})$'.format(
separator=RGX_TIMESTAMP_MAGNITUDE_DELIM,
field=RGX_POSITIVE_INT,
optional_field=RGX_POSITIVE_INT_OPTIONAL
)
)
def _process(proc_data: List[JSONDictType]) -> List[JSONDictType]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (Dictionary) raw structured data to process
Returns:
List of Dictionaries representing an SRT document.
"""
int_list = {'index'}
timestamp_list = {"start", "end"}
timestamp_int_list = {"hours", "minutes", "seconds", "milliseconds"}
for entry in proc_data:
# Converting {"index"} to int.
for key in entry:
if key in int_list:
entry[key] = jc.utils.convert_to_int(entry[key])
# Converting {"hours", "minutes", "seconds", "milliseconds"} to int.
if key in timestamp_list:
timestamp = entry[key]
for timestamp_key in timestamp:
if timestamp_key in timestamp_int_list:
timestamp[timestamp_key] = jc.utils.convert_to_int(
timestamp[timestamp_key])
return proc_data
def parse_timestamp(timestamp: str) -> Dict:
"""
timestamp: "hours:minutes:seconds,milliseconds" --->
{
"hours": "hours",
"minutes": "minutes",
"seconds": "seconds",
"milliseconds": "milliseconds",
"timestamp": "hours:minutes:seconds,milliseconds"
}
"""
ts_match = TIMESTAMP_REGEX.match(timestamp)
if ts_match:
hours, minutes, seconds, milliseconds = ts_match.groups()
return {
"hours": hours,
"minutes": minutes,
"seconds": seconds,
"milliseconds": milliseconds,
"timestamp": timestamp
}
return {}
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[JSONDictType]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List[Dict] = []
if not jc.utils.has_data(data):
return raw_output
for subtitle in SRT_REGEX.finditer(data):
index, start, end, content = subtitle.groups()
raw_output.append(
{
"index": index,
"start": parse_timestamp(start),
"end": parse_timestamp(end),
"content": content.replace("\r\n", "\n")
}
)
return raw_output if raw else _process(raw_output)

View File

@@ -1,8 +1,5 @@
"""jc - JSON Convert `ss` command output parser
Extended information options like `-e` and `-p` are not supported and may
cause parsing irregularities.
Usage (cli):
$ ss | jc --ss
@@ -23,21 +20,29 @@ field names
[
{
"netid": string,
"state": string,
"recv_q": integer,
"send_q": integer,
"local_address": string,
"local_port": string,
"local_port_num": integer,
"peer_address": string,
"peer_port": string,
"peer_port_num": integer,
"interface": string,
"link_layer" string,
"channel": string,
"path": string,
"pid": integer
"netid": string,
"state": string,
"recv_q": integer,
"send_q": integer,
"local_address": string,
"local_port": string,
"local_port_num": integer,
"peer_address": string,
"peer_port": string,
"peer_port_num": integer,
"interface": string,
"link_layer" string,
"channel": string,
"path": string,
"pid": integer,
"opts": {
"process_id": {
"<process_id>": {
"user": string,
"file_descriptor": string
}
}
}
}
]
@@ -275,13 +280,15 @@ Examples:
}
]
"""
import re
import ast
import string
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.6'
version = '1.7'
description = '`ss` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -324,6 +331,57 @@ def _process(proc_data):
return proc_data
def _parse_opts(proc_data):
""" Process extra options -e, -o, -p
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
Structured data dictionary for extra/optional headerless options.
"""
o_field = proc_data.split(' ')
opts = {}
for item in o_field:
# -e option:
item = re.sub(
'uid', 'uid_number',
re.sub('sk', 'cookie', re.sub('ino', 'inode_number', item)))
if ":" in item:
key, val = item.split(':')
# -o option
if key == "timer":
val = val.replace('(', '[').replace(')', ']')
val = ast.literal_eval(re.sub(r'([a-z0-9\.]+)', '"\\1"', val))
val = {
'timer_name': val[0],
'expire_time': val[1],
'retrans': val[2]
}
opts[key] = val
# -p option
if key == "users":
key = 'process_id'
val = val.replace('(', '[').replace(')', ']')
val = ast.literal_eval(re.sub(r'([a-z]+=[0-9]+)', '"\\1"', val))
data = {}
for rec in val:
params = {}
params['user'] = rec[0]
for i in [x for x in rec if '=' in x]:
k, v = i.split('=')
params[k] = v
data.update({
params['pid']: {
'user': params['user'],
'file_descriptor': params['fd']
}
})
val = data
opts[key] = val
return opts
def parse(data, raw=False, quiet=False):
"""
@@ -357,15 +415,20 @@ def parse(data, raw=False, quiet=False):
header_text = header_text.replace('-', '_')
header_list = header_text.split()
extra_opts = False
for entry in cleandata[1:]:
output_line = {}
if entry[0] not in string.whitespace:
# fix weird ss bug where first two columns have no space between them sometimes
entry = entry[:5] + ' ' + entry[5:]
entry = entry[:5] + ' ' + entry[5:]
entry_list = entry.split()
entry_list = re.split(r'[ ]{1,}',entry.strip())
if len(entry_list) > len(header_list) or extra_opts == True:
entry_list = re.split(r'[ ]{2,}',entry.strip())
extra_opts = True
if entry_list[0] in contains_colon and ':' in entry_list[4]:
l_field = entry_list[4].rsplit(':', maxsplit=1)
@@ -381,6 +444,10 @@ def parse(data, raw=False, quiet=False):
entry_list[6] = p_address
entry_list.insert(7, p_port)
if re.search(r'ino:|uid:|sk:|users:|timer:',entry_list[-1]):
header_list.append('opts')
entry_list[-1] = _parse_opts(entry_list[-1])
output_line = dict(zip(header_list, entry_list))
# some post processing to pull out fields: interface, link_layer, path, pid, channel

688
jc/parsers/ssh_conf.py Normal file
View File

@@ -0,0 +1,688 @@
"""jc - JSON Convert `ssh` configuration file and `ssh -G` command output parser
This parser will work with `ssh` configuration files or the output of
`ssh -G`. Any `Match` blocks in the `ssh` configuration file will be
ignored.
Usage (cli):
$ ssh -G hostname | jc --ssh-conf
or
$ jc ssh -G hostname
or
$ cat ~/.ssh/config | jc --ssh-conf
Usage (module):
import jc
result = jc.parse('ssh_conf', ssh_conf_output)
Schema:
[
{
"host": string,
"host_list": [
string
],
"addkeystoagent": string,
"addressfamily": string,
"batchmode": string,
"bindaddress": string,
"bindinterface": string,
"canonicaldomains": [
string
],
"canonicalizefallbacklocal": string,
"canonicalizehostname": string,
"canonicalizemaxdots": integer,
"canonicalizepermittedcnames": [
string
],
"casignaturealgorithms": [
string
],
"certificatefile": [
string
],
"checkhostip": string,
"ciphers": [
string
],
"clearallforwardings": string,
"compression": string,
"connectionattempts": integer,
"connecttimeout": integer,
"controlmaster": string,
"controlpath": string,
"controlpersist": string,
"dynamicforward": string,
"enableescapecommandline": string,
"enablesshkeysign": string,
"escapechar": string,
"exitonforwardfailure": string,
"fingerprinthash": string,
"forkafterauthentication": string,
"forwardagent": string,
"forwardx11": string,
"forwardx11timeout": integer,
"forwardx11trusted": string,
"gatewayports": string,
"globalknownhostsfile": [
string
],
"gssapiauthentication": string,
"gssapidelegatecredentials": string,
"hashknownhosts": string,
"hostbasedacceptedalgorithms": [
string
],
"hostbasedauthentication": string,
"hostkeyalgorithms": [
string
],
"hostkeyalias": string,
"hostname": string,
"identitiesonly": string,
"identityagent": string,
"identityfile": [
string
],
"ignoreunknown": string,
"include": [
string
],
"ipqos": [
string
],
"kbdinteractiveauthentication": string,
"kbdinteractivedevices": [
string
],
"kexalgorithms": [
string
],
"kexalgorithms_strategy": string,
"knownhostscommand": string,
"localcommand": string,
"localforward": [
string
],
"loglevel": string,
"logverbose": [
string
],
"macs": [
string
],
"macs_strategy": string,
"nohostauthenticationforlocalhost": string,
"numberofpasswordprompts": integer,
"passwordauthentication": string,
"permitlocalcommand": string,
"permitremoteopen": [
string
],
"pkcs11provider": string,
"port": integer,
"preferredauthentications": [
string
],
"protocol": integer,
"proxycommand": string,
"proxyjump": [
string
],
"proxyusefdpass": string,
"pubkeyacceptedalgorithms": [
string
],
"pubkeyacceptedalgorithms_strategy": string,
"pubkeyauthentication": string,
"rekeylimit": string,
"remotecommand": string,
"remoteforward": string,
"requesttty": string,
"requiredrsasize": integer,
"revokedhostkeys": string,
"securitykeyprovider": string,
"sendenv": [
string
],
"serveralivecountmax": integer,
"serveraliveinterval": integer,
"sessiontype": string,
"setenv": [
string
],
"stdinnull": string,
"streamlocalbindmask": string,
"streamlocalbindunlink": string,
"stricthostkeychecking": string,
"syslogfacility": string,
"tcpkeepalive": string,
"tunnel": string,
"tunneldevice": string,
"updatehostkeys": string,
"user": string,
"userknownhostsfile": [
string
],
"verifyhostkeydns": string,
"visualhostkey": string,
"xauthlocation": string
}
]
Examples:
$ ssh -G - | jc --ssh-conf -p
[
{
"user": "foo",
"hostname": "-",
"port": 22,
"addressfamily": "any",
"batchmode": "no",
"canonicalizefallbacklocal": "yes",
"canonicalizehostname": "false",
"checkhostip": "no",
"compression": "no",
"controlmaster": "false",
"enablesshkeysign": "no",
"clearallforwardings": "no",
"exitonforwardfailure": "no",
"fingerprinthash": "SHA256",
"forwardx11": "no",
"forwardx11trusted": "no",
"gatewayports": "no",
"gssapiauthentication": "no",
"gssapidelegatecredentials": "no",
"hashknownhosts": "no",
"hostbasedauthentication": "no",
"identitiesonly": "no",
"kbdinteractiveauthentication": "yes",
"nohostauthenticationforlocalhost": "no",
"passwordauthentication": "yes",
"permitlocalcommand": "no",
"proxyusefdpass": "no",
"pubkeyauthentication": "true",
"requesttty": "auto",
"sessiontype": "default",
"stdinnull": "no",
"forkafterauthentication": "no",
"streamlocalbindunlink": "no",
"stricthostkeychecking": "ask",
"tcpkeepalive": "yes",
"tunnel": "false",
"verifyhostkeydns": "false",
"visualhostkey": "no",
"updatehostkeys": "true",
"applemultipath": "no",
"canonicalizemaxdots": 1,
"connectionattempts": 1,
"forwardx11timeout": 1200,
"numberofpasswordprompts": 3,
"serveralivecountmax": 3,
"serveraliveinterval": 0,
"ciphers": [
"chacha20-poly1305@openssh.com",
"aes128-ctr",
"aes192-ctr",
"aes256-ctr",
"aes128-gcm@openssh.com",
"aes256-gcm@openssh.com"
],
"hostkeyalgorithms": [
"ssh-ed25519-cert-v01@openssh.com",
"ecdsa-sha2-nistp256-cert-v01@openssh.com",
"ecdsa-sha2-nistp384-cert-v01@openssh.com",
"ecdsa-sha2-nistp521-cert-v01@openssh.com",
"rsa-sha2-512-cert-v01@openssh.com",
"rsa-sha2-256-cert-v01@openssh.com",
"ssh-ed25519",
"ecdsa-sha2-nistp256",
"ecdsa-sha2-nistp384",
"ecdsa-sha2-nistp521",
"rsa-sha2-512",
"rsa-sha2-256"
],
"hostbasedacceptedalgorithms": [
"ssh-ed25519-cert-v01@openssh.com",
"ecdsa-sha2-nistp256-cert-v01@openssh.com",
"ecdsa-sha2-nistp384-cert-v01@openssh.com",
"ecdsa-sha2-nistp521-cert-v01@openssh.com",
"rsa-sha2-512-cert-v01@openssh.com",
"rsa-sha2-256-cert-v01@openssh.com",
"ssh-ed25519",
"ecdsa-sha2-nistp256",
"ecdsa-sha2-nistp384",
"ecdsa-sha2-nistp521",
"rsa-sha2-512",
"rsa-sha2-256"
],
"kexalgorithms": [
"sntrup761x25519-sha512@openssh.com",
"curve25519-sha256",
"curve25519-sha256@libssh.org",
"ecdh-sha2-nistp256",
"ecdh-sha2-nistp384",
"ecdh-sha2-nistp521",
"diffie-hellman-group-exchange-sha256",
"diffie-hellman-group16-sha512",
"diffie-hellman-group18-sha512",
"diffie-hellman-group14-sha256"
],
"casignaturealgorithms": [
"ssh-ed25519",
"ecdsa-sha2-nistp256",
"ecdsa-sha2-nistp384",
"ecdsa-sha2-nistp521",
"rsa-sha2-512",
"rsa-sha2-256"
],
"loglevel": "INFO",
"macs": [
"umac-64-etm@openssh.com",
"umac-128-etm@openssh.com",
"hmac-sha2-256-etm@openssh.com",
"hmac-sha2-512-etm@openssh.com",
"hmac-sha1-etm@openssh.com",
"umac-64@openssh.com",
"umac-128@openssh.com",
"hmac-sha2-256",
"hmac-sha2-512",
"hmac-sha1"
],
"securitykeyprovider": "$SSH_SK_PROVIDER",
"pubkeyacceptedalgorithms": [
"ssh-ed25519-cert-v01@openssh.com",
"ecdsa-sha2-nistp256-cert-v01@openssh.com",
"ecdsa-sha2-nistp384-cert-v01@openssh.com",
"ecdsa-sha2-nistp521-cert-v01@openssh.com",
"rsa-sha2-512-cert-v01@openssh.com",
"rsa-sha2-256-cert-v01@openssh.com",
"ssh-ed25519",
"ecdsa-sha2-nistp256",
"ecdsa-sha2-nistp384",
"ecdsa-sha2-nistp521",
"rsa-sha2-512",
"rsa-sha2-256"
],
"xauthlocation": "/usr/X11R6/bin/xauth",
"identityfile": [
"~/.ssh/id_rsa",
"~/.ssh/id_ecdsa",
"~/.ssh/id_ecdsa_sk",
"~/.ssh/id_ed25519",
"~/.ssh/id_ed25519_sk",
"~/.ssh/id_xmss",
"~/.ssh/id_dsa"
],
"canonicaldomains": [
"none"
],
"globalknownhostsfile": [
"/etc/ssh/ssh_known_hosts",
"/etc/ssh/ssh_known_hosts2"
],
"userknownhostsfile": [
"/Users/foo/.ssh/known_hosts",
"/Users/foo/.ssh/known_hosts2"
],
"sendenv": [
"LANG",
"LC_*"
],
"logverbose": [
"none"
],
"permitremoteopen": [
"any"
],
"addkeystoagent": "false",
"forwardagent": "no",
"connecttimeout": null,
"tunneldevice": "any:any",
"canonicalizepermittedcnames": [
"none"
],
"controlpersist": "no",
"escapechar": "~",
"ipqos": [
"af21",
"cs1"
],
"rekeylimit": "0 0",
"streamlocalbindmask": "0177",
"syslogfacility": "USER"
}
]
$ cat ~/.ssh/config | jc --ssh-conf -p
[
{
"host": "server1",
"host_list": [
"server1"
],
"hostname": "server1.cyberciti.biz",
"user": "nixcraft",
"port": 4242,
"identityfile": [
"/nfs/shared/users/nixcraft/keys/server1/id_rsa"
]
},
{
"host": "nas01",
"host_list": [
"nas01"
],
"hostname": "192.168.1.100",
"user": "root",
"identityfile": [
"~/.ssh/nas01.key"
]
},
{
"host": "aws.apache",
"host_list": [
"aws.apache"
],
"hostname": "1.2.3.4",
"user": "wwwdata",
"identityfile": [
"~/.ssh/aws.apache.key"
]
},
{
"host": "uk.gw.lan uk.lan",
"host_list": [
"uk.gw.lan",
"uk.lan"
],
"hostname": "192.168.0.251",
"user": "nixcraft",
"proxycommand": "ssh nixcraft@gateway.uk.cyberciti.biz nc %h %p 2> /dev/null"
},
{
"host": "proxyus",
"host_list": [
"proxyus"
],
"hostname": "vps1.cyberciti.biz",
"user": "breakfree",
"identityfile": [
"~/.ssh/vps1.cyberciti.biz.key"
],
"localforward": [
"3128 127.0.0.1:3128"
]
},
{
"host": "*",
"host_list": [
"*"
],
"forwardagent": "no",
"forwardx11": "no",
"forwardx11trusted": "yes",
"user": "nixcraft",
"port": 22,
"protocol": 2,
"serveraliveinterval": 60,
"serveralivecountmax": 30
}
]
$ cat ~/.ssh/config | jc --ssh-conf -p -r
[
{
"host": "server1",
"host_list": [
"server1"
],
"hostname": "server1.cyberciti.biz",
"user": "nixcraft",
"port": "4242",
"identityfile": [
"/nfs/shared/users/nixcraft/keys/server1/id_rsa"
]
},
{
"host": "nas01",
"host_list": [
"nas01"
],
"hostname": "192.168.1.100",
"user": "root",
"identityfile": [
"~/.ssh/nas01.key"
]
},
{
"host": "aws.apache",
"host_list": [
"aws.apache"
],
"hostname": "1.2.3.4",
"user": "wwwdata",
"identityfile": [
"~/.ssh/aws.apache.key"
]
},
{
"host": "uk.gw.lan uk.lan",
"host_list": [
"uk.gw.lan",
"uk.lan"
],
"hostname": "192.168.0.251",
"user": "nixcraft",
"proxycommand": "ssh nixcraft@gateway.uk.cyberciti.biz nc %h %p 2> /dev/null"
},
{
"host": "proxyus",
"host_list": [
"proxyus"
],
"hostname": "vps1.cyberciti.biz",
"user": "breakfree",
"identityfile": [
"~/.ssh/vps1.cyberciti.biz.key"
],
"localforward": [
"3128 127.0.0.1:3128"
]
},
{
"host": "*",
"host_list": [
"*"
],
"forwardagent": "no",
"forwardx11": "no",
"forwardx11trusted": "yes",
"user": "nixcraft",
"port": "22",
"protocol": "2",
"serveraliveinterval": "60",
"serveralivecountmax": "30"
}
]
"""
from typing import Set, List, Dict
from jc.jc_types import JSONDictType
import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = '`ssh` config file and `ssh -G` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux', 'darwin', 'freebsd']
magic_commands = ['ssh -G']
tags = ['command', 'file']
__version__ = info.version
def _process(proc_data: List[JSONDictType]) -> List[JSONDictType]:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured to conform to the schema.
"""
split_fields_space: Set[str] = {
'canonicaldomains', 'globalknownhostsfile', 'include', 'ipqos',
'permitremoteopen', 'sendenv', 'setenv', 'userknownhostsfile'
}
split_fields_comma: Set[str] = {
'canonicalizepermittedcnames', 'casignaturealgorithms', 'ciphers',
'hostbasedacceptedalgorithms', 'hostkeyalgorithms',
'kbdinteractivedevices', 'kexalgorithms', 'logverbose', 'macs',
'preferredauthentications', 'proxyjump', 'pubkeyacceptedalgorithms'
}
int_list: Set[str] = {
'canonicalizemaxdots', 'connectionattempts', 'connecttimeout',
'forwardx11timeout', 'numberofpasswordprompts', 'port', 'protocol',
'requiredrsasize', 'serveralivecountmax', 'serveraliveinterval'
}
for host in proc_data:
dict_copy = host.copy()
for key, val in dict_copy.items():
# these are list values
if key == 'sendenv' or key == 'setenv' or key == 'include':
new_list: List[str] = []
for item in val:
new_list.extend(item.split())
host[key] = new_list
continue
if key in split_fields_space:
host[key] = val.split()
continue
if key in split_fields_comma:
host[key] = val.split(',')
continue
for key, val in host.items():
if key in int_list:
host[key] = jc.utils.convert_to_int(val)
return proc_data
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> List[JSONDictType]:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: List = []
host: Dict = {}
multi_fields: Set[str] = {
'certificatefile', 'identityfile', 'include', 'localforward',
'sendenv', 'setenv'
}
modified_fields: Set[str] = {
'casignaturealgorithms', 'ciphers', 'hostbasedacceptedalgorithms',
'HostKeyAlgorithms', 'kexalgorithms', 'macs',
'pubkeyacceptedalgorithms'
}
modifiers: Set[str] = {'+', '-', '^'}
match_block_found = False
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
# skip any lines with only whitespace
if not line.strip():
continue
# support configuration file by skipping commented lines
if line.strip().startswith('#'):
continue
if line.strip().startswith('Host '):
if host:
raw_output.append(host)
hostnames = line.split(maxsplit=1)[1]
host = {
'host': hostnames,
'host_list': hostnames.split()
}
# support configuration file by ignoring all lines between
# Match xxx and Match any
if line.strip().startswith('Match all'):
match_block_found = False
continue
if line.strip().startswith('Match'):
match_block_found = True
continue
if match_block_found:
continue
key, val = line.split(maxsplit=1)
# support configuration file by converting to lower case
key = key.lower()
if key in multi_fields:
if key not in host:
host[key] = []
host[key].append(val)
continue
if key in modified_fields and val[0] in modifiers:
host[key] = val[1:]
host[key + '_strategy'] = val[0]
continue
host[key] = val
continue
if host:
raw_output.append(host)
return raw_output if raw else _process(raw_output)

View File

@@ -1,4 +1,4 @@
"""jc - JSON Convert sshd configuration file and `sshd -T` command output parser
"""jc - JSON Convert `sshd` configuration file and `sshd -T` command output parser
This parser will work with `sshd` configuration files or the output of
`sshd -T`. Any `Match` blocks in the `sshd` configuration file will be
@@ -483,13 +483,13 @@ import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = 'sshd config file and `sshd -T` command parser'
version = '1.1'
description = '`sshd` config file and `sshd -T` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
compatible = ['linux', 'darwin', 'freebsd']
magic_commands = ['sshd -T']
tags = ['file']
tags = ['command', 'file']
__version__ = info.version
@@ -622,6 +622,10 @@ def parse(
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
# skip any lines with only whitespace
if not line.strip():
continue
# support configuration file by skipping commented lines
if line.strip().startswith('#'):
continue

View File

@@ -171,7 +171,7 @@ import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.12'
version = '1.13'
description = '`stat` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -238,112 +238,115 @@ def parse(data, raw=False, quiet=False):
# Clear any blank lines
cleandata = list(filter(None, data.splitlines()))
if jc.utils.has_data(data):
# linux output
if cleandata[0].startswith(' File: '):
# stats output contains 8 lines
for line in cleandata:
# line #1
if line.find('File:') == 2:
output_line = {}
line_list = line.split(maxsplit=1)
output_line['file'] = line_list[1]
# populate link_to field if -> found
if ' -> ' in output_line['file']:
filename = output_line['file'].split(' -> ')[0].strip('\u2018').rstrip('\u2019')
link = output_line['file'].split(' -> ')[1].strip('\u2018').rstrip('\u2019')
output_line['file'] = filename
output_line['link_to'] = link
else:
filename = output_line['file'].split(' -> ')[0].strip('\u2018').rstrip('\u2019')
output_line['file'] = filename
continue
# line #2
if line.find('Size:') == 2:
line_list = line.split(maxsplit=7)
output_line['size'] = line_list[1]
output_line['blocks'] = line_list[3]
output_line['io_blocks'] = line_list[6]
output_line['type'] = line_list[7]
continue
# line #3
if line.startswith('Device:'):
line_list = line.split()
output_line['device'] = line_list[1]
output_line['inode'] = line_list[3]
output_line['links'] = line_list[5]
continue
# line #4
if line.startswith('Access: ('):
line = line.replace('(', ' ').replace(')', ' ').replace('/', ' ')
line_list = line.split()
output_line['access'] = line_list[1]
output_line['flags'] = line_list[2]
output_line['uid'] = line_list[4]
output_line['user'] = line_list[5]
output_line['gid'] = line_list[7]
output_line['group'] = line_list[8]
continue
# line #5
if line.startswith('Access: 2'):
line_list = line.split(maxsplit=1)
output_line['access_time'] = line_list[1]
continue
# line #6
if line.startswith('Modify:'):
line_list = line.split(maxsplit=1)
output_line['modify_time'] = line_list[1]
continue
# line #7
if line.startswith('Change:'):
line_list = line.split(maxsplit=1)
output_line['change_time'] = line_list[1]
continue
# line #8
if line.find('Birth:') == 1:
line_list = line.split(maxsplit=1)
output_line['birth_time'] = line_list[1]
raw_output.append(output_line)
continue
# FreeBSD/OSX output
else:
for line in cleandata:
value = shlex.split(line)
output_line = {
'file': ' '.join(value[15:]),
'unix_device': value[0],
'inode': value[1],
'flags': value[2],
'links': value[3],
'user': value[4],
'group': value[5],
'rdev': value[6],
'size': value[7],
'access_time': value[8],
'modify_time': value[9],
'change_time': value[10],
'birth_time': value[11],
'block_size': value[12],
'blocks': value[13],
'unix_flags': value[14]
}
raw_output.append(output_line)
if raw:
if not jc.utils.has_data(data):
return raw_output
# linux output
if cleandata[0].startswith(' File: '):
output_line = {}
# stats output contains 8 lines
for line in cleandata:
# line #1
if line.find('File:') == 2:
if output_line: # Reached a new file stat info.
raw_output.append(output_line)
output_line = {}
line_list = line.split(maxsplit=1)
output_line['file'] = line_list[1]
# populate link_to field if -> found
if ' -> ' in output_line['file']:
filename = output_line['file'].split(' -> ')[0].strip('\u2018').rstrip('\u2019')
link = output_line['file'].split(' -> ')[1].strip('\u2018').rstrip('\u2019')
output_line['file'] = filename
output_line['link_to'] = link
else:
filename = output_line['file'].split(' -> ')[0].strip('\u2018').rstrip('\u2019')
output_line['file'] = filename
continue
# line #2
if line.startswith(' Size:'):
line_list = line.split(maxsplit=7)
output_line['size'] = line_list[1]
output_line['blocks'] = line_list[3]
output_line['io_blocks'] = line_list[6]
output_line['type'] = line_list[7]
continue
# line #3
if line.startswith('Device:'):
line_list = line.split()
output_line['device'] = line_list[1]
output_line['inode'] = line_list[3]
output_line['links'] = line_list[5]
continue
# line #4
if line.startswith('Access: ('):
line = line.replace('(', ' ').replace(')', ' ').replace('/', ' ')
line_list = line.split()
output_line['access'] = line_list[1]
output_line['flags'] = line_list[2]
output_line['uid'] = line_list[4]
output_line['user'] = line_list[5]
output_line['gid'] = line_list[7]
output_line['group'] = line_list[8]
continue
# line #5
if line.startswith('Access: 2'):
line_list = line.split(maxsplit=1)
output_line['access_time'] = line_list[1]
continue
# line #6
if line.startswith('Modify:'):
line_list = line.split(maxsplit=1)
output_line['modify_time'] = line_list[1]
continue
# line #7
if line.startswith('Change:'):
line_list = line.split(maxsplit=1)
output_line['change_time'] = line_list[1]
continue
# line #8
if line.startswith(' Birth:'):
line_list = line.split(maxsplit=1)
output_line['birth_time'] = line_list[1]
continue
if output_line:
raw_output.append(output_line)
# FreeBSD/OSX output
else:
return _process(raw_output)
for line in cleandata:
value = shlex.split(line)
output_line = {
'file': ' '.join(value[15:]),
'unix_device': value[0],
'inode': value[1],
'flags': value[2],
'links': value[3],
'user': value[4],
'group': value[5],
'rdev': value[6],
'size': value[7],
'access_time': value[8],
'modify_time': value[9],
'change_time': value[10],
'birth_time': value[11],
'block_size': value[12],
'blocks': value[13],
'unix_flags': value[14]
}
raw_output.append(output_line)
return raw_output if raw else _process(raw_output)

View File

@@ -1,5 +1,7 @@
"""jc - JSON Convert `timedatectl` command output parser
Also supports the `timesync-status` option.
The `epoch_utc` calculated timestamp field is timezone-aware and is only
available if the `universal_time` field is available.
@@ -29,7 +31,24 @@ Schema:
"system_clock_synchronized": boolean,
"systemd-timesyncd.service_active": boolean,
"rtc_in_local_tz": boolean,
"dst_active": boolean
"dst_active": boolean,
"server": string,
"poll_interval": string,
"leap": string,
"version": integer,
"stratum": integer,
"reference": string,
"precision": string,
"root_distance": string,
"offset": float,
"offset_unit": string,
"delay": float,
"delay_unit": string,
"jitter": float,
"jitter_unit": string,
"packet_count": integer,
"frequency": float,
"frequency_unit": string
}
Examples:
@@ -64,7 +83,7 @@ import jc.utils
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.7'
version = '1.8'
description = '`timedatectl status` command parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -90,11 +109,26 @@ def _process(proc_data):
"""
bool_list = {'ntp_enabled', 'ntp_synchronized', 'rtc_in_local_tz', 'dst_active',
'system_clock_synchronized', 'systemd-timesyncd.service_active'}
int_list = {'version', 'stratum', 'packet_count'}
float_list = {'offset', 'delay', 'jitter', 'frequency'}
for key in ['offset', 'delay', 'jitter']:
if key in proc_data:
proc_data[key + '_unit'] = proc_data[key][-2:]
if 'frequency' in proc_data:
proc_data['frequency_unit'] = proc_data['frequency'][-3:]
for key in proc_data:
if key in bool_list:
proc_data[key] = jc.utils.convert_to_bool(proc_data[key])
if key in int_list:
proc_data[key] = jc.utils.convert_to_int(proc_data[key])
if key in float_list:
proc_data[key] = jc.utils.convert_to_float(proc_data[key])
if 'universal_time' in proc_data:
ts = jc.utils.timestamp(proc_data['universal_time'], format_hint=(7300,))
proc_data['epoch_utc'] = ts.utc
@@ -120,17 +154,27 @@ def parse(data, raw=False, quiet=False):
jc.utils.input_type_check(data)
raw_output = {}
valid_fields = {
'local time', 'universal time', 'rtc time', 'time zone', 'ntp enabled',
'ntp synchronized', 'rtc in local tz', 'dst active',
'system clock synchronized', 'ntp service',
'systemd-timesyncd.service active', 'server', 'poll interval', 'leap',
'version', 'stratum', 'reference', 'precision', 'root distance',
'offset', 'delay', 'jitter', 'packet count', 'frequency'
}
if jc.utils.has_data(data):
for line in filter(None, data.splitlines()):
linedata = line.split(':', maxsplit=1)
raw_output[linedata[0].strip().lower().replace(' ', '_')] = linedata[1].strip()
try:
key, val = line.split(':', maxsplit=1)
key = key.lower().strip()
val = val.strip()
except ValueError:
continue
if linedata[0].strip() == 'DST active':
break
if key in valid_fields:
keyname = key.replace(' ', '_')
raw_output[keyname] = val
if raw:
return raw_output
else:
return _process(raw_output)
return raw_output if raw else _process(raw_output)

149
jc/parsers/toml.py Normal file
View File

@@ -0,0 +1,149 @@
"""jc - JSON Convert TOML file parser
Usage (cli):
$ cat file.toml | jc --toml
Usage (module):
import jc
result = jc.parse('toml', toml_file_output)
Schema:
TOML Document converted to a Dictionary.
See https://toml.io/en/ for details.
{
"key1": string/int/float/boolean/null/array/object,
"key2": string/int/float/boolean/null/array/object
}
Examples:
$ cat file.toml
title = "TOML Example"
[owner]
name = "Tom Preston-Werner"
dob = 1979-05-27T07:32:00-08:00
[database]
enabled = true
ports = [ 8000, 8001, 8002 ]
$ cat file.toml | jc --toml -p
{
"title": "TOML Example",
"owner": {
"name": "Tom Preston-Werner",
"dob": 296667120,
"dob_iso": "1979-05-27T07:32:00-08:00"
},
"database": {
"enabled": true,
"ports": [
8000,
8001,
8002
]
}
}
"""
from typing import Any
from jc.jc_types import JSONDictType
import jc.utils
from jc.parsers import tomli
from datetime import datetime
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.0'
description = 'TOML file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
details = 'Using the tomli library at https://github.com/hukkin/tomli.'
compatible = ['linux', 'darwin', 'cygwin', 'win32', 'aix', 'freebsd']
tags = ['standard', 'file', 'string']
__version__ = info.version
def _process(proc_data: JSONDictType) -> JSONDictType:
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
Dictionary. Structured to conform to the schema.
"""
return proc_data
def _fix_objects(obj: Any) -> JSONDictType:
"""
Recursively traverse the nested dictionary or list and convert objects
into JSON serializable types.
"""
if isinstance(obj, dict):
for k, v in obj.copy().items():
if isinstance(v, datetime):
iso = v.isoformat()
v = int(round(v.timestamp()))
obj.update({k: v, f'{k}_iso': iso})
continue
if isinstance(v, dict):
obj.update({k: _fix_objects(v)})
continue
if isinstance(v, list):
newlist = []
for i in v:
newlist.append(_fix_objects(i))
obj.update({k: newlist})
continue
if isinstance(obj, list):
new_list = []
for i in obj:
new_list.append(_fix_objects(i))
obj = new_list
return obj
def parse(
data: str,
raw: bool = False,
quiet: bool = False
) -> JSONDictType:
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) unprocessed output if True
quiet: (boolean) suppress warning messages if True
Returns:
Dictionary. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output: JSONDictType = {}
if jc.utils.has_data(data):
raw_output = _fix_objects(tomli.loads(data))
return raw_output if raw else _process(raw_output)

21
jc/parsers/tomli/LICENSE Normal file
View File

@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2021 Taneli Hukkinen
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

View File

@@ -0,0 +1,11 @@
# SPDX-License-Identifier: MIT
# SPDX-FileCopyrightText: 2021 Taneli Hukkinen
# Licensed to PSF under a Contributor Agreement.
__all__ = ("loads", "load", "TOMLDecodeError")
__version__ = "2.0.1" # DO NOT EDIT THIS LINE MANUALLY. LET bump2version UTILITY DO IT
from ._parser import TOMLDecodeError, load, loads
# Pretend this exception was created here.
TOMLDecodeError.__module__ = __name__

691
jc/parsers/tomli/_parser.py Normal file
View File

@@ -0,0 +1,691 @@
# SPDX-License-Identifier: MIT
# SPDX-FileCopyrightText: 2021 Taneli Hukkinen
# Licensed to PSF under a Contributor Agreement.
# from __future__ import annotations
from collections import namedtuple
import string
# from types import MappingProxyType
# from typing import Any, BinaryIO, NamedTuple
from ._re import (
RE_DATETIME,
RE_LOCALTIME,
RE_NUMBER,
match_to_datetime,
match_to_localtime,
match_to_number,
)
# from ._types import Key, ParseFloat, Pos
ASCII_CTRL = frozenset(chr(i) for i in range(32)) | frozenset(chr(127))
# Neither of these sets include quotation mark or backslash. They are
# currently handled as separate cases in the parser functions.
ILLEGAL_BASIC_STR_CHARS = ASCII_CTRL - frozenset("\t")
ILLEGAL_MULTILINE_BASIC_STR_CHARS = ASCII_CTRL - frozenset("\t\n")
ILLEGAL_LITERAL_STR_CHARS = ILLEGAL_BASIC_STR_CHARS
ILLEGAL_MULTILINE_LITERAL_STR_CHARS = ILLEGAL_MULTILINE_BASIC_STR_CHARS
ILLEGAL_COMMENT_CHARS = ILLEGAL_BASIC_STR_CHARS
TOML_WS = frozenset(" \t")
TOML_WS_AND_NEWLINE = TOML_WS | frozenset("\n")
BARE_KEY_CHARS = frozenset(string.ascii_letters + string.digits + "-_")
KEY_INITIAL_CHARS = BARE_KEY_CHARS | frozenset("\"'")
HEXDIGIT_CHARS = frozenset(string.hexdigits)
BASIC_STR_ESCAPE_REPLACEMENTS = {
"\\b": "\u0008", # backspace
"\\t": "\u0009", # tab
"\\n": "\u000A", # linefeed
"\\f": "\u000C", # form feed
"\\r": "\u000D", # carriage return
'\\"': "\u0022", # quote
"\\\\": "\u005C", # backslash
}
class TOMLDecodeError(ValueError):
"""An error raised if a document is not valid TOML."""
def load(__fp, *, parse_float=float):
"""Parse TOML from a binary file object."""
b = __fp.read()
try:
s = b.decode()
except AttributeError:
raise TypeError(
"File must be opened in binary mode, e.g. use `open('foo.toml', 'rb')`"
) from None
return loads(s, parse_float=parse_float)
def loads(__s, *, parse_float=float):
"""Parse TOML from a string."""
# The spec allows converting "\r\n" to "\n", even in string
# literals. Let's do so to simplify parsing.
src = __s.replace("\r\n", "\n")
pos = 0
out = Output(NestedDict(), Flags())
header = ()
parse_float = make_safe_parse_float(parse_float)
# Parse one statement at a time
# (typically means one line in TOML source)
while True:
# 1. Skip line leading whitespace
pos = skip_chars(src, pos, TOML_WS)
# 2. Parse rules. Expect one of the following:
# - end of file
# - end of line
# - comment
# - key/value pair
# - append dict to list (and move to its namespace)
# - create dict (and move to its namespace)
# Skip trailing whitespace when applicable.
try:
char = src[pos]
except IndexError:
break
if char == "\n":
pos += 1
continue
if char in KEY_INITIAL_CHARS:
pos = key_value_rule(src, pos, out, header, parse_float)
pos = skip_chars(src, pos, TOML_WS)
elif char == "[":
try:
second_char = src[pos + 1]
except IndexError:
second_char = None
out.flags.finalize_pending()
if second_char == "[":
pos, header = create_list_rule(src, pos, out)
else:
pos, header = create_dict_rule(src, pos, out)
pos = skip_chars(src, pos, TOML_WS)
elif char != "#":
raise suffixed_err(src, pos, "Invalid statement")
# 3. Skip comment
pos = skip_comment(src, pos)
# 4. Expect end of line or end of file
try:
char = src[pos]
except IndexError:
break
if char != "\n":
raise suffixed_err(
src, pos, "Expected newline or end of document after a statement"
)
pos += 1
return out.data.dict
class Flags:
"""Flags that map to parsed keys/namespaces."""
# Marks an immutable namespace (inline array or inline table).
FROZEN = 0
# Marks a nest that has been explicitly created and can no longer
# be opened using the "[table]" syntax.
EXPLICIT_NEST = 1
def __init__(self):
self._flags = {}
self._pending_flags = set()
def add_pending(self, key, flag):
self._pending_flags.add((key, flag))
def finalize_pending(self):
for key, flag in self._pending_flags:
self.set(key, flag, recursive=False)
self._pending_flags.clear()
def unset_all(self, key):
cont = self._flags
for k in key[:-1]:
if k not in cont:
return
cont = cont[k]["nested"]
cont.pop(key[-1], None)
def set(self, key, flag, *, recursive):
cont = self._flags
key_parent, key_stem = key[:-1], key[-1]
for k in key_parent:
if k not in cont:
cont[k] = {"flags": set(), "recursive_flags": set(), "nested": {}}
cont = cont[k]["nested"]
if key_stem not in cont:
cont[key_stem] = {"flags": set(), "recursive_flags": set(), "nested": {}}
cont[key_stem]["recursive_flags" if recursive else "flags"].add(flag)
def is_(self, key, flag):
if not key:
return False # document root has no flags
cont = self._flags
for k in key[:-1]:
if k not in cont:
return False
inner_cont = cont[k]
if flag in inner_cont["recursive_flags"]:
return True
cont = inner_cont["nested"]
key_stem = key[-1]
if key_stem in cont:
cont = cont[key_stem]
return flag in cont["flags"] or flag in cont["recursive_flags"]
return False
class NestedDict:
def __init__(self):
# The parsed content of the TOML document
self.dict = {}
def get_or_create_nest(
self,
key,
*,
access_lists: bool = True,
) -> dict:
cont = self.dict
for k in key:
if k not in cont:
cont[k] = {}
cont = cont[k]
if access_lists and isinstance(cont, list):
cont = cont[-1]
if not isinstance(cont, dict):
raise KeyError("There is no nest behind this key")
return cont
def append_nest_to_list(self, key):
cont = self.get_or_create_nest(key[:-1])
last_key = key[-1]
if last_key in cont:
list_ = cont[last_key]
if not isinstance(list_, list):
raise KeyError("An object other than list found behind this key")
list_.append({})
else:
cont[last_key] = [{}]
# class Output(namedtuple):
# data: NestedDict
# flags: Flags
Output = namedtuple('Output', ['data', 'flags'])
def skip_chars(src: str, pos, chars):
try:
while src[pos] in chars:
pos += 1
except IndexError:
pass
return pos
def skip_until(
src,
pos,
expect,
*,
error_on,
error_on_eof,
):
try:
new_pos = src.index(expect, pos)
except ValueError:
new_pos = len(src)
if error_on_eof:
raise suffixed_err(src, new_pos, f"Expected {expect!r}") from None
if not error_on.isdisjoint(src[pos:new_pos]):
while src[pos] not in error_on:
pos += 1
raise suffixed_err(src, pos, f"Found invalid character {src[pos]!r}")
return new_pos
def skip_comment(src, pos):
try:
char = src[pos]
except IndexError:
char = None
if char == "#":
return skip_until(
src, pos + 1, "\n", error_on=ILLEGAL_COMMENT_CHARS, error_on_eof=False
)
return pos
def skip_comments_and_array_ws(src, pos):
while True:
pos_before_skip = pos
pos = skip_chars(src, pos, TOML_WS_AND_NEWLINE)
pos = skip_comment(src, pos)
if pos == pos_before_skip:
return pos
def create_dict_rule(src, pos, out):
pos += 1 # Skip "["
pos = skip_chars(src, pos, TOML_WS)
pos, key = parse_key(src, pos)
if out.flags.is_(key, Flags.EXPLICIT_NEST) or out.flags.is_(key, Flags.FROZEN):
raise suffixed_err(src, pos, f"Cannot declare {key} twice")
out.flags.set(key, Flags.EXPLICIT_NEST, recursive=False)
try:
out.data.get_or_create_nest(key)
except KeyError:
raise suffixed_err(src, pos, "Cannot overwrite a value") from None
if not src.startswith("]", pos):
raise suffixed_err(src, pos, "Expected ']' at the end of a table declaration")
return pos + 1, key
def create_list_rule(src: str, pos, out):
pos += 2 # Skip "[["
pos = skip_chars(src, pos, TOML_WS)
pos, key = parse_key(src, pos)
if out.flags.is_(key, Flags.FROZEN):
raise suffixed_err(src, pos, f"Cannot mutate immutable namespace {key}")
# Free the namespace now that it points to another empty list item...
out.flags.unset_all(key)
# ...but this key precisely is still prohibited from table declaration
out.flags.set(key, Flags.EXPLICIT_NEST, recursive=False)
try:
out.data.append_nest_to_list(key)
except KeyError:
raise suffixed_err(src, pos, "Cannot overwrite a value") from None
if not src.startswith("]]", pos):
raise suffixed_err(src, pos, "Expected ']]' at the end of an array declaration")
return pos + 2, key
def key_value_rule(
src, pos, out, header, parse_float
):
pos, key, value = parse_key_value_pair(src, pos, parse_float)
key_parent, key_stem = key[:-1], key[-1]
abs_key_parent = header + key_parent
relative_path_cont_keys = (header + key[:i] for i in range(1, len(key)))
for cont_key in relative_path_cont_keys:
# Check that dotted key syntax does not redefine an existing table
if out.flags.is_(cont_key, Flags.EXPLICIT_NEST):
raise suffixed_err(src, pos, f"Cannot redefine namespace {cont_key}")
# Containers in the relative path can't be opened with the table syntax or
# dotted key/value syntax in following table sections.
out.flags.add_pending(cont_key, Flags.EXPLICIT_NEST)
if out.flags.is_(abs_key_parent, Flags.FROZEN):
raise suffixed_err(
src, pos, f"Cannot mutate immutable namespace {abs_key_parent}"
)
try:
nest = out.data.get_or_create_nest(abs_key_parent)
except KeyError:
raise suffixed_err(src, pos, "Cannot overwrite a value") from None
if key_stem in nest:
raise suffixed_err(src, pos, "Cannot overwrite a value")
# Mark inline table and array namespaces recursively immutable
if isinstance(value, (dict, list)):
out.flags.set(header + key, Flags.FROZEN, recursive=True)
nest[key_stem] = value
return pos
def parse_key_value_pair(
src, pos, parse_float
):
pos, key = parse_key(src, pos)
try:
char = src[pos]
except IndexError:
char = None
if char != "=":
raise suffixed_err(src, pos, "Expected '=' after a key in a key/value pair")
pos += 1
pos = skip_chars(src, pos, TOML_WS)
pos, value = parse_value(src, pos, parse_float)
return pos, key, value
def parse_key(src, pos):
pos, key_part = parse_key_part(src, pos)
key = (key_part,)
pos = skip_chars(src, pos, TOML_WS)
while True:
try:
char = src[pos]
except IndexError:
char = None
if char != ".":
return pos, key
pos += 1
pos = skip_chars(src, pos, TOML_WS)
pos, key_part = parse_key_part(src, pos)
key += (key_part,)
pos = skip_chars(src, pos, TOML_WS)
def parse_key_part(src, pos):
try:
char = src[pos]
except IndexError:
char = None
if char in BARE_KEY_CHARS:
start_pos = pos
pos = skip_chars(src, pos, BARE_KEY_CHARS)
return pos, src[start_pos:pos]
if char == "'":
return parse_literal_str(src, pos)
if char == '"':
return parse_one_line_basic_str(src, pos)
raise suffixed_err(src, pos, "Invalid initial character for a key part")
def parse_one_line_basic_str(src, pos):
pos += 1
return parse_basic_str(src, pos, multiline=False)
def parse_array(src, pos, parse_float):
pos += 1
array = []
pos = skip_comments_and_array_ws(src, pos)
if src.startswith("]", pos):
return pos + 1, array
while True:
pos, val = parse_value(src, pos, parse_float)
array.append(val)
pos = skip_comments_and_array_ws(src, pos)
c = src[pos : pos + 1]
if c == "]":
return pos + 1, array
if c != ",":
raise suffixed_err(src, pos, "Unclosed array")
pos += 1
pos = skip_comments_and_array_ws(src, pos)
if src.startswith("]", pos):
return pos + 1, array
def parse_inline_table(src, pos, parse_float):
pos += 1
nested_dict = NestedDict()
flags = Flags()
pos = skip_chars(src, pos, TOML_WS)
if src.startswith("}", pos):
return pos + 1, nested_dict.dict
while True:
pos, key, value = parse_key_value_pair(src, pos, parse_float)
key_parent, key_stem = key[:-1], key[-1]
if flags.is_(key, Flags.FROZEN):
raise suffixed_err(src, pos, f"Cannot mutate immutable namespace {key}")
try:
nest = nested_dict.get_or_create_nest(key_parent, access_lists=False)
except KeyError:
raise suffixed_err(src, pos, "Cannot overwrite a value") from None
if key_stem in nest:
raise suffixed_err(src, pos, f"Duplicate inline table key {key_stem!r}")
nest[key_stem] = value
pos = skip_chars(src, pos, TOML_WS)
c = src[pos : pos + 1]
if c == "}":
return pos + 1, nested_dict.dict
if c != ",":
raise suffixed_err(src, pos, "Unclosed inline table")
if isinstance(value, (dict, list)):
flags.set(key, Flags.FROZEN, recursive=True)
pos += 1
pos = skip_chars(src, pos, TOML_WS)
def parse_basic_str_escape(
src, pos, *, multiline = False
):
escape_id = src[pos : pos + 2]
pos += 2
if multiline and escape_id in {"\\ ", "\\\t", "\\\n"}:
# Skip whitespace until next non-whitespace character or end of
# the doc. Error if non-whitespace is found before newline.
if escape_id != "\\\n":
pos = skip_chars(src, pos, TOML_WS)
try:
char = src[pos]
except IndexError:
return pos, ""
if char != "\n":
raise suffixed_err(src, pos, "Unescaped '\\' in a string")
pos += 1
pos = skip_chars(src, pos, TOML_WS_AND_NEWLINE)
return pos, ""
if escape_id == "\\u":
return parse_hex_char(src, pos, 4)
if escape_id == "\\U":
return parse_hex_char(src, pos, 8)
try:
return pos, BASIC_STR_ESCAPE_REPLACEMENTS[escape_id]
except KeyError:
raise suffixed_err(src, pos, "Unescaped '\\' in a string") from None
def parse_basic_str_escape_multiline(src, pos):
return parse_basic_str_escape(src, pos, multiline=True)
def parse_hex_char(src, pos, hex_len):
hex_str = src[pos : pos + hex_len]
if len(hex_str) != hex_len or not HEXDIGIT_CHARS.issuperset(hex_str):
raise suffixed_err(src, pos, "Invalid hex value")
pos += hex_len
hex_int = int(hex_str, 16)
if not is_unicode_scalar_value(hex_int):
raise suffixed_err(src, pos, "Escaped character is not a Unicode scalar value")
return pos, chr(hex_int)
def parse_literal_str(src, pos):
pos += 1 # Skip starting apostrophe
start_pos = pos
pos = skip_until(
src, pos, "'", error_on=ILLEGAL_LITERAL_STR_CHARS, error_on_eof=True
)
return pos + 1, src[start_pos:pos] # Skip ending apostrophe
def parse_multiline_str(src, pos, *, literal):
pos += 3
if src.startswith("\n", pos):
pos += 1
if literal:
delim = "'"
end_pos = skip_until(
src,
pos,
"'''",
error_on=ILLEGAL_MULTILINE_LITERAL_STR_CHARS,
error_on_eof=True,
)
result = src[pos:end_pos]
pos = end_pos + 3
else:
delim = '"'
pos, result = parse_basic_str(src, pos, multiline=True)
# Add at maximum two extra apostrophes/quotes if the end sequence
# is 4 or 5 chars long instead of just 3.
if not src.startswith(delim, pos):
return pos, result
pos += 1
if not src.startswith(delim, pos):
return pos, result + delim
pos += 1
return pos, result + (delim * 2)
def parse_basic_str(src, pos, *, multiline):
if multiline:
error_on = ILLEGAL_MULTILINE_BASIC_STR_CHARS
parse_escapes = parse_basic_str_escape_multiline
else:
error_on = ILLEGAL_BASIC_STR_CHARS
parse_escapes = parse_basic_str_escape
result = ""
start_pos = pos
while True:
try:
char = src[pos]
except IndexError:
raise suffixed_err(src, pos, "Unterminated string") from None
if char == '"':
if not multiline:
return pos + 1, result + src[start_pos:pos]
if src.startswith('"""', pos):
return pos + 3, result + src[start_pos:pos]
pos += 1
continue
if char == "\\":
result += src[start_pos:pos]
pos, parsed_escape = parse_escapes(src, pos)
result += parsed_escape
start_pos = pos
continue
if char in error_on:
raise suffixed_err(src, pos, f"Illegal character {char!r}")
pos += 1
def parse_value( # noqa: C901
src, pos, parse_float
):
try:
char = src[pos]
except IndexError:
char = None
# IMPORTANT: order conditions based on speed of checking and likelihood
# Basic strings
if char == '"':
if src.startswith('"""', pos):
return parse_multiline_str(src, pos, literal=False)
return parse_one_line_basic_str(src, pos)
# Literal strings
if char == "'":
if src.startswith("'''", pos):
return parse_multiline_str(src, pos, literal=True)
return parse_literal_str(src, pos)
# Booleans
if char == "t":
if src.startswith("true", pos):
return pos + 4, True
if char == "f":
if src.startswith("false", pos):
return pos + 5, False
# Arrays
if char == "[":
return parse_array(src, pos, parse_float)
# Inline tables
if char == "{":
return parse_inline_table(src, pos, parse_float)
# Dates and times
datetime_match = RE_DATETIME.match(src, pos)
if datetime_match:
try:
datetime_obj = match_to_datetime(datetime_match)
except ValueError as e:
raise suffixed_err(src, pos, "Invalid date or datetime") from e
return datetime_match.end(), datetime_obj
localtime_match = RE_LOCALTIME.match(src, pos)
if localtime_match:
return localtime_match.end(), match_to_localtime(localtime_match)
# Integers and "normal" floats.
# The regex will greedily match any type starting with a decimal
# char, so needs to be located after handling of dates and times.
number_match = RE_NUMBER.match(src, pos)
if number_match:
return number_match.end(), match_to_number(number_match, parse_float)
# Special floats
first_three = src[pos : pos + 3]
if first_three in {"inf", "nan"}:
return pos + 3, parse_float(first_three)
first_four = src[pos : pos + 4]
if first_four in {"-inf", "+inf", "-nan", "+nan"}:
return pos + 4, parse_float(first_four)
raise suffixed_err(src, pos, "Invalid value")
def suffixed_err(src, pos, msg):
"""Return a `TOMLDecodeError` where error message is suffixed with
coordinates in source."""
def coord_repr(src, pos):
if pos >= len(src):
return "end of document"
line = src.count("\n", 0, pos) + 1
if line == 1:
column = pos + 1
else:
column = pos - src.rindex("\n", 0, pos)
return f"line {line}, column {column}"
return TOMLDecodeError(f"{msg} (at {coord_repr(src, pos)})")
def is_unicode_scalar_value(codepoint: int) -> bool:
return (0 <= codepoint <= 55295) or (57344 <= codepoint <= 1114111)
def make_safe_parse_float(parse_float):
"""A decorator to make `parse_float` safe.
`parse_float` must not return dicts or lists, because these types
would be mixed with parsed TOML tables and arrays, thus confusing
the parser. The returned decorated callable raises `ValueError`
instead of returning illegal types.
"""
# The default `float` callable never returns illegal types. Optimize it.
if parse_float is float:
return float
def safe_parse_float(float_str):
float_value = parse_float(float_str)
if isinstance(float_value, (dict, list)):
raise ValueError("parse_float must not return dicts or lists")
return float_value
return safe_parse_float

Some files were not shown because too many files have changed in this diff Show More