mirror of
https://github.com/kellyjonbrazil/jc.git
synced 2026-04-03 17:44:07 +02:00
Compare commits
190 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
9b33a50bb0 | ||
|
|
b0e9d9fa9d | ||
|
|
991b612b62 | ||
|
|
855c9363a5 | ||
|
|
625544f53e | ||
|
|
f179da8cd1 | ||
|
|
eeee776555 | ||
|
|
d0bde14a2a | ||
|
|
69b0e407ad | ||
|
|
ad2b9ab76c | ||
|
|
32483de4f2 | ||
|
|
ec713cfa1b | ||
|
|
eac8c1c1a0 | ||
|
|
4e15ab7924 | ||
|
|
5882755e7c | ||
|
|
13388df603 | ||
|
|
430385fa63 | ||
|
|
7bfebdcfeb | ||
|
|
e472c3b140 | ||
|
|
43ac3d951c | ||
|
|
207fa232c3 | ||
|
|
667dc1976a | ||
|
|
7437750de5 | ||
|
|
aab47c9e80 | ||
|
|
f1ed803525 | ||
|
|
093c1703d7 | ||
|
|
1f2e16aeea | ||
|
|
6f89745a58 | ||
|
|
9b710b0b6f | ||
|
|
992120b861 | ||
|
|
229bef5f82 | ||
|
|
c8a4890fc7 | ||
|
|
10a9848ac8 | ||
|
|
95189e37ba | ||
|
|
fe1f09f08d | ||
|
|
6adfa15742 | ||
|
|
edef264f17 | ||
|
|
9911b3fc9d | ||
|
|
1675aa7a59 | ||
|
|
7fd67fda13 | ||
|
|
7583f315ce | ||
|
|
7fd70d2088 | ||
|
|
337ee73844 | ||
|
|
43702a260b | ||
|
|
40208a9f76 | ||
|
|
b958358389 | ||
|
|
c23aacedad | ||
|
|
4f148469d7 | ||
|
|
247c43278c | ||
|
|
45f45e0511 | ||
|
|
c0c469ae9b | ||
|
|
4b86fd8d8a | ||
|
|
8fba47f449 | ||
|
|
f9ae964280 | ||
|
|
ca95615d7f | ||
|
|
747b255e34 | ||
|
|
c0da9ebd6c | ||
|
|
5c40b38a05 | ||
|
|
7bc03dcf06 | ||
|
|
f62e6168fd | ||
|
|
39c1470ea6 | ||
|
|
e48b99f1c1 | ||
|
|
71ae545907 | ||
|
|
b9a5eda187 | ||
|
|
231a2039c2 | ||
|
|
a415bc23fa | ||
|
|
79add35fc1 | ||
|
|
901763fc39 | ||
|
|
1034cb1ea2 | ||
|
|
020093fd67 | ||
|
|
f17b3fbd32 | ||
|
|
15b58e3a6b | ||
|
|
da6c98826b | ||
|
|
26415e2978 | ||
|
|
d849bd3b66 | ||
|
|
5996192455 | ||
|
|
a38b6b5522 | ||
|
|
1d6bc40bff | ||
|
|
437fa62cb1 | ||
|
|
6cde26d9ed | ||
|
|
f6dd5a68cd | ||
|
|
b1507dc576 | ||
|
|
a5d5b1554f | ||
|
|
ec8efebc94 | ||
|
|
f86cbf5527 | ||
|
|
57142d899c | ||
|
|
504a04279e | ||
|
|
3b4ef4a814 | ||
|
|
a7e3d2fc86 | ||
|
|
776ee66bae | ||
|
|
9bf2cd0691 | ||
|
|
fec74cf305 | ||
|
|
d3d7fbca61 | ||
|
|
550862a415 | ||
|
|
1763b530da | ||
|
|
8307150cae | ||
|
|
d2223c45d1 | ||
|
|
472ee5e295 | ||
|
|
1d0aebd836 | ||
|
|
8240626043 | ||
|
|
2cd14235c8 | ||
|
|
f774513554 | ||
|
|
388da9f003 | ||
|
|
fa18243491 | ||
|
|
5df1c1702b | ||
|
|
b6557802f4 | ||
|
|
15e3a511b6 | ||
|
|
08fbde0e8f | ||
|
|
28ebb4e8dd | ||
|
|
04fda57cbe | ||
|
|
b24495136c | ||
|
|
7d3fa55571 | ||
|
|
a76b5db8db | ||
|
|
667dd01ac7 | ||
|
|
559bee962a | ||
|
|
9f977d06e0 | ||
|
|
f67a916940 | ||
|
|
8a1308948f | ||
|
|
4e130d11a3 | ||
|
|
51aa5b268f | ||
|
|
6869133f53 | ||
|
|
ad3d88e47f | ||
|
|
6d29f8ba74 | ||
|
|
c7cb89e91d | ||
|
|
27f5118abd | ||
|
|
ac5a6e516d | ||
|
|
5d5bfbc1d1 | ||
|
|
9f4ba80000 | ||
|
|
c21c334b73 | ||
|
|
14093d9d43 | ||
|
|
176ca2f75d | ||
|
|
cfb71c7dad | ||
|
|
722eab83b1 | ||
|
|
d30edb2dae | ||
|
|
cba461ce8a | ||
|
|
f876505e25 | ||
|
|
5b9fcd5852 | ||
|
|
8a122cd9e1 | ||
|
|
557e68225c | ||
|
|
dc0947b87e | ||
|
|
af2c06cd28 | ||
|
|
67a4c6f797 | ||
|
|
0d1c857410 | ||
|
|
c7684dc94d | ||
|
|
a247572f64 | ||
|
|
306111303a | ||
|
|
c12b48537a | ||
|
|
0bf69713ab | ||
|
|
c05f1475ed | ||
|
|
416c262f02 | ||
|
|
40a5976f11 | ||
|
|
dc582fbec8 | ||
|
|
7047f0a449 | ||
|
|
d7a884a567 | ||
|
|
28de57792c | ||
|
|
a230cdbd21 | ||
|
|
251a261679 | ||
|
|
3baef254fd | ||
|
|
9e69e928b2 | ||
|
|
6eb595d2ca | ||
|
|
acf07a5144 | ||
|
|
7f53c58057 | ||
|
|
a069dc4855 | ||
|
|
febf544202 | ||
|
|
f2dd7b8815 | ||
|
|
d758e37fe3 | ||
|
|
9bc02a5623 | ||
|
|
eb0ec265d0 | ||
|
|
1f236dc02a | ||
|
|
86c279cbb2 | ||
|
|
4a7f1bed3a | ||
|
|
a730ae18c8 | ||
|
|
930bf439c0 | ||
|
|
f5f3133b87 | ||
|
|
cbca96de84 | ||
|
|
4d8ae3f124 | ||
|
|
653127431d | ||
|
|
22acef3765 | ||
|
|
0d5bf11f0d | ||
|
|
a824ccaff3 | ||
|
|
97ac965ba5 | ||
|
|
3e7f284df5 | ||
|
|
5c749fe26f | ||
|
|
aa48b46f48 | ||
|
|
6eaa4ae176 | ||
|
|
60c330b342 | ||
|
|
fdeb994121 | ||
|
|
65ed92fe7b | ||
|
|
9c8730786b | ||
|
|
7608823ea7 |
54
CHANGELOG
54
CHANGELOG
@@ -1,31 +1,63 @@
|
||||
jc changelog
|
||||
|
||||
20220705 v1.20.2
|
||||
- Add `gpg --with-colons` parser tested on linux
|
||||
- Add DER and PEM encoded X.509 Certificate parser
|
||||
- Add Bash and Zsh completion scripts to DEB and RPM packages
|
||||
|
||||
20220615 v1.20.1
|
||||
- Add `postconf -M` parser tested on linux
|
||||
- Update `asciitable` and `asciitable-m` parsers to preserve case in key
|
||||
names when using the `-r` or `raw=True` options.
|
||||
- Add long options (e.g. `--help`, `--about`, `--pretty`, etc.)
|
||||
- Add shell completions for Bash and Zsh
|
||||
- Fix `id` parser for cases where the user or group name is not present
|
||||
|
||||
20220531 v1.20.0
|
||||
- Add YAML output option with `-y`
|
||||
- Add `top -b` standard and streaming parsers tested on linux
|
||||
- Add `plugin_parser_count`, `standard_parser_count`, and `streaming_parser_count`
|
||||
keys to `jc -a` output
|
||||
- Add `is_compatible` function to the `utils` module
|
||||
- Fix `pip-show` parser for packages with a multi-line license field
|
||||
- Fix ASCII Table parser for cases where centered headers cause mis-aligned fields
|
||||
|
||||
20220513 v1.19.0
|
||||
- Add `chage --list` command parser tested on linux
|
||||
- Add `git log` command streaming parser
|
||||
- Fix `git log` standard parser for corner-cases where hash values are in messages
|
||||
- Fix `df` command parser for rare instances when a newline is found at the end
|
||||
- Allow jc to pip install on unsupported python version 3.6
|
||||
- Fix `asciitable-m` parser to skip some rows that contain column separator
|
||||
characters in cell data. A warning message will be printed to STDERR
|
||||
unless `-q` or `quiet=True` is used.
|
||||
|
||||
20220427 v1.18.8
|
||||
- Fix update-alternatives --query parser for cases where `slaves` are not present
|
||||
- Fix `update-alternatives --query` parser for cases where `slaves` are not present
|
||||
- Fix UnicodeEncodeError on some systems where LANG=C is set and unicode
|
||||
characters are in the output
|
||||
- Update history parser: do not drop non-ASCII characters if the system
|
||||
- Update `history` parser: do not drop non-ASCII characters if the system
|
||||
is configured for UTF-8 encoding
|
||||
- Enhance "magic syntax" to always use UTF-8 encoding
|
||||
|
||||
20220425 v1.18.7
|
||||
- Add git log command parser
|
||||
- Add update-alternatives --query parser
|
||||
- Add update-alternatives --get-selections parser
|
||||
- Add `git log` command parser
|
||||
- Add `update-alternatives --query` parser
|
||||
- Add `update-alternatives --get-selections` parser
|
||||
- Fix key/value and ini parsers to allow duplicate keys
|
||||
- Fix yaml file parser for files including timestamp objects
|
||||
- Update xrandr parser: add a 'rotation' field
|
||||
- Update `xrandr` parser: add a 'rotation' field
|
||||
- Fix failing tests by moving template files
|
||||
- Add python interpreter version and path to -v and -a output
|
||||
|
||||
20220325 v1.18.6
|
||||
- Add pidstat command parser tested on linux
|
||||
- Add pidstat command streaming parser tested on linux
|
||||
- Add mpstat command parser tested on linux
|
||||
- Add mpstat command streaming parser tested on linux
|
||||
- Add `pidstat` command parser tested on linux
|
||||
- Add `pidstat` command streaming parser tested on linux
|
||||
- Add `mpstat` command parser tested on linux
|
||||
- Add `mpstat` command streaming parser tested on linux
|
||||
- Add single-line ASCII and Unicode table parser
|
||||
- Add multi-line ASCII and Unicode table parser
|
||||
- Add documentation option to parser_info() and all_parser_info()
|
||||
- Add documentation option to `parser_info()` and `all_parser_info()`
|
||||
|
||||
20220305 v1.18.5
|
||||
- Fix date parser to ensure AM/PM period string is always uppercase
|
||||
|
||||
301
EXAMPLES.md
301
EXAMPLES.md
@@ -265,6 +265,21 @@ blkid -o udev -ip /dev/sda2 | jc --blkid -p # or: jc -p blkid -o udev
|
||||
}
|
||||
]
|
||||
```
|
||||
### chage --list
|
||||
```bash
|
||||
chage --list joeuser | jc --chage -p # or: jc -p chage --list joeuser
|
||||
```
|
||||
```json
|
||||
{
|
||||
"password_last_changed": "never",
|
||||
"password_expires": "never",
|
||||
"password_inactive": "never",
|
||||
"account_expires": "never",
|
||||
"min_days_between_password_change": 0,
|
||||
"max_days_between_password_change": 99999,
|
||||
"warning_days_before_password_expires": 7
|
||||
}
|
||||
```
|
||||
### cksum
|
||||
```bash
|
||||
cksum * | jc --cksum -p # or: jc -p cksum *
|
||||
@@ -1059,6 +1074,84 @@ cat /etc/fstab | jc --fstab -p
|
||||
}
|
||||
]
|
||||
```
|
||||
### git log
|
||||
```bash
|
||||
git log --stat | jc --git-log -p or: jc -p git log --stat
|
||||
```
|
||||
```json
|
||||
[
|
||||
{
|
||||
"commit": "728d882ed007b3c8b785018874a0eb06e1143b66",
|
||||
"author": "Kelly Brazil",
|
||||
"author_email": "kellyjonbrazil@gmail.com",
|
||||
"date": "Wed Apr 20 09:50:19 2022 -0400",
|
||||
"stats": {
|
||||
"files_changed": 2,
|
||||
"insertions": 90,
|
||||
"deletions": 12,
|
||||
"files": [
|
||||
"docs/parsers/git_log.md",
|
||||
"jc/parsers/git_log.py"
|
||||
]
|
||||
},
|
||||
"message": "add timestamp docs and examples",
|
||||
"epoch": 1650462619,
|
||||
"epoch_utc": null
|
||||
},
|
||||
{
|
||||
"commit": "b53e42aca623181aa9bc72194e6eeef1e9a3a237",
|
||||
"author": "Kelly Brazil",
|
||||
"author_email": "kellyjonbrazil@gmail.com",
|
||||
"date": "Wed Apr 20 09:44:42 2022 -0400",
|
||||
"stats": {
|
||||
"files_changed": 5,
|
||||
"insertions": 29,
|
||||
"deletions": 6,
|
||||
"files": [
|
||||
"docs/parsers/git_log.md",
|
||||
"docs/utils.md",
|
||||
"jc/parsers/git_log.py",
|
||||
"jc/utils.py",
|
||||
"man/jc.1"
|
||||
]
|
||||
},
|
||||
"message": "add calculated timestamp",
|
||||
"epoch": 1650462282,
|
||||
"epoch_utc": null
|
||||
}
|
||||
]
|
||||
```
|
||||
### gpg --with-colons
|
||||
```bash
|
||||
gpg --with-colons --show-keys file.gpg | jc --gpg -p # or jc -p gpg --with-colons --show-keys file.gpg
|
||||
```
|
||||
```json
|
||||
[
|
||||
{
|
||||
"type": "pub",
|
||||
"validity": "f",
|
||||
"key_length": "1024",
|
||||
"pub_key_alg": "17",
|
||||
"key_id": "6C7EE1B8621CC013",
|
||||
"creation_date": "899817715",
|
||||
"expiration_date": "1055898235",
|
||||
"certsn_uidhash_trustinfo": null,
|
||||
"owner_trust": "m",
|
||||
"user_id": null,
|
||||
"signature_class": null,
|
||||
"key_capabilities": "scESC",
|
||||
"cert_fingerprint_other": null,
|
||||
"flag": null,
|
||||
"token_sn": null,
|
||||
"hash_alg": null,
|
||||
"curve_name": null,
|
||||
"compliance_flags": null,
|
||||
"last_update_date": null,
|
||||
"origin": null,
|
||||
"comment": null
|
||||
}
|
||||
]
|
||||
```
|
||||
### /etc/group file
|
||||
```bash
|
||||
cat /etc/group | jc --group -p
|
||||
@@ -2666,6 +2759,36 @@ pip show wrapt wheel | jc --pip-show -p # or: jc -p pip show wrapt whe
|
||||
}
|
||||
]
|
||||
```
|
||||
### postconf -M
|
||||
```bash
|
||||
postconf -M | jc --postconf -p # or jc -p postconf -M
|
||||
```
|
||||
```json
|
||||
[
|
||||
{
|
||||
"service_name": "smtp",
|
||||
"service_type": "inet",
|
||||
"private": false,
|
||||
"unprivileged": null,
|
||||
"chroot": true,
|
||||
"wake_up_time": null,
|
||||
"process_limit": null,
|
||||
"command": "smtpd",
|
||||
"no_wake_up_before_first_use": null
|
||||
},
|
||||
{
|
||||
"service_name": "pickup",
|
||||
"service_type": "unix",
|
||||
"private": false,
|
||||
"unprivileged": null,
|
||||
"chroot": true,
|
||||
"wake_up_time": 60,
|
||||
"process_limit": 1,
|
||||
"command": "pickup",
|
||||
"no_wake_up_before_first_use": false
|
||||
}
|
||||
]
|
||||
```
|
||||
### ps
|
||||
```bash
|
||||
ps -ef | jc --ps -p # or: jc -p ps -ef
|
||||
@@ -3403,6 +3526,105 @@ timedatectl | jc --timedatectl -p # or: jc -p timedatectl
|
||||
"epoch_utc": 1583888001
|
||||
}
|
||||
```
|
||||
### tob -b
|
||||
```bash
|
||||
top -b -n 1 | jc --top -p # or jc -p tob -b -n 1
|
||||
```
|
||||
```json
|
||||
[
|
||||
{
|
||||
"time": "11:20:43",
|
||||
"uptime": 118,
|
||||
"users": 2,
|
||||
"load_1m": 0.0,
|
||||
"load_5m": 0.01,
|
||||
"load_15m": 0.05,
|
||||
"tasks_total": 108,
|
||||
"tasks_running": 2,
|
||||
"tasks_sleeping": 106,
|
||||
"tasks_stopped": 0,
|
||||
"tasks_zombie": 0,
|
||||
"cpu_user": 5.6,
|
||||
"cpu_sys": 11.1,
|
||||
"cpu_nice": 0.0,
|
||||
"cpu_idle": 83.3,
|
||||
"cpu_wait": 0.0,
|
||||
"cpu_hardware": 0.0,
|
||||
"cpu_software": 0.0,
|
||||
"cpu_steal": 0.0,
|
||||
"mem_total": 3.7,
|
||||
"mem_free": 3.3,
|
||||
"mem_used": 0.2,
|
||||
"mem_buff_cache": 0.2,
|
||||
"swap_total": 2.0,
|
||||
"swap_free": 2.0,
|
||||
"swap_used": 0.0,
|
||||
"mem_available": 3.3,
|
||||
"processes": [
|
||||
{
|
||||
"pid": 2225,
|
||||
"user": "kbrazil",
|
||||
"priority": 20,
|
||||
"nice": 0,
|
||||
"virtual_mem": 158.1,
|
||||
"resident_mem": 2.2,
|
||||
"shared_mem": 1.6,
|
||||
"status": "running",
|
||||
"percent_cpu": 12.5,
|
||||
"percent_mem": 0.1,
|
||||
"time_hundredths": "0:00.02",
|
||||
"command": "top",
|
||||
"parent_pid": 1884,
|
||||
"uid": 1000,
|
||||
"real_uid": 1000,
|
||||
"real_user": "kbrazil",
|
||||
"saved_uid": 1000,
|
||||
"saved_user": "kbrazil",
|
||||
"gid": 1000,
|
||||
"group": "kbrazil",
|
||||
"pgrp": 2225,
|
||||
"tty": "pts/0",
|
||||
"tty_process_gid": 2225,
|
||||
"session_id": 1884,
|
||||
"thread_count": 1,
|
||||
"last_used_processor": 0,
|
||||
"time": "0:00",
|
||||
"swap": 0.0,
|
||||
"code": 0.1,
|
||||
"data": 1.0,
|
||||
"major_page_fault_count": 0,
|
||||
"minor_page_fault_count": 736,
|
||||
"dirty_pages_count": 0,
|
||||
"sleeping_in_function": null,
|
||||
"flags": "..4.2...",
|
||||
"cgroups": "1:name=systemd:/user.slice/user-1000.+",
|
||||
"supplementary_gids": [
|
||||
10,
|
||||
1000
|
||||
],
|
||||
"supplementary_groups": [
|
||||
"wheel",
|
||||
"kbrazil"
|
||||
],
|
||||
"thread_gid": 2225,
|
||||
"environment_variables": [
|
||||
"XDG_SESSION_ID=2",
|
||||
"HOSTNAME=localhost"
|
||||
],
|
||||
"major_page_fault_count_delta": 0,
|
||||
"minor_page_fault_count_delta": 4,
|
||||
"used": 2.2,
|
||||
"ipc_namespace_inode": 4026531839,
|
||||
"mount_namespace_inode": 4026531840,
|
||||
"net_namespace_inode": 4026531956,
|
||||
"pid_namespace_inode": 4026531836,
|
||||
"user_namespace_inode": 4026531837,
|
||||
"nts_namespace_inode": 4026531838
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
```
|
||||
### tracepath
|
||||
```bash
|
||||
tracepath6 3ffe:2400:0:109::2 | jc --tracepath -p
|
||||
@@ -3900,6 +4122,85 @@ who -a | jc --who -p # or: jc -p who -a
|
||||
}
|
||||
]
|
||||
```
|
||||
### X.509 PEM and DER certificate files
|
||||
```bash
|
||||
cat entrust.pem | jc --x509-cert -p
|
||||
```
|
||||
```json
|
||||
[
|
||||
{
|
||||
"tbs_certificate": {
|
||||
"version": "v3",
|
||||
"serial_number": "a6:8b:79:29:00:00:00:00:50:d0:91:f9",
|
||||
"signature": {
|
||||
"algorithm": "sha384_ecdsa",
|
||||
"parameters": null
|
||||
},
|
||||
"issuer": {
|
||||
"country_name": "US",
|
||||
"organization_name": "Entrust, Inc.",
|
||||
"organizational_unit_name": [
|
||||
"See www.entrust.net/legal-terms",
|
||||
"(c) 2012 Entrust, Inc. - for authorized use only"
|
||||
],
|
||||
"common_name": "Entrust Root Certification Authority - EC1"
|
||||
},
|
||||
"validity": {
|
||||
"not_before": 1355844336,
|
||||
"not_after": 2144764536,
|
||||
"not_before_iso": "2012-12-18T15:25:36+00:00",
|
||||
"not_after_iso": "2037-12-18T15:55:36+00:00"
|
||||
},
|
||||
"subject": {
|
||||
"country_name": "US",
|
||||
"organization_name": "Entrust, Inc.",
|
||||
"organizational_unit_name": [
|
||||
"See www.entrust.net/legal-terms",
|
||||
"(c) 2012 Entrust, Inc. - for authorized use only"
|
||||
],
|
||||
"common_name": "Entrust Root Certification Authority - EC1"
|
||||
},
|
||||
"subject_public_key_info": {
|
||||
"algorithm": {
|
||||
"algorithm": "ec",
|
||||
"parameters": "secp384r1"
|
||||
},
|
||||
"public_key": "04:84:13:c9:d0:ba:6d:41:7b:e2:6c:d0:eb:55:5f:66:02:1a:24:f4:5b:89:69:47:e3:b8:c2:7d:f1:f2:02:c5:9f:a0:f6:5b:d5:8b:06:19:86:4f:53:10:6d:07:24:27:a1:a0:f8:d5:47:19:61:4c:7d:ca:93:27:ea:74:0c:ef:6f:96:09:fe:63:ec:70:5d:36:ad:67:77:ae:c9:9d:7c:55:44:3a:a2:63:51:1f:f5:e3:62:d4:a9:47:07:3e:cc:20"
|
||||
},
|
||||
"issuer_unique_id": null,
|
||||
"subject_unique_id": null,
|
||||
"extensions": [
|
||||
{
|
||||
"extn_id": "key_usage",
|
||||
"critical": true,
|
||||
"extn_value": [
|
||||
"key_cert_sign",
|
||||
"crl_sign"
|
||||
]
|
||||
},
|
||||
{
|
||||
"extn_id": "basic_constraints",
|
||||
"critical": true,
|
||||
"extn_value": {
|
||||
"ca": true,
|
||||
"path_len_constraint": null
|
||||
}
|
||||
},
|
||||
{
|
||||
"extn_id": "key_identifier",
|
||||
"critical": false,
|
||||
"extn_value": "b7:63:e7:1a:dd:8d:e9:08:a6:55:83:a4:e0:6a:50:41:65:11:42:49"
|
||||
}
|
||||
]
|
||||
},
|
||||
"signature_algorithm": {
|
||||
"algorithm": "sha384_ecdsa",
|
||||
"parameters": null
|
||||
},
|
||||
"signature_value": "30:64:02:30:61:79:d8:e5:42:47:df:1c:ae:53:99:17:b6:6f:1c:7d:e1:bf:11:94:d1:03:88:75:e4:8d:89:a4:8a:77:46:de:6d:61:ef:02:f5:fb:b5:df:cc:fe:4e:ff:fe:a9:e6:a7:02:30:5b:99:d7:85:37:06:b5:7b:08:fd:eb:27:8b:4a:94:f9:e1:fa:a7:8e:26:08:e8:7c:92:68:6d:73:d8:6f:26:ac:21:02:b8:99:b7:26:41:5b:25:60:ae:d0:48:1a:ee:06"
|
||||
}
|
||||
]
|
||||
```
|
||||
### XML files
|
||||
```bash
|
||||
cat cd_catalog.xml
|
||||
|
||||
290
README.md
290
README.md
@@ -44,8 +44,10 @@ $ jc dig example.com | jq -r '.[].answer[].data'
|
||||
93.184.216.34
|
||||
```
|
||||
|
||||
The `jc` parsers can also be used as python modules. In this case the output
|
||||
will be a python dictionary, or list of dictionaries, instead of JSON:
|
||||
`jc` can also be used as a python library. In this case the output will be
|
||||
a python dictionary, a list of dictionaries, or even a
|
||||
[lazy iterable of dictionaries](#using-streaming-parsers-as-python-modules)
|
||||
instead of JSON:
|
||||
```python
|
||||
>>> import subprocess
|
||||
>>> import jc
|
||||
@@ -111,13 +113,13 @@ pip3 install jc
|
||||
| Debian/Ubuntu linux | `apt-get install jc` |
|
||||
| Fedora linux | `dnf install jc` |
|
||||
| openSUSE linux | `zypper install jc` |
|
||||
| Archlinux User Repositories (AUR) | `paru -S jc` or `aura -A jc` or `yay -S jc` |
|
||||
| Archlinux Community Repository | `paru -S jc` or `aura -S jc` or `yay -S jc` |
|
||||
| NixOS linux | `nix-env -iA nixpkgs.jc` or `nix-env -iA nixos.jc` |
|
||||
| Guix System linux | `guix install jc` |
|
||||
| Gentoo Linux | `emerge dev-python/jc` |
|
||||
| macOS | `brew install jc` |
|
||||
| FreeBSD | `portsnap fetch update && cd /usr/ports/textproc/py-jc && make install clean` |
|
||||
| Ansible filter plugin | `ansible-galaxy collection install community.general` |
|
||||
| Gentoo Linux | `emerge dev-python/jc` |
|
||||
|
||||
> For more OS Packages, see https://repology.org/project/jc/versions.
|
||||
|
||||
@@ -144,121 +146,131 @@ option.
|
||||
|
||||
### Parsers
|
||||
|
||||
- `--acpi` enables the `acpi` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/acpi))
|
||||
- `--airport` enables the `airport -I` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/airport))
|
||||
- `--airport-s` enables the `airport -s` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/airport_s))
|
||||
- `--arp` enables the `arp` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/arp))
|
||||
- `--asciitable` enables the ASCII and Unicode table parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/asciitable))
|
||||
- `--asciitable-m` enables the multi-line ASCII and Unicode table parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/asciitable_m))
|
||||
- `--blkid` enables the `blkid` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/blkid))
|
||||
- `--cksum` enables the `cksum` and `sum` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/cksum))
|
||||
- `--crontab` enables the `crontab` command and file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab))
|
||||
- `--crontab-u` enables the `crontab` file parser with user support ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab_u))
|
||||
- `--csv` enables the CSV file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/csv))
|
||||
- `--csv-s` enables the CSV file streaming parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/csv_s))
|
||||
- `--date` enables the `date` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/date))
|
||||
- `--df` enables the `df` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/df))
|
||||
- `--dig` enables the `dig` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/dig))
|
||||
- `--dir` enables the `dir` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/dir))
|
||||
- `--dmidecode` enables the `dmidecode` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/dmidecode))
|
||||
- `--dpkg-l` enables the `dpkg -l` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/dpkg_l))
|
||||
- `--du` enables the `du` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/du))
|
||||
- `--env` enables the `env` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/env))
|
||||
- `--file` enables the `file` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/file))
|
||||
- `--finger` enables the `finger` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/finger))
|
||||
- `--free` enables the `free` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/free))
|
||||
- `--fstab` enables the `/etc/fstab` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/fstab))
|
||||
- `--git-log` enables the `git log` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/git_log))
|
||||
- `--group` enables the `/etc/group` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/group))
|
||||
- `--gshadow` enables the `/etc/gshadow` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/gshadow))
|
||||
- `--hash` enables the `hash` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/hash))
|
||||
- `--hashsum` enables the hashsum command parser (`md5sum`, `shasum`, etc.) ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/hashsum))
|
||||
- `--hciconfig` enables the `hciconfig` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/hciconfig))
|
||||
- `--history` enables the `history` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/history))
|
||||
- `--hosts` enables the `/etc/hosts` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/hosts))
|
||||
- `--id` enables the `id` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/id))
|
||||
- `--ifconfig` enables the `ifconfig` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ifconfig))
|
||||
- `--ini` enables the INI file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ini))
|
||||
- `--iostat` enables the `iostat` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/iostat))
|
||||
- `--iostat-s` enables the `iostat` command streaming parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/iostat_s))
|
||||
- `--iptables` enables the `iptables` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/iptables))
|
||||
- `--iw-scan` enables the `iw dev [device] scan` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/iw_scan))
|
||||
- `--jar-manifest` enables the MANIFEST.MF file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/jar_manifest))
|
||||
- `--jobs` enables the `jobs` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/jobs))
|
||||
- `--kv` enables the Key/Value file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/kv))
|
||||
- `--last` enables the `last` and `lastb` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/last))
|
||||
- `--ls` enables the `ls` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ls))
|
||||
- `--ls-s` enables the `ls` command streaming parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ls_s))
|
||||
- `--lsblk` enables the `lsblk` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/lsblk))
|
||||
- `--lsmod` enables the `lsmod` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/lsmod))
|
||||
- `--lsof` enables the `lsof` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/lsof))
|
||||
- `--lsusb` enables the `lsusb` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/lsusb))
|
||||
- `--mount` enables the `mount` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/mount))
|
||||
- `--mpstat` enables the `mpstat` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/mpstat))
|
||||
- `--mpstat-s` enables the `mpstat` command streaming parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/mpstat_s))
|
||||
- `--netstat` enables the `netstat` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/netstat))
|
||||
- `--nmcli` enables the `nmcli` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/nmcli))
|
||||
- `--ntpq` enables the `ntpq -p` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ntpq))
|
||||
- `--passwd` enables the `/etc/passwd` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/passwd))
|
||||
- `--pidstat` enables the `pidstat` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/pidstat))
|
||||
- `--pidstat-s` enables the `pidstat` command streaming parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/pidstat_s))
|
||||
- `--ping` enables the `ping` and `ping6` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ping))
|
||||
- `--ping-s` enables the `ping` and `ping6` command streaming parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ping_s))
|
||||
- `--pip-list` enables the `pip list` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_list))
|
||||
- `--pip-show` enables the `pip show` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_show))
|
||||
- `--ps` enables the `ps` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ps))
|
||||
- `--route` enables the `route` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/route))
|
||||
- `--rpm-qi` enables the `rpm -qi` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/rpm_qi))
|
||||
- `--rsync` enables the `rsync` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/rsync))
|
||||
- `--rsync-s` enables the `rsync` command streaming parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/rsync_s))
|
||||
- `--sfdisk` enables the `sfdisk` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/sfdisk))
|
||||
- `--shadow` enables the `/etc/shadow` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/shadow))
|
||||
- `--ss` enables the `ss` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ss))
|
||||
- `--stat` enables the `stat` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/stat))
|
||||
- `--stat-s` enables the `stat` command streaming parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/stat_s))
|
||||
- `--sysctl` enables the `sysctl` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/sysctl))
|
||||
- `--systemctl` enables the `systemctl` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl))
|
||||
- `--systemctl-lj` enables the `systemctl list-jobs` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_lj))
|
||||
- `--systemctl-ls` enables the `systemctl list-sockets` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_ls))
|
||||
- `--systemctl-luf` enables the `systemctl list-unit-files` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_luf))
|
||||
- `--systeminfo` enables the `systeminfo` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/systeminfo))
|
||||
- `--time` enables the `/usr/bin/time` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/time))
|
||||
- `--timedatectl` enables the `timedatectl status` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/timedatectl))
|
||||
- `--tracepath` enables the `tracepath` and `tracepath6` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/tracepath))
|
||||
- `--traceroute` enables the `traceroute` and `traceroute6` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/traceroute))
|
||||
- `--ufw` enables the `ufw status` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ufw))
|
||||
- `--ufw-appinfo` enables the `ufw app info [application]` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ufw_appinfo))
|
||||
- `--uname` enables the `uname -a` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/uname))
|
||||
- `--update-alt-gs` enables the `update-alternatives --get-selections` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/update_alt_gs))
|
||||
- `--update-alt-q` enables the `update-alternatives --query` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/update_alt_q))
|
||||
- `--upower` enables the `upower` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/upower))
|
||||
- `--uptime` enables the `uptime` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/uptime))
|
||||
- `--vmstat` enables the `vmstat` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/vmstat))
|
||||
- `--vmstat-s` enables the `vmstat` command streaming parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/vmstat_s))
|
||||
- `--w` enables the `w` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/w))
|
||||
- `--wc` enables the `wc` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/wc))
|
||||
- `--who` enables the `who` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/who))
|
||||
- `--xml` enables the XML file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/xml))
|
||||
- `--xrandr` enables the `xrandr` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/xrandr))
|
||||
- `--yaml` enables the YAML file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/yaml))
|
||||
- `--zipinfo` enables the `zipinfo` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/zipinfo))
|
||||
| Argument | Command or Filetype | Documentation |
|
||||
|-------------------|---------------------------------------------------------|----------------------------------------------------------------------------|
|
||||
| ` --acpi` | `acpi` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/acpi) |
|
||||
| ` --airport` | `airport -I` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/airport) |
|
||||
| ` --airport-s` | `airport -s` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/airport_s) |
|
||||
| ` --arp` | `arp` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/arp) |
|
||||
| ` --asciitable` | ASCII and Unicode table parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/asciitable) |
|
||||
| ` --asciitable-m` | multi-line ASCII and Unicode table parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/asciitable_m) |
|
||||
| ` --blkid` | `blkid` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/blkid) |
|
||||
| ` --chage` | `chage --list` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/chage) |
|
||||
| ` --cksum` | `cksum` and `sum` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/cksum) |
|
||||
| ` --crontab` | `crontab` command and file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab) |
|
||||
| ` --crontab-u` | `crontab` file parser with user support | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab_u) |
|
||||
| ` --csv` | CSV file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/csv) |
|
||||
| ` --csv-s` | CSV file streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/csv_s) |
|
||||
| ` --date` | `date` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/date) |
|
||||
| ` --df` | `df` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/df) |
|
||||
| ` --dig` | `dig` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/dig) |
|
||||
| ` --dir` | `dir` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/dir) |
|
||||
| ` --dmidecode` | `dmidecode` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/dmidecode) |
|
||||
| ` --dpkg-l` | `dpkg -l` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/dpkg_l) |
|
||||
| ` --du` | `du` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/du) |
|
||||
| ` --env` | `env` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/env) |
|
||||
| ` --file` | `file` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/file) |
|
||||
| ` --finger` | `finger` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/finger) |
|
||||
| ` --free` | `free` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/free) |
|
||||
| ` --fstab` | `/etc/fstab` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/fstab) |
|
||||
| ` --git-log` | `git log` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/git_log) |
|
||||
| ` --git-log-s` | `git log` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/git_log_s) |
|
||||
| ` --gpg` | `gpg --with-colons` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/gpg) |
|
||||
| ` --group` | `/etc/group` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/group) |
|
||||
| ` --gshadow` | `/etc/gshadow` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/gshadow) |
|
||||
| ` --hash` | `hash` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/hash) |
|
||||
| ` --hashsum` | hashsum command parser (`md5sum`, `shasum`, etc.) | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/hashsum) |
|
||||
| ` --hciconfig` | `hciconfig` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/hciconfig) |
|
||||
| ` --history` | `history` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/history) |
|
||||
| ` --hosts` | `/etc/hosts` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/hosts) |
|
||||
| ` --id` | `id` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/id) |
|
||||
| ` --ifconfig` | `ifconfig` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ifconfig) |
|
||||
| ` --ini` | INI file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ini) |
|
||||
| ` --iostat` | `iostat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/iostat) |
|
||||
| ` --iostat-s` | `iostat` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/iostat_s) |
|
||||
| ` --iptables` | `iptables` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/iptables) |
|
||||
| ` --iw-scan` | `iw dev [device] scan` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/iw_scan) |
|
||||
| ` --jar-manifest` | MANIFEST.MF file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/jar_manifest) |
|
||||
| ` --jobs` | `jobs` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/jobs) |
|
||||
| ` --kv` | Key/Value file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/kv) |
|
||||
| ` --last` | `last` and `lastb` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/last) |
|
||||
| ` --ls` | `ls` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ls) |
|
||||
| ` --ls-s` | `ls` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ls_s) |
|
||||
| ` --lsblk` | `lsblk` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lsblk) |
|
||||
| ` --lsmod` | `lsmod` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lsmod) |
|
||||
| ` --lsof` | `lsof` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lsof) |
|
||||
| ` --lsusb` | `lsusb` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/lsusb) |
|
||||
| ` --mount` | `mount` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/mount) |
|
||||
| ` --mpstat` | `mpstat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/mpstat) |
|
||||
| ` --mpstat-s` | `mpstat` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/mpstat_s) |
|
||||
| ` --netstat` | `netstat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/netstat) |
|
||||
| ` --nmcli` | `nmcli` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/nmcli) |
|
||||
| ` --ntpq` | `ntpq -p` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ntpq) |
|
||||
| ` --passwd` | `/etc/passwd` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/passwd) |
|
||||
| ` --pidstat` | `pidstat -h` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pidstat) |
|
||||
| ` --pidstat-s` | `pidstat -h` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pidstat_s) |
|
||||
| ` --ping` | `ping` and `ping6` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ping) |
|
||||
| ` --ping-s` | `ping` and `ping6` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ping_s) |
|
||||
| ` --pip-list` | `pip list` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_list) |
|
||||
| ` --pip-show` | `pip show` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_show) |
|
||||
| ` --postconf` | `postconf -M` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/postconf) |
|
||||
| ` --ps` | `ps` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ps) |
|
||||
| ` --route` | `route` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/route) |
|
||||
| ` --rpm-qi` | `rpm -qi` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/rpm_qi) |
|
||||
| ` --rsync` | `rsync` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/rsync) |
|
||||
| ` --rsync-s` | `rsync` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/rsync_s) |
|
||||
| ` --sfdisk` | `sfdisk` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/sfdisk) |
|
||||
| ` --shadow` | `/etc/shadow` file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/shadow) |
|
||||
| ` --ss` | `ss` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ss) |
|
||||
| ` --stat` | `stat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/stat) |
|
||||
| ` --stat-s` | `stat` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/stat_s) |
|
||||
| ` --sysctl` | `sysctl` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/sysctl) |
|
||||
| ` --systemctl` | `systemctl` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl) |
|
||||
| ` --systemctl-lj` | `systemctl list-jobs` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_lj) |
|
||||
| ` --systemctl-ls` | `systemctl list-sockets` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_ls) |
|
||||
| `--systemctl-luf` | `systemctl list-unit-files` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_luf) |
|
||||
| ` --systeminfo` | `systeminfo` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/systeminfo) |
|
||||
| ` --time` | `/usr/bin/time` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/time) |
|
||||
| ` --timedatectl` | `timedatectl status` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/timedatectl) |
|
||||
| ` --top` | `top -b` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/top) |
|
||||
| ` --top-s` | `top -b` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/top_s) |
|
||||
| ` --tracepath` | `tracepath` and `tracepath6` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/tracepath) |
|
||||
| ` --traceroute` | `traceroute` and `traceroute6` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/traceroute) |
|
||||
| ` --ufw` | `ufw status` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ufw) |
|
||||
| ` --ufw-appinfo` | `ufw app info [application]` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/ufw_appinfo) |
|
||||
| ` --uname` | `uname -a` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/uname) |
|
||||
| `--update-alt-gs` | `update-alternatives --get-selections` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/update_alt_gs) |
|
||||
| ` --update-alt-q` | `update-alternatives --query` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/update_alt_q) |
|
||||
| ` --upower` | `upower` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/upower) |
|
||||
| ` --uptime` | `uptime` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/uptime) |
|
||||
| ` --vmstat` | `vmstat` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/vmstat) |
|
||||
| ` --vmstat-s` | `vmstat` command streaming parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/vmstat_s) |
|
||||
| ` --w` | `w` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/w) |
|
||||
| ` --wc` | `wc` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/wc) |
|
||||
| ` --who` | `who` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/who) |
|
||||
| ` --x509-cert` | X.509 PEM and DER certificate file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/x509_cert) |
|
||||
| ` --xml` | XML file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/xml) |
|
||||
| ` --xrandr` | `xrandr` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/xrandr) |
|
||||
| ` --yaml` | YAML file parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/yaml) |
|
||||
| ` --zipinfo` | `zipinfo` command parser | [details](https://kellyjonbrazil.github.io/jc/docs/parsers/zipinfo) |
|
||||
|
||||
### Options
|
||||
- `-a` about `jc`. Prints information about `jc` and the parsers (in JSON, of
|
||||
course!)
|
||||
- `-C` force color output even when using pipes (overrides `-m` and the
|
||||
`NO_COLOR` env variable)
|
||||
- `-d` debug mode. Prints trace messages if parsing issues are encountered (use
|
||||
`-dd` for verbose debugging)
|
||||
- `-h` help. Use `jc -h --parser_name` for parser documentation
|
||||
- `-m` monochrome JSON output
|
||||
- `-p` pretty format the JSON output
|
||||
- `-q` quiet mode. Suppresses parser warning messages (use `-qq` to ignore
|
||||
streaming parser errors)
|
||||
- `-r` raw output. Provides a more literal JSON output, typically with string
|
||||
values and no additional semantic processing
|
||||
- `-u` unbuffer output
|
||||
- `-v` version information
|
||||
|
||||
| Short | Long | Description |
|
||||
|-------|-----------------|---------------------------------------------------------------------------------------------------------------------|
|
||||
| `-a` | `--about` | About `jc`. Prints information about `jc` and the parsers (in JSON or YAML, of course!) |
|
||||
| `-C` | `--force-color` | Force color output even when using pipes (overrides `-m` and the `NO_COLOR` env variable) |
|
||||
| `-d` | `--debug` | Debug mode. Prints trace messages if parsing issues are encountered (use`-dd` for verbose debugging) |
|
||||
| `-h` | `--help` | Help. Use `jc -h --parser_name` for parser documentation |
|
||||
| `-m` | `--monochrome` | Monochrome output |
|
||||
| `-p` | `--pretty` | Pretty format the JSON output |
|
||||
| `-q` | `--quiet` | Quiet mode. Suppresses parser warning messages (use `-qq` to ignore streaming parser errors) |
|
||||
| `-r` | `--raw` | Raw output. Provides more literal output, typically with string values and no additional semantic processing |
|
||||
| `-u` | `--unbuffer` | Unbuffer output |
|
||||
| `-v` | `--version` | Version information |
|
||||
| `-y` | `--yaml-out` | YAML output |
|
||||
| `-B` | `--bash-comp` | Generate Bash shell completion script ([more info](https://github.com/kellyjonbrazil/jc/wiki/Shell-Completions)) |
|
||||
| `-Z` | `--zsh-comp` | Generate Zsh shell completion script ([more info](https://github.com/kellyjonbrazil/jc/wiki/Shell-Completions)) |
|
||||
|
||||
### Exit Codes
|
||||
Any fatal errors within `jc` will generate an exit code of `100`, otherwise the
|
||||
@@ -305,11 +317,11 @@ color output will override both the `NO_COLOR` environment variable and the `-m`
|
||||
option.
|
||||
|
||||
### Streaming Parsers
|
||||
Most parsers load all of the data from STDIN, parse it, then output the entire
|
||||
Most parsers load all of the data from `STDIN`, parse it, then output the entire
|
||||
JSON document serially. There are some streaming parsers (e.g. `ls-s` and
|
||||
`ping-s`) that immediately start processing and outputing the data line-by-line
|
||||
as [JSON Lines](https://jsonlines.org/) (aka [NDJSON](http://ndjson.org/)) while
|
||||
it is being received from STDIN. This can significantly reduce the amount of
|
||||
it is being received from `STDIN`. This can significantly reduce the amount of
|
||||
memory required to parse large amounts of command output (e.g. `ls -lR /`) and
|
||||
can sometimes process the data more quickly. Streaming parsers have slightly
|
||||
different behavior than standard parsers as outlined below.
|
||||
@@ -374,14 +386,14 @@ $ ping 1.1.1.1 | jc --ping-s -u | jq
|
||||
|
||||
#### Using Streaming Parsers as Python Modules
|
||||
|
||||
Streaming parsers accept any iterable object and return an iterator object
|
||||
(generator) allowing lazy processing of the data. The input data should
|
||||
iterate on lines of string data. Examples of good input data are `sys.stdin` or
|
||||
Streaming parsers accept any iterable object and return an iterable object
|
||||
allowing lazy processing of the data. The input data should iterate on lines
|
||||
of string data. Examples of good input data are `sys.stdin` or
|
||||
`str.splitlines()`.
|
||||
|
||||
To use the generator object in your code, simply loop through it or use the
|
||||
[next()](https://docs.python.org/3/library/functions.html#next) builtin
|
||||
function:
|
||||
To use the returned iterable object in your code, simply loop through it or
|
||||
use the [next()](https://docs.python.org/3/library/functions.html#next)
|
||||
builtin function:
|
||||
```python
|
||||
import jc
|
||||
|
||||
@@ -404,8 +416,8 @@ or [`jc/parsers/foo_s.py (streaming)`](https://github.com/kellyjonbrazil/jc/blob
|
||||
parser as a template and simply place a `.py` file in the `jcparsers` subfolder.
|
||||
|
||||
Local plugin filenames must be valid python module names and therefore must
|
||||
start with a letter and consist entirely of alphanumerics. Local plugins
|
||||
may override default parsers.
|
||||
start with a letter and consist entirely of alphanumerics and underscores.
|
||||
Local plugins may override default parsers.
|
||||
|
||||
> Note: The application data directory follows the
|
||||
[XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html)
|
||||
@@ -438,15 +450,30 @@ If a UTC timezone can be detected in the text of the command output, the
|
||||
timestamp will be timezone aware and have a `_utc` suffix on the key name.
|
||||
(e.g. `epoch_utc`) No other timezones are supported for aware timestamps.
|
||||
|
||||
## Use In Other Shells
|
||||
`jc` can be used in most any shell. Some modern shells have JSON deserialization
|
||||
and filtering capabilities built-in which makes using `jc` even more convenient.
|
||||
|
||||
For example, the following is possible in [NGS](https://ngs-lang.org/)
|
||||
(Next Generation Shell):
|
||||
```bash
|
||||
myvar = ``jc dig www.google.com``[0].answer[0].data
|
||||
```
|
||||
This runs `jc`, parses the output JSON, and assigs the resulting data structure
|
||||
to a variable in a single line of code.
|
||||
|
||||
For more examples of how to use `jc` in other shells, see this
|
||||
[wiki page](https://github.com/kellyjonbrazil/jc/wiki/Using-jc-With-Different-Shells).
|
||||
|
||||
## Compatibility
|
||||
Some parsers like `dig`, `xml`, `csv`, etc. will work on any platform. Other
|
||||
parsers that convert platform-specific output will generate a warning message if
|
||||
they are run on an unsupported platform. To see all parser information,
|
||||
including compatibility, run `jc -ap`.
|
||||
|
||||
You may still use a parser on an unsupported platform - for example, you may want
|
||||
to parse a file with linux `lsof` output on an macOS or Windows laptop. In that
|
||||
case you can suppress the warning message with the `-q` cli option or the
|
||||
You may still use a parser on an unsupported platform - for example, you may
|
||||
want to parse a file with linux `lsof` output on an macOS or Windows laptop. In
|
||||
that case you can suppress the warning message with the `-q` cli option or the
|
||||
`quiet=True` function parameter in `parse()`:
|
||||
|
||||
macOS:
|
||||
@@ -475,7 +502,8 @@ Tested on:
|
||||
## Contributions
|
||||
Feel free to add/improve code or parsers! You can use the
|
||||
[`jc/parsers/foo.py`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py)
|
||||
or [`jc/parsers/foo_s.py (streaming)`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo_s.py) parsers as a template and submit your parser with a pull request.
|
||||
or [`jc/parsers/foo_s.py (streaming)`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo_s.py) parsers as a template and submit your parser with a pull
|
||||
request.
|
||||
|
||||
Please see the [Contributing Guidelines](https://github.com/kellyjonbrazil/jc/blob/master/CONTRIBUTING.md) for more information.
|
||||
|
||||
|
||||
@@ -1 +1,2 @@
|
||||
theme: jekyll-theme-cayman
|
||||
theme: jekyll-theme-cayman
|
||||
markdown: GFM
|
||||
|
||||
9
build-completions.py
Executable file
9
build-completions.py
Executable file
@@ -0,0 +1,9 @@
|
||||
#!/usr/bin/env python3
|
||||
# build Bash and Zsh completion scripts and add to the completions folder
|
||||
from jc.shell_completions import bash_completion, zsh_completion
|
||||
|
||||
with open('completions/jc_bash_completion.sh', 'w') as f:
|
||||
print(bash_completion(), file=f)
|
||||
|
||||
with open('completions/jc_zsh_completion.sh', 'w') as f:
|
||||
print(zsh_completion(), file=f)
|
||||
84
completions/jc_bash_completion.sh
Normal file
84
completions/jc_bash_completion.sh
Normal file
@@ -0,0 +1,84 @@
|
||||
_jc()
|
||||
{
|
||||
local cur prev words cword jc_commands jc_parsers jc_options \
|
||||
jc_about_options jc_about_mod_options jc_help_options jc_special_options
|
||||
|
||||
jc_commands=(acpi airport arp blkid chage cksum crontab date df dig dmidecode dpkg du env file finger free git gpg hciconfig id ifconfig iostat iptables iw jobs last lastb ls lsblk lsmod lsof lsusb md5 md5sum mount mpstat netstat nmcli ntpq pidstat ping ping6 pip pip3 postconf printenv ps route rpm rsync sfdisk sha1sum sha224sum sha256sum sha384sum sha512sum shasum ss stat sum sysctl systemctl systeminfo timedatectl top tracepath tracepath6 traceroute traceroute6 ufw uname update-alternatives upower uptime vdir vmstat w wc who xrandr zipinfo)
|
||||
jc_parsers=(--acpi --airport --airport-s --arp --asciitable --asciitable-m --blkid --chage --cksum --crontab --crontab-u --csv --csv-s --date --df --dig --dir --dmidecode --dpkg-l --du --env --file --finger --free --fstab --git-log --git-log-s --gpg --group --gshadow --hash --hashsum --hciconfig --history --hosts --id --ifconfig --ini --iostat --iostat-s --iptables --iw-scan --jar-manifest --jobs --kv --last --ls --ls-s --lsblk --lsmod --lsof --lsusb --mount --mpstat --mpstat-s --netstat --nmcli --ntpq --passwd --pidstat --pidstat-s --ping --ping-s --pip-list --pip-show --postconf --ps --route --rpm-qi --rsync --rsync-s --sfdisk --shadow --ss --stat --stat-s --sysctl --systemctl --systemctl-lj --systemctl-ls --systemctl-luf --systeminfo --time --timedatectl --top --top-s --tracepath --traceroute --ufw --ufw-appinfo --uname --update-alt-gs --update-alt-q --upower --uptime --vmstat --vmstat-s --w --wc --who --x509-cert --xml --xrandr --yaml --zipinfo)
|
||||
jc_options=(--force-color -C --debug -d --monochrome -m --pretty -p --quiet -q --raw -r --unbuffer -u --yaml-out -y)
|
||||
jc_about_options=(--about -a)
|
||||
jc_about_mod_options=(--pretty -p --yaml-out -y --monochrome -m --force-color -C)
|
||||
jc_help_options=(--help -h)
|
||||
jc_special_options=(--version -v --bash-comp -B --zsh-comp -Z)
|
||||
|
||||
COMPREPLY=()
|
||||
_get_comp_words_by_ref cur prev words cword
|
||||
|
||||
# if jc_about_options are found anywhere in the line, then only complete from jc_about_mod_options
|
||||
for i in "${words[@]::${#words[@]}-1}"; do
|
||||
if [[ " ${jc_about_options[*]} " =~ " ${i} " ]]; then
|
||||
COMPREPLY=( $( compgen -W "${jc_about_mod_options[*]}" \
|
||||
-- "${cur}" ) )
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
|
||||
# if jc_help_options and a parser are found anywhere in the line, then no more completions
|
||||
if
|
||||
(
|
||||
for i in "${words[@]::${#words[@]}-1}"; do
|
||||
if [[ " ${jc_help_options[*]} " =~ " ${i} " ]]; then
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
) && (
|
||||
for i in "${words[@]::${#words[@]}-1}"; do
|
||||
if [[ " ${jc_parsers[*]} " =~ " ${i} " ]]; then
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
); then
|
||||
return 0
|
||||
fi
|
||||
|
||||
# if jc_help_options are found anywhere in the line, then only complete with parsers
|
||||
for i in "${words[@]::${#words[@]}-1}"; do
|
||||
if [[ " ${jc_help_options[*]} " =~ " ${i} " ]]; then
|
||||
COMPREPLY=( $( compgen -W "${jc_parsers[*]}" \
|
||||
-- "${cur}" ) )
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
|
||||
# if special options are found anywhere in the line, then no more completions
|
||||
for i in "${words[@]::${#words[@]}-1}"; do
|
||||
if [[ " ${jc_special_options[*]} " =~ " ${i} " ]]; then
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
|
||||
# if magic command is found anywhere in the line, use called command's autocompletion
|
||||
for i in "${words[@]::${#words[@]}-1}"; do
|
||||
if [[ " ${jc_commands[*]} " =~ " ${i} " ]]; then
|
||||
_command
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
|
||||
# if a parser arg is found anywhere in the line, only show options and help options
|
||||
for i in "${words[@]::${#words[@]}-1}"; do
|
||||
if [[ " ${jc_parsers[*]} " =~ " ${i} " ]]; then
|
||||
COMPREPLY=( $( compgen -W "${jc_options[*]} ${jc_help_options[*]}" \
|
||||
-- "${cur}" ) )
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
|
||||
# default completion
|
||||
COMPREPLY=( $( compgen -W "${jc_options[*]} ${jc_about_options[*]} ${jc_help_options[*]} ${jc_special_options[*]} ${jc_parsers[*]} ${jc_commands[*]}" \
|
||||
-- "${cur}" ) )
|
||||
} &&
|
||||
complete -F _jc jc
|
||||
|
||||
325
completions/jc_zsh_completion.sh
Normal file
325
completions/jc_zsh_completion.sh
Normal file
@@ -0,0 +1,325 @@
|
||||
#compdef jc
|
||||
|
||||
_jc() {
|
||||
local -a jc_commands jc_commands_describe \
|
||||
jc_parsers jc_parsers_describe \
|
||||
jc_options jc_options_describe \
|
||||
jc_about_options jc_about_options_describe \
|
||||
jc_about_mod_options jc_about_mod_options_describe \
|
||||
jc_help_options jc_help_options_describe \
|
||||
jc_special_options jc_special_options_describe
|
||||
|
||||
jc_commands=(acpi airport arp blkid chage cksum crontab date df dig dmidecode dpkg du env file finger free git gpg hciconfig id ifconfig iostat iptables iw jobs last lastb ls lsblk lsmod lsof lsusb md5 md5sum mount mpstat netstat nmcli ntpq pidstat ping ping6 pip pip3 postconf printenv ps route rpm rsync sfdisk sha1sum sha224sum sha256sum sha384sum sha512sum shasum ss stat sum sysctl systemctl systeminfo timedatectl top tracepath tracepath6 traceroute traceroute6 ufw uname update-alternatives upower uptime vdir vmstat w wc who xrandr zipinfo)
|
||||
jc_commands_describe=(
|
||||
'acpi:run "acpi" command with magic syntax.'
|
||||
'airport:run "airport" command with magic syntax.'
|
||||
'arp:run "arp" command with magic syntax.'
|
||||
'blkid:run "blkid" command with magic syntax.'
|
||||
'chage:run "chage" command with magic syntax.'
|
||||
'cksum:run "cksum" command with magic syntax.'
|
||||
'crontab:run "crontab" command with magic syntax.'
|
||||
'date:run "date" command with magic syntax.'
|
||||
'df:run "df" command with magic syntax.'
|
||||
'dig:run "dig" command with magic syntax.'
|
||||
'dmidecode:run "dmidecode" command with magic syntax.'
|
||||
'dpkg:run "dpkg" command with magic syntax.'
|
||||
'du:run "du" command with magic syntax.'
|
||||
'env:run "env" command with magic syntax.'
|
||||
'file:run "file" command with magic syntax.'
|
||||
'finger:run "finger" command with magic syntax.'
|
||||
'free:run "free" command with magic syntax.'
|
||||
'git:run "git" command with magic syntax.'
|
||||
'gpg:run "gpg" command with magic syntax.'
|
||||
'hciconfig:run "hciconfig" command with magic syntax.'
|
||||
'id:run "id" command with magic syntax.'
|
||||
'ifconfig:run "ifconfig" command with magic syntax.'
|
||||
'iostat:run "iostat" command with magic syntax.'
|
||||
'iptables:run "iptables" command with magic syntax.'
|
||||
'iw:run "iw" command with magic syntax.'
|
||||
'jobs:run "jobs" command with magic syntax.'
|
||||
'last:run "last" command with magic syntax.'
|
||||
'lastb:run "lastb" command with magic syntax.'
|
||||
'ls:run "ls" command with magic syntax.'
|
||||
'lsblk:run "lsblk" command with magic syntax.'
|
||||
'lsmod:run "lsmod" command with magic syntax.'
|
||||
'lsof:run "lsof" command with magic syntax.'
|
||||
'lsusb:run "lsusb" command with magic syntax.'
|
||||
'md5:run "md5" command with magic syntax.'
|
||||
'md5sum:run "md5sum" command with magic syntax.'
|
||||
'mount:run "mount" command with magic syntax.'
|
||||
'mpstat:run "mpstat" command with magic syntax.'
|
||||
'netstat:run "netstat" command with magic syntax.'
|
||||
'nmcli:run "nmcli" command with magic syntax.'
|
||||
'ntpq:run "ntpq" command with magic syntax.'
|
||||
'pidstat:run "pidstat" command with magic syntax.'
|
||||
'ping:run "ping" command with magic syntax.'
|
||||
'ping6:run "ping6" command with magic syntax.'
|
||||
'pip:run "pip" command with magic syntax.'
|
||||
'pip3:run "pip3" command with magic syntax.'
|
||||
'postconf:run "postconf" command with magic syntax.'
|
||||
'printenv:run "printenv" command with magic syntax.'
|
||||
'ps:run "ps" command with magic syntax.'
|
||||
'route:run "route" command with magic syntax.'
|
||||
'rpm:run "rpm" command with magic syntax.'
|
||||
'rsync:run "rsync" command with magic syntax.'
|
||||
'sfdisk:run "sfdisk" command with magic syntax.'
|
||||
'sha1sum:run "sha1sum" command with magic syntax.'
|
||||
'sha224sum:run "sha224sum" command with magic syntax.'
|
||||
'sha256sum:run "sha256sum" command with magic syntax.'
|
||||
'sha384sum:run "sha384sum" command with magic syntax.'
|
||||
'sha512sum:run "sha512sum" command with magic syntax.'
|
||||
'shasum:run "shasum" command with magic syntax.'
|
||||
'ss:run "ss" command with magic syntax.'
|
||||
'stat:run "stat" command with magic syntax.'
|
||||
'sum:run "sum" command with magic syntax.'
|
||||
'sysctl:run "sysctl" command with magic syntax.'
|
||||
'systemctl:run "systemctl" command with magic syntax.'
|
||||
'systeminfo:run "systeminfo" command with magic syntax.'
|
||||
'timedatectl:run "timedatectl" command with magic syntax.'
|
||||
'top:run "top" command with magic syntax.'
|
||||
'tracepath:run "tracepath" command with magic syntax.'
|
||||
'tracepath6:run "tracepath6" command with magic syntax.'
|
||||
'traceroute:run "traceroute" command with magic syntax.'
|
||||
'traceroute6:run "traceroute6" command with magic syntax.'
|
||||
'ufw:run "ufw" command with magic syntax.'
|
||||
'uname:run "uname" command with magic syntax.'
|
||||
'update-alternatives:run "update-alternatives" command with magic syntax.'
|
||||
'upower:run "upower" command with magic syntax.'
|
||||
'uptime:run "uptime" command with magic syntax.'
|
||||
'vdir:run "vdir" command with magic syntax.'
|
||||
'vmstat:run "vmstat" command with magic syntax.'
|
||||
'w:run "w" command with magic syntax.'
|
||||
'wc:run "wc" command with magic syntax.'
|
||||
'who:run "who" command with magic syntax.'
|
||||
'xrandr:run "xrandr" command with magic syntax.'
|
||||
'zipinfo:run "zipinfo" command with magic syntax.'
|
||||
)
|
||||
jc_parsers=(--acpi --airport --airport-s --arp --asciitable --asciitable-m --blkid --chage --cksum --crontab --crontab-u --csv --csv-s --date --df --dig --dir --dmidecode --dpkg-l --du --env --file --finger --free --fstab --git-log --git-log-s --gpg --group --gshadow --hash --hashsum --hciconfig --history --hosts --id --ifconfig --ini --iostat --iostat-s --iptables --iw-scan --jar-manifest --jobs --kv --last --ls --ls-s --lsblk --lsmod --lsof --lsusb --mount --mpstat --mpstat-s --netstat --nmcli --ntpq --passwd --pidstat --pidstat-s --ping --ping-s --pip-list --pip-show --postconf --ps --route --rpm-qi --rsync --rsync-s --sfdisk --shadow --ss --stat --stat-s --sysctl --systemctl --systemctl-lj --systemctl-ls --systemctl-luf --systeminfo --time --timedatectl --top --top-s --tracepath --traceroute --ufw --ufw-appinfo --uname --update-alt-gs --update-alt-q --upower --uptime --vmstat --vmstat-s --w --wc --who --x509-cert --xml --xrandr --yaml --zipinfo)
|
||||
jc_parsers_describe=(
|
||||
'--acpi:`acpi` command parser'
|
||||
'--airport:`airport -I` command parser'
|
||||
'--airport-s:`airport -s` command parser'
|
||||
'--arp:`arp` command parser'
|
||||
'--asciitable:ASCII and Unicode table parser'
|
||||
'--asciitable-m:multi-line ASCII and Unicode table parser'
|
||||
'--blkid:`blkid` command parser'
|
||||
'--chage:`chage --list` command parser'
|
||||
'--cksum:`cksum` and `sum` command parser'
|
||||
'--crontab:`crontab` command and file parser'
|
||||
'--crontab-u:`crontab` file parser with user support'
|
||||
'--csv:CSV file parser'
|
||||
'--csv-s:CSV file streaming parser'
|
||||
'--date:`date` command parser'
|
||||
'--df:`df` command parser'
|
||||
'--dig:`dig` command parser'
|
||||
'--dir:`dir` command parser'
|
||||
'--dmidecode:`dmidecode` command parser'
|
||||
'--dpkg-l:`dpkg -l` command parser'
|
||||
'--du:`du` command parser'
|
||||
'--env:`env` command parser'
|
||||
'--file:`file` command parser'
|
||||
'--finger:`finger` command parser'
|
||||
'--free:`free` command parser'
|
||||
'--fstab:`/etc/fstab` file parser'
|
||||
'--git-log:`git log` command parser'
|
||||
'--git-log-s:`git log` command streaming parser'
|
||||
'--gpg:`gpg --with-colons` command parser'
|
||||
'--group:`/etc/group` file parser'
|
||||
'--gshadow:`/etc/gshadow` file parser'
|
||||
'--hash:`hash` command parser'
|
||||
'--hashsum:hashsum command parser (`md5sum`, `shasum`, etc.)'
|
||||
'--hciconfig:`hciconfig` command parser'
|
||||
'--history:`history` command parser'
|
||||
'--hosts:`/etc/hosts` file parser'
|
||||
'--id:`id` command parser'
|
||||
'--ifconfig:`ifconfig` command parser'
|
||||
'--ini:INI file parser'
|
||||
'--iostat:`iostat` command parser'
|
||||
'--iostat-s:`iostat` command streaming parser'
|
||||
'--iptables:`iptables` command parser'
|
||||
'--iw-scan:`iw dev [device] scan` command parser'
|
||||
'--jar-manifest:MANIFEST.MF file parser'
|
||||
'--jobs:`jobs` command parser'
|
||||
'--kv:Key/Value file parser'
|
||||
'--last:`last` and `lastb` command parser'
|
||||
'--ls:`ls` command parser'
|
||||
'--ls-s:`ls` command streaming parser'
|
||||
'--lsblk:`lsblk` command parser'
|
||||
'--lsmod:`lsmod` command parser'
|
||||
'--lsof:`lsof` command parser'
|
||||
'--lsusb:`lsusb` command parser'
|
||||
'--mount:`mount` command parser'
|
||||
'--mpstat:`mpstat` command parser'
|
||||
'--mpstat-s:`mpstat` command streaming parser'
|
||||
'--netstat:`netstat` command parser'
|
||||
'--nmcli:`nmcli` command parser'
|
||||
'--ntpq:`ntpq -p` command parser'
|
||||
'--passwd:`/etc/passwd` file parser'
|
||||
'--pidstat:`pidstat -h` command parser'
|
||||
'--pidstat-s:`pidstat -h` command streaming parser'
|
||||
'--ping:`ping` and `ping6` command parser'
|
||||
'--ping-s:`ping` and `ping6` command streaming parser'
|
||||
'--pip-list:`pip list` command parser'
|
||||
'--pip-show:`pip show` command parser'
|
||||
'--postconf:`postconf -M` command parser'
|
||||
'--ps:`ps` command parser'
|
||||
'--route:`route` command parser'
|
||||
'--rpm-qi:`rpm -qi` command parser'
|
||||
'--rsync:`rsync` command parser'
|
||||
'--rsync-s:`rsync` command streaming parser'
|
||||
'--sfdisk:`sfdisk` command parser'
|
||||
'--shadow:`/etc/shadow` file parser'
|
||||
'--ss:`ss` command parser'
|
||||
'--stat:`stat` command parser'
|
||||
'--stat-s:`stat` command streaming parser'
|
||||
'--sysctl:`sysctl` command parser'
|
||||
'--systemctl:`systemctl` command parser'
|
||||
'--systemctl-lj:`systemctl list-jobs` command parser'
|
||||
'--systemctl-ls:`systemctl list-sockets` command parser'
|
||||
'--systemctl-luf:`systemctl list-unit-files` command parser'
|
||||
'--systeminfo:`systeminfo` command parser'
|
||||
'--time:`/usr/bin/time` command parser'
|
||||
'--timedatectl:`timedatectl status` command parser'
|
||||
'--top:`top -b` command parser'
|
||||
'--top-s:`top -b` command streaming parser'
|
||||
'--tracepath:`tracepath` and `tracepath6` command parser'
|
||||
'--traceroute:`traceroute` and `traceroute6` command parser'
|
||||
'--ufw:`ufw status` command parser'
|
||||
'--ufw-appinfo:`ufw app info [application]` command parser'
|
||||
'--uname:`uname -a` command parser'
|
||||
'--update-alt-gs:`update-alternatives --get-selections` command parser'
|
||||
'--update-alt-q:`update-alternatives --query` command parser'
|
||||
'--upower:`upower` command parser'
|
||||
'--uptime:`uptime` command parser'
|
||||
'--vmstat:`vmstat` command parser'
|
||||
'--vmstat-s:`vmstat` command streaming parser'
|
||||
'--w:`w` command parser'
|
||||
'--wc:`wc` command parser'
|
||||
'--who:`who` command parser'
|
||||
'--x509-cert:X.509 PEM and DER certificate file parser'
|
||||
'--xml:XML file parser'
|
||||
'--xrandr:`xrandr` command parser'
|
||||
'--yaml:YAML file parser'
|
||||
'--zipinfo:`zipinfo` command parser'
|
||||
)
|
||||
jc_options=(--force-color -C --debug -d --monochrome -m --pretty -p --quiet -q --raw -r --unbuffer -u --yaml-out -y)
|
||||
jc_options_describe=(
|
||||
'--force-color:force color output even when using pipes (overrides -m)'
|
||||
'-C:force color output even when using pipes (overrides -m)'
|
||||
'--debug:debug (double for verbose debug)'
|
||||
'-d:debug (double for verbose debug)'
|
||||
'--monochrome:monochrome output'
|
||||
'-m:monochrome output'
|
||||
'--pretty:pretty print output'
|
||||
'-p:pretty print output'
|
||||
'--quiet:suppress warnings (double to ignore streaming errors)'
|
||||
'-q:suppress warnings (double to ignore streaming errors)'
|
||||
'--raw:raw output'
|
||||
'-r:raw output'
|
||||
'--unbuffer:unbuffer output'
|
||||
'-u:unbuffer output'
|
||||
'--yaml-out:YAML output'
|
||||
'-y:YAML output'
|
||||
)
|
||||
jc_about_options=(--about -a)
|
||||
jc_about_options_describe=(
|
||||
'--about:about jc'
|
||||
'-a:about jc'
|
||||
)
|
||||
jc_about_mod_options=(--pretty -p --yaml-out -y --monochrome -m --force-color -C)
|
||||
jc_about_mod_options_describe=(
|
||||
'--pretty:pretty print output'
|
||||
'-p:pretty print output'
|
||||
'--yaml-out:YAML output'
|
||||
'-y:YAML output'
|
||||
'--monochrome:monochrome output'
|
||||
'-m:monochrome output'
|
||||
'--force-color:force color output even when using pipes (overrides -m)'
|
||||
'-C:force color output even when using pipes (overrides -m)'
|
||||
)
|
||||
jc_help_options=(--help -h)
|
||||
jc_help_options_describe=(
|
||||
'--help:help (--help --parser_name for parser documentation)'
|
||||
'-h:help (--help --parser_name for parser documentation)'
|
||||
)
|
||||
jc_special_options=(--version -v --bash-comp -B --zsh-comp -Z)
|
||||
jc_special_options_describe=(
|
||||
'--version:version info'
|
||||
'-v:version info'
|
||||
'--bash-comp:gen Bash completion: jc -B > /etc/bash_completion.d/jc'
|
||||
'-B:gen Bash completion: jc -B > /etc/bash_completion.d/jc'
|
||||
'--zsh-comp:gen Zsh completion: jc -Z > "${fpath[1]}/_jc"'
|
||||
'-Z:gen Zsh completion: jc -Z > "${fpath[1]}/_jc"'
|
||||
)
|
||||
|
||||
# if jc_about_options are found anywhere in the line, then only complete from jc_about_mod_options
|
||||
for i in ${words:0:-1}; do
|
||||
if (( $jc_about_options[(Ie)${i}] )); then
|
||||
_describe 'commands' jc_about_mod_options_describe
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
|
||||
# if jc_help_options and a parser are found anywhere in the line, then no more completions
|
||||
if
|
||||
(
|
||||
for i in ${words:0:-1}; do
|
||||
if (( $jc_help_options[(Ie)${i}] )); then
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
) && (
|
||||
for i in ${words:0:-1}; do
|
||||
if (( $jc_parsers[(Ie)${i}] )); then
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
); then
|
||||
return 0
|
||||
fi
|
||||
|
||||
# if jc_help_options are found anywhere in the line, then only complete with parsers
|
||||
for i in ${words:0:-1}; do
|
||||
if (( $jc_help_options[(Ie)${i}] )); then
|
||||
_describe 'commands' jc_parsers_describe
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
|
||||
# if special options are found anywhere in the line, then no more completions
|
||||
for i in ${words:0:-1}; do
|
||||
if (( $jc_special_options[(Ie)${i}] )); then
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
|
||||
# if magic command is found anywhere in the line, use called command's autocompletion
|
||||
for i in ${words:0:-1}; do
|
||||
if (( $jc_commands[(Ie)${i}] )); then
|
||||
# hack to remove options between jc and the magic command
|
||||
shift $(( ${#words} - 2 )) words
|
||||
words[1,0]=(jc)
|
||||
CURRENT=${#words}
|
||||
|
||||
# run the magic command's completions
|
||||
_arguments '*::arguments:_normal'
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
|
||||
# if a parser arg is found anywhere in the line, only show options and help options
|
||||
for i in ${words:0:-1}; do
|
||||
if (( $jc_parsers[(Ie)${i}] )); then
|
||||
_describe 'commands' jc_options_describe -- jc_help_options_describe
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
|
||||
# default completion
|
||||
_describe 'commands' jc_options_describe -- jc_about_options_describe -- jc_help_options_describe -- jc_special_options_describe -- jc_parsers_describe -- jc_commands_describe
|
||||
}
|
||||
|
||||
_jc
|
||||
|
||||
@@ -8,7 +8,12 @@ jc - JSON Convert `asciitable` parser
|
||||
This parser converts ASCII and Unicode text tables with single-line rows.
|
||||
|
||||
Column headers must be at least two spaces apart from each other and must
|
||||
be unique.
|
||||
be unique. For best results, column headers should be left-justified. If
|
||||
column separators are present, then non-left-justified headers will be fixed
|
||||
automatically.
|
||||
|
||||
Row separators are optional and are ignored. Each non-row-separator line is
|
||||
considered a separate row in the table.
|
||||
|
||||
For example:
|
||||
|
||||
@@ -54,6 +59,9 @@ etc...
|
||||
Headers (keys) are converted to snake-case. All values are returned as
|
||||
strings, except empty strings, which are converted to None/null.
|
||||
|
||||
> Note: To preserve the case of the keys use the `-r` cli option or
|
||||
> `raw=True` argument in `parse()`.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ cat table.txt | jc --asciitable
|
||||
@@ -136,4 +144,4 @@ Returns:
|
||||
### Parser Information
|
||||
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
|
||||
|
||||
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
Version 1.2 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
|
||||
@@ -29,6 +29,15 @@ Headers (keys) are converted to snake-case and newlines between multi-line
|
||||
headers are joined with an underscore. All values are returned as strings,
|
||||
except empty strings, which are converted to None/null.
|
||||
|
||||
> Note: To preserve the case of the keys use the `-r` cli option or
|
||||
> `raw=True` argument in `parse()`.
|
||||
|
||||
> Note: table column separator characters (e.g. `|`) cannot be present
|
||||
> inside the cell data. If detected, a warning message will be printed to
|
||||
> `STDERR` and the line will be skipped. The warning message can be
|
||||
> suppressed by using the `-q` command option or by setting `quiet=True` in
|
||||
> `parse()`.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ cat table.txt | jc --asciitable-m
|
||||
@@ -120,4 +129,4 @@ Returns:
|
||||
### Parser Information
|
||||
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
|
||||
|
||||
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
Version 1.2 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
|
||||
82
docs/parsers/chage.md
Normal file
82
docs/parsers/chage.md
Normal file
@@ -0,0 +1,82 @@
|
||||
[Home](https://kellyjonbrazil.github.io/jc/)
|
||||
<a id="jc.parsers.chage"></a>
|
||||
|
||||
# jc.parsers.chage
|
||||
|
||||
jc - JSON Convert `chage --list` command output parser
|
||||
|
||||
Supports `chage -l <username>` or `chage --list <username>`
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ chage -l johndoe | jc --chage
|
||||
|
||||
or
|
||||
|
||||
$ jc chage -l johndoe
|
||||
|
||||
Usage (module):
|
||||
|
||||
import jc
|
||||
result = jc.parse('chage', chage_command_output)
|
||||
|
||||
Schema:
|
||||
|
||||
{
|
||||
"password_last_changed": string,
|
||||
"password_expires": string,
|
||||
"password_inactive": string,
|
||||
"account_expires": string,
|
||||
"min_days_between_password_change": integer,
|
||||
"max_days_between_password_change": integer,
|
||||
"warning_days_before_password_expires": integer
|
||||
}
|
||||
|
||||
Examples:
|
||||
|
||||
$ chage --list joeuser | jc --chage -p
|
||||
{
|
||||
"password_last_changed": "never",
|
||||
"password_expires": "never",
|
||||
"password_inactive": "never",
|
||||
"account_expires": "never",
|
||||
"min_days_between_password_change": 0,
|
||||
"max_days_between_password_change": 99999,
|
||||
"warning_days_before_password_expires": 7
|
||||
}
|
||||
|
||||
$ chage --list joeuser | jc --chage -p -r
|
||||
{
|
||||
"password_last_changed": "never",
|
||||
"password_expires": "never",
|
||||
"password_inactive": "never",
|
||||
"account_expires": "never",
|
||||
"min_days_between_password_change": "0",
|
||||
"max_days_between_password_change": "99999",
|
||||
"warning_days_before_password_expires": "7"
|
||||
}
|
||||
|
||||
<a id="jc.parsers.chage.parse"></a>
|
||||
|
||||
### parse
|
||||
|
||||
```python
|
||||
def parse(data: str, raw: bool = False, quiet: bool = False) -> Dict
|
||||
```
|
||||
|
||||
Main text parsing function
|
||||
|
||||
Parameters:
|
||||
|
||||
data: (string) text data to parse
|
||||
raw: (boolean) unprocessed output if True
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
|
||||
Returns:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux
|
||||
|
||||
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
@@ -5,15 +5,15 @@
|
||||
|
||||
jc - JSON Convert `csv` file streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns a Generator
|
||||
iterator of Dictionaries (module)
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
The `csv` streaming parser will attempt to automatically detect the
|
||||
delimiter character. If the delimiter cannot be detected it will default
|
||||
to comma. The first row of the file must be a header row.
|
||||
|
||||
Note: The first 100 rows are read into memory to enable delimiter detection,
|
||||
then the rest of the rows are loaded lazily.
|
||||
> Note: The first 100 rows are read into memory to enable delimiter
|
||||
> detection, then the rest of the rows are loaded lazily.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
@@ -68,7 +68,7 @@ Examples:
|
||||
def parse(data, raw=False, quiet=False, ignore_exceptions=False)
|
||||
```
|
||||
|
||||
Main text parsing generator function. Returns an iterator object.
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
@@ -79,13 +79,9 @@ Parameters:
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
Yields:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
|
||||
Returns:
|
||||
|
||||
Iterator object (generator)
|
||||
Iterable of Dictionaries
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
|
||||
|
||||
@@ -120,4 +120,4 @@ Returns:
|
||||
### Parser Information
|
||||
Compatibility: linux, darwin, freebsd
|
||||
|
||||
Version 1.9 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
Version 1.10 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
|
||||
@@ -19,7 +19,7 @@ time of the system the parser is run on)
|
||||
|
||||
Usage (cli):
|
||||
|
||||
C:> dir | jc --dir
|
||||
C:\> dir | jc --dir
|
||||
|
||||
Usage (module):
|
||||
|
||||
@@ -42,7 +42,7 @@ Schema:
|
||||
|
||||
Examples:
|
||||
|
||||
C:> dir | jc --dir -p
|
||||
C:\> dir | jc --dir -p
|
||||
[
|
||||
{
|
||||
"date": "03/24/2021",
|
||||
@@ -83,7 +83,7 @@ Examples:
|
||||
...
|
||||
]
|
||||
|
||||
C:> dir | jc --dir -p -r
|
||||
C:\> dir | jc --dir -p -r
|
||||
[
|
||||
{
|
||||
"date": "03/24/2021",
|
||||
|
||||
@@ -43,8 +43,8 @@ Schema:
|
||||
"author": string,
|
||||
"author_email": string,
|
||||
"date": string,
|
||||
"epoch": integer, [0]
|
||||
"epoch_utc": integer, [1]
|
||||
"epoch": integer, # [0]
|
||||
"epoch_utc": integer, # [1]
|
||||
"commit_by": string,
|
||||
"commit_by_email": string,
|
||||
"commit_by_date": string,
|
||||
@@ -172,4 +172,4 @@ Returns:
|
||||
### Parser Information
|
||||
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
|
||||
|
||||
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
Version 1.1 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
|
||||
111
docs/parsers/git_log_s.md
Normal file
111
docs/parsers/git_log_s.md
Normal file
@@ -0,0 +1,111 @@
|
||||
[Home](https://kellyjonbrazil.github.io/jc/)
|
||||
<a id="jc.parsers.git_log_s"></a>
|
||||
|
||||
# jc.parsers.git\_log\_s
|
||||
|
||||
jc - JSON Convert `git log` command output streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
Can be used with the following format options:
|
||||
- `oneline`
|
||||
- `short`
|
||||
- `medium`
|
||||
- `full`
|
||||
- `fuller`
|
||||
|
||||
Additional options supported:
|
||||
- `--stat`
|
||||
- `--shortstat`
|
||||
|
||||
The `epoch` calculated timestamp field is naive. (i.e. based on the
|
||||
local time of the system the parser is run on)
|
||||
|
||||
The `epoch_utc` calculated timestamp field is timezone-aware and is
|
||||
only available if the timezone field is UTC.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ git log | jc --git-log-s
|
||||
|
||||
Usage (module):
|
||||
|
||||
import jc
|
||||
|
||||
result = jc.parse('git_log_s', git_log_command_output.splitlines())
|
||||
for item in result:
|
||||
# do something
|
||||
|
||||
Schema:
|
||||
|
||||
{
|
||||
"commit": string,
|
||||
"author": string,
|
||||
"author_email": string,
|
||||
"date": string,
|
||||
"epoch": integer, # [0]
|
||||
"epoch_utc": integer, # [1]
|
||||
"commit_by": string,
|
||||
"commit_by_email": string,
|
||||
"commit_by_date": string,
|
||||
"message": string,
|
||||
"stats" : {
|
||||
"files_changed": integer,
|
||||
"insertions": integer,
|
||||
"deletions": integer,
|
||||
"files": [
|
||||
string
|
||||
]
|
||||
}
|
||||
|
||||
# below object only exists if using -qq or ignore_exceptions=True
|
||||
"_jc_meta": {
|
||||
"success": boolean, # false if error parsing
|
||||
"error": string, # exists if "success" is false
|
||||
"line": string # exists if "success" is false
|
||||
}
|
||||
}
|
||||
|
||||
[0] naive timestamp if "date" field is parsable, else null
|
||||
[1] timezone aware timestamp availabe for UTC, else null
|
||||
|
||||
Examples:
|
||||
|
||||
$ git log | jc --git-log-s
|
||||
{"commit":"a730ae18c8e81c5261db132df73cd74f272a0a26","author":"Kelly...}
|
||||
{"commit":"930bf439c06c48a952baec05a9896c8d92b7693e","author":"Kelly...}
|
||||
...
|
||||
|
||||
<a id="jc.parsers.git_log_s.parse"></a>
|
||||
|
||||
### parse
|
||||
|
||||
```python
|
||||
@add_jc_meta
|
||||
def parse(data: Iterable[str],
|
||||
raw: bool = False,
|
||||
quiet: bool = False,
|
||||
ignore_exceptions: bool = False) -> Union[Iterable[Dict], tuple]
|
||||
```
|
||||
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
data: (iterable) line-based text data to parse
|
||||
(e.g. sys.stdin or str.splitlines())
|
||||
|
||||
raw: (boolean) unprocessed output if True
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
|
||||
Returns:
|
||||
|
||||
Iterable of Dictionaries
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
|
||||
|
||||
Version 1.1 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
145
docs/parsers/gpg.md
Normal file
145
docs/parsers/gpg.md
Normal file
@@ -0,0 +1,145 @@
|
||||
[Home](https://kellyjonbrazil.github.io/jc/)
|
||||
<a id="jc.parsers.gpg"></a>
|
||||
|
||||
# jc.parsers.gpg
|
||||
|
||||
jc - JSON Convert `gpg --with-colons` command output parser
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ gpg --with-colons --show-keys file.gpg | jc --gpg
|
||||
|
||||
or
|
||||
|
||||
$ jc gpg --with-colons --show-keys file.gpg
|
||||
|
||||
Usage (module):
|
||||
|
||||
import jc
|
||||
result = jc.parse('gpg', gpg_command_output)
|
||||
|
||||
Schema:
|
||||
|
||||
Field definitions from https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob_plain;f=doc/DETAILS
|
||||
|
||||
> Note: Number values are not converted to integers because many field
|
||||
> specifications are overloaded and future augmentations are implied in the
|
||||
> documentation.
|
||||
|
||||
[
|
||||
{
|
||||
"type": string,
|
||||
"validity": string,
|
||||
"key_length": string,
|
||||
"pub_key_alg": string,
|
||||
"key_id": string,
|
||||
"creation_date": string,
|
||||
"expiration_date": string,
|
||||
"certsn_uidhash_trustinfo": string,
|
||||
"owner_trust": string,
|
||||
"user_id": string,
|
||||
"signature_class": string,
|
||||
"key_capabilities": string,
|
||||
"cert_fingerprint_other": string,
|
||||
"flag": string,
|
||||
"token_sn": string,
|
||||
"hash_alg": string,
|
||||
"curve_name": string,
|
||||
"compliance_flags": string,
|
||||
"last_update_date": string,
|
||||
"origin": string,
|
||||
"comment": string,
|
||||
"index": string, # [0]
|
||||
"bits": string, # [0]
|
||||
"value": string, # [0]
|
||||
"version": string, # [1], [4]
|
||||
"signature_count": string, # [1]
|
||||
"encryption_count": string, # [1]
|
||||
"policy": string, # [1]
|
||||
"signature_first_seen": string, # [1]
|
||||
"signature_most_recent_seen": string, # [1]
|
||||
"encryption_first_done": string, # [1]
|
||||
"encryption_most_recent_done": string, # [1]
|
||||
"staleness_reason": string, # [2]
|
||||
"trust_model": string, # [2]
|
||||
"trust_db_created": string, # [2]
|
||||
"trust_db_expires": string, # [2]
|
||||
"marginally_trusted_users": string, # [2]
|
||||
"completely_trusted_users": string, # [2]
|
||||
"cert_chain_max_depth": string, # [2]
|
||||
"subpacket_number": string, # [3]
|
||||
"hex_flags": string, # [3]
|
||||
"subpacket_length": string, # [3]
|
||||
"subpacket_data": string, # [3]
|
||||
"pubkey": string, # [4]
|
||||
"cipher": string, # [4]
|
||||
"digest": string, # [4]
|
||||
"compress": string, # [4]
|
||||
"group": string, # [4]
|
||||
"members": string, # [4]
|
||||
"curve_names": string, # [4]
|
||||
}
|
||||
]
|
||||
|
||||
All blank values are converted to null/None.
|
||||
|
||||
[0] for 'pkd' type
|
||||
[1] for 'tfs' type
|
||||
[2] for 'tru' type
|
||||
[3] for 'skp' type
|
||||
[4] for 'cfg' type
|
||||
|
||||
Examples:
|
||||
|
||||
$ gpg --with-colons --show-keys file.gpg | jc --gpg -p
|
||||
[
|
||||
{
|
||||
"type": "pub",
|
||||
"validity": "f",
|
||||
"key_length": "1024",
|
||||
"pub_key_alg": "17",
|
||||
"key_id": "6C7EE1B8621CC013",
|
||||
"creation_date": "899817715",
|
||||
"expiration_date": "1055898235",
|
||||
"certsn_uidhash_trustinfo": null,
|
||||
"owner_trust": "m",
|
||||
"user_id": null,
|
||||
"signature_class": null,
|
||||
"key_capabilities": "scESC",
|
||||
"cert_fingerprint_other": null,
|
||||
"flag": null,
|
||||
"token_sn": null,
|
||||
"hash_alg": null,
|
||||
"curve_name": null,
|
||||
"compliance_flags": null,
|
||||
"last_update_date": null,
|
||||
"origin": null,
|
||||
"comment": null
|
||||
},
|
||||
...
|
||||
]
|
||||
|
||||
<a id="jc.parsers.gpg.parse"></a>
|
||||
|
||||
### parse
|
||||
|
||||
```python
|
||||
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
|
||||
```
|
||||
|
||||
Main text parsing function
|
||||
|
||||
Parameters:
|
||||
|
||||
data: (string) text data to parse
|
||||
raw: (boolean) unprocessed output if True
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
|
||||
Returns:
|
||||
|
||||
List of Dictionaries. Raw or processed structured data.
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux
|
||||
|
||||
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
@@ -128,4 +128,4 @@ Returns:
|
||||
### Parser Information
|
||||
Compatibility: linux, darwin, aix, freebsd
|
||||
|
||||
Version 1.4 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
Version 1.5 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
jc - JSON Convert `ifconfig` command output parser
|
||||
|
||||
Note: No `ifconfig` options are supported.
|
||||
> Note: No `ifconfig` options are supported.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
|
||||
@@ -11,9 +11,10 @@ Parses standard `INI` files and files containing simple key/value pairs.
|
||||
- Comment prefix can be `#` or `;`. Comments must be on their own line.
|
||||
- If duplicate keys are found, only the last value will be used.
|
||||
|
||||
> Note: Values starting and ending with quotation marks will have the marks
|
||||
removed. If you would like to keep the quotation marks, use the `-r`
|
||||
command-line argument or the `raw=True` argument in `parse()`.
|
||||
> Note: Values starting and ending with double or single quotation marks
|
||||
> will have the marks removed. If you would like to keep the quotation
|
||||
> marks, use the `-r` command-line argument or the `raw=True` argument in
|
||||
> `parse()`.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
@@ -26,8 +27,8 @@ Usage (module):
|
||||
|
||||
Schema:
|
||||
|
||||
ini or key/value document converted to a dictionary - see the
|
||||
configparser standard library documentation for more details.
|
||||
ini or key/value document converted to a dictionary - see the configparser
|
||||
standard library documentation for more details.
|
||||
|
||||
{
|
||||
"key1": string,
|
||||
@@ -91,4 +92,4 @@ Returns:
|
||||
### Parser Information
|
||||
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
|
||||
|
||||
Version 1.6 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
Version 1.7 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
jc - JSON Convert `iostat` command output parser
|
||||
|
||||
Note: `iostat` version 11 and higher include a JSON output option
|
||||
> Note: `iostat` version 11 and higher include a JSON output option
|
||||
|
||||
Usage (cli):
|
||||
|
||||
|
||||
@@ -5,21 +5,21 @@
|
||||
|
||||
jc - JSON Convert `iostat` command output streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns a Generator
|
||||
iterator of Dictionaries (module)
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
Note: `iostat` version 11 and higher include a JSON output option
|
||||
> Note: `iostat` version 11 and higher include a JSON output option
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ iostat | jc --iostat-s
|
||||
|
||||
> Note: When piping `jc` converted `iostat` output to other processes it may
|
||||
appear the output is hanging due to the OS pipe buffers. This is because
|
||||
`iostat` output is too small to quickly fill up the buffer. Use the `-u`
|
||||
option to unbuffer the `jc` output if you would like immediate output. See
|
||||
the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
|
||||
for more information.
|
||||
> appear the output is hanging due to the OS pipe buffers. This is because
|
||||
> `iostat` output is too small to quickly fill up the buffer. Use the `-u`
|
||||
> option to unbuffer the `jc` output if you would like immediate output. See
|
||||
> the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
|
||||
> for more information.
|
||||
|
||||
Usage (module):
|
||||
|
||||
@@ -112,7 +112,7 @@ Examples:
|
||||
def parse(data, raw=False, quiet=False, ignore_exceptions=False)
|
||||
```
|
||||
|
||||
Main text parsing generator function. Returns an iterator object.
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
@@ -123,13 +123,9 @@ Parameters:
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
Yields:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
|
||||
Returns:
|
||||
|
||||
Iterator object (generator)
|
||||
Iterable of Dictionaries
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux
|
||||
|
||||
@@ -186,4 +186,4 @@ Returns:
|
||||
### Parser Information
|
||||
Compatibility: linux
|
||||
|
||||
Version 1.7 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
Version 1.8 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
|
||||
@@ -12,8 +12,8 @@ Supports files containing simple key/value pairs.
|
||||
- If duplicate keys are found, only the last value will be used.
|
||||
|
||||
> Note: Values starting and ending with quotation marks will have the marks
|
||||
removed. If you would like to keep the quotation marks, use the `-r`
|
||||
command-line argument or the `raw=True` argument in `parse()`.
|
||||
> removed. If you would like to keep the quotation marks, use the `-r`
|
||||
> command-line argument or the `raw=True` argument in `parse()`.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
@@ -26,8 +26,8 @@ Usage (module):
|
||||
|
||||
Schema:
|
||||
|
||||
key/value document converted to a dictionary - see the
|
||||
configparser standard library documentation for more details.
|
||||
key/value document converted to a dictionary - see the configparser standard
|
||||
library documentation for more details.
|
||||
|
||||
{
|
||||
"key1": string,
|
||||
|
||||
@@ -9,11 +9,12 @@ Options supported:
|
||||
- `lbaR1`
|
||||
- `--time-style=full-iso`
|
||||
|
||||
Note: The `-1`, `-l`, or `-b` option of `ls` should be used to correctly
|
||||
parse filenames that include newline characters. Since `ls` does not encode
|
||||
newlines in filenames when outputting to a pipe it will cause `jc` to see
|
||||
multiple files instead of a single file if `-1`, `-l`, or `-b` is not used.
|
||||
Alternatively, `vdir` can be used, which is the same as running `ls -lb`.
|
||||
> Note: The `-1`, `-l`, or `-b` option of `ls` should be used to correctly
|
||||
> parse filenames that include newline characters. Since `ls` does not
|
||||
> encode newlines in filenames when outputting to a pipe it will cause `jc`
|
||||
> to see multiple files instead of a single file if `-1`, `-l`, or `-b` is
|
||||
> not used. Alternatively, `vdir` can be used, which is the same as running
|
||||
> `ls -lb`.
|
||||
|
||||
The `epoch` calculated timestamp field is naive. (i.e. based on the local
|
||||
time of the system the parser is run on)
|
||||
|
||||
@@ -5,8 +5,8 @@
|
||||
|
||||
jc - JSON Convert `ls` and `vdir` command output streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns a Generator
|
||||
iterator of Dictionaries (module)
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
Requires the `-l` option to be used on `ls`. If there are newline characters
|
||||
in the filename, then make sure to use the `-b` option on `ls`.
|
||||
@@ -81,7 +81,7 @@ Examples:
|
||||
def parse(data, raw=False, quiet=False, ignore_exceptions=False)
|
||||
```
|
||||
|
||||
Main text parsing generator function. Returns an iterator object.
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
@@ -92,13 +92,9 @@ Parameters:
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
Yields:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
|
||||
Returns:
|
||||
|
||||
Iterator object (generator)
|
||||
Iterable of Dictionaries
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux, darwin, cygwin, aix, freebsd
|
||||
|
||||
@@ -22,9 +22,9 @@ Usage (module):
|
||||
|
||||
Schema:
|
||||
|
||||
Note: <item> object keynames are assigned directly from the lsusb
|
||||
output. If there are duplicate <item> names in a section, only the
|
||||
last one is converted.
|
||||
> Note: <item> object keynames are assigned directly from the lsusb
|
||||
> output. If there are duplicate <item> names in a section, only the
|
||||
> last one is converted.
|
||||
|
||||
[
|
||||
{
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
jc - JSON Convert `mpstat` command output parser
|
||||
|
||||
Note: Latest versions of `mpstat` support JSON output (v11.5.1+)
|
||||
> Note: Latest versions of `mpstat` support JSON output (v11.5.1+)
|
||||
|
||||
Usage (cli):
|
||||
|
||||
|
||||
@@ -5,10 +5,10 @@
|
||||
|
||||
jc - JSON Convert `mpstat` command output streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns a Generator
|
||||
iterator of Dictionaries (module)
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
Note: Latest versions of `mpstat` support JSON output (v11.5.1+)
|
||||
> Note: Latest versions of `mpstat` support JSON output (v11.5.1+)
|
||||
|
||||
Usage (cli):
|
||||
|
||||
@@ -101,15 +101,13 @@ Examples:
|
||||
|
||||
```python
|
||||
@add_jc_meta
|
||||
def parse(
|
||||
data: Iterable[str],
|
||||
raw: bool = False,
|
||||
quiet: bool = False,
|
||||
ignore_exceptions: bool = False
|
||||
) -> Union[Generator[Dict, None, None], tuple]
|
||||
def parse(data: Iterable[str],
|
||||
raw: bool = False,
|
||||
quiet: bool = False,
|
||||
ignore_exceptions: bool = False) -> Union[Iterable[Dict], tuple]
|
||||
```
|
||||
|
||||
Main text parsing generator function. Returns an iterator object.
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
@@ -120,13 +118,9 @@ Parameters:
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
Yields:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
|
||||
Returns:
|
||||
|
||||
Iterator object (generator)
|
||||
Iterable of Dictionaries
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux
|
||||
|
||||
@@ -29,18 +29,18 @@ Usage (module):
|
||||
|
||||
Schema:
|
||||
|
||||
Because there are so many options, the schema is not strictly defined.
|
||||
Integer and Float value conversions are attempted and the original
|
||||
values are kept if they fail. If you don't want automatic conversion,
|
||||
then use the -r or raw=True option to disable it.
|
||||
Because there are so many options, the schema is not strictly defined.
|
||||
Integer and Float value conversions are attempted and the original
|
||||
values are kept if they fail. If you don't want automatic conversion,
|
||||
then use the `-r` or `raw=True` option to disable it.
|
||||
|
||||
The structure is flat, for the most part, but there are a couple of
|
||||
"well-known" keys that are further parsed into objects for convenience.
|
||||
These are documented below.
|
||||
The structure is flat, for the most part, but there are a couple of
|
||||
"well-known" keys that are further parsed into objects for convenience.
|
||||
These are documented below.
|
||||
|
||||
[
|
||||
{
|
||||
"<key>": string/integer/float, [0]
|
||||
"<key>": string/integer/float, # [0]
|
||||
"dhcp4_option_x": {
|
||||
"name": string,
|
||||
"value": string/integer/float,
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
|
||||
# jc.parsers.pidstat
|
||||
|
||||
jc - JSON Convert `pidstat` command output parser
|
||||
jc - JSON Convert `pidstat -h` command output parser
|
||||
|
||||
Must use the `-h` option in `pidstat`. All other `pidstat` options are
|
||||
supported in combination with `-h`.
|
||||
|
||||
@@ -3,10 +3,10 @@
|
||||
|
||||
# jc.parsers.pidstat\_s
|
||||
|
||||
jc - JSON Convert `pidstat` command output streaming parser
|
||||
jc - JSON Convert `pidstat -h` command output streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns a Generator
|
||||
iterator of Dictionaries (module)
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
Must use the `-h` option in `pidstat`. All other `pidstat` options are
|
||||
supported in combination with `-h`.
|
||||
@@ -16,11 +16,11 @@ Usage (cli):
|
||||
$ pidstat | jc --pidstat-s
|
||||
|
||||
> Note: When piping `jc` converted `pidstat` output to other processes it
|
||||
may appear the output is hanging due to the OS pipe buffers. This is
|
||||
because `pidstat` output is too small to quickly fill up the buffer. Use
|
||||
the `-u` option to unbuffer the `jc` output if you would like immediate
|
||||
output. See the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
|
||||
for more information.
|
||||
> may appear the output is hanging due to the OS pipe buffers. This is
|
||||
> because `pidstat` output is too small to quickly fill up the buffer. Use
|
||||
> the `-u` option to unbuffer the `jc` output if you would like immediate
|
||||
> output. See the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
|
||||
> for more information.
|
||||
|
||||
Usage (module):
|
||||
|
||||
@@ -83,15 +83,13 @@ Examples:
|
||||
|
||||
```python
|
||||
@add_jc_meta
|
||||
def parse(
|
||||
data: Iterable[str],
|
||||
raw: bool = False,
|
||||
quiet: bool = False,
|
||||
ignore_exceptions: bool = False
|
||||
) -> Union[Generator[Dict, None, None], tuple]
|
||||
def parse(data: Iterable[str],
|
||||
raw: bool = False,
|
||||
quiet: bool = False,
|
||||
ignore_exceptions: bool = False) -> Union[Iterable[Dict], tuple]
|
||||
```
|
||||
|
||||
Main text parsing generator function. Returns an iterator object.
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
@@ -102,13 +100,9 @@ Parameters:
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
Yields:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
|
||||
Returns:
|
||||
|
||||
Iterator object (generator)
|
||||
Iterable of Dictionaries
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux
|
||||
|
||||
@@ -9,8 +9,8 @@ Supports `ping` and `ping6` output.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
Note: Use the ping `-c` (count) option, otherwise data will not be
|
||||
piped to `jc`.
|
||||
> Note: Use the ping `-c` (count) option, otherwise data will not be
|
||||
> piped to `jc`.
|
||||
|
||||
$ ping -c 3 1.2.3.4 | jc --ping
|
||||
|
||||
|
||||
@@ -5,21 +5,21 @@
|
||||
|
||||
jc - JSON Convert `ping` command output streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns a Generator
|
||||
iterator of Dictionaries (module)
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
Supports `ping` and `ping6` output.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ ping | jc --ping-s
|
||||
$ ping 1.2.3.4 | jc --ping-s
|
||||
|
||||
> Note: When piping `jc` converted `ping` output to other processes it may
|
||||
appear the output is hanging due to the OS pipe buffers. This is because
|
||||
`ping` output is too small to quickly fill up the buffer. Use the `-u`
|
||||
option to unbuffer the `jc` output if you would like immediate output.
|
||||
See the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
|
||||
for more information.
|
||||
> appear the output is hanging due to the OS pipe buffers. This is because
|
||||
> `ping` output is too small to quickly fill up the buffer. Use the `-u`
|
||||
> option to unbuffer the `jc` output if you would like immediate output.
|
||||
> See the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
|
||||
> for more information.
|
||||
|
||||
Usage (module):
|
||||
|
||||
@@ -88,7 +88,7 @@ Examples:
|
||||
def parse(data, raw=False, quiet=False, ignore_exceptions=False)
|
||||
```
|
||||
|
||||
Main text parsing generator function. Returns an iterator object.
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
@@ -99,13 +99,9 @@ Parameters:
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
Yields:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
|
||||
Returns:
|
||||
|
||||
Iterator object (generator)
|
||||
Iterable of Dictionaries
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux, darwin, freebsd
|
||||
|
||||
@@ -70,7 +70,7 @@ Examples:
|
||||
### parse
|
||||
|
||||
```python
|
||||
def parse(data, raw=False, quiet=False)
|
||||
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
|
||||
```
|
||||
|
||||
Main text parsing function
|
||||
@@ -88,4 +88,4 @@ Returns:
|
||||
### Parser Information
|
||||
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
|
||||
|
||||
Version 1.3 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
Version 1.4 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
|
||||
115
docs/parsers/postconf.md
Normal file
115
docs/parsers/postconf.md
Normal file
@@ -0,0 +1,115 @@
|
||||
[Home](https://kellyjonbrazil.github.io/jc/)
|
||||
<a id="jc.parsers.postconf"></a>
|
||||
|
||||
# jc.parsers.postconf
|
||||
|
||||
jc - JSON Convert `postconf -M` command output parser
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ postconf -M | jc --postconf
|
||||
|
||||
or
|
||||
|
||||
$ jc postconf -M
|
||||
|
||||
Usage (module):
|
||||
|
||||
import jc
|
||||
result = jc.parse('postconf', postconf_command_output)
|
||||
|
||||
Schema:
|
||||
|
||||
[
|
||||
{
|
||||
"service_name": string,
|
||||
"service_type": string,
|
||||
"private": boolean/null, # [0]
|
||||
"unprivileged": boolean/null, # [0]
|
||||
"chroot": boolean/null, # [0]
|
||||
"wake_up_time": integer/null, # [0]
|
||||
"no_wake_up_before_first_use": boolean/null, # [1]
|
||||
"process_limit": integer/null, # [0]
|
||||
"command": string
|
||||
}
|
||||
]
|
||||
|
||||
[0] '-' converted to null/None
|
||||
[1] null/None if `wake_up_time` is null/None
|
||||
|
||||
Examples:
|
||||
|
||||
$ postconf -M | jc --postconf -p
|
||||
[
|
||||
{
|
||||
"service_name": "smtp",
|
||||
"service_type": "inet",
|
||||
"private": false,
|
||||
"unprivileged": null,
|
||||
"chroot": true,
|
||||
"wake_up_time": null,
|
||||
"process_limit": null,
|
||||
"command": "smtpd",
|
||||
"no_wake_up_before_first_use": null
|
||||
},
|
||||
{
|
||||
"service_name": "pickup",
|
||||
"service_type": "unix",
|
||||
"private": false,
|
||||
"unprivileged": null,
|
||||
"chroot": true,
|
||||
"wake_up_time": 60,
|
||||
"process_limit": 1,
|
||||
"command": "pickup",
|
||||
"no_wake_up_before_first_use": false
|
||||
}
|
||||
]
|
||||
|
||||
$ postconf -M | jc --postconf -p -r
|
||||
[
|
||||
{
|
||||
"service_name": "smtp",
|
||||
"service_type": "inet",
|
||||
"private": "n",
|
||||
"unprivileged": "-",
|
||||
"chroot": "y",
|
||||
"wake_up_time": "-",
|
||||
"process_limit": "-",
|
||||
"command": "smtpd"
|
||||
},
|
||||
{
|
||||
"service_name": "pickup",
|
||||
"service_type": "unix",
|
||||
"private": "n",
|
||||
"unprivileged": "-",
|
||||
"chroot": "y",
|
||||
"wake_up_time": "60",
|
||||
"process_limit": "1",
|
||||
"command": "pickup"
|
||||
}
|
||||
]
|
||||
|
||||
<a id="jc.parsers.postconf.parse"></a>
|
||||
|
||||
### parse
|
||||
|
||||
```python
|
||||
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
|
||||
```
|
||||
|
||||
Main text parsing function
|
||||
|
||||
Parameters:
|
||||
|
||||
data: (string) text data to parse
|
||||
raw: (boolean) unprocessed output if True
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
|
||||
Returns:
|
||||
|
||||
List of Dictionaries. Raw or processed structured data.
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux
|
||||
|
||||
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
@@ -22,20 +22,20 @@ Schema:
|
||||
|
||||
[
|
||||
{
|
||||
"destination": string,
|
||||
"gateway": string,
|
||||
"genmask": string,
|
||||
"flags": string,
|
||||
"destination": string,
|
||||
"gateway": string,
|
||||
"genmask": string,
|
||||
"flags": string,
|
||||
"flags_pretty": [
|
||||
string
|
||||
string
|
||||
]
|
||||
"metric": integer,
|
||||
"ref": integer,
|
||||
"use": integer,
|
||||
"mss": integer,
|
||||
"window": integer,
|
||||
"irtt": integer,
|
||||
"iface": string
|
||||
"metric": integer,
|
||||
"ref": integer,
|
||||
"use": integer,
|
||||
"mss": integer,
|
||||
"window": integer,
|
||||
"irtt": integer,
|
||||
"iface": string
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
@@ -10,7 +10,7 @@ Works with `rpm -qi [package]` or `rpm -qia`.
|
||||
The `..._epoch` calculated timestamp fields are naive. (i.e. based on the
|
||||
local time of the system the parser is run on)
|
||||
|
||||
The `..._epoch_utc` calculated timestamp fields are timezone-aware and is
|
||||
The `..._epoch_utc` calculated timestamp fields are timezone-aware and are
|
||||
only available if the timezone field is UTC.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
jc - JSON Convert `rsync` command output parser
|
||||
|
||||
Supports the `-i` or `--itemize-changes` options with all levels of
|
||||
verbosity. This parser will process the STDOUT output or a log file
|
||||
verbosity. This parser will process the `STDOUT` output or a log file
|
||||
generated with the `--log-file` option.
|
||||
|
||||
Usage (cli):
|
||||
@@ -51,8 +51,8 @@ Schema:
|
||||
"time": string,
|
||||
"process": integer,
|
||||
"metadata": string,
|
||||
"update_type": string/null, [0]
|
||||
"file_type": string/null, [1]
|
||||
"update_type": string/null, # [0]
|
||||
"file_type": string/null, # [1]
|
||||
"checksum_or_value_different": bool/null,
|
||||
"size_different": bool/null,
|
||||
"modification_time_different": bool/null,
|
||||
@@ -61,7 +61,7 @@ Schema:
|
||||
"group_different": bool/null,
|
||||
"acl_different": bool/null,
|
||||
"extended_attribute_different": bool/null,
|
||||
"epoch": integer, [2]
|
||||
"epoch": integer, # [2]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -5,11 +5,11 @@
|
||||
|
||||
jc - JSON Convert `rsync` command output streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns a Generator
|
||||
iterator of Dictionaries (module)
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
Supports the `-i` or `--itemize-changes` options with all levels of
|
||||
verbosity. This parser will process the STDOUT output or a log file
|
||||
verbosity. This parser will process the `STDOUT` output or a log file
|
||||
generated with the `--log-file` option.
|
||||
|
||||
Usage (cli):
|
||||
@@ -49,8 +49,8 @@ Schema:
|
||||
"time": string,
|
||||
"process": integer,
|
||||
"metadata": string,
|
||||
"update_type": string/null, [0]
|
||||
"file_type": string/null, [1]
|
||||
"update_type": string/null, # [0]
|
||||
"file_type": string/null, # [1]
|
||||
"checksum_or_value_different": bool/null,
|
||||
"size_different": bool/null,
|
||||
"modification_time_different": bool/null,
|
||||
@@ -59,7 +59,7 @@ Schema:
|
||||
"group_different": bool/null,
|
||||
"acl_different": bool/null,
|
||||
"extended_attribute_different": bool/null,
|
||||
"epoch": integer, [2]
|
||||
"epoch": integer, # [2]
|
||||
|
||||
# below object only exists if using -qq or ignore_exceptions=True
|
||||
"_jc_meta": {
|
||||
@@ -90,15 +90,13 @@ Examples:
|
||||
|
||||
```python
|
||||
@add_jc_meta
|
||||
def parse(
|
||||
data: Iterable[str],
|
||||
raw: bool = False,
|
||||
quiet: bool = False,
|
||||
ignore_exceptions: bool = False
|
||||
) -> Union[Generator[Dict, None, None], tuple]
|
||||
def parse(data: Iterable[str],
|
||||
raw: bool = False,
|
||||
quiet: bool = False,
|
||||
ignore_exceptions: bool = False) -> Union[Iterable[Dict], tuple]
|
||||
```
|
||||
|
||||
Main text parsing generator function. Returns an iterator object.
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
@@ -109,13 +107,9 @@ Parameters:
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
Yields:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
|
||||
Returns:
|
||||
|
||||
Iterator object (generator)
|
||||
Iterable of Dictionaries
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux, darwin, freebsd
|
||||
|
||||
@@ -5,8 +5,8 @@
|
||||
|
||||
jc - JSON Convert `ss` command output parser
|
||||
|
||||
Extended information options like -e and -p are not supported and may cause
|
||||
parsing irregularities.
|
||||
Extended information options like `-e` and `-p` are not supported and may
|
||||
cause parsing irregularities.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
@@ -23,8 +23,8 @@ Usage (module):
|
||||
|
||||
Schema:
|
||||
|
||||
Information from https://www.cyberciti.biz/files/ss.html used to define
|
||||
field names
|
||||
Information from https://www.cyberciti.biz/files/ss.html used to define
|
||||
field names
|
||||
|
||||
[
|
||||
{
|
||||
|
||||
@@ -5,8 +5,8 @@
|
||||
|
||||
jc - JSON Convert `stat` command output streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns a Generator
|
||||
iterator of Dictionaries (module)
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
The `xxx_epoch` calculated timestamp fields are naive. (i.e. based on the
|
||||
local time of the system the parser is run on).
|
||||
@@ -86,7 +86,7 @@ Examples:
|
||||
def parse(data, raw=False, quiet=False, ignore_exceptions=False)
|
||||
```
|
||||
|
||||
Main text parsing generator function. Returns an iterator object.
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
@@ -97,13 +97,9 @@ Parameters:
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
Yields:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
|
||||
Returns:
|
||||
|
||||
Iterator object (generator)
|
||||
Iterable of Dictionaries
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux, darwin, freebsd
|
||||
|
||||
@@ -5,10 +5,10 @@
|
||||
|
||||
jc - JSON Convert `sysctl -a` command output parser
|
||||
|
||||
Note: Since `sysctl` output is not easily parsable only a very simple
|
||||
key/value object will be output. An attempt is made to convert obvious
|
||||
integers and floats. If no conversion is desired, use the `-r`
|
||||
command-line argument or the `raw=True` argument in `parse()`.
|
||||
> Note: Since `sysctl` output is not easily parsable only a very simple
|
||||
> key/value object will be output. An attempt is made to convert obvious
|
||||
> integers and floats. If no conversion is desired, use the `-r`
|
||||
> command-line argument or the `raw=True` argument in `parse()`.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
|
||||
@@ -11,8 +11,8 @@ used to redirect the output to a file that can be read by `jc`.
|
||||
Alternatively, the output from `/usr/bin/time` can be redirected to `STDOUT`
|
||||
so `jc` can receive it.
|
||||
|
||||
Note: `/usr/bin/time` is similar but different from the Bash builtin
|
||||
`time` command.
|
||||
> Note: `/usr/bin/time` is similar but different from the Bash builtin
|
||||
> `time` command.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
@@ -26,8 +26,8 @@ Usage (module):
|
||||
|
||||
Schema:
|
||||
|
||||
Source: https://www.freebsd.org/cgi/man.cgi?query=getrusage
|
||||
https://man7.org/linux/man-pages/man1/time.1.html
|
||||
Source: https://www.freebsd.org/cgi/man.cgi?query=getrusage,
|
||||
https://man7.org/linux/man-pages/man1/time.1.html
|
||||
|
||||
{
|
||||
"real_time": float,
|
||||
|
||||
339
docs/parsers/top.md
Normal file
339
docs/parsers/top.md
Normal file
@@ -0,0 +1,339 @@
|
||||
[Home](https://kellyjonbrazil.github.io/jc/)
|
||||
<a id="jc.parsers.top"></a>
|
||||
|
||||
# jc.parsers.top
|
||||
|
||||
jc - JSON Convert `top -b` command output parser
|
||||
|
||||
Requires batch mode (`-b`). The `-n` option must also be used to limit
|
||||
the number of times `top` is run.
|
||||
|
||||
Warning messages will be printed to `STDERR` if truncated fields are
|
||||
detected. These warnings can be suppressed with the `-q` or `quiet=True`
|
||||
option.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ top -b -n 3 | jc --top
|
||||
|
||||
or
|
||||
|
||||
$ jc top -b -n 3
|
||||
|
||||
Usage (module):
|
||||
|
||||
import jc
|
||||
result = jc.parse('top', top_command_output)
|
||||
|
||||
Schema:
|
||||
|
||||
All `-` values are converted to `null`
|
||||
|
||||
[
|
||||
{
|
||||
"time": string,
|
||||
"uptime": integer,
|
||||
"users": integer,
|
||||
"load_1m": float,
|
||||
"load_5m": float,
|
||||
"load_15m": float,
|
||||
"tasks_total": integer,
|
||||
"tasks_running": integer,
|
||||
"tasks_sleeping": integer,
|
||||
"tasks_stopped": integer,
|
||||
"tasks_zombie": integer,
|
||||
"cpu_user": float,
|
||||
"cpu_sys": float,
|
||||
"cpu_nice": float,
|
||||
"cpu_idle": float,
|
||||
"cpu_wait": float,
|
||||
"cpu_hardware": float,
|
||||
"cpu_software": float,
|
||||
"cpu_steal": float,
|
||||
"mem_total": float, # [0]
|
||||
"mem_free": float, # [0]
|
||||
"mem_used": float, # [0]
|
||||
"mem_buff_cache": float, # [0]
|
||||
"swap_total": float, # [0]
|
||||
"swap_free": float, # [0]
|
||||
"swap_used": float, # [0]
|
||||
"mem_available": float, # [0]
|
||||
"processes": [
|
||||
{
|
||||
"pid": integer,
|
||||
"user": string,
|
||||
"priority": integer,
|
||||
"nice": integer,
|
||||
"virtual_mem": float, # [1]
|
||||
"resident_mem": float, # [1]
|
||||
"shared_mem": float, # [1]
|
||||
"status": string,
|
||||
"percent_cpu": float,
|
||||
"percent_mem": float,
|
||||
"time_hundredths": string,
|
||||
"command": string,
|
||||
"parent_pid": integer,
|
||||
"uid": integer,
|
||||
"real_uid": integer,
|
||||
"real_user": string,
|
||||
"saved_uid": integer,
|
||||
"saved_user": string,
|
||||
"gid": integer,
|
||||
"group": string,
|
||||
"pgrp": integer,
|
||||
"tty": string,
|
||||
"tty_process_gid": integer,
|
||||
"session_id": integer,
|
||||
"thread_count": integer,
|
||||
"last_used_processor": integer,
|
||||
"time": string,
|
||||
"swap": float, # [1]
|
||||
"code": float, # [1]
|
||||
"data": float, # [1]
|
||||
"major_page_fault_count": integer,
|
||||
"minor_page_fault_count": integer,
|
||||
"dirty_pages_count": integer,
|
||||
"sleeping_in_function": string,
|
||||
"flags": string,
|
||||
"cgroups": string,
|
||||
"supplementary_gids": [
|
||||
integer
|
||||
],
|
||||
"supplementary_groups": [
|
||||
string
|
||||
],
|
||||
"thread_gid": integer,
|
||||
"environment_variables": [
|
||||
string
|
||||
]
|
||||
"major_page_fault_count_delta": integer,
|
||||
"minor_page_fault_count_delta": integer,
|
||||
"used": float, # [1]
|
||||
"ipc_namespace_inode": integer,
|
||||
"mount_namespace_inode": integer,
|
||||
"net_namespace_inode": integer,
|
||||
"pid_namespace_inode": integer,
|
||||
"user_namespace_inode": integer,
|
||||
"nts_namespace_inode": integer,
|
||||
"control_group_name": string,
|
||||
"lxc_container_name": string,
|
||||
"numa_node": integer,
|
||||
"out_of_mem_adjustment": integer,
|
||||
"out_of_mem_score": integer,
|
||||
"resident_anon_mem": integer,
|
||||
"resident_file_backed_mem": integer,
|
||||
"resident_locked_mem": integer,
|
||||
"resident_shared_mem": integer
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
[0] Values are in the units output by `top`
|
||||
[1] Unit suffix stripped during float conversion
|
||||
|
||||
Examples:
|
||||
|
||||
$ top -b -n 3 | jc --top -p
|
||||
[
|
||||
{
|
||||
"time": "11:20:43",
|
||||
"uptime": 118,
|
||||
"users": 2,
|
||||
"load_1m": 0.0,
|
||||
"load_5m": 0.01,
|
||||
"load_15m": 0.05,
|
||||
"tasks_total": 108,
|
||||
"tasks_running": 2,
|
||||
"tasks_sleeping": 106,
|
||||
"tasks_stopped": 0,
|
||||
"tasks_zombie": 0,
|
||||
"cpu_user": 5.6,
|
||||
"cpu_sys": 11.1,
|
||||
"cpu_nice": 0.0,
|
||||
"cpu_idle": 83.3,
|
||||
"cpu_wait": 0.0,
|
||||
"cpu_hardware": 0.0,
|
||||
"cpu_software": 0.0,
|
||||
"cpu_steal": 0.0,
|
||||
"mem_total": 3.7,
|
||||
"mem_free": 3.3,
|
||||
"mem_used": 0.2,
|
||||
"mem_buff_cache": 0.2,
|
||||
"swap_total": 2.0,
|
||||
"swap_free": 2.0,
|
||||
"swap_used": 0.0,
|
||||
"mem_available": 3.3,
|
||||
"processes": [
|
||||
{
|
||||
"pid": 2225,
|
||||
"user": "kbrazil",
|
||||
"priority": 20,
|
||||
"nice": 0,
|
||||
"virtual_mem": 158.1,
|
||||
"resident_mem": 2.2,
|
||||
"shared_mem": 1.6,
|
||||
"status": "running",
|
||||
"percent_cpu": 12.5,
|
||||
"percent_mem": 0.1,
|
||||
"time_hundredths": "0:00.02",
|
||||
"command": "top",
|
||||
"parent_pid": 1884,
|
||||
"uid": 1000,
|
||||
"real_uid": 1000,
|
||||
"real_user": "kbrazil",
|
||||
"saved_uid": 1000,
|
||||
"saved_user": "kbrazil",
|
||||
"gid": 1000,
|
||||
"group": "kbrazil",
|
||||
"pgrp": 2225,
|
||||
"tty": "pts/0",
|
||||
"tty_process_gid": 2225,
|
||||
"session_id": 1884,
|
||||
"thread_count": 1,
|
||||
"last_used_processor": 0,
|
||||
"time": "0:00",
|
||||
"swap": 0.0,
|
||||
"code": 0.1,
|
||||
"data": 1.0,
|
||||
"major_page_fault_count": 0,
|
||||
"minor_page_fault_count": 736,
|
||||
"dirty_pages_count": 0,
|
||||
"sleeping_in_function": null,
|
||||
"flags": "..4.2...",
|
||||
"cgroups": "1:name=systemd:/user.slice/user-1000.+",
|
||||
"supplementary_gids": [
|
||||
10,
|
||||
1000
|
||||
],
|
||||
"supplementary_groups": [
|
||||
"wheel",
|
||||
"kbrazil"
|
||||
],
|
||||
"thread_gid": 2225,
|
||||
"environment_variables": [
|
||||
"XDG_SESSION_ID=2",
|
||||
"HOSTNAME=localhost"
|
||||
],
|
||||
"major_page_fault_count_delta": 0,
|
||||
"minor_page_fault_count_delta": 4,
|
||||
"used": 2.2,
|
||||
"ipc_namespace_inode": 4026531839,
|
||||
"mount_namespace_inode": 4026531840,
|
||||
"net_namespace_inode": 4026531956,
|
||||
"pid_namespace_inode": 4026531836,
|
||||
"user_namespace_inode": 4026531837,
|
||||
"nts_namespace_inode": 4026531838
|
||||
},
|
||||
...
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
$ top -b -n 3 | jc --top -p -r
|
||||
[
|
||||
{
|
||||
"time": "11:20:43",
|
||||
"uptime": "1:18",
|
||||
"users": "2",
|
||||
"load_1m": "0.00",
|
||||
"load_5m": "0.01",
|
||||
"load_15m": "0.05",
|
||||
"tasks_total": "108",
|
||||
"tasks_running": "2",
|
||||
"tasks_sleeping": "106",
|
||||
"tasks_stopped": "0",
|
||||
"tasks_zombie": "0",
|
||||
"cpu_user": "5.6",
|
||||
"cpu_sys": "11.1",
|
||||
"cpu_nice": "0.0",
|
||||
"cpu_idle": "83.3",
|
||||
"cpu_wait": "0.0",
|
||||
"cpu_hardware": "0.0",
|
||||
"cpu_software": "0.0",
|
||||
"cpu_steal": "0.0",
|
||||
"swap_total": "2.0",
|
||||
"swap_free": "2.0",
|
||||
"swap_used": "0.0",
|
||||
"mem_available": "3.3",
|
||||
"processes": [
|
||||
{
|
||||
"PID": "2225",
|
||||
"USER": "kbrazil",
|
||||
"PR": "20",
|
||||
"NI": "0",
|
||||
"VIRT": "158.1m",
|
||||
"RES": "2.2m",
|
||||
"SHR": "1.6m",
|
||||
"S": "R",
|
||||
"%CPU": "12.5",
|
||||
"%MEM": "0.1",
|
||||
"TIME+": "0:00.02",
|
||||
"COMMAND": "top",
|
||||
"PPID": "1884",
|
||||
"UID": "1000",
|
||||
"RUID": "1000",
|
||||
"RUSER": "kbrazil",
|
||||
"SUID": "1000",
|
||||
"SUSER": "kbrazil",
|
||||
"GID": "1000",
|
||||
"GROUP": "kbrazil",
|
||||
"PGRP": "2225",
|
||||
"TTY": "pts/0",
|
||||
"TPGID": "2225",
|
||||
"SID": "1884",
|
||||
"nTH": "1",
|
||||
"P": "0",
|
||||
"TIME": "0:00",
|
||||
"SWAP": "0.0m",
|
||||
"CODE": "0.1m",
|
||||
"DATA": "1.0m",
|
||||
"nMaj": "0",
|
||||
"nMin": "736",
|
||||
"nDRT": "0",
|
||||
"WCHAN": "-",
|
||||
"Flags": "..4.2...",
|
||||
"CGROUPS": "1:name=systemd:/user.slice/user-1000.+",
|
||||
"SUPGIDS": "10,1000",
|
||||
"SUPGRPS": "wheel,kbrazil",
|
||||
"TGID": "2225",
|
||||
"ENVIRON": "XDG_SESSION_ID=2 HOSTNAME=localhost S+",
|
||||
"vMj": "0",
|
||||
"vMn": "4",
|
||||
"USED": "2.2m",
|
||||
"nsIPC": "4026531839",
|
||||
"nsMNT": "4026531840",
|
||||
"nsNET": "4026531956",
|
||||
"nsPID": "4026531836",
|
||||
"nsUSER": "4026531837",
|
||||
"nsUTS": "4026531838"
|
||||
},
|
||||
...
|
||||
]
|
||||
}
|
||||
]
|
||||
|
||||
<a id="jc.parsers.top.parse"></a>
|
||||
|
||||
### parse
|
||||
|
||||
```python
|
||||
def parse(data: str, raw: bool = False, quiet: bool = False) -> List[Dict]
|
||||
```
|
||||
|
||||
Main text parsing function
|
||||
|
||||
Parameters:
|
||||
|
||||
data: (string) text data to parse
|
||||
raw: (boolean) unprocessed output if True
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
|
||||
Returns:
|
||||
|
||||
List of Dictionaries. Raw or processed structured data.
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux
|
||||
|
||||
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
180
docs/parsers/top_s.md
Normal file
180
docs/parsers/top_s.md
Normal file
@@ -0,0 +1,180 @@
|
||||
[Home](https://kellyjonbrazil.github.io/jc/)
|
||||
<a id="jc.parsers.top_s"></a>
|
||||
|
||||
# jc.parsers.top\_s
|
||||
|
||||
jc - JSON Convert `top -b` command output streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
Requires batch mode (`-b`).
|
||||
|
||||
Warning messages will be printed to `STDERR` if truncated fields are
|
||||
detected. These warnings can be suppressed with the `-q` or `quiet=True`
|
||||
option.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ top -b | jc --top-s
|
||||
|
||||
Usage (module):
|
||||
|
||||
import jc
|
||||
|
||||
result = jc.parse('top_s', top_command_output.splitlines())
|
||||
for item in result:
|
||||
# do something
|
||||
|
||||
Schema:
|
||||
|
||||
{
|
||||
"time": string,
|
||||
"uptime": integer,
|
||||
"users": integer,
|
||||
"load_1m": float,
|
||||
"load_5m": float,
|
||||
"load_15m": float,
|
||||
"tasks_total": integer,
|
||||
"tasks_running": integer,
|
||||
"tasks_sleeping": integer,
|
||||
"tasks_stopped": integer,
|
||||
"tasks_zombie": integer,
|
||||
"cpu_user": float,
|
||||
"cpu_sys": float,
|
||||
"cpu_nice": float,
|
||||
"cpu_idle": float,
|
||||
"cpu_wait": float,
|
||||
"cpu_hardware": float,
|
||||
"cpu_software": float,
|
||||
"cpu_steal": float,
|
||||
"mem_total": float, # [0]
|
||||
"mem_free": float, # [0]
|
||||
"mem_used": float, # [0]
|
||||
"mem_buff_cache": float, # [0]
|
||||
"swap_total": float, # [0]
|
||||
"swap_free": float, # [0]
|
||||
"swap_used": float, # [0]
|
||||
"mem_available": float, # [0]
|
||||
"processes": [
|
||||
{
|
||||
"pid": integer,
|
||||
"user": string,
|
||||
"priority": integer,
|
||||
"nice": integer,
|
||||
"virtual_mem": float, # [1]
|
||||
"resident_mem": float, # [1]
|
||||
"shared_mem": float, # [1]
|
||||
"status": string,
|
||||
"percent_cpu": float,
|
||||
"percent_mem": float,
|
||||
"time_hundredths": string,
|
||||
"command": string,
|
||||
"parent_pid": integer,
|
||||
"uid": integer,
|
||||
"real_uid": integer,
|
||||
"real_user": string,
|
||||
"saved_uid": integer,
|
||||
"saved_user": string,
|
||||
"gid": integer,
|
||||
"group": string,
|
||||
"pgrp": integer,
|
||||
"tty": string,
|
||||
"tty_process_gid": integer,
|
||||
"session_id": integer,
|
||||
"thread_count": integer,
|
||||
"last_used_processor": integer,
|
||||
"time": string,
|
||||
"swap": float, # [1]
|
||||
"code": float, # [1]
|
||||
"data": float, # [1]
|
||||
"major_page_fault_count": integer,
|
||||
"minor_page_fault_count": integer,
|
||||
"dirty_pages_count": integer,
|
||||
"sleeping_in_function": string,
|
||||
"flags": string,
|
||||
"cgroups": string,
|
||||
"supplementary_gids": [
|
||||
integer
|
||||
],
|
||||
"supplementary_groups": [
|
||||
string
|
||||
],
|
||||
"thread_gid": integer,
|
||||
"environment_variables": [
|
||||
string
|
||||
]
|
||||
"major_page_fault_count_delta": integer,
|
||||
"minor_page_fault_count_delta": integer,
|
||||
"used": float, # [1]
|
||||
"ipc_namespace_inode": integer,
|
||||
"mount_namespace_inode": integer,
|
||||
"net_namespace_inode": integer,
|
||||
"pid_namespace_inode": integer,
|
||||
"user_namespace_inode": integer,
|
||||
"nts_namespace_inode": integer,
|
||||
"control_group_name": string,
|
||||
"lxc_container_name": string,
|
||||
"numa_node": integer,
|
||||
"out_of_mem_adjustment": integer,
|
||||
"out_of_mem_score": integer,
|
||||
"resident_anon_mem": integer,
|
||||
"resident_file_backed_mem": integer,
|
||||
"resident_locked_mem": integer,
|
||||
"resident_shared_mem": integer
|
||||
}
|
||||
],
|
||||
|
||||
# below object only exists if using -qq or ignore_exceptions=True
|
||||
"_jc_meta": {
|
||||
"success": boolean, # false if error parsing
|
||||
"error": string, # exists if "success" is false
|
||||
"line": string # exists if "success" is false
|
||||
}
|
||||
}
|
||||
|
||||
[0] Values are in the units output by `top`
|
||||
[1] Unit suffix stripped during float conversion
|
||||
|
||||
Examples:
|
||||
|
||||
$ top -b | jc --top-s
|
||||
{"time":"11:24:50","uptime":2,"users":2,"load_1m":0.23,"load_5m":...}
|
||||
...
|
||||
|
||||
$ top -b | jc --top-s -r
|
||||
{"time":"11:24:50","uptime":"2 min","users":"2","load_1m":"0.23","lo...}
|
||||
...
|
||||
|
||||
<a id="jc.parsers.top_s.parse"></a>
|
||||
|
||||
### parse
|
||||
|
||||
```python
|
||||
@add_jc_meta
|
||||
def parse(data: Iterable[str],
|
||||
raw: bool = False,
|
||||
quiet: bool = False,
|
||||
ignore_exceptions: bool = False) -> Union[Iterable[Dict], tuple]
|
||||
```
|
||||
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
data: (iterable) line-based text data to parse
|
||||
(e.g. sys.stdin or str.splitlines())
|
||||
|
||||
raw: (boolean) unprocessed output if True
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
|
||||
Returns:
|
||||
|
||||
Iterable of Dictionaries
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux
|
||||
|
||||
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
@@ -7,12 +7,12 @@ jc - JSON Convert `traceroute` command output parser
|
||||
|
||||
Supports `traceroute` and `traceroute6` output.
|
||||
|
||||
Note: On some operating systems you will need to redirect `STDERR` to
|
||||
`STDOUT` for destination info since the header line is sent to
|
||||
`STDERR`. A warning message will be printed to `STDERR` if the
|
||||
header row is not found.
|
||||
|
||||
e.g. `$ traceroute 8.8.8.8 2>&1 | jc --traceroute`
|
||||
> Note: On some operating systems you will need to redirect `STDERR` to
|
||||
> `STDOUT` for destination info since the header line is sent to
|
||||
> `STDERR`. A warning message will be printed to `STDERR` if the
|
||||
> header row is not found.
|
||||
>
|
||||
> e.g. `$ traceroute 8.8.8.8 2>&1 | jc --traceroute`
|
||||
|
||||
Usage (cli):
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
jc - JSON Convert `uname -a` command output parser
|
||||
|
||||
Note: Must use `uname -a`
|
||||
> Note: Must use `uname -a`
|
||||
|
||||
Usage (cli):
|
||||
|
||||
|
||||
@@ -5,8 +5,8 @@
|
||||
|
||||
jc - JSON Convert `vmstat` command output streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns a Generator
|
||||
iterator of Dictionaries (module)
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
Options supported: `-a`, `-w`, `-d`, `-t`
|
||||
|
||||
@@ -21,11 +21,11 @@ Usage (cli):
|
||||
$ vmstat | jc --vmstat-s
|
||||
|
||||
> Note: When piping `jc` converted `vmstat` output to other processes it may
|
||||
appear the output is hanging due to the OS pipe buffers. This is because
|
||||
`vmstat` output is too small to quickly fill up the buffer. Use the `-u`
|
||||
option to unbuffer the `jc` output if you would like immediate output. See
|
||||
the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
|
||||
for more information.
|
||||
> appear the output is hanging due to the OS pipe buffers. This is because
|
||||
> `vmstat` output is too small to quickly fill up the buffer. Use the `-u`
|
||||
> option to unbuffer the `jc` output if you would like immediate output. See
|
||||
> the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
|
||||
> for more information.
|
||||
|
||||
Usage (module):
|
||||
|
||||
@@ -75,9 +75,9 @@ Schema:
|
||||
|
||||
# below object only exists if using -qq or ignore_exceptions=True
|
||||
"_jc_meta": {
|
||||
"success": boolean, # [2]
|
||||
"error": string, # [3]
|
||||
"line": string # [3]
|
||||
"success": boolean, # [2]
|
||||
"error": string, # [3]
|
||||
"line": string # [3]
|
||||
}
|
||||
}
|
||||
|
||||
@@ -105,7 +105,7 @@ Examples:
|
||||
def parse(data, raw=False, quiet=False, ignore_exceptions=False)
|
||||
```
|
||||
|
||||
Main text parsing generator function. Returns an iterator object.
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
@@ -116,13 +116,9 @@ Parameters:
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
Yields:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
|
||||
Returns:
|
||||
|
||||
Iterator object (generator)
|
||||
Iterable of Dictionaries
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux
|
||||
|
||||
225
docs/parsers/x509_cert.md
Normal file
225
docs/parsers/x509_cert.md
Normal file
@@ -0,0 +1,225 @@
|
||||
[Home](https://kellyjonbrazil.github.io/jc/)
|
||||
<a id="jc.parsers.x509_cert"></a>
|
||||
|
||||
# jc.parsers.x509\_cert
|
||||
|
||||
jc - JSON Convert X.509 Certificate format file parser
|
||||
|
||||
This parser will convert DER and PEM encoded X.509 certificate files.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ cat certificate.pem | jc --x509-cert
|
||||
|
||||
Usage (module):
|
||||
|
||||
import jc
|
||||
result = jc.parse('x509_cert', x509_cert_file_output)
|
||||
|
||||
Schema:
|
||||
|
||||
[
|
||||
{
|
||||
"tbs_certificate": {
|
||||
"version": string,
|
||||
"serial_number": string, # [0]
|
||||
"signature": {
|
||||
"algorithm": string,
|
||||
"parameters": string/null,
|
||||
},
|
||||
"issuer": {
|
||||
"country_name": string,
|
||||
"state_or_province_name" string,
|
||||
"locality_name": string,
|
||||
"organization_name": array/string,
|
||||
"organizational_unit_name": array/string,
|
||||
"common_name": string,
|
||||
"email_address": string
|
||||
},
|
||||
"validity": {
|
||||
"not_before": integer, # [1]
|
||||
"not_after": integer, # [1]
|
||||
"not_before_iso": string,
|
||||
"not_after_iso": string
|
||||
},
|
||||
"subject": {
|
||||
"country_name": string,
|
||||
"state_or_province_name": string,
|
||||
"locality_name": string,
|
||||
"organization_name": array/string,
|
||||
"organizational_unit_name": array/string,
|
||||
"common_name": string,
|
||||
"email_address": string
|
||||
},
|
||||
"subject_public_key_info": {
|
||||
"algorithm": {
|
||||
"algorithm": string,
|
||||
"parameters": string/null,
|
||||
},
|
||||
"public_key": {
|
||||
"modulus": string, # [0]
|
||||
"public_exponent": integer
|
||||
}
|
||||
},
|
||||
"issuer_unique_id": string/null,
|
||||
"subject_unique_id": string/null,
|
||||
"extensions": [
|
||||
{
|
||||
"extn_id": string,
|
||||
"critical": boolean,
|
||||
"extn_value": array/object/string/integer # [2]
|
||||
}
|
||||
]
|
||||
},
|
||||
"signature_algorithm": {
|
||||
"algorithm": string,
|
||||
"parameters": string/null
|
||||
},
|
||||
"signature_value": string # [0]
|
||||
}
|
||||
]
|
||||
|
||||
[0] in colon-delimited hex notation
|
||||
[1] time-zone-aware (UTC) epoch timestamp
|
||||
[2] See below for well-known Extension schemas:
|
||||
|
||||
Basic Constraints:
|
||||
{
|
||||
"extn_id": "basic_constraints",
|
||||
"critical": boolean,
|
||||
"extn_value": {
|
||||
"ca": boolean,
|
||||
"path_len_constraint": string/null
|
||||
}
|
||||
}
|
||||
|
||||
Key Usage:
|
||||
{
|
||||
"extn_id": "key_usage",
|
||||
"critical": boolean,
|
||||
"extn_value": [
|
||||
string
|
||||
]
|
||||
}
|
||||
|
||||
Key Identifier:
|
||||
{
|
||||
"extn_id": "key_identifier",
|
||||
"critical": boolean,
|
||||
"extn_value": string # [0]
|
||||
}
|
||||
|
||||
Authority Key Identifier:
|
||||
{
|
||||
"extn_id": "authority_key_identifier",
|
||||
"critical": boolean,
|
||||
"extn_value": {
|
||||
"key_identifier": string, # [0]
|
||||
"authority_cert_issuer": string/null,
|
||||
"authority_cert_serial_number": string/null
|
||||
}
|
||||
}
|
||||
|
||||
Examples:
|
||||
|
||||
$ cat entrust-ec1.pem| jc --x509-cert -p
|
||||
[
|
||||
{
|
||||
"tbs_certificate": {
|
||||
"version": "v3",
|
||||
"serial_number": "a6:8b:79:29:00:00:00:00:50:d0:91:f9",
|
||||
"signature": {
|
||||
"algorithm": "sha384_ecdsa",
|
||||
"parameters": null
|
||||
},
|
||||
"issuer": {
|
||||
"country_name": "US",
|
||||
"organization_name": "Entrust, Inc.",
|
||||
"organizational_unit_name": [
|
||||
"See www.entrust.net/legal-terms",
|
||||
"(c) 2012 Entrust, Inc. - for authorized use only"
|
||||
],
|
||||
"common_name": "Entrust Root Certification Authority - EC1"
|
||||
},
|
||||
"validity": {
|
||||
"not_before": 1355844336,
|
||||
"not_after": 2144764536,
|
||||
"not_before_iso": "2012-12-18T15:25:36+00:00",
|
||||
"not_after_iso": "2037-12-18T15:55:36+00:00"
|
||||
},
|
||||
"subject": {
|
||||
"country_name": "US",
|
||||
"organization_name": "Entrust, Inc.",
|
||||
"organizational_unit_name": [
|
||||
"See www.entrust.net/legal-terms",
|
||||
"(c) 2012 Entrust, Inc. - for authorized use only"
|
||||
],
|
||||
"common_name": "Entrust Root Certification Authority - EC1"
|
||||
},
|
||||
"subject_public_key_info": {
|
||||
"algorithm": {
|
||||
"algorithm": "ec",
|
||||
"parameters": "secp384r1"
|
||||
},
|
||||
"public_key": "04:84:13:c9:d0:ba:6d:41:7b:e2:6c:d0:eb:55:..."
|
||||
},
|
||||
"issuer_unique_id": null,
|
||||
"subject_unique_id": null,
|
||||
"extensions": [
|
||||
{
|
||||
"extn_id": "key_usage",
|
||||
"critical": true,
|
||||
"extn_value": [
|
||||
"crl_sign",
|
||||
"key_cert_sign"
|
||||
]
|
||||
},
|
||||
{
|
||||
"extn_id": "basic_constraints",
|
||||
"critical": true,
|
||||
"extn_value": {
|
||||
"ca": true,
|
||||
"path_len_constraint": null
|
||||
}
|
||||
},
|
||||
{
|
||||
"extn_id": "key_identifier",
|
||||
"critical": false,
|
||||
"extn_value": "b7:63:e7:1a:dd:8d:e9:08:a6:55:83:a4:e0:6a:..."
|
||||
}
|
||||
]
|
||||
},
|
||||
"signature_algorithm": {
|
||||
"algorithm": "sha384_ecdsa",
|
||||
"parameters": null
|
||||
},
|
||||
"signature_value": "30:64:02:30:61:79:d8:e5:42:47:df:1c:ae:53:..."
|
||||
}
|
||||
]
|
||||
|
||||
<a id="jc.parsers.x509_cert.parse"></a>
|
||||
|
||||
### parse
|
||||
|
||||
```python
|
||||
def parse(data: Union[str, bytes],
|
||||
raw: bool = False,
|
||||
quiet: bool = False) -> List[Dict]
|
||||
```
|
||||
|
||||
Main text parsing function
|
||||
|
||||
Parameters:
|
||||
|
||||
data: (string) text data to parse
|
||||
raw: (boolean) unprocessed output if True
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
|
||||
Returns:
|
||||
|
||||
List of Dictionaries. Raw or processed structured data.
|
||||
|
||||
### Parser Information
|
||||
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
|
||||
|
||||
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)
|
||||
@@ -16,8 +16,8 @@ Usage (module):
|
||||
|
||||
Schema:
|
||||
|
||||
XML Document converted to a Dictionary
|
||||
See https://github.com/martinblech/xmltodict for details
|
||||
XML Document converted to a Dictionary. See https://github.com/martinblech/xmltodict
|
||||
for details.
|
||||
|
||||
{
|
||||
"key1": string/object,
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
|
||||
jc - JSON Convert `YAML` file parser
|
||||
|
||||
Note: datetime objects will be converted to strings.
|
||||
> Note: `datetime` objects will be converted to strings.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
|
||||
@@ -5,10 +5,7 @@
|
||||
|
||||
jc - JSON Convert `zipinfo` command output parser
|
||||
|
||||
Options supported:
|
||||
- none
|
||||
|
||||
Note: The default listing format.
|
||||
> Note: No `zipinfo` options are supported.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
|
||||
@@ -26,11 +26,12 @@ https://github.com/kellyjonbrazil/jc/tree/master/docs
|
||||
|
||||
### Specific Version
|
||||
|
||||
Replace `<full_version_number>` - e.g. `1.17.7`:
|
||||
|
||||
`https://github.com/kellyjonbrazil/jc/tree/v<full_version_number>/docs`
|
||||
|
||||
Specific versions can also be selected by tag in the branch dropdown menu.
|
||||
> Replace `<full_version_number>` - e.g. `1.18.0`:
|
||||
|
||||
Specific versions can also be selected by tag in the Github branch dropdown
|
||||
menu.
|
||||
|
||||
## Usage Example
|
||||
|
||||
@@ -93,14 +94,14 @@ its module name.
|
||||
|
||||
### parser_mod_list
|
||||
|
||||
parser_mod_list() -> list
|
||||
parser_mod_list() -> list[str]
|
||||
|
||||
Get a list of all available parser module names to be used in
|
||||
`parse()`, `parser_info()`, and `get_help()`.
|
||||
|
||||
### plugin_parser_mod_list
|
||||
|
||||
plugin_parser_mod_list() -> list
|
||||
plugin_parser_mod_list() -> list[str]
|
||||
|
||||
Get a list of plugin parser module names to be used in
|
||||
`parse()`, `parser_info()`, and `get_help()`. This list is a subset of
|
||||
@@ -108,7 +109,7 @@ Get a list of plugin parser module names to be used in
|
||||
|
||||
### standard_parser_mod_list
|
||||
|
||||
standard_parser_mod_list() -> list
|
||||
standard_parser_mod_list() -> list[str]
|
||||
|
||||
Get a list of standard parser module names to be used in
|
||||
`parse()`, `parser_info()`, and `get_help()`. This list is a subset of
|
||||
@@ -116,7 +117,7 @@ Get a list of standard parser module names to be used in
|
||||
|
||||
### streaming_parser_mod_list
|
||||
|
||||
streaming_parser_mod_list() -> list
|
||||
streaming_parser_mod_list() -> list[str]
|
||||
|
||||
Get a list of streaming parser module names to be used in
|
||||
`parse()`, `parser_info()`, and `get_help()`. This list is a subset of
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
* [jc.utils](#jc.utils)
|
||||
* [warning\_message](#jc.utils.warning_message)
|
||||
* [error\_message](#jc.utils.error_message)
|
||||
* [is\_compatible](#jc.utils.is_compatible)
|
||||
* [compatibility](#jc.utils.compatibility)
|
||||
* [has\_data](#jc.utils.has_data)
|
||||
* [convert\_to\_int](#jc.utils.convert_to_int)
|
||||
@@ -26,8 +27,8 @@ jc - JSON Convert utils
|
||||
def warning_message(message_lines: List[str]) -> None
|
||||
```
|
||||
|
||||
Prints warning message for non-fatal issues. The first line is
|
||||
prepended with 'jc: Warning - ' and subsequent lines are indented.
|
||||
Prints warning message to `STDERR` for non-fatal issues. The first line
|
||||
is prepended with 'jc: Warning - ' and subsequent lines are indented.
|
||||
Wraps text as needed based on the terminal width.
|
||||
|
||||
Parameters:
|
||||
@@ -46,7 +47,7 @@ Returns:
|
||||
def error_message(message_lines: List[str]) -> None
|
||||
```
|
||||
|
||||
Prints an error message for fatal issues. The first line is
|
||||
Prints an error message to `STDERR` for fatal issues. The first line is
|
||||
prepended with 'jc: Error - ' and subsequent lines are indented.
|
||||
Wraps text as needed based on the terminal width.
|
||||
|
||||
@@ -58,6 +59,16 @@ Returns:
|
||||
|
||||
None - just prints output to STDERR
|
||||
|
||||
<a id="jc.utils.is_compatible"></a>
|
||||
|
||||
### is\_compatible
|
||||
|
||||
```python
|
||||
def is_compatible(compatible: List) -> bool
|
||||
```
|
||||
|
||||
Returns True if the parser is compatible with the running OS platform.
|
||||
|
||||
<a id="jc.utils.compatibility"></a>
|
||||
|
||||
### compatibility
|
||||
@@ -68,8 +79,9 @@ def compatibility(mod_name: str,
|
||||
quiet: bool = False) -> None
|
||||
```
|
||||
|
||||
Checks for the parser's compatibility with the running OS
|
||||
platform.
|
||||
Checks for the parser's compatibility with the running OS platform and
|
||||
prints a warning message to `STDERR` if not compatible and
|
||||
`quiet=False.`
|
||||
|
||||
Parameters:
|
||||
|
||||
@@ -90,20 +102,23 @@ Returns:
|
||||
### has\_data
|
||||
|
||||
```python
|
||||
def has_data(data: str) -> bool
|
||||
def has_data(data: Union[str, bytes]) -> bool
|
||||
```
|
||||
|
||||
Checks if the input contains data. If there are any non-whitespace
|
||||
characters then return True, else return False.
|
||||
Checks if the string input contains data. If there are any
|
||||
non-whitespace characters then return `True`, else return `False`.
|
||||
|
||||
For bytes, returns True if there is any data.
|
||||
|
||||
Parameters:
|
||||
|
||||
data: (string) input to check whether it contains data
|
||||
data: (string, bytes) input to check whether it contains data
|
||||
|
||||
Returns:
|
||||
|
||||
Boolean True if input string (data) contains non-whitespace
|
||||
characters, otherwise False
|
||||
characters, otherwise False. For bytes data, returns
|
||||
True if there is any data, otherwise False.
|
||||
|
||||
<a id="jc.utils.convert_to_int"></a>
|
||||
|
||||
|
||||
@@ -22,11 +22,12 @@ https://github.com/kellyjonbrazil/jc/tree/master/docs
|
||||
|
||||
### Specific Version
|
||||
|
||||
Replace `<full_version_number>` - e.g. `1.17.7`:
|
||||
|
||||
`https://github.com/kellyjonbrazil/jc/tree/v<full_version_number>/docs`
|
||||
|
||||
Specific versions can also be selected by tag in the branch dropdown menu.
|
||||
> Replace `<full_version_number>` - e.g. `1.18.0`:
|
||||
|
||||
Specific versions can also be selected by tag in the Github branch dropdown
|
||||
menu.
|
||||
|
||||
## Usage Example
|
||||
|
||||
@@ -89,14 +90,14 @@ its module name.
|
||||
|
||||
### parser_mod_list
|
||||
|
||||
parser_mod_list() -> list
|
||||
parser_mod_list() -> list[str]
|
||||
|
||||
Get a list of all available parser module names to be used in
|
||||
`parse()`, `parser_info()`, and `get_help()`.
|
||||
|
||||
### plugin_parser_mod_list
|
||||
|
||||
plugin_parser_mod_list() -> list
|
||||
plugin_parser_mod_list() -> list[str]
|
||||
|
||||
Get a list of plugin parser module names to be used in
|
||||
`parse()`, `parser_info()`, and `get_help()`. This list is a subset of
|
||||
@@ -104,7 +105,7 @@ Get a list of plugin parser module names to be used in
|
||||
|
||||
### standard_parser_mod_list
|
||||
|
||||
standard_parser_mod_list() -> list
|
||||
standard_parser_mod_list() -> list[str]
|
||||
|
||||
Get a list of standard parser module names to be used in
|
||||
`parse()`, `parser_info()`, and `get_help()`. This list is a subset of
|
||||
@@ -112,7 +113,7 @@ Get a list of standard parser module names to be used in
|
||||
|
||||
### streaming_parser_mod_list
|
||||
|
||||
streaming_parser_mod_list() -> list
|
||||
streaming_parser_mod_list() -> list[str]
|
||||
|
||||
Get a list of streaming parser module names to be used in
|
||||
`parse()`, `parser_info()`, and `get_help()`. This list is a subset of
|
||||
|
||||
295
jc/cli.py
295
jc/cli.py
@@ -2,16 +2,20 @@
|
||||
JC cli module
|
||||
"""
|
||||
|
||||
import io
|
||||
import sys
|
||||
import os
|
||||
import textwrap
|
||||
import signal
|
||||
import shlex
|
||||
import subprocess
|
||||
import json
|
||||
from typing import List, Dict
|
||||
from .lib import (__version__, parser_info, all_parser_info, parsers,
|
||||
_get_parser, _parser_is_streaming)
|
||||
_get_parser, _parser_is_streaming, standard_parser_mod_list,
|
||||
plugin_parser_mod_list, streaming_parser_mod_list)
|
||||
from . import utils
|
||||
from .cli_data import long_options_map, new_pygments_colors, old_pygments_colors
|
||||
from .shell_completions import bash_completion, zsh_completion
|
||||
from . import tracebackplus
|
||||
from .exceptions import LibraryNotInstalled, ParseError
|
||||
|
||||
@@ -21,13 +25,12 @@ try:
|
||||
from pygments import highlight
|
||||
from pygments.style import Style
|
||||
from pygments.token import (Name, Number, String, Keyword)
|
||||
from pygments.lexers import JsonLexer
|
||||
from pygments.lexers.data import JsonLexer, YamlLexer
|
||||
from pygments.formatters import Terminal256Formatter
|
||||
PYGMENTS_INSTALLED = True
|
||||
except Exception:
|
||||
PYGMENTS_INSTALLED = False
|
||||
|
||||
|
||||
JC_ERROR_EXIT = 100
|
||||
|
||||
|
||||
@@ -45,63 +48,9 @@ class info():
|
||||
# startswith is sufficient and avoids potential exceptions from split and int.
|
||||
if PYGMENTS_INSTALLED:
|
||||
if pygments.__version__.startswith('2.3.'):
|
||||
PYGMENT_COLOR = {
|
||||
'black': '#ansiblack',
|
||||
'red': '#ansidarkred',
|
||||
'green': '#ansidarkgreen',
|
||||
'yellow': '#ansibrown',
|
||||
'blue': '#ansidarkblue',
|
||||
'magenta': '#ansipurple',
|
||||
'cyan': '#ansiteal',
|
||||
'gray': '#ansilightgray',
|
||||
'brightblack': '#ansidarkgray',
|
||||
'brightred': '#ansired',
|
||||
'brightgreen': '#ansigreen',
|
||||
'brightyellow': '#ansiyellow',
|
||||
'brightblue': '#ansiblue',
|
||||
'brightmagenta': '#ansifuchsia',
|
||||
'brightcyan': '#ansiturquoise',
|
||||
'white': '#ansiwhite',
|
||||
}
|
||||
PYGMENT_COLOR = old_pygments_colors
|
||||
else:
|
||||
PYGMENT_COLOR = {
|
||||
'black': 'ansiblack',
|
||||
'red': 'ansired',
|
||||
'green': 'ansigreen',
|
||||
'yellow': 'ansiyellow',
|
||||
'blue': 'ansiblue',
|
||||
'magenta': 'ansimagenta',
|
||||
'cyan': 'ansicyan',
|
||||
'gray': 'ansigray',
|
||||
'brightblack': 'ansibrightblack',
|
||||
'brightred': 'ansibrightred',
|
||||
'brightgreen': 'ansibrightgreen',
|
||||
'brightyellow': 'ansibrightyellow',
|
||||
'brightblue': 'ansibrightblue',
|
||||
'brightmagenta': 'ansibrightmagenta',
|
||||
'brightcyan': 'ansibrightcyan',
|
||||
'white': 'ansiwhite',
|
||||
}
|
||||
|
||||
|
||||
def safe_print_json(string, pretty=None, env_colors=None, mono=None,
|
||||
piped_out=None, flush=None):
|
||||
"""Safely prints JSON output in both UTF-8 and ASCII systems"""
|
||||
try:
|
||||
print(json_out(string,
|
||||
pretty=pretty,
|
||||
env_colors=env_colors,
|
||||
mono=mono,
|
||||
piped_out=piped_out),
|
||||
flush=flush)
|
||||
except UnicodeEncodeError:
|
||||
print(json_out(string,
|
||||
pretty=pretty,
|
||||
env_colors=env_colors,
|
||||
mono=mono,
|
||||
piped_out=piped_out,
|
||||
ascii_only=True),
|
||||
flush=flush)
|
||||
PYGMENT_COLOR = new_pygments_colors
|
||||
|
||||
|
||||
def set_env_colors(env_colors=None):
|
||||
@@ -154,7 +103,7 @@ def set_env_colors(env_colors=None):
|
||||
|
||||
def piped_output(force_color):
|
||||
"""
|
||||
Return False if stdout is a TTY. True if output is being piped to
|
||||
Return False if `STDOUT` is a TTY. True if output is being piped to
|
||||
another program and foce_color is True. This allows forcing of ANSI
|
||||
color codes even when using pipes.
|
||||
"""
|
||||
@@ -186,6 +135,22 @@ def parsers_text(indent=0, pad=0):
|
||||
return ptext
|
||||
|
||||
|
||||
def options_text(indent=0, pad=0):
|
||||
"""Return the argument and description information from each option"""
|
||||
otext = ''
|
||||
padding_char = ' '
|
||||
for option in long_options_map:
|
||||
o_short = '-' + long_options_map[option][0]
|
||||
o_desc = long_options_map[option][1]
|
||||
o_combined = o_short + ', ' + option
|
||||
padding = pad - len(o_combined)
|
||||
indent_text = padding_char * indent
|
||||
padding_text = padding_char * padding
|
||||
otext += indent_text + o_combined + padding_text + o_desc + '\n'
|
||||
|
||||
return otext
|
||||
|
||||
|
||||
def about_jc():
|
||||
"""Return jc info and the contents of each parser.info as a dictionary"""
|
||||
return {
|
||||
@@ -200,48 +165,44 @@ def about_jc():
|
||||
'python_version': '.'.join((str(sys.version_info.major), str(sys.version_info.minor), str(sys.version_info.micro))),
|
||||
'python_path': sys.executable,
|
||||
'parser_count': len(all_parser_info()),
|
||||
'standard_parser_count': len(standard_parser_mod_list()),
|
||||
'streaming_parser_count': len(streaming_parser_mod_list()),
|
||||
'plugin_parser_count': len(plugin_parser_mod_list()),
|
||||
'parsers': all_parser_info()
|
||||
}
|
||||
|
||||
|
||||
def helptext():
|
||||
"""Return the help text with the list of parsers"""
|
||||
parsers_string = parsers_text(indent=12, pad=17)
|
||||
parsers_string = parsers_text(indent=4, pad=20)
|
||||
options_string = options_text(indent=4, pad=20)
|
||||
|
||||
helptext_string = f'''\
|
||||
jc converts the output of many commands and file-types to JSON
|
||||
jc converts the output of many commands and file-types to JSON or YAML
|
||||
|
||||
Usage: COMMAND | jc PARSER [OPTIONS]
|
||||
Usage:
|
||||
COMMAND | jc PARSER [OPTIONS]
|
||||
|
||||
or magic syntax:
|
||||
or magic syntax:
|
||||
|
||||
jc [OPTIONS] COMMAND
|
||||
jc [OPTIONS] COMMAND
|
||||
|
||||
Parsers:
|
||||
Parsers:
|
||||
{parsers_string}
|
||||
Options:
|
||||
-a about jc
|
||||
-C force color output even when using pipes (overrides -m)
|
||||
-d debug (-dd for verbose debug)
|
||||
-h help (-h --parser_name for parser documentation)
|
||||
-m monochrome output
|
||||
-p pretty print output
|
||||
-q quiet - suppress parser warnings (-qq to ignore streaming errors)
|
||||
-r raw JSON output
|
||||
-u unbuffer output
|
||||
-v version info
|
||||
Options:
|
||||
{options_string}
|
||||
Examples:
|
||||
Standard Syntax:
|
||||
$ dig www.google.com | jc --dig --pretty
|
||||
|
||||
Examples:
|
||||
Standard Syntax:
|
||||
$ dig www.google.com | jc --dig -p
|
||||
Magic Syntax:
|
||||
$ jc --pretty dig www.google.com
|
||||
|
||||
Magic Syntax:
|
||||
$ jc -p dig www.google.com
|
||||
Parser Documentation:
|
||||
$ jc --help --dig
|
||||
'''
|
||||
|
||||
Parser Documentation:
|
||||
$ jc -h --dig
|
||||
'''
|
||||
return textwrap.dedent(helptext_string)
|
||||
return helptext_string
|
||||
|
||||
|
||||
def help_doc(options):
|
||||
@@ -283,11 +244,52 @@ def versiontext():
|
||||
return textwrap.dedent(versiontext_string)
|
||||
|
||||
|
||||
def yaml_out(data, pretty=False, env_colors=None, mono=False, piped_out=False, ascii_only=False):
|
||||
"""
|
||||
Return a YAML formatted string. String may include color codes. If the
|
||||
YAML library is not installed, output will fall back to JSON with a
|
||||
warning message to STDERR"""
|
||||
# make ruamel.yaml import optional
|
||||
try:
|
||||
from ruamel.yaml import YAML
|
||||
YAML_INSTALLED = True
|
||||
except Exception:
|
||||
YAML_INSTALLED = False
|
||||
|
||||
if YAML_INSTALLED:
|
||||
y_string_buf = io.BytesIO()
|
||||
# monkey patch to disable plugins since we don't use them and in
|
||||
# ruamel.yaml versions prior to 0.17.0 the use of __file__ in the
|
||||
# plugin code is incompatible with the pyoxidizer packager
|
||||
YAML.official_plug_ins = lambda a: []
|
||||
yaml = YAML()
|
||||
yaml.default_flow_style = False
|
||||
yaml.explicit_start = True
|
||||
yaml.allow_unicode = not ascii_only
|
||||
yaml.encoding = 'utf-8'
|
||||
yaml.dump(data, y_string_buf)
|
||||
y_string = y_string_buf.getvalue().decode('utf-8')[:-1]
|
||||
|
||||
if not mono and not piped_out:
|
||||
# set colors
|
||||
class JcStyle(Style):
|
||||
styles = set_env_colors(env_colors)
|
||||
|
||||
return str(highlight(y_string, YamlLexer(), Terminal256Formatter(style=JcStyle))[0:-1])
|
||||
|
||||
return y_string
|
||||
|
||||
utils.warning_message(['YAML Library not installed. Reverting to JSON output.'])
|
||||
return json_out(data, pretty=pretty, env_colors=env_colors, mono=mono, piped_out=piped_out, ascii_only=ascii_only)
|
||||
|
||||
|
||||
def json_out(data, pretty=False, env_colors=None, mono=False, piped_out=False, ascii_only=False):
|
||||
"""
|
||||
Return a JSON formatted string. String may include color codes or be
|
||||
pretty printed.
|
||||
"""
|
||||
import json
|
||||
|
||||
separators = (',', ':')
|
||||
indent = None
|
||||
|
||||
@@ -307,6 +309,44 @@ def json_out(data, pretty=False, env_colors=None, mono=False, piped_out=False, a
|
||||
return j_string
|
||||
|
||||
|
||||
def safe_print_out(list_or_dict, pretty=None, env_colors=None, mono=None,
|
||||
piped_out=None, flush=None, yaml=None):
|
||||
"""Safely prints JSON or YAML output in both UTF-8 and ASCII systems"""
|
||||
if yaml:
|
||||
try:
|
||||
print(yaml_out(list_or_dict,
|
||||
pretty=pretty,
|
||||
env_colors=env_colors,
|
||||
mono=mono,
|
||||
piped_out=piped_out),
|
||||
flush=flush)
|
||||
except UnicodeEncodeError:
|
||||
print(yaml_out(list_or_dict,
|
||||
pretty=pretty,
|
||||
env_colors=env_colors,
|
||||
mono=mono,
|
||||
piped_out=piped_out,
|
||||
ascii_only=True),
|
||||
flush=flush)
|
||||
|
||||
else:
|
||||
try:
|
||||
print(json_out(list_or_dict,
|
||||
pretty=pretty,
|
||||
env_colors=env_colors,
|
||||
mono=mono,
|
||||
piped_out=piped_out),
|
||||
flush=flush)
|
||||
except UnicodeEncodeError:
|
||||
print(json_out(list_or_dict,
|
||||
pretty=pretty,
|
||||
env_colors=env_colors,
|
||||
mono=mono,
|
||||
piped_out=piped_out,
|
||||
ascii_only=True),
|
||||
flush=flush)
|
||||
|
||||
|
||||
def magic_parser(args):
|
||||
"""
|
||||
Parse command arguments for magic syntax: jc -p ls -al
|
||||
@@ -318,7 +358,7 @@ def magic_parser(args):
|
||||
jc_options (list) list of jc options
|
||||
"""
|
||||
# bail immediately if there are no args or a parser is defined
|
||||
if len(args) <= 1 or args[1].startswith('--'):
|
||||
if len(args) <= 1 or (args[1].startswith('--') and args[1] not in long_options_map):
|
||||
return False, None, None, []
|
||||
|
||||
args_given = args[1:]
|
||||
@@ -326,6 +366,12 @@ def magic_parser(args):
|
||||
|
||||
# find the options
|
||||
for arg in list(args_given):
|
||||
# long option found - populate option list
|
||||
if arg in long_options_map:
|
||||
options.extend(long_options_map[arg][0])
|
||||
args_given.pop(0)
|
||||
continue
|
||||
|
||||
# parser found - use standard syntax
|
||||
if arg.startswith('--'):
|
||||
return False, None, None, []
|
||||
@@ -420,6 +466,9 @@ def main():
|
||||
# find options if magic_parser did not find a command
|
||||
if not valid_command:
|
||||
for opt in sys.argv:
|
||||
if opt in long_options_map:
|
||||
options.extend(long_options_map[opt][0])
|
||||
|
||||
if opt.startswith('-') and not opt.startswith('--'):
|
||||
options.extend(opt[1:])
|
||||
|
||||
@@ -435,6 +484,9 @@ def main():
|
||||
raw = 'r' in options
|
||||
unbuffer = 'u' in options
|
||||
version_info = 'v' in options
|
||||
yaml_out = 'y' in options
|
||||
bash_comp = 'B' in options
|
||||
zsh_comp = 'Z' in options
|
||||
|
||||
if verbose_debug:
|
||||
tracebackplus.enable(context=11)
|
||||
@@ -443,11 +495,12 @@ def main():
|
||||
mono = True
|
||||
|
||||
if about:
|
||||
safe_print_json(about_jc(),
|
||||
pretty=pretty,
|
||||
env_colors=jc_colors,
|
||||
mono=mono,
|
||||
piped_out=piped_output(force_color))
|
||||
safe_print_out(about_jc(),
|
||||
pretty=pretty,
|
||||
env_colors=jc_colors,
|
||||
mono=mono,
|
||||
piped_out=piped_output(force_color),
|
||||
yaml=yaml_out)
|
||||
sys.exit(0)
|
||||
|
||||
if help_me:
|
||||
@@ -458,6 +511,14 @@ def main():
|
||||
utils._safe_print(versiontext())
|
||||
sys.exit(0)
|
||||
|
||||
if bash_comp:
|
||||
utils._safe_print(bash_completion())
|
||||
sys.exit(0)
|
||||
|
||||
if zsh_comp:
|
||||
utils._safe_print(zsh_completion())
|
||||
sys.exit(0)
|
||||
|
||||
# if magic syntax used, try to run the command and error if it's not found, etc.
|
||||
magic_stdout, magic_stderr, magic_exit_code = None, None, 0
|
||||
if run_command:
|
||||
@@ -527,35 +588,45 @@ def main():
|
||||
try:
|
||||
# differentiate between regular and streaming parsers
|
||||
|
||||
# streaming
|
||||
# streaming (only supports UTF-8 string data for now)
|
||||
if _parser_is_streaming(parser):
|
||||
result = parser.parse(sys.stdin,
|
||||
raw=raw,
|
||||
quiet=quiet,
|
||||
ignore_exceptions=ignore_exceptions)
|
||||
for line in result:
|
||||
safe_print_json(line,
|
||||
pretty=pretty,
|
||||
env_colors=jc_colors,
|
||||
mono=mono,
|
||||
piped_out=piped_output(force_color),
|
||||
flush=unbuffer)
|
||||
safe_print_out(line,
|
||||
pretty=pretty,
|
||||
env_colors=jc_colors,
|
||||
mono=mono,
|
||||
piped_out=piped_output(force_color),
|
||||
flush=unbuffer,
|
||||
yaml=yaml_out)
|
||||
|
||||
sys.exit(combined_exit_code(magic_exit_code, 0))
|
||||
|
||||
# regular
|
||||
# regular (supports binary and UTF-8 string data)
|
||||
else:
|
||||
data = magic_stdout or sys.stdin.read()
|
||||
data = magic_stdout or sys.stdin.buffer.read()
|
||||
|
||||
# convert to UTF-8, if possible. Otherwise, leave as bytes
|
||||
try:
|
||||
if isinstance(data, bytes):
|
||||
data = data.decode('utf-8')
|
||||
except UnicodeDecodeError:
|
||||
pass
|
||||
|
||||
result = parser.parse(data,
|
||||
raw=raw,
|
||||
quiet=quiet)
|
||||
|
||||
safe_print_json(result,
|
||||
pretty=pretty,
|
||||
env_colors=jc_colors,
|
||||
mono=mono,
|
||||
piped_out=piped_output(force_color),
|
||||
flush=unbuffer)
|
||||
safe_print_out(result,
|
||||
pretty=pretty,
|
||||
env_colors=jc_colors,
|
||||
mono=mono,
|
||||
piped_out=piped_output(force_color),
|
||||
flush=unbuffer,
|
||||
yaml=yaml_out)
|
||||
|
||||
sys.exit(combined_exit_code(magic_exit_code, 0))
|
||||
|
||||
@@ -570,14 +641,6 @@ def main():
|
||||
])
|
||||
sys.exit(combined_exit_code(magic_exit_code, JC_ERROR_EXIT))
|
||||
|
||||
except json.JSONDecodeError:
|
||||
if debug:
|
||||
raise
|
||||
|
||||
utils.error_message(['There was an issue generating the JSON output.',
|
||||
'For details use the -d or -dd option.'])
|
||||
sys.exit(combined_exit_code(magic_exit_code, JC_ERROR_EXIT))
|
||||
|
||||
except Exception:
|
||||
if debug:
|
||||
raise
|
||||
|
||||
56
jc/cli_data.py
Normal file
56
jc/cli_data.py
Normal file
@@ -0,0 +1,56 @@
|
||||
"""jc - JSON Convert cli_data module"""
|
||||
from typing import List, Dict
|
||||
|
||||
long_options_map: Dict[str, List[str]] = {
|
||||
'--about': ['a', 'about jc'],
|
||||
'--force-color': ['C', 'force color output even when using pipes (overrides -m)'],
|
||||
'--debug': ['d', 'debug (double for verbose debug)'],
|
||||
'--help': ['h', 'help (--help --parser_name for parser documentation)'],
|
||||
'--monochrome': ['m', 'monochrome output'],
|
||||
'--pretty': ['p', 'pretty print output'],
|
||||
'--quiet': ['q', 'suppress warnings (double to ignore streaming errors)'],
|
||||
'--raw': ['r', 'raw output'],
|
||||
'--unbuffer': ['u', 'unbuffer output'],
|
||||
'--version': ['v', 'version info'],
|
||||
'--yaml-out': ['y', 'YAML output'],
|
||||
'--bash-comp': ['B', 'gen Bash completion: jc -B > /etc/bash_completion.d/jc'],
|
||||
'--zsh-comp': ['Z', 'gen Zsh completion: jc -Z > "${fpath[1]}/_jc"']
|
||||
}
|
||||
|
||||
new_pygments_colors = {
|
||||
'black': 'ansiblack',
|
||||
'red': 'ansired',
|
||||
'green': 'ansigreen',
|
||||
'yellow': 'ansiyellow',
|
||||
'blue': 'ansiblue',
|
||||
'magenta': 'ansimagenta',
|
||||
'cyan': 'ansicyan',
|
||||
'gray': 'ansigray',
|
||||
'brightblack': 'ansibrightblack',
|
||||
'brightred': 'ansibrightred',
|
||||
'brightgreen': 'ansibrightgreen',
|
||||
'brightyellow': 'ansibrightyellow',
|
||||
'brightblue': 'ansibrightblue',
|
||||
'brightmagenta': 'ansibrightmagenta',
|
||||
'brightcyan': 'ansibrightcyan',
|
||||
'white': 'ansiwhite',
|
||||
}
|
||||
|
||||
old_pygments_colors = {
|
||||
'black': '#ansiblack',
|
||||
'red': '#ansidarkred',
|
||||
'green': '#ansidarkgreen',
|
||||
'yellow': '#ansibrown',
|
||||
'blue': '#ansidarkblue',
|
||||
'magenta': '#ansipurple',
|
||||
'cyan': '#ansiteal',
|
||||
'gray': '#ansilightgray',
|
||||
'brightblack': '#ansidarkgray',
|
||||
'brightred': '#ansired',
|
||||
'brightgreen': '#ansigreen',
|
||||
'brightyellow': '#ansiyellow',
|
||||
'brightblue': '#ansiblue',
|
||||
'brightmagenta': '#ansifuchsia',
|
||||
'brightcyan': '#ansiturquoise',
|
||||
'white': '#ansiwhite',
|
||||
}
|
||||
@@ -6,7 +6,7 @@ import importlib
|
||||
from typing import Dict, List, Iterable, Union, Iterator
|
||||
from jc import appdirs
|
||||
|
||||
__version__ = '1.18.8'
|
||||
__version__ = '1.20.2'
|
||||
|
||||
parsers = [
|
||||
'acpi',
|
||||
@@ -16,6 +16,7 @@ parsers = [
|
||||
'asciitable',
|
||||
'asciitable-m',
|
||||
'blkid',
|
||||
'chage',
|
||||
'cksum',
|
||||
'crontab',
|
||||
'crontab-u',
|
||||
@@ -34,6 +35,8 @@ parsers = [
|
||||
'free',
|
||||
'fstab',
|
||||
'git-log',
|
||||
'git-log-s',
|
||||
'gpg',
|
||||
'group',
|
||||
'gshadow',
|
||||
'hash',
|
||||
@@ -71,6 +74,7 @@ parsers = [
|
||||
'ping-s',
|
||||
'pip-list',
|
||||
'pip-show',
|
||||
'postconf',
|
||||
'ps',
|
||||
'route',
|
||||
'rpm-qi',
|
||||
@@ -89,6 +93,8 @@ parsers = [
|
||||
'systeminfo',
|
||||
'time',
|
||||
'timedatectl',
|
||||
'top',
|
||||
'top-s',
|
||||
'tracepath',
|
||||
'traceroute',
|
||||
'ufw',
|
||||
@@ -103,6 +109,7 @@ parsers = [
|
||||
'w',
|
||||
'wc',
|
||||
'who',
|
||||
'x509-cert',
|
||||
'xml',
|
||||
'xrandr',
|
||||
'yaml',
|
||||
|
||||
@@ -3,7 +3,12 @@
|
||||
This parser converts ASCII and Unicode text tables with single-line rows.
|
||||
|
||||
Column headers must be at least two spaces apart from each other and must
|
||||
be unique.
|
||||
be unique. For best results, column headers should be left-justified. If
|
||||
column separators are present, then non-left-justified headers will be fixed
|
||||
automatically.
|
||||
|
||||
Row separators are optional and are ignored. Each non-row-separator line is
|
||||
considered a separate row in the table.
|
||||
|
||||
For example:
|
||||
|
||||
@@ -49,6 +54,9 @@ etc...
|
||||
Headers (keys) are converted to snake-case. All values are returned as
|
||||
strings, except empty strings, which are converted to None/null.
|
||||
|
||||
> Note: To preserve the case of the keys use the `-r` cli option or
|
||||
> `raw=True` argument in `parse()`.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ cat table.txt | jc --asciitable
|
||||
@@ -117,7 +125,7 @@ from jc.parsers.universal import sparse_table_parse
|
||||
|
||||
class info():
|
||||
"""Provides parser metadata (version, author, etc.)"""
|
||||
version = '1.0'
|
||||
version = '1.2'
|
||||
description = 'ASCII and Unicode table parser'
|
||||
author = 'Kelly Brazil'
|
||||
author_email = 'kellyjonbrazil@gmail.com'
|
||||
@@ -139,6 +147,12 @@ def _process(proc_data: List[Dict]) -> List[Dict]:
|
||||
|
||||
List of Dictionaries. Structured to conform to the schema.
|
||||
"""
|
||||
# normalize keys: convert to lowercase
|
||||
for item in proc_data:
|
||||
for key in item.copy():
|
||||
k_new = key.lower()
|
||||
item[k_new] = item.pop(key)
|
||||
|
||||
return proc_data
|
||||
|
||||
|
||||
@@ -222,18 +236,19 @@ def _is_separator(line: str) -> bool:
|
||||
|
||||
def _snake_case(line: str) -> str:
|
||||
"""
|
||||
replace spaces between words and special characters with an underscore
|
||||
and set to lowercase
|
||||
Replace spaces between words and special characters with an underscore.
|
||||
Ignore the replacement char (�) used for header padding.
|
||||
"""
|
||||
line = re.sub(r'[^a-zA-Z0-9 ]', '_', line)
|
||||
return re.sub(r'\b \b', '_', line).lower()
|
||||
line = re.sub(r'[^a-zA-Z0-9� ]', '_', line) # special characters
|
||||
line = re.sub(r'\b \b', '_', line) # spaces between words
|
||||
return line
|
||||
|
||||
|
||||
def _normalize_rows(table: str) -> List[str]:
|
||||
"""
|
||||
returns a List of row strings. Header is snake-cased
|
||||
"""
|
||||
result = []
|
||||
result: List[str] = []
|
||||
for line in table.splitlines():
|
||||
# skip blank lines
|
||||
if not line.strip():
|
||||
@@ -243,7 +258,36 @@ def _normalize_rows(table: str) -> List[str]:
|
||||
if _is_separator(line):
|
||||
continue
|
||||
|
||||
# data row - remove column separators
|
||||
# header or data row found - remove column separators
|
||||
if not result: # this is the header row
|
||||
# normalize the separator
|
||||
line = line.replace('│', '|')\
|
||||
.replace('┃', '|')\
|
||||
.replace('┆', '|')\
|
||||
.replace('┇', '|')\
|
||||
.replace('┊', '|')\
|
||||
.replace('┋', '|')\
|
||||
.replace('╎', '|')\
|
||||
.replace('╏', '|')\
|
||||
.replace('║', '|')
|
||||
|
||||
# find the number of chars to pad in front of headers that are too
|
||||
# far away from the separator. Replace spaces with unicode char: �
|
||||
# we will remove this char from headers after sparse_table_parse
|
||||
problem_header_pattern = re.compile(r'(?:\| )( +)([^|]+)')
|
||||
problem_headers = problem_header_pattern.findall(line)
|
||||
if problem_headers:
|
||||
for p_header in problem_headers:
|
||||
old_header = p_header[0] + p_header[1]
|
||||
sub_chars = '�' * len(p_header[0])
|
||||
new_header = sub_chars + p_header[1]
|
||||
line = line.replace(old_header, new_header)
|
||||
|
||||
line = line.replace('|', ' ')
|
||||
result.append(_snake_case(line))
|
||||
continue
|
||||
|
||||
# this is a data row
|
||||
line = line.replace('|', ' ')\
|
||||
.replace('│', ' ')\
|
||||
.replace('┃', ' ')\
|
||||
@@ -256,7 +300,6 @@ def _normalize_rows(table: str) -> List[str]:
|
||||
.replace('║', ' ')
|
||||
result.append(line)
|
||||
|
||||
result[0] = _snake_case(result[0])
|
||||
return result
|
||||
|
||||
|
||||
@@ -266,7 +309,8 @@ def _fixup_headers(table: List[Dict]) -> List[Dict]:
|
||||
for row in table:
|
||||
new_row = row.copy()
|
||||
for k in row:
|
||||
k_new = k
|
||||
# remove replacement character
|
||||
k_new = k.replace('�', '')
|
||||
# remove consecutive underscores
|
||||
k_new = re.sub(r'__+', '_', k_new)
|
||||
# remove trailing underscores
|
||||
|
||||
@@ -24,6 +24,15 @@ Headers (keys) are converted to snake-case and newlines between multi-line
|
||||
headers are joined with an underscore. All values are returned as strings,
|
||||
except empty strings, which are converted to None/null.
|
||||
|
||||
> Note: To preserve the case of the keys use the `-r` cli option or
|
||||
> `raw=True` argument in `parse()`.
|
||||
|
||||
> Note: table column separator characters (e.g. `|`) cannot be present
|
||||
> inside the cell data. If detected, a warning message will be printed to
|
||||
> `STDERR` and the line will be skipped. The warning message can be
|
||||
> suppressed by using the `-q` command option or by setting `quiet=True` in
|
||||
> `parse()`.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ cat table.txt | jc --asciitable-m
|
||||
@@ -101,7 +110,7 @@ from jc.exceptions import ParseError
|
||||
|
||||
class info():
|
||||
"""Provides parser metadata (version, author, etc.)"""
|
||||
version = '1.0'
|
||||
version = '1.2'
|
||||
description = 'multi-line ASCII and Unicode table parser'
|
||||
author = 'Kelly Brazil'
|
||||
author_email = 'kellyjonbrazil@gmail.com'
|
||||
@@ -123,6 +132,12 @@ def _process(proc_data: List[Dict]) -> List[Dict]:
|
||||
|
||||
List of Dictionaries. Structured to conform to the schema.
|
||||
"""
|
||||
# normalize keys: convert to lowercase
|
||||
for item in proc_data:
|
||||
for key in item.copy():
|
||||
k_new = key.lower()
|
||||
item[k_new] = item.pop(key)
|
||||
|
||||
return proc_data
|
||||
|
||||
|
||||
@@ -227,12 +242,11 @@ def _is_separator(line: str) -> bool:
|
||||
|
||||
def _snake_case(line: str) -> str:
|
||||
"""
|
||||
replace spaces between words and special characters with an underscore
|
||||
and set to lowercase
|
||||
replace spaces between words and special characters with an underscore.
|
||||
"""
|
||||
# must include all column separator characters in regex
|
||||
line = re.sub(r'[^a-zA-Z0-9 |│┃┆┇┊┋╎╏║]', '_', line)
|
||||
return re.sub(r'\b \b', '_', line).lower()
|
||||
return re.sub(r'\b \b', '_', line)
|
||||
|
||||
|
||||
def _fixup_separators(line: str) -> str:
|
||||
@@ -377,20 +391,27 @@ def _collapse_headers(table: List[List[str]]) -> List[str]:
|
||||
return result
|
||||
|
||||
|
||||
def _collapse_data(table: List[List[List[str]]]) -> List[List[str]]:
|
||||
def _collapse_data(table: List[List[List[str]]], quiet=False) -> List[List[str]]:
|
||||
"""combine data rows to return a simple list of lists"""
|
||||
result: List[List[str]] = []
|
||||
|
||||
for row in table:
|
||||
new_row: List[str] = []
|
||||
for line in row:
|
||||
if new_row:
|
||||
for i, item in enumerate(line):
|
||||
new_row[i] = (new_row[i] + '\n' + item).strip()
|
||||
else:
|
||||
new_row = line
|
||||
for index, row in enumerate(table):
|
||||
try:
|
||||
new_row: List[str] = []
|
||||
for line in row:
|
||||
if new_row:
|
||||
for i, item in enumerate(line):
|
||||
new_row[i] = (new_row[i] + '\n' + item).strip()
|
||||
else:
|
||||
new_row = line
|
||||
|
||||
result.append(new_row)
|
||||
result.append(new_row)
|
||||
except IndexError:
|
||||
if not quiet:
|
||||
row_string = '\n'.join([' | '.join(l) for l in row])
|
||||
jc.utils.warning_message(
|
||||
[f'Possible table separator character found in row {index}: {row_string}. Skipping.']
|
||||
)
|
||||
|
||||
return result
|
||||
|
||||
@@ -409,14 +430,14 @@ def _create_table_dict(header: List[str], data: List[List[str]]) -> List[Dict[st
|
||||
return table_list_dict
|
||||
|
||||
|
||||
def _parse_pretty(string: str) -> List[Dict[str, Optional[str]]]:
|
||||
def _parse_pretty(string: str, quiet: bool =False) -> List[Dict[str, Optional[str]]]:
|
||||
string_lines: List[str] = string.splitlines()
|
||||
clean: List[Tuple[int, List[str]]] = _normalize_rows(string_lines)
|
||||
raw_headers: List[List[str]] = _get_headers(clean)
|
||||
raw_data: List[List[List[str]]] = _get_data(clean)
|
||||
|
||||
new_headers: List[str] = _collapse_headers(raw_headers)
|
||||
new_data: List[List[str]] = _collapse_data(raw_data)
|
||||
new_data: List[List[str]] = _collapse_data(raw_data, quiet)
|
||||
final_table: List[Dict[str, Optional[str]]] = _create_table_dict(new_headers, new_data)
|
||||
|
||||
return final_table
|
||||
@@ -452,7 +473,7 @@ def parse(
|
||||
table_type = _table_sniff(data)
|
||||
|
||||
if table_type == 'pretty':
|
||||
raw_output = _parse_pretty(data)
|
||||
raw_output = _parse_pretty(data, quiet)
|
||||
elif table_type == 'markdown':
|
||||
raise ParseError('Only "pretty" tables supported with multiline. "markdown" table detected. Please try the "asciitable" parser.')
|
||||
else:
|
||||
|
||||
47
jc/parsers/asn1crypto/__init__.py
Normal file
47
jc/parsers/asn1crypto/__init__.py
Normal file
@@ -0,0 +1,47 @@
|
||||
# coding: utf-8
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
from .version import __version__, __version_info__
|
||||
|
||||
__all__ = [
|
||||
'__version__',
|
||||
'__version_info__',
|
||||
'load_order',
|
||||
]
|
||||
|
||||
|
||||
def load_order():
|
||||
"""
|
||||
Returns a list of the module and sub-module names for asn1crypto in
|
||||
dependency load order, for the sake of live reloading code
|
||||
|
||||
:return:
|
||||
A list of unicode strings of module names, as they would appear in
|
||||
sys.modules, ordered by which module should be reloaded first
|
||||
"""
|
||||
|
||||
return [
|
||||
'asn1crypto._errors',
|
||||
'asn1crypto._int',
|
||||
'asn1crypto._ordereddict',
|
||||
'asn1crypto._teletex_codec',
|
||||
'asn1crypto._types',
|
||||
'asn1crypto._inet',
|
||||
'asn1crypto._iri',
|
||||
'asn1crypto.version',
|
||||
'asn1crypto.pem',
|
||||
'asn1crypto.util',
|
||||
'asn1crypto.parser',
|
||||
'asn1crypto.core',
|
||||
'asn1crypto.algos',
|
||||
'asn1crypto.keys',
|
||||
'asn1crypto.x509',
|
||||
'asn1crypto.crl',
|
||||
'asn1crypto.csr',
|
||||
'asn1crypto.ocsp',
|
||||
'asn1crypto.cms',
|
||||
'asn1crypto.pdf',
|
||||
'asn1crypto.pkcs12',
|
||||
'asn1crypto.tsp',
|
||||
'asn1crypto',
|
||||
]
|
||||
54
jc/parsers/asn1crypto/_errors.py
Normal file
54
jc/parsers/asn1crypto/_errors.py
Normal file
@@ -0,0 +1,54 @@
|
||||
# coding: utf-8
|
||||
|
||||
"""
|
||||
Exports the following items:
|
||||
|
||||
- unwrap()
|
||||
- APIException()
|
||||
"""
|
||||
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
import re
|
||||
import textwrap
|
||||
|
||||
|
||||
class APIException(Exception):
|
||||
"""
|
||||
An exception indicating an API has been removed from asn1crypto
|
||||
"""
|
||||
|
||||
pass
|
||||
|
||||
|
||||
def unwrap(string, *params):
|
||||
"""
|
||||
Takes a multi-line string and does the following:
|
||||
|
||||
- dedents
|
||||
- converts newlines with text before and after into a single line
|
||||
- strips leading and trailing whitespace
|
||||
|
||||
:param string:
|
||||
The string to format
|
||||
|
||||
:param *params:
|
||||
Params to interpolate into the string
|
||||
|
||||
:return:
|
||||
The formatted string
|
||||
"""
|
||||
|
||||
output = textwrap.dedent(string)
|
||||
|
||||
# Unwrap lines, taking into account bulleted lists, ordered lists and
|
||||
# underlines consisting of = signs
|
||||
if output.find('\n') != -1:
|
||||
output = re.sub('(?<=\\S)\n(?=[^ \n\t\\d\\*\\-=])', ' ', output)
|
||||
|
||||
if params:
|
||||
output = output % params
|
||||
|
||||
output = output.strip()
|
||||
|
||||
return output
|
||||
170
jc/parsers/asn1crypto/_inet.py
Normal file
170
jc/parsers/asn1crypto/_inet.py
Normal file
@@ -0,0 +1,170 @@
|
||||
# coding: utf-8
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
import socket
|
||||
import struct
|
||||
|
||||
from ._errors import unwrap
|
||||
from ._types import byte_cls, bytes_to_list, str_cls, type_name
|
||||
|
||||
|
||||
def inet_ntop(address_family, packed_ip):
|
||||
"""
|
||||
Windows compatibility shim for socket.inet_ntop().
|
||||
|
||||
:param address_family:
|
||||
socket.AF_INET for IPv4 or socket.AF_INET6 for IPv6
|
||||
|
||||
:param packed_ip:
|
||||
A byte string of the network form of an IP address
|
||||
|
||||
:return:
|
||||
A unicode string of the IP address
|
||||
"""
|
||||
|
||||
if address_family not in set([socket.AF_INET, socket.AF_INET6]):
|
||||
raise ValueError(unwrap(
|
||||
'''
|
||||
address_family must be socket.AF_INET (%s) or socket.AF_INET6 (%s),
|
||||
not %s
|
||||
''',
|
||||
repr(socket.AF_INET),
|
||||
repr(socket.AF_INET6),
|
||||
repr(address_family)
|
||||
))
|
||||
|
||||
if not isinstance(packed_ip, byte_cls):
|
||||
raise TypeError(unwrap(
|
||||
'''
|
||||
packed_ip must be a byte string, not %s
|
||||
''',
|
||||
type_name(packed_ip)
|
||||
))
|
||||
|
||||
required_len = 4 if address_family == socket.AF_INET else 16
|
||||
if len(packed_ip) != required_len:
|
||||
raise ValueError(unwrap(
|
||||
'''
|
||||
packed_ip must be %d bytes long - is %d
|
||||
''',
|
||||
required_len,
|
||||
len(packed_ip)
|
||||
))
|
||||
|
||||
if address_family == socket.AF_INET:
|
||||
return '%d.%d.%d.%d' % tuple(bytes_to_list(packed_ip))
|
||||
|
||||
octets = struct.unpack(b'!HHHHHHHH', packed_ip)
|
||||
|
||||
runs_of_zero = {}
|
||||
longest_run = 0
|
||||
zero_index = None
|
||||
for i, octet in enumerate(octets + (-1,)):
|
||||
if octet != 0:
|
||||
if zero_index is not None:
|
||||
length = i - zero_index
|
||||
if length not in runs_of_zero:
|
||||
runs_of_zero[length] = zero_index
|
||||
longest_run = max(longest_run, length)
|
||||
zero_index = None
|
||||
elif zero_index is None:
|
||||
zero_index = i
|
||||
|
||||
hexed = [hex(o)[2:] for o in octets]
|
||||
|
||||
if longest_run < 2:
|
||||
return ':'.join(hexed)
|
||||
|
||||
zero_start = runs_of_zero[longest_run]
|
||||
zero_end = zero_start + longest_run
|
||||
|
||||
return ':'.join(hexed[:zero_start]) + '::' + ':'.join(hexed[zero_end:])
|
||||
|
||||
|
||||
def inet_pton(address_family, ip_string):
|
||||
"""
|
||||
Windows compatibility shim for socket.inet_ntop().
|
||||
|
||||
:param address_family:
|
||||
socket.AF_INET for IPv4 or socket.AF_INET6 for IPv6
|
||||
|
||||
:param ip_string:
|
||||
A unicode string of an IP address
|
||||
|
||||
:return:
|
||||
A byte string of the network form of the IP address
|
||||
"""
|
||||
|
||||
if address_family not in set([socket.AF_INET, socket.AF_INET6]):
|
||||
raise ValueError(unwrap(
|
||||
'''
|
||||
address_family must be socket.AF_INET (%s) or socket.AF_INET6 (%s),
|
||||
not %s
|
||||
''',
|
||||
repr(socket.AF_INET),
|
||||
repr(socket.AF_INET6),
|
||||
repr(address_family)
|
||||
))
|
||||
|
||||
if not isinstance(ip_string, str_cls):
|
||||
raise TypeError(unwrap(
|
||||
'''
|
||||
ip_string must be a unicode string, not %s
|
||||
''',
|
||||
type_name(ip_string)
|
||||
))
|
||||
|
||||
if address_family == socket.AF_INET:
|
||||
octets = ip_string.split('.')
|
||||
error = len(octets) != 4
|
||||
if not error:
|
||||
ints = []
|
||||
for o in octets:
|
||||
o = int(o)
|
||||
if o > 255 or o < 0:
|
||||
error = True
|
||||
break
|
||||
ints.append(o)
|
||||
|
||||
if error:
|
||||
raise ValueError(unwrap(
|
||||
'''
|
||||
ip_string must be a dotted string with four integers in the
|
||||
range of 0 to 255, got %s
|
||||
''',
|
||||
repr(ip_string)
|
||||
))
|
||||
|
||||
return struct.pack(b'!BBBB', *ints)
|
||||
|
||||
error = False
|
||||
omitted = ip_string.count('::')
|
||||
if omitted > 1:
|
||||
error = True
|
||||
elif omitted == 0:
|
||||
octets = ip_string.split(':')
|
||||
error = len(octets) != 8
|
||||
else:
|
||||
begin, end = ip_string.split('::')
|
||||
begin_octets = begin.split(':')
|
||||
end_octets = end.split(':')
|
||||
missing = 8 - len(begin_octets) - len(end_octets)
|
||||
octets = begin_octets + (['0'] * missing) + end_octets
|
||||
|
||||
if not error:
|
||||
ints = []
|
||||
for o in octets:
|
||||
o = int(o, 16)
|
||||
if o > 65535 or o < 0:
|
||||
error = True
|
||||
break
|
||||
ints.append(o)
|
||||
|
||||
return struct.pack(b'!HHHHHHHH', *ints)
|
||||
|
||||
raise ValueError(unwrap(
|
||||
'''
|
||||
ip_string must be a valid ipv6 string, got %s
|
||||
''',
|
||||
repr(ip_string)
|
||||
))
|
||||
22
jc/parsers/asn1crypto/_int.py
Normal file
22
jc/parsers/asn1crypto/_int.py
Normal file
@@ -0,0 +1,22 @@
|
||||
# coding: utf-8
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
|
||||
def fill_width(bytes_, width):
|
||||
"""
|
||||
Ensure a byte string representing a positive integer is a specific width
|
||||
(in bytes)
|
||||
|
||||
:param bytes_:
|
||||
The integer byte string
|
||||
|
||||
:param width:
|
||||
The desired width as an integer
|
||||
|
||||
:return:
|
||||
A byte string of the width specified
|
||||
"""
|
||||
|
||||
while len(bytes_) < width:
|
||||
bytes_ = b'\x00' + bytes_
|
||||
return bytes_
|
||||
291
jc/parsers/asn1crypto/_iri.py
Normal file
291
jc/parsers/asn1crypto/_iri.py
Normal file
@@ -0,0 +1,291 @@
|
||||
# coding: utf-8
|
||||
|
||||
"""
|
||||
Functions to convert unicode IRIs into ASCII byte string URIs and back. Exports
|
||||
the following items:
|
||||
|
||||
- iri_to_uri()
|
||||
- uri_to_iri()
|
||||
"""
|
||||
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
from encodings import idna # noqa
|
||||
import codecs
|
||||
import re
|
||||
import sys
|
||||
|
||||
from ._errors import unwrap
|
||||
from ._types import byte_cls, str_cls, type_name, bytes_to_list, int_types
|
||||
|
||||
if sys.version_info < (3,):
|
||||
from urlparse import urlsplit, urlunsplit
|
||||
from urllib import (
|
||||
quote as urlquote,
|
||||
unquote as unquote_to_bytes,
|
||||
)
|
||||
|
||||
else:
|
||||
from urllib.parse import (
|
||||
quote as urlquote,
|
||||
unquote_to_bytes,
|
||||
urlsplit,
|
||||
urlunsplit,
|
||||
)
|
||||
|
||||
|
||||
def iri_to_uri(value, normalize=False):
|
||||
"""
|
||||
Encodes a unicode IRI into an ASCII byte string URI
|
||||
|
||||
:param value:
|
||||
A unicode string of an IRI
|
||||
|
||||
:param normalize:
|
||||
A bool that controls URI normalization
|
||||
|
||||
:return:
|
||||
A byte string of the ASCII-encoded URI
|
||||
"""
|
||||
|
||||
if not isinstance(value, str_cls):
|
||||
raise TypeError(unwrap(
|
||||
'''
|
||||
value must be a unicode string, not %s
|
||||
''',
|
||||
type_name(value)
|
||||
))
|
||||
|
||||
scheme = None
|
||||
# Python 2.6 doesn't split properly is the URL doesn't start with http:// or https://
|
||||
if sys.version_info < (2, 7) and not value.startswith('http://') and not value.startswith('https://'):
|
||||
real_prefix = None
|
||||
prefix_match = re.match('^[^:]*://', value)
|
||||
if prefix_match:
|
||||
real_prefix = prefix_match.group(0)
|
||||
value = 'http://' + value[len(real_prefix):]
|
||||
parsed = urlsplit(value)
|
||||
if real_prefix:
|
||||
value = real_prefix + value[7:]
|
||||
scheme = _urlquote(real_prefix[:-3])
|
||||
else:
|
||||
parsed = urlsplit(value)
|
||||
|
||||
if scheme is None:
|
||||
scheme = _urlquote(parsed.scheme)
|
||||
hostname = parsed.hostname
|
||||
if hostname is not None:
|
||||
hostname = hostname.encode('idna')
|
||||
# RFC 3986 allows userinfo to contain sub-delims
|
||||
username = _urlquote(parsed.username, safe='!$&\'()*+,;=')
|
||||
password = _urlquote(parsed.password, safe='!$&\'()*+,;=')
|
||||
port = parsed.port
|
||||
if port is not None:
|
||||
port = str_cls(port).encode('ascii')
|
||||
|
||||
netloc = b''
|
||||
if username is not None:
|
||||
netloc += username
|
||||
if password:
|
||||
netloc += b':' + password
|
||||
netloc += b'@'
|
||||
if hostname is not None:
|
||||
netloc += hostname
|
||||
if port is not None:
|
||||
default_http = scheme == b'http' and port == b'80'
|
||||
default_https = scheme == b'https' and port == b'443'
|
||||
if not normalize or (not default_http and not default_https):
|
||||
netloc += b':' + port
|
||||
|
||||
# RFC 3986 allows a path to contain sub-delims, plus "@" and ":"
|
||||
path = _urlquote(parsed.path, safe='/!$&\'()*+,;=@:')
|
||||
# RFC 3986 allows the query to contain sub-delims, plus "@", ":" , "/" and "?"
|
||||
query = _urlquote(parsed.query, safe='/?!$&\'()*+,;=@:')
|
||||
# RFC 3986 allows the fragment to contain sub-delims, plus "@", ":" , "/" and "?"
|
||||
fragment = _urlquote(parsed.fragment, safe='/?!$&\'()*+,;=@:')
|
||||
|
||||
if normalize and query is None and fragment is None and path == b'/':
|
||||
path = None
|
||||
|
||||
# Python 2.7 compat
|
||||
if path is None:
|
||||
path = ''
|
||||
|
||||
output = urlunsplit((scheme, netloc, path, query, fragment))
|
||||
if isinstance(output, str_cls):
|
||||
output = output.encode('latin1')
|
||||
return output
|
||||
|
||||
|
||||
def uri_to_iri(value):
|
||||
"""
|
||||
Converts an ASCII URI byte string into a unicode IRI
|
||||
|
||||
:param value:
|
||||
An ASCII-encoded byte string of the URI
|
||||
|
||||
:return:
|
||||
A unicode string of the IRI
|
||||
"""
|
||||
|
||||
if not isinstance(value, byte_cls):
|
||||
raise TypeError(unwrap(
|
||||
'''
|
||||
value must be a byte string, not %s
|
||||
''',
|
||||
type_name(value)
|
||||
))
|
||||
|
||||
parsed = urlsplit(value)
|
||||
|
||||
scheme = parsed.scheme
|
||||
if scheme is not None:
|
||||
scheme = scheme.decode('ascii')
|
||||
|
||||
username = _urlunquote(parsed.username, remap=[':', '@'])
|
||||
password = _urlunquote(parsed.password, remap=[':', '@'])
|
||||
hostname = parsed.hostname
|
||||
if hostname:
|
||||
hostname = hostname.decode('idna')
|
||||
port = parsed.port
|
||||
if port and not isinstance(port, int_types):
|
||||
port = port.decode('ascii')
|
||||
|
||||
netloc = ''
|
||||
if username is not None:
|
||||
netloc += username
|
||||
if password:
|
||||
netloc += ':' + password
|
||||
netloc += '@'
|
||||
if hostname is not None:
|
||||
netloc += hostname
|
||||
if port is not None:
|
||||
netloc += ':' + str_cls(port)
|
||||
|
||||
path = _urlunquote(parsed.path, remap=['/'], preserve=True)
|
||||
query = _urlunquote(parsed.query, remap=['&', '='], preserve=True)
|
||||
fragment = _urlunquote(parsed.fragment)
|
||||
|
||||
return urlunsplit((scheme, netloc, path, query, fragment))
|
||||
|
||||
|
||||
def _iri_utf8_errors_handler(exc):
|
||||
"""
|
||||
Error handler for decoding UTF-8 parts of a URI into an IRI. Leaves byte
|
||||
sequences encoded in %XX format, but as part of a unicode string.
|
||||
|
||||
:param exc:
|
||||
The UnicodeDecodeError exception
|
||||
|
||||
:return:
|
||||
A 2-element tuple of (replacement unicode string, integer index to
|
||||
resume at)
|
||||
"""
|
||||
|
||||
bytes_as_ints = bytes_to_list(exc.object[exc.start:exc.end])
|
||||
replacements = ['%%%02x' % num for num in bytes_as_ints]
|
||||
return (''.join(replacements), exc.end)
|
||||
|
||||
|
||||
codecs.register_error('iriutf8', _iri_utf8_errors_handler)
|
||||
|
||||
|
||||
def _urlquote(string, safe=''):
|
||||
"""
|
||||
Quotes a unicode string for use in a URL
|
||||
|
||||
:param string:
|
||||
A unicode string
|
||||
|
||||
:param safe:
|
||||
A unicode string of character to not encode
|
||||
|
||||
:return:
|
||||
None (if string is None) or an ASCII byte string of the quoted string
|
||||
"""
|
||||
|
||||
if string is None or string == '':
|
||||
return None
|
||||
|
||||
# Anything already hex quoted is pulled out of the URL and unquoted if
|
||||
# possible
|
||||
escapes = []
|
||||
if re.search('%[0-9a-fA-F]{2}', string):
|
||||
# Try to unquote any percent values, restoring them if they are not
|
||||
# valid UTF-8. Also, requote any safe chars since encoded versions of
|
||||
# those are functionally different than the unquoted ones.
|
||||
def _try_unescape(match):
|
||||
byte_string = unquote_to_bytes(match.group(0))
|
||||
unicode_string = byte_string.decode('utf-8', 'iriutf8')
|
||||
for safe_char in list(safe):
|
||||
unicode_string = unicode_string.replace(safe_char, '%%%02x' % ord(safe_char))
|
||||
return unicode_string
|
||||
string = re.sub('(?:%[0-9a-fA-F]{2})+', _try_unescape, string)
|
||||
|
||||
# Once we have the minimal set of hex quoted values, removed them from
|
||||
# the string so that they are not double quoted
|
||||
def _extract_escape(match):
|
||||
escapes.append(match.group(0).encode('ascii'))
|
||||
return '\x00'
|
||||
string = re.sub('%[0-9a-fA-F]{2}', _extract_escape, string)
|
||||
|
||||
output = urlquote(string.encode('utf-8'), safe=safe.encode('utf-8'))
|
||||
if not isinstance(output, byte_cls):
|
||||
output = output.encode('ascii')
|
||||
|
||||
# Restore the existing quoted values that we extracted
|
||||
if len(escapes) > 0:
|
||||
def _return_escape(_):
|
||||
return escapes.pop(0)
|
||||
output = re.sub(b'%00', _return_escape, output)
|
||||
|
||||
return output
|
||||
|
||||
|
||||
def _urlunquote(byte_string, remap=None, preserve=None):
|
||||
"""
|
||||
Unquotes a URI portion from a byte string into unicode using UTF-8
|
||||
|
||||
:param byte_string:
|
||||
A byte string of the data to unquote
|
||||
|
||||
:param remap:
|
||||
A list of characters (as unicode) that should be re-mapped to a
|
||||
%XX encoding. This is used when characters are not valid in part of a
|
||||
URL.
|
||||
|
||||
:param preserve:
|
||||
A bool - indicates that the chars to be remapped if they occur in
|
||||
non-hex form, should be preserved. E.g. / for URL path.
|
||||
|
||||
:return:
|
||||
A unicode string
|
||||
"""
|
||||
|
||||
if byte_string is None:
|
||||
return byte_string
|
||||
|
||||
if byte_string == b'':
|
||||
return ''
|
||||
|
||||
if preserve:
|
||||
replacements = ['\x1A', '\x1C', '\x1D', '\x1E', '\x1F']
|
||||
preserve_unmap = {}
|
||||
for char in remap:
|
||||
replacement = replacements.pop(0)
|
||||
preserve_unmap[replacement] = char
|
||||
byte_string = byte_string.replace(char.encode('ascii'), replacement.encode('ascii'))
|
||||
|
||||
byte_string = unquote_to_bytes(byte_string)
|
||||
|
||||
if remap:
|
||||
for char in remap:
|
||||
byte_string = byte_string.replace(char.encode('ascii'), ('%%%02x' % ord(char)).encode('ascii'))
|
||||
|
||||
output = byte_string.decode('utf-8', 'iriutf8')
|
||||
|
||||
if preserve:
|
||||
for replacement, original in preserve_unmap.items():
|
||||
output = output.replace(replacement, original)
|
||||
|
||||
return output
|
||||
135
jc/parsers/asn1crypto/_ordereddict.py
Normal file
135
jc/parsers/asn1crypto/_ordereddict.py
Normal file
@@ -0,0 +1,135 @@
|
||||
# Copyright (c) 2009 Raymond Hettinger
|
||||
#
|
||||
# Permission is hereby granted, free of charge, to any person
|
||||
# obtaining a copy of this software and associated documentation files
|
||||
# (the "Software"), to deal in the Software without restriction,
|
||||
# including without limitation the rights to use, copy, modify, merge,
|
||||
# publish, distribute, sublicense, and/or sell copies of the Software,
|
||||
# and to permit persons to whom the Software is furnished to do so,
|
||||
# subject to the following conditions:
|
||||
#
|
||||
# The above copyright notice and this permission notice shall be
|
||||
# included in all copies or substantial portions of the Software.
|
||||
#
|
||||
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
||||
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
|
||||
# OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
||||
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
|
||||
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
|
||||
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
|
||||
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
||||
# OTHER DEALINGS IN THE SOFTWARE.
|
||||
|
||||
import sys
|
||||
|
||||
if not sys.version_info < (2, 7):
|
||||
|
||||
from collections import OrderedDict
|
||||
|
||||
else:
|
||||
|
||||
from UserDict import DictMixin
|
||||
|
||||
class OrderedDict(dict, DictMixin):
|
||||
|
||||
def __init__(self, *args, **kwds):
|
||||
if len(args) > 1:
|
||||
raise TypeError('expected at most 1 arguments, got %d' % len(args))
|
||||
try:
|
||||
self.__end
|
||||
except AttributeError:
|
||||
self.clear()
|
||||
self.update(*args, **kwds)
|
||||
|
||||
def clear(self):
|
||||
self.__end = end = []
|
||||
end += [None, end, end] # sentinel node for doubly linked list
|
||||
self.__map = {} # key --> [key, prev, next]
|
||||
dict.clear(self)
|
||||
|
||||
def __setitem__(self, key, value):
|
||||
if key not in self:
|
||||
end = self.__end
|
||||
curr = end[1]
|
||||
curr[2] = end[1] = self.__map[key] = [key, curr, end]
|
||||
dict.__setitem__(self, key, value)
|
||||
|
||||
def __delitem__(self, key):
|
||||
dict.__delitem__(self, key)
|
||||
key, prev, next_ = self.__map.pop(key)
|
||||
prev[2] = next_
|
||||
next_[1] = prev
|
||||
|
||||
def __iter__(self):
|
||||
end = self.__end
|
||||
curr = end[2]
|
||||
while curr is not end:
|
||||
yield curr[0]
|
||||
curr = curr[2]
|
||||
|
||||
def __reversed__(self):
|
||||
end = self.__end
|
||||
curr = end[1]
|
||||
while curr is not end:
|
||||
yield curr[0]
|
||||
curr = curr[1]
|
||||
|
||||
def popitem(self, last=True):
|
||||
if not self:
|
||||
raise KeyError('dictionary is empty')
|
||||
if last:
|
||||
key = reversed(self).next()
|
||||
else:
|
||||
key = iter(self).next()
|
||||
value = self.pop(key)
|
||||
return key, value
|
||||
|
||||
def __reduce__(self):
|
||||
items = [[k, self[k]] for k in self]
|
||||
tmp = self.__map, self.__end
|
||||
del self.__map, self.__end
|
||||
inst_dict = vars(self).copy()
|
||||
self.__map, self.__end = tmp
|
||||
if inst_dict:
|
||||
return (self.__class__, (items,), inst_dict)
|
||||
return self.__class__, (items,)
|
||||
|
||||
def keys(self):
|
||||
return list(self)
|
||||
|
||||
setdefault = DictMixin.setdefault
|
||||
update = DictMixin.update
|
||||
pop = DictMixin.pop
|
||||
values = DictMixin.values
|
||||
items = DictMixin.items
|
||||
iterkeys = DictMixin.iterkeys
|
||||
itervalues = DictMixin.itervalues
|
||||
iteritems = DictMixin.iteritems
|
||||
|
||||
def __repr__(self):
|
||||
if not self:
|
||||
return '%s()' % (self.__class__.__name__,)
|
||||
return '%s(%r)' % (self.__class__.__name__, self.items())
|
||||
|
||||
def copy(self):
|
||||
return self.__class__(self)
|
||||
|
||||
@classmethod
|
||||
def fromkeys(cls, iterable, value=None):
|
||||
d = cls()
|
||||
for key in iterable:
|
||||
d[key] = value
|
||||
return d
|
||||
|
||||
def __eq__(self, other):
|
||||
if isinstance(other, OrderedDict):
|
||||
if len(self) != len(other):
|
||||
return False
|
||||
for p, q in zip(self.items(), other.items()):
|
||||
if p != q:
|
||||
return False
|
||||
return True
|
||||
return dict.__eq__(self, other)
|
||||
|
||||
def __ne__(self, other):
|
||||
return not self == other
|
||||
331
jc/parsers/asn1crypto/_teletex_codec.py
Normal file
331
jc/parsers/asn1crypto/_teletex_codec.py
Normal file
@@ -0,0 +1,331 @@
|
||||
# coding: utf-8
|
||||
|
||||
"""
|
||||
Implementation of the teletex T.61 codec. Exports the following items:
|
||||
|
||||
- register()
|
||||
"""
|
||||
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
import codecs
|
||||
|
||||
|
||||
class TeletexCodec(codecs.Codec):
|
||||
|
||||
def encode(self, input_, errors='strict'):
|
||||
return codecs.charmap_encode(input_, errors, ENCODING_TABLE)
|
||||
|
||||
def decode(self, input_, errors='strict'):
|
||||
return codecs.charmap_decode(input_, errors, DECODING_TABLE)
|
||||
|
||||
|
||||
class TeletexIncrementalEncoder(codecs.IncrementalEncoder):
|
||||
|
||||
def encode(self, input_, final=False):
|
||||
return codecs.charmap_encode(input_, self.errors, ENCODING_TABLE)[0]
|
||||
|
||||
|
||||
class TeletexIncrementalDecoder(codecs.IncrementalDecoder):
|
||||
|
||||
def decode(self, input_, final=False):
|
||||
return codecs.charmap_decode(input_, self.errors, DECODING_TABLE)[0]
|
||||
|
||||
|
||||
class TeletexStreamWriter(TeletexCodec, codecs.StreamWriter):
|
||||
|
||||
pass
|
||||
|
||||
|
||||
class TeletexStreamReader(TeletexCodec, codecs.StreamReader):
|
||||
|
||||
pass
|
||||
|
||||
|
||||
def teletex_search_function(name):
|
||||
"""
|
||||
Search function for teletex codec that is passed to codecs.register()
|
||||
"""
|
||||
|
||||
if name != 'teletex':
|
||||
return None
|
||||
|
||||
return codecs.CodecInfo(
|
||||
name='teletex',
|
||||
encode=TeletexCodec().encode,
|
||||
decode=TeletexCodec().decode,
|
||||
incrementalencoder=TeletexIncrementalEncoder,
|
||||
incrementaldecoder=TeletexIncrementalDecoder,
|
||||
streamreader=TeletexStreamReader,
|
||||
streamwriter=TeletexStreamWriter,
|
||||
)
|
||||
|
||||
|
||||
def register():
|
||||
"""
|
||||
Registers the teletex codec
|
||||
"""
|
||||
|
||||
codecs.register(teletex_search_function)
|
||||
|
||||
|
||||
# http://en.wikipedia.org/wiki/ITU_T.61
|
||||
DECODING_TABLE = (
|
||||
'\u0000'
|
||||
'\u0001'
|
||||
'\u0002'
|
||||
'\u0003'
|
||||
'\u0004'
|
||||
'\u0005'
|
||||
'\u0006'
|
||||
'\u0007'
|
||||
'\u0008'
|
||||
'\u0009'
|
||||
'\u000A'
|
||||
'\u000B'
|
||||
'\u000C'
|
||||
'\u000D'
|
||||
'\u000E'
|
||||
'\u000F'
|
||||
'\u0010'
|
||||
'\u0011'
|
||||
'\u0012'
|
||||
'\u0013'
|
||||
'\u0014'
|
||||
'\u0015'
|
||||
'\u0016'
|
||||
'\u0017'
|
||||
'\u0018'
|
||||
'\u0019'
|
||||
'\u001A'
|
||||
'\u001B'
|
||||
'\u001C'
|
||||
'\u001D'
|
||||
'\u001E'
|
||||
'\u001F'
|
||||
'\u0020'
|
||||
'\u0021'
|
||||
'\u0022'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\u0025'
|
||||
'\u0026'
|
||||
'\u0027'
|
||||
'\u0028'
|
||||
'\u0029'
|
||||
'\u002A'
|
||||
'\u002B'
|
||||
'\u002C'
|
||||
'\u002D'
|
||||
'\u002E'
|
||||
'\u002F'
|
||||
'\u0030'
|
||||
'\u0031'
|
||||
'\u0032'
|
||||
'\u0033'
|
||||
'\u0034'
|
||||
'\u0035'
|
||||
'\u0036'
|
||||
'\u0037'
|
||||
'\u0038'
|
||||
'\u0039'
|
||||
'\u003A'
|
||||
'\u003B'
|
||||
'\u003C'
|
||||
'\u003D'
|
||||
'\u003E'
|
||||
'\u003F'
|
||||
'\u0040'
|
||||
'\u0041'
|
||||
'\u0042'
|
||||
'\u0043'
|
||||
'\u0044'
|
||||
'\u0045'
|
||||
'\u0046'
|
||||
'\u0047'
|
||||
'\u0048'
|
||||
'\u0049'
|
||||
'\u004A'
|
||||
'\u004B'
|
||||
'\u004C'
|
||||
'\u004D'
|
||||
'\u004E'
|
||||
'\u004F'
|
||||
'\u0050'
|
||||
'\u0051'
|
||||
'\u0052'
|
||||
'\u0053'
|
||||
'\u0054'
|
||||
'\u0055'
|
||||
'\u0056'
|
||||
'\u0057'
|
||||
'\u0058'
|
||||
'\u0059'
|
||||
'\u005A'
|
||||
'\u005B'
|
||||
'\ufffe'
|
||||
'\u005D'
|
||||
'\ufffe'
|
||||
'\u005F'
|
||||
'\ufffe'
|
||||
'\u0061'
|
||||
'\u0062'
|
||||
'\u0063'
|
||||
'\u0064'
|
||||
'\u0065'
|
||||
'\u0066'
|
||||
'\u0067'
|
||||
'\u0068'
|
||||
'\u0069'
|
||||
'\u006A'
|
||||
'\u006B'
|
||||
'\u006C'
|
||||
'\u006D'
|
||||
'\u006E'
|
||||
'\u006F'
|
||||
'\u0070'
|
||||
'\u0071'
|
||||
'\u0072'
|
||||
'\u0073'
|
||||
'\u0074'
|
||||
'\u0075'
|
||||
'\u0076'
|
||||
'\u0077'
|
||||
'\u0078'
|
||||
'\u0079'
|
||||
'\u007A'
|
||||
'\ufffe'
|
||||
'\u007C'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\u007F'
|
||||
'\u0080'
|
||||
'\u0081'
|
||||
'\u0082'
|
||||
'\u0083'
|
||||
'\u0084'
|
||||
'\u0085'
|
||||
'\u0086'
|
||||
'\u0087'
|
||||
'\u0088'
|
||||
'\u0089'
|
||||
'\u008A'
|
||||
'\u008B'
|
||||
'\u008C'
|
||||
'\u008D'
|
||||
'\u008E'
|
||||
'\u008F'
|
||||
'\u0090'
|
||||
'\u0091'
|
||||
'\u0092'
|
||||
'\u0093'
|
||||
'\u0094'
|
||||
'\u0095'
|
||||
'\u0096'
|
||||
'\u0097'
|
||||
'\u0098'
|
||||
'\u0099'
|
||||
'\u009A'
|
||||
'\u009B'
|
||||
'\u009C'
|
||||
'\u009D'
|
||||
'\u009E'
|
||||
'\u009F'
|
||||
'\u00A0'
|
||||
'\u00A1'
|
||||
'\u00A2'
|
||||
'\u00A3'
|
||||
'\u0024'
|
||||
'\u00A5'
|
||||
'\u0023'
|
||||
'\u00A7'
|
||||
'\u00A4'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\u00AB'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\u00B0'
|
||||
'\u00B1'
|
||||
'\u00B2'
|
||||
'\u00B3'
|
||||
'\u00D7'
|
||||
'\u00B5'
|
||||
'\u00B6'
|
||||
'\u00B7'
|
||||
'\u00F7'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\u00BB'
|
||||
'\u00BC'
|
||||
'\u00BD'
|
||||
'\u00BE'
|
||||
'\u00BF'
|
||||
'\ufffe'
|
||||
'\u0300'
|
||||
'\u0301'
|
||||
'\u0302'
|
||||
'\u0303'
|
||||
'\u0304'
|
||||
'\u0306'
|
||||
'\u0307'
|
||||
'\u0308'
|
||||
'\ufffe'
|
||||
'\u030A'
|
||||
'\u0327'
|
||||
'\u0332'
|
||||
'\u030B'
|
||||
'\u0328'
|
||||
'\u030C'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\ufffe'
|
||||
'\u2126'
|
||||
'\u00C6'
|
||||
'\u00D0'
|
||||
'\u00AA'
|
||||
'\u0126'
|
||||
'\ufffe'
|
||||
'\u0132'
|
||||
'\u013F'
|
||||
'\u0141'
|
||||
'\u00D8'
|
||||
'\u0152'
|
||||
'\u00BA'
|
||||
'\u00DE'
|
||||
'\u0166'
|
||||
'\u014A'
|
||||
'\u0149'
|
||||
'\u0138'
|
||||
'\u00E6'
|
||||
'\u0111'
|
||||
'\u00F0'
|
||||
'\u0127'
|
||||
'\u0131'
|
||||
'\u0133'
|
||||
'\u0140'
|
||||
'\u0142'
|
||||
'\u00F8'
|
||||
'\u0153'
|
||||
'\u00DF'
|
||||
'\u00FE'
|
||||
'\u0167'
|
||||
'\u014B'
|
||||
'\ufffe'
|
||||
)
|
||||
ENCODING_TABLE = codecs.charmap_build(DECODING_TABLE)
|
||||
46
jc/parsers/asn1crypto/_types.py
Normal file
46
jc/parsers/asn1crypto/_types.py
Normal file
@@ -0,0 +1,46 @@
|
||||
# coding: utf-8
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
import inspect
|
||||
import sys
|
||||
|
||||
|
||||
if sys.version_info < (3,):
|
||||
str_cls = unicode # noqa
|
||||
byte_cls = str
|
||||
int_types = (int, long) # noqa
|
||||
|
||||
def bytes_to_list(byte_string):
|
||||
return [ord(b) for b in byte_string]
|
||||
|
||||
chr_cls = chr
|
||||
|
||||
else:
|
||||
str_cls = str
|
||||
byte_cls = bytes
|
||||
int_types = int
|
||||
|
||||
bytes_to_list = list
|
||||
|
||||
def chr_cls(num):
|
||||
return bytes([num])
|
||||
|
||||
|
||||
def type_name(value):
|
||||
"""
|
||||
Returns a user-readable name for the type of an object
|
||||
|
||||
:param value:
|
||||
A value to get the type name of
|
||||
|
||||
:return:
|
||||
A unicode string of the object's type name
|
||||
"""
|
||||
|
||||
if inspect.isclass(value):
|
||||
cls = value
|
||||
else:
|
||||
cls = value.__class__
|
||||
if cls.__module__ in set(['builtins', '__builtin__']):
|
||||
return cls.__name__
|
||||
return '%s.%s' % (cls.__module__, cls.__name__)
|
||||
1189
jc/parsers/asn1crypto/algos.py
Normal file
1189
jc/parsers/asn1crypto/algos.py
Normal file
File diff suppressed because it is too large
Load Diff
1003
jc/parsers/asn1crypto/cms.py
Normal file
1003
jc/parsers/asn1crypto/cms.py
Normal file
File diff suppressed because it is too large
Load Diff
5676
jc/parsers/asn1crypto/core.py
Normal file
5676
jc/parsers/asn1crypto/core.py
Normal file
File diff suppressed because it is too large
Load Diff
536
jc/parsers/asn1crypto/crl.py
Normal file
536
jc/parsers/asn1crypto/crl.py
Normal file
@@ -0,0 +1,536 @@
|
||||
# coding: utf-8
|
||||
|
||||
"""
|
||||
ASN.1 type classes for certificate revocation lists (CRL). Exports the
|
||||
following items:
|
||||
|
||||
- CertificateList()
|
||||
|
||||
Other type classes are defined that help compose the types listed above.
|
||||
"""
|
||||
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
import hashlib
|
||||
|
||||
from .algos import SignedDigestAlgorithm
|
||||
from .core import (
|
||||
Boolean,
|
||||
Enumerated,
|
||||
GeneralizedTime,
|
||||
Integer,
|
||||
ObjectIdentifier,
|
||||
OctetBitString,
|
||||
ParsableOctetString,
|
||||
Sequence,
|
||||
SequenceOf,
|
||||
)
|
||||
from .x509 import (
|
||||
AuthorityInfoAccessSyntax,
|
||||
AuthorityKeyIdentifier,
|
||||
CRLDistributionPoints,
|
||||
DistributionPointName,
|
||||
GeneralNames,
|
||||
Name,
|
||||
ReasonFlags,
|
||||
Time,
|
||||
)
|
||||
|
||||
|
||||
# The structures in this file are taken from https://tools.ietf.org/html/rfc5280
|
||||
|
||||
|
||||
class Version(Integer):
|
||||
_map = {
|
||||
0: 'v1',
|
||||
1: 'v2',
|
||||
2: 'v3',
|
||||
}
|
||||
|
||||
|
||||
class IssuingDistributionPoint(Sequence):
|
||||
_fields = [
|
||||
('distribution_point', DistributionPointName, {'explicit': 0, 'optional': True}),
|
||||
('only_contains_user_certs', Boolean, {'implicit': 1, 'default': False}),
|
||||
('only_contains_ca_certs', Boolean, {'implicit': 2, 'default': False}),
|
||||
('only_some_reasons', ReasonFlags, {'implicit': 3, 'optional': True}),
|
||||
('indirect_crl', Boolean, {'implicit': 4, 'default': False}),
|
||||
('only_contains_attribute_certs', Boolean, {'implicit': 5, 'default': False}),
|
||||
]
|
||||
|
||||
|
||||
class TBSCertListExtensionId(ObjectIdentifier):
|
||||
_map = {
|
||||
'2.5.29.18': 'issuer_alt_name',
|
||||
'2.5.29.20': 'crl_number',
|
||||
'2.5.29.27': 'delta_crl_indicator',
|
||||
'2.5.29.28': 'issuing_distribution_point',
|
||||
'2.5.29.35': 'authority_key_identifier',
|
||||
'2.5.29.46': 'freshest_crl',
|
||||
'1.3.6.1.5.5.7.1.1': 'authority_information_access',
|
||||
}
|
||||
|
||||
|
||||
class TBSCertListExtension(Sequence):
|
||||
_fields = [
|
||||
('extn_id', TBSCertListExtensionId),
|
||||
('critical', Boolean, {'default': False}),
|
||||
('extn_value', ParsableOctetString),
|
||||
]
|
||||
|
||||
_oid_pair = ('extn_id', 'extn_value')
|
||||
_oid_specs = {
|
||||
'issuer_alt_name': GeneralNames,
|
||||
'crl_number': Integer,
|
||||
'delta_crl_indicator': Integer,
|
||||
'issuing_distribution_point': IssuingDistributionPoint,
|
||||
'authority_key_identifier': AuthorityKeyIdentifier,
|
||||
'freshest_crl': CRLDistributionPoints,
|
||||
'authority_information_access': AuthorityInfoAccessSyntax,
|
||||
}
|
||||
|
||||
|
||||
class TBSCertListExtensions(SequenceOf):
|
||||
_child_spec = TBSCertListExtension
|
||||
|
||||
|
||||
class CRLReason(Enumerated):
|
||||
_map = {
|
||||
0: 'unspecified',
|
||||
1: 'key_compromise',
|
||||
2: 'ca_compromise',
|
||||
3: 'affiliation_changed',
|
||||
4: 'superseded',
|
||||
5: 'cessation_of_operation',
|
||||
6: 'certificate_hold',
|
||||
8: 'remove_from_crl',
|
||||
9: 'privilege_withdrawn',
|
||||
10: 'aa_compromise',
|
||||
}
|
||||
|
||||
@property
|
||||
def human_friendly(self):
|
||||
"""
|
||||
:return:
|
||||
A unicode string with revocation description that is suitable to
|
||||
show to end-users. Starts with a lower case letter and phrased in
|
||||
such a way that it makes sense after the phrase "because of" or
|
||||
"due to".
|
||||
"""
|
||||
|
||||
return {
|
||||
'unspecified': 'an unspecified reason',
|
||||
'key_compromise': 'a compromised key',
|
||||
'ca_compromise': 'the CA being compromised',
|
||||
'affiliation_changed': 'an affiliation change',
|
||||
'superseded': 'certificate supersession',
|
||||
'cessation_of_operation': 'a cessation of operation',
|
||||
'certificate_hold': 'a certificate hold',
|
||||
'remove_from_crl': 'removal from the CRL',
|
||||
'privilege_withdrawn': 'privilege withdrawl',
|
||||
'aa_compromise': 'the AA being compromised',
|
||||
}[self.native]
|
||||
|
||||
|
||||
class CRLEntryExtensionId(ObjectIdentifier):
|
||||
_map = {
|
||||
'2.5.29.21': 'crl_reason',
|
||||
'2.5.29.23': 'hold_instruction_code',
|
||||
'2.5.29.24': 'invalidity_date',
|
||||
'2.5.29.29': 'certificate_issuer',
|
||||
}
|
||||
|
||||
|
||||
class CRLEntryExtension(Sequence):
|
||||
_fields = [
|
||||
('extn_id', CRLEntryExtensionId),
|
||||
('critical', Boolean, {'default': False}),
|
||||
('extn_value', ParsableOctetString),
|
||||
]
|
||||
|
||||
_oid_pair = ('extn_id', 'extn_value')
|
||||
_oid_specs = {
|
||||
'crl_reason': CRLReason,
|
||||
'hold_instruction_code': ObjectIdentifier,
|
||||
'invalidity_date': GeneralizedTime,
|
||||
'certificate_issuer': GeneralNames,
|
||||
}
|
||||
|
||||
|
||||
class CRLEntryExtensions(SequenceOf):
|
||||
_child_spec = CRLEntryExtension
|
||||
|
||||
|
||||
class RevokedCertificate(Sequence):
|
||||
_fields = [
|
||||
('user_certificate', Integer),
|
||||
('revocation_date', Time),
|
||||
('crl_entry_extensions', CRLEntryExtensions, {'optional': True}),
|
||||
]
|
||||
|
||||
_processed_extensions = False
|
||||
_critical_extensions = None
|
||||
_crl_reason_value = None
|
||||
_invalidity_date_value = None
|
||||
_certificate_issuer_value = None
|
||||
_issuer_name = False
|
||||
|
||||
def _set_extensions(self):
|
||||
"""
|
||||
Sets common named extensions to private attributes and creates a list
|
||||
of critical extensions
|
||||
"""
|
||||
|
||||
self._critical_extensions = set()
|
||||
|
||||
for extension in self['crl_entry_extensions']:
|
||||
name = extension['extn_id'].native
|
||||
attribute_name = '_%s_value' % name
|
||||
if hasattr(self, attribute_name):
|
||||
setattr(self, attribute_name, extension['extn_value'].parsed)
|
||||
if extension['critical'].native:
|
||||
self._critical_extensions.add(name)
|
||||
|
||||
self._processed_extensions = True
|
||||
|
||||
@property
|
||||
def critical_extensions(self):
|
||||
"""
|
||||
Returns a set of the names (or OID if not a known extension) of the
|
||||
extensions marked as critical
|
||||
|
||||
:return:
|
||||
A set of unicode strings
|
||||
"""
|
||||
|
||||
if not self._processed_extensions:
|
||||
self._set_extensions()
|
||||
return self._critical_extensions
|
||||
|
||||
@property
|
||||
def crl_reason_value(self):
|
||||
"""
|
||||
This extension indicates the reason that a certificate was revoked.
|
||||
|
||||
:return:
|
||||
None or a CRLReason object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._crl_reason_value
|
||||
|
||||
@property
|
||||
def invalidity_date_value(self):
|
||||
"""
|
||||
This extension indicates the suspected date/time the private key was
|
||||
compromised or the certificate became invalid. This would usually be
|
||||
before the revocation date, which is when the CA processed the
|
||||
revocation.
|
||||
|
||||
:return:
|
||||
None or a GeneralizedTime object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._invalidity_date_value
|
||||
|
||||
@property
|
||||
def certificate_issuer_value(self):
|
||||
"""
|
||||
This extension indicates the issuer of the certificate in question,
|
||||
and is used in indirect CRLs. CRL entries without this extension are
|
||||
for certificates issued from the last seen issuer.
|
||||
|
||||
:return:
|
||||
None or an x509.GeneralNames object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._certificate_issuer_value
|
||||
|
||||
@property
|
||||
def issuer_name(self):
|
||||
"""
|
||||
:return:
|
||||
None, or an asn1crypto.x509.Name object for the issuer of the cert
|
||||
"""
|
||||
|
||||
if self._issuer_name is False:
|
||||
self._issuer_name = None
|
||||
if self.certificate_issuer_value:
|
||||
for general_name in self.certificate_issuer_value:
|
||||
if general_name.name == 'directory_name':
|
||||
self._issuer_name = general_name.chosen
|
||||
break
|
||||
return self._issuer_name
|
||||
|
||||
|
||||
class RevokedCertificates(SequenceOf):
|
||||
_child_spec = RevokedCertificate
|
||||
|
||||
|
||||
class TbsCertList(Sequence):
|
||||
_fields = [
|
||||
('version', Version, {'optional': True}),
|
||||
('signature', SignedDigestAlgorithm),
|
||||
('issuer', Name),
|
||||
('this_update', Time),
|
||||
('next_update', Time, {'optional': True}),
|
||||
('revoked_certificates', RevokedCertificates, {'optional': True}),
|
||||
('crl_extensions', TBSCertListExtensions, {'explicit': 0, 'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class CertificateList(Sequence):
|
||||
_fields = [
|
||||
('tbs_cert_list', TbsCertList),
|
||||
('signature_algorithm', SignedDigestAlgorithm),
|
||||
('signature', OctetBitString),
|
||||
]
|
||||
|
||||
_processed_extensions = False
|
||||
_critical_extensions = None
|
||||
_issuer_alt_name_value = None
|
||||
_crl_number_value = None
|
||||
_delta_crl_indicator_value = None
|
||||
_issuing_distribution_point_value = None
|
||||
_authority_key_identifier_value = None
|
||||
_freshest_crl_value = None
|
||||
_authority_information_access_value = None
|
||||
_issuer_cert_urls = None
|
||||
_delta_crl_distribution_points = None
|
||||
_sha1 = None
|
||||
_sha256 = None
|
||||
|
||||
def _set_extensions(self):
|
||||
"""
|
||||
Sets common named extensions to private attributes and creates a list
|
||||
of critical extensions
|
||||
"""
|
||||
|
||||
self._critical_extensions = set()
|
||||
|
||||
for extension in self['tbs_cert_list']['crl_extensions']:
|
||||
name = extension['extn_id'].native
|
||||
attribute_name = '_%s_value' % name
|
||||
if hasattr(self, attribute_name):
|
||||
setattr(self, attribute_name, extension['extn_value'].parsed)
|
||||
if extension['critical'].native:
|
||||
self._critical_extensions.add(name)
|
||||
|
||||
self._processed_extensions = True
|
||||
|
||||
@property
|
||||
def critical_extensions(self):
|
||||
"""
|
||||
Returns a set of the names (or OID if not a known extension) of the
|
||||
extensions marked as critical
|
||||
|
||||
:return:
|
||||
A set of unicode strings
|
||||
"""
|
||||
|
||||
if not self._processed_extensions:
|
||||
self._set_extensions()
|
||||
return self._critical_extensions
|
||||
|
||||
@property
|
||||
def issuer_alt_name_value(self):
|
||||
"""
|
||||
This extension allows associating one or more alternative names with
|
||||
the issuer of the CRL.
|
||||
|
||||
:return:
|
||||
None or an x509.GeneralNames object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._issuer_alt_name_value
|
||||
|
||||
@property
|
||||
def crl_number_value(self):
|
||||
"""
|
||||
This extension adds a monotonically increasing number to the CRL and is
|
||||
used to distinguish different versions of the CRL.
|
||||
|
||||
:return:
|
||||
None or an Integer object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._crl_number_value
|
||||
|
||||
@property
|
||||
def delta_crl_indicator_value(self):
|
||||
"""
|
||||
This extension indicates a CRL is a delta CRL, and contains the CRL
|
||||
number of the base CRL that it is a delta from.
|
||||
|
||||
:return:
|
||||
None or an Integer object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._delta_crl_indicator_value
|
||||
|
||||
@property
|
||||
def issuing_distribution_point_value(self):
|
||||
"""
|
||||
This extension includes information about what types of revocations
|
||||
and certificates are part of the CRL.
|
||||
|
||||
:return:
|
||||
None or an IssuingDistributionPoint object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._issuing_distribution_point_value
|
||||
|
||||
@property
|
||||
def authority_key_identifier_value(self):
|
||||
"""
|
||||
This extension helps in identifying the public key with which to
|
||||
validate the authenticity of the CRL.
|
||||
|
||||
:return:
|
||||
None or an AuthorityKeyIdentifier object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._authority_key_identifier_value
|
||||
|
||||
@property
|
||||
def freshest_crl_value(self):
|
||||
"""
|
||||
This extension is used in complete CRLs to indicate where a delta CRL
|
||||
may be located.
|
||||
|
||||
:return:
|
||||
None or a CRLDistributionPoints object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._freshest_crl_value
|
||||
|
||||
@property
|
||||
def authority_information_access_value(self):
|
||||
"""
|
||||
This extension is used to provide a URL with which to download the
|
||||
certificate used to sign this CRL.
|
||||
|
||||
:return:
|
||||
None or an AuthorityInfoAccessSyntax object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._authority_information_access_value
|
||||
|
||||
@property
|
||||
def issuer(self):
|
||||
"""
|
||||
:return:
|
||||
An asn1crypto.x509.Name object for the issuer of the CRL
|
||||
"""
|
||||
|
||||
return self['tbs_cert_list']['issuer']
|
||||
|
||||
@property
|
||||
def authority_key_identifier(self):
|
||||
"""
|
||||
:return:
|
||||
None or a byte string of the key_identifier from the authority key
|
||||
identifier extension
|
||||
"""
|
||||
|
||||
if not self.authority_key_identifier_value:
|
||||
return None
|
||||
|
||||
return self.authority_key_identifier_value['key_identifier'].native
|
||||
|
||||
@property
|
||||
def issuer_cert_urls(self):
|
||||
"""
|
||||
:return:
|
||||
A list of unicode strings that are URLs that should contain either
|
||||
an individual DER-encoded X.509 certificate, or a DER-encoded CMS
|
||||
message containing multiple certificates
|
||||
"""
|
||||
|
||||
if self._issuer_cert_urls is None:
|
||||
self._issuer_cert_urls = []
|
||||
if self.authority_information_access_value:
|
||||
for entry in self.authority_information_access_value:
|
||||
if entry['access_method'].native == 'ca_issuers':
|
||||
location = entry['access_location']
|
||||
if location.name != 'uniform_resource_identifier':
|
||||
continue
|
||||
url = location.native
|
||||
if url.lower()[0:7] == 'http://':
|
||||
self._issuer_cert_urls.append(url)
|
||||
return self._issuer_cert_urls
|
||||
|
||||
@property
|
||||
def delta_crl_distribution_points(self):
|
||||
"""
|
||||
Returns delta CRL URLs - only applies to complete CRLs
|
||||
|
||||
:return:
|
||||
A list of zero or more DistributionPoint objects
|
||||
"""
|
||||
|
||||
if self._delta_crl_distribution_points is None:
|
||||
self._delta_crl_distribution_points = []
|
||||
|
||||
if self.freshest_crl_value is not None:
|
||||
for distribution_point in self.freshest_crl_value:
|
||||
distribution_point_name = distribution_point['distribution_point']
|
||||
# RFC 5280 indicates conforming CA should not use the relative form
|
||||
if distribution_point_name.name == 'name_relative_to_crl_issuer':
|
||||
continue
|
||||
# This library is currently only concerned with HTTP-based CRLs
|
||||
for general_name in distribution_point_name.chosen:
|
||||
if general_name.name == 'uniform_resource_identifier':
|
||||
self._delta_crl_distribution_points.append(distribution_point)
|
||||
|
||||
return self._delta_crl_distribution_points
|
||||
|
||||
@property
|
||||
def signature(self):
|
||||
"""
|
||||
:return:
|
||||
A byte string of the signature
|
||||
"""
|
||||
|
||||
return self['signature'].native
|
||||
|
||||
@property
|
||||
def sha1(self):
|
||||
"""
|
||||
:return:
|
||||
The SHA1 hash of the DER-encoded bytes of this certificate list
|
||||
"""
|
||||
|
||||
if self._sha1 is None:
|
||||
self._sha1 = hashlib.sha1(self.dump()).digest()
|
||||
return self._sha1
|
||||
|
||||
@property
|
||||
def sha256(self):
|
||||
"""
|
||||
:return:
|
||||
The SHA-256 hash of the DER-encoded bytes of this certificate list
|
||||
"""
|
||||
|
||||
if self._sha256 is None:
|
||||
self._sha256 = hashlib.sha256(self.dump()).digest()
|
||||
return self._sha256
|
||||
133
jc/parsers/asn1crypto/csr.py
Normal file
133
jc/parsers/asn1crypto/csr.py
Normal file
@@ -0,0 +1,133 @@
|
||||
# coding: utf-8
|
||||
|
||||
"""
|
||||
ASN.1 type classes for certificate signing requests (CSR). Exports the
|
||||
following items:
|
||||
|
||||
- CertificationRequest()
|
||||
|
||||
Other type classes are defined that help compose the types listed above.
|
||||
"""
|
||||
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
from .algos import SignedDigestAlgorithm
|
||||
from .core import (
|
||||
Any,
|
||||
BitString,
|
||||
BMPString,
|
||||
Integer,
|
||||
ObjectIdentifier,
|
||||
OctetBitString,
|
||||
Sequence,
|
||||
SetOf,
|
||||
UTF8String
|
||||
)
|
||||
from .keys import PublicKeyInfo
|
||||
from .x509 import DirectoryString, Extensions, Name
|
||||
|
||||
|
||||
# The structures in this file are taken from https://tools.ietf.org/html/rfc2986
|
||||
# and https://tools.ietf.org/html/rfc2985
|
||||
|
||||
|
||||
class Version(Integer):
|
||||
_map = {
|
||||
0: 'v1',
|
||||
}
|
||||
|
||||
|
||||
class CSRAttributeType(ObjectIdentifier):
|
||||
_map = {
|
||||
'1.2.840.113549.1.9.7': 'challenge_password',
|
||||
'1.2.840.113549.1.9.9': 'extended_certificate_attributes',
|
||||
'1.2.840.113549.1.9.14': 'extension_request',
|
||||
# https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-wcce/a5eaae36-e9f3-4dc5-a687-bfa7115954f1
|
||||
'1.3.6.1.4.1.311.13.2.2': 'microsoft_enrollment_csp_provider',
|
||||
# https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-wcce/7c677cba-030d-48be-ba2b-01e407705f34
|
||||
'1.3.6.1.4.1.311.13.2.3': 'microsoft_os_version',
|
||||
# https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-wcce/64e5ff6d-c6dd-4578-92f7-b3d895f9b9c7
|
||||
'1.3.6.1.4.1.311.21.20': 'microsoft_request_client_info',
|
||||
}
|
||||
|
||||
|
||||
class SetOfDirectoryString(SetOf):
|
||||
_child_spec = DirectoryString
|
||||
|
||||
|
||||
class Attribute(Sequence):
|
||||
_fields = [
|
||||
('type', ObjectIdentifier),
|
||||
('values', SetOf, {'spec': Any}),
|
||||
]
|
||||
|
||||
|
||||
class SetOfAttributes(SetOf):
|
||||
_child_spec = Attribute
|
||||
|
||||
|
||||
class SetOfExtensions(SetOf):
|
||||
_child_spec = Extensions
|
||||
|
||||
|
||||
class MicrosoftEnrollmentCSProvider(Sequence):
|
||||
_fields = [
|
||||
('keyspec', Integer),
|
||||
('cspname', BMPString), # cryptographic service provider name
|
||||
('signature', BitString),
|
||||
]
|
||||
|
||||
|
||||
class SetOfMicrosoftEnrollmentCSProvider(SetOf):
|
||||
_child_spec = MicrosoftEnrollmentCSProvider
|
||||
|
||||
|
||||
class MicrosoftRequestClientInfo(Sequence):
|
||||
_fields = [
|
||||
('clientid', Integer),
|
||||
('machinename', UTF8String),
|
||||
('username', UTF8String),
|
||||
('processname', UTF8String),
|
||||
]
|
||||
|
||||
|
||||
class SetOfMicrosoftRequestClientInfo(SetOf):
|
||||
_child_spec = MicrosoftRequestClientInfo
|
||||
|
||||
|
||||
class CRIAttribute(Sequence):
|
||||
_fields = [
|
||||
('type', CSRAttributeType),
|
||||
('values', Any),
|
||||
]
|
||||
|
||||
_oid_pair = ('type', 'values')
|
||||
_oid_specs = {
|
||||
'challenge_password': SetOfDirectoryString,
|
||||
'extended_certificate_attributes': SetOfAttributes,
|
||||
'extension_request': SetOfExtensions,
|
||||
'microsoft_enrollment_csp_provider': SetOfMicrosoftEnrollmentCSProvider,
|
||||
'microsoft_os_version': SetOfDirectoryString,
|
||||
'microsoft_request_client_info': SetOfMicrosoftRequestClientInfo,
|
||||
}
|
||||
|
||||
|
||||
class CRIAttributes(SetOf):
|
||||
_child_spec = CRIAttribute
|
||||
|
||||
|
||||
class CertificationRequestInfo(Sequence):
|
||||
_fields = [
|
||||
('version', Version),
|
||||
('subject', Name),
|
||||
('subject_pk_info', PublicKeyInfo),
|
||||
('attributes', CRIAttributes, {'implicit': 0, 'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class CertificationRequest(Sequence):
|
||||
_fields = [
|
||||
('certification_request_info', CertificationRequestInfo),
|
||||
('signature_algorithm', SignedDigestAlgorithm),
|
||||
('signature', OctetBitString),
|
||||
]
|
||||
1301
jc/parsers/asn1crypto/keys.py
Normal file
1301
jc/parsers/asn1crypto/keys.py
Normal file
File diff suppressed because it is too large
Load Diff
703
jc/parsers/asn1crypto/ocsp.py
Normal file
703
jc/parsers/asn1crypto/ocsp.py
Normal file
@@ -0,0 +1,703 @@
|
||||
# coding: utf-8
|
||||
|
||||
"""
|
||||
ASN.1 type classes for the online certificate status protocol (OCSP). Exports
|
||||
the following items:
|
||||
|
||||
- OCSPRequest()
|
||||
- OCSPResponse()
|
||||
|
||||
Other type classes are defined that help compose the types listed above.
|
||||
"""
|
||||
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
from ._errors import unwrap
|
||||
from .algos import DigestAlgorithm, SignedDigestAlgorithm
|
||||
from .core import (
|
||||
Boolean,
|
||||
Choice,
|
||||
Enumerated,
|
||||
GeneralizedTime,
|
||||
IA5String,
|
||||
Integer,
|
||||
Null,
|
||||
ObjectIdentifier,
|
||||
OctetBitString,
|
||||
OctetString,
|
||||
ParsableOctetString,
|
||||
Sequence,
|
||||
SequenceOf,
|
||||
)
|
||||
from .crl import AuthorityInfoAccessSyntax, CRLReason
|
||||
from .keys import PublicKeyAlgorithm
|
||||
from .x509 import Certificate, GeneralName, GeneralNames, Name
|
||||
|
||||
|
||||
# The structures in this file are taken from https://tools.ietf.org/html/rfc6960
|
||||
|
||||
|
||||
class Version(Integer):
|
||||
_map = {
|
||||
0: 'v1'
|
||||
}
|
||||
|
||||
|
||||
class CertId(Sequence):
|
||||
_fields = [
|
||||
('hash_algorithm', DigestAlgorithm),
|
||||
('issuer_name_hash', OctetString),
|
||||
('issuer_key_hash', OctetString),
|
||||
('serial_number', Integer),
|
||||
]
|
||||
|
||||
|
||||
class ServiceLocator(Sequence):
|
||||
_fields = [
|
||||
('issuer', Name),
|
||||
('locator', AuthorityInfoAccessSyntax),
|
||||
]
|
||||
|
||||
|
||||
class RequestExtensionId(ObjectIdentifier):
|
||||
_map = {
|
||||
'1.3.6.1.5.5.7.48.1.7': 'service_locator',
|
||||
}
|
||||
|
||||
|
||||
class RequestExtension(Sequence):
|
||||
_fields = [
|
||||
('extn_id', RequestExtensionId),
|
||||
('critical', Boolean, {'default': False}),
|
||||
('extn_value', ParsableOctetString),
|
||||
]
|
||||
|
||||
_oid_pair = ('extn_id', 'extn_value')
|
||||
_oid_specs = {
|
||||
'service_locator': ServiceLocator,
|
||||
}
|
||||
|
||||
|
||||
class RequestExtensions(SequenceOf):
|
||||
_child_spec = RequestExtension
|
||||
|
||||
|
||||
class Request(Sequence):
|
||||
_fields = [
|
||||
('req_cert', CertId),
|
||||
('single_request_extensions', RequestExtensions, {'explicit': 0, 'optional': True}),
|
||||
]
|
||||
|
||||
_processed_extensions = False
|
||||
_critical_extensions = None
|
||||
_service_locator_value = None
|
||||
|
||||
def _set_extensions(self):
|
||||
"""
|
||||
Sets common named extensions to private attributes and creates a list
|
||||
of critical extensions
|
||||
"""
|
||||
|
||||
self._critical_extensions = set()
|
||||
|
||||
for extension in self['single_request_extensions']:
|
||||
name = extension['extn_id'].native
|
||||
attribute_name = '_%s_value' % name
|
||||
if hasattr(self, attribute_name):
|
||||
setattr(self, attribute_name, extension['extn_value'].parsed)
|
||||
if extension['critical'].native:
|
||||
self._critical_extensions.add(name)
|
||||
|
||||
self._processed_extensions = True
|
||||
|
||||
@property
|
||||
def critical_extensions(self):
|
||||
"""
|
||||
Returns a set of the names (or OID if not a known extension) of the
|
||||
extensions marked as critical
|
||||
|
||||
:return:
|
||||
A set of unicode strings
|
||||
"""
|
||||
|
||||
if not self._processed_extensions:
|
||||
self._set_extensions()
|
||||
return self._critical_extensions
|
||||
|
||||
@property
|
||||
def service_locator_value(self):
|
||||
"""
|
||||
This extension is used when communicating with an OCSP responder that
|
||||
acts as a proxy for OCSP requests
|
||||
|
||||
:return:
|
||||
None or a ServiceLocator object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._service_locator_value
|
||||
|
||||
|
||||
class Requests(SequenceOf):
|
||||
_child_spec = Request
|
||||
|
||||
|
||||
class ResponseType(ObjectIdentifier):
|
||||
_map = {
|
||||
'1.3.6.1.5.5.7.48.1.1': 'basic_ocsp_response',
|
||||
}
|
||||
|
||||
|
||||
class AcceptableResponses(SequenceOf):
|
||||
_child_spec = ResponseType
|
||||
|
||||
|
||||
class PreferredSignatureAlgorithm(Sequence):
|
||||
_fields = [
|
||||
('sig_identifier', SignedDigestAlgorithm),
|
||||
('cert_identifier', PublicKeyAlgorithm, {'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class PreferredSignatureAlgorithms(SequenceOf):
|
||||
_child_spec = PreferredSignatureAlgorithm
|
||||
|
||||
|
||||
class TBSRequestExtensionId(ObjectIdentifier):
|
||||
_map = {
|
||||
'1.3.6.1.5.5.7.48.1.2': 'nonce',
|
||||
'1.3.6.1.5.5.7.48.1.4': 'acceptable_responses',
|
||||
'1.3.6.1.5.5.7.48.1.8': 'preferred_signature_algorithms',
|
||||
}
|
||||
|
||||
|
||||
class TBSRequestExtension(Sequence):
|
||||
_fields = [
|
||||
('extn_id', TBSRequestExtensionId),
|
||||
('critical', Boolean, {'default': False}),
|
||||
('extn_value', ParsableOctetString),
|
||||
]
|
||||
|
||||
_oid_pair = ('extn_id', 'extn_value')
|
||||
_oid_specs = {
|
||||
'nonce': OctetString,
|
||||
'acceptable_responses': AcceptableResponses,
|
||||
'preferred_signature_algorithms': PreferredSignatureAlgorithms,
|
||||
}
|
||||
|
||||
|
||||
class TBSRequestExtensions(SequenceOf):
|
||||
_child_spec = TBSRequestExtension
|
||||
|
||||
|
||||
class TBSRequest(Sequence):
|
||||
_fields = [
|
||||
('version', Version, {'explicit': 0, 'default': 'v1'}),
|
||||
('requestor_name', GeneralName, {'explicit': 1, 'optional': True}),
|
||||
('request_list', Requests),
|
||||
('request_extensions', TBSRequestExtensions, {'explicit': 2, 'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class Certificates(SequenceOf):
|
||||
_child_spec = Certificate
|
||||
|
||||
|
||||
class Signature(Sequence):
|
||||
_fields = [
|
||||
('signature_algorithm', SignedDigestAlgorithm),
|
||||
('signature', OctetBitString),
|
||||
('certs', Certificates, {'explicit': 0, 'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class OCSPRequest(Sequence):
|
||||
_fields = [
|
||||
('tbs_request', TBSRequest),
|
||||
('optional_signature', Signature, {'explicit': 0, 'optional': True}),
|
||||
]
|
||||
|
||||
_processed_extensions = False
|
||||
_critical_extensions = None
|
||||
_nonce_value = None
|
||||
_acceptable_responses_value = None
|
||||
_preferred_signature_algorithms_value = None
|
||||
|
||||
def _set_extensions(self):
|
||||
"""
|
||||
Sets common named extensions to private attributes and creates a list
|
||||
of critical extensions
|
||||
"""
|
||||
|
||||
self._critical_extensions = set()
|
||||
|
||||
for extension in self['tbs_request']['request_extensions']:
|
||||
name = extension['extn_id'].native
|
||||
attribute_name = '_%s_value' % name
|
||||
if hasattr(self, attribute_name):
|
||||
setattr(self, attribute_name, extension['extn_value'].parsed)
|
||||
if extension['critical'].native:
|
||||
self._critical_extensions.add(name)
|
||||
|
||||
self._processed_extensions = True
|
||||
|
||||
@property
|
||||
def critical_extensions(self):
|
||||
"""
|
||||
Returns a set of the names (or OID if not a known extension) of the
|
||||
extensions marked as critical
|
||||
|
||||
:return:
|
||||
A set of unicode strings
|
||||
"""
|
||||
|
||||
if not self._processed_extensions:
|
||||
self._set_extensions()
|
||||
return self._critical_extensions
|
||||
|
||||
@property
|
||||
def nonce_value(self):
|
||||
"""
|
||||
This extension is used to prevent replay attacks by including a unique,
|
||||
random value with each request/response pair
|
||||
|
||||
:return:
|
||||
None or an OctetString object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._nonce_value
|
||||
|
||||
@property
|
||||
def acceptable_responses_value(self):
|
||||
"""
|
||||
This extension is used to allow the client and server to communicate
|
||||
with alternative response formats other than just basic_ocsp_response,
|
||||
although no other formats are defined in the standard.
|
||||
|
||||
:return:
|
||||
None or an AcceptableResponses object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._acceptable_responses_value
|
||||
|
||||
@property
|
||||
def preferred_signature_algorithms_value(self):
|
||||
"""
|
||||
This extension is used by the client to define what signature algorithms
|
||||
are preferred, including both the hash algorithm and the public key
|
||||
algorithm, with a level of detail down to even the public key algorithm
|
||||
parameters, such as curve name.
|
||||
|
||||
:return:
|
||||
None or a PreferredSignatureAlgorithms object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._preferred_signature_algorithms_value
|
||||
|
||||
|
||||
class OCSPResponseStatus(Enumerated):
|
||||
_map = {
|
||||
0: 'successful',
|
||||
1: 'malformed_request',
|
||||
2: 'internal_error',
|
||||
3: 'try_later',
|
||||
5: 'sign_required',
|
||||
6: 'unauthorized',
|
||||
}
|
||||
|
||||
|
||||
class ResponderId(Choice):
|
||||
_alternatives = [
|
||||
('by_name', Name, {'explicit': 1}),
|
||||
('by_key', OctetString, {'explicit': 2}),
|
||||
]
|
||||
|
||||
|
||||
# Custom class to return a meaningful .native attribute from CertStatus()
|
||||
class StatusGood(Null):
|
||||
def set(self, value):
|
||||
"""
|
||||
Sets the value of the object
|
||||
|
||||
:param value:
|
||||
None or 'good'
|
||||
"""
|
||||
|
||||
if value is not None and value != 'good' and not isinstance(value, Null):
|
||||
raise ValueError(unwrap(
|
||||
'''
|
||||
value must be one of None, "good", not %s
|
||||
''',
|
||||
repr(value)
|
||||
))
|
||||
|
||||
self.contents = b''
|
||||
|
||||
@property
|
||||
def native(self):
|
||||
return 'good'
|
||||
|
||||
|
||||
# Custom class to return a meaningful .native attribute from CertStatus()
|
||||
class StatusUnknown(Null):
|
||||
def set(self, value):
|
||||
"""
|
||||
Sets the value of the object
|
||||
|
||||
:param value:
|
||||
None or 'unknown'
|
||||
"""
|
||||
|
||||
if value is not None and value != 'unknown' and not isinstance(value, Null):
|
||||
raise ValueError(unwrap(
|
||||
'''
|
||||
value must be one of None, "unknown", not %s
|
||||
''',
|
||||
repr(value)
|
||||
))
|
||||
|
||||
self.contents = b''
|
||||
|
||||
@property
|
||||
def native(self):
|
||||
return 'unknown'
|
||||
|
||||
|
||||
class RevokedInfo(Sequence):
|
||||
_fields = [
|
||||
('revocation_time', GeneralizedTime),
|
||||
('revocation_reason', CRLReason, {'explicit': 0, 'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class CertStatus(Choice):
|
||||
_alternatives = [
|
||||
('good', StatusGood, {'implicit': 0}),
|
||||
('revoked', RevokedInfo, {'implicit': 1}),
|
||||
('unknown', StatusUnknown, {'implicit': 2}),
|
||||
]
|
||||
|
||||
|
||||
class CrlId(Sequence):
|
||||
_fields = [
|
||||
('crl_url', IA5String, {'explicit': 0, 'optional': True}),
|
||||
('crl_num', Integer, {'explicit': 1, 'optional': True}),
|
||||
('crl_time', GeneralizedTime, {'explicit': 2, 'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class SingleResponseExtensionId(ObjectIdentifier):
|
||||
_map = {
|
||||
'1.3.6.1.5.5.7.48.1.3': 'crl',
|
||||
'1.3.6.1.5.5.7.48.1.6': 'archive_cutoff',
|
||||
# These are CRLEntryExtension values from
|
||||
# https://tools.ietf.org/html/rfc5280
|
||||
'2.5.29.21': 'crl_reason',
|
||||
'2.5.29.24': 'invalidity_date',
|
||||
'2.5.29.29': 'certificate_issuer',
|
||||
# https://tools.ietf.org/html/rfc6962.html#page-13
|
||||
'1.3.6.1.4.1.11129.2.4.5': 'signed_certificate_timestamp_list',
|
||||
}
|
||||
|
||||
|
||||
class SingleResponseExtension(Sequence):
|
||||
_fields = [
|
||||
('extn_id', SingleResponseExtensionId),
|
||||
('critical', Boolean, {'default': False}),
|
||||
('extn_value', ParsableOctetString),
|
||||
]
|
||||
|
||||
_oid_pair = ('extn_id', 'extn_value')
|
||||
_oid_specs = {
|
||||
'crl': CrlId,
|
||||
'archive_cutoff': GeneralizedTime,
|
||||
'crl_reason': CRLReason,
|
||||
'invalidity_date': GeneralizedTime,
|
||||
'certificate_issuer': GeneralNames,
|
||||
'signed_certificate_timestamp_list': OctetString,
|
||||
}
|
||||
|
||||
|
||||
class SingleResponseExtensions(SequenceOf):
|
||||
_child_spec = SingleResponseExtension
|
||||
|
||||
|
||||
class SingleResponse(Sequence):
|
||||
_fields = [
|
||||
('cert_id', CertId),
|
||||
('cert_status', CertStatus),
|
||||
('this_update', GeneralizedTime),
|
||||
('next_update', GeneralizedTime, {'explicit': 0, 'optional': True}),
|
||||
('single_extensions', SingleResponseExtensions, {'explicit': 1, 'optional': True}),
|
||||
]
|
||||
|
||||
_processed_extensions = False
|
||||
_critical_extensions = None
|
||||
_crl_value = None
|
||||
_archive_cutoff_value = None
|
||||
_crl_reason_value = None
|
||||
_invalidity_date_value = None
|
||||
_certificate_issuer_value = None
|
||||
|
||||
def _set_extensions(self):
|
||||
"""
|
||||
Sets common named extensions to private attributes and creates a list
|
||||
of critical extensions
|
||||
"""
|
||||
|
||||
self._critical_extensions = set()
|
||||
|
||||
for extension in self['single_extensions']:
|
||||
name = extension['extn_id'].native
|
||||
attribute_name = '_%s_value' % name
|
||||
if hasattr(self, attribute_name):
|
||||
setattr(self, attribute_name, extension['extn_value'].parsed)
|
||||
if extension['critical'].native:
|
||||
self._critical_extensions.add(name)
|
||||
|
||||
self._processed_extensions = True
|
||||
|
||||
@property
|
||||
def critical_extensions(self):
|
||||
"""
|
||||
Returns a set of the names (or OID if not a known extension) of the
|
||||
extensions marked as critical
|
||||
|
||||
:return:
|
||||
A set of unicode strings
|
||||
"""
|
||||
|
||||
if not self._processed_extensions:
|
||||
self._set_extensions()
|
||||
return self._critical_extensions
|
||||
|
||||
@property
|
||||
def crl_value(self):
|
||||
"""
|
||||
This extension is used to locate the CRL that a certificate's revocation
|
||||
is contained within.
|
||||
|
||||
:return:
|
||||
None or a CrlId object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._crl_value
|
||||
|
||||
@property
|
||||
def archive_cutoff_value(self):
|
||||
"""
|
||||
This extension is used to indicate the date at which an archived
|
||||
(historical) certificate status entry will no longer be available.
|
||||
|
||||
:return:
|
||||
None or a GeneralizedTime object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._archive_cutoff_value
|
||||
|
||||
@property
|
||||
def crl_reason_value(self):
|
||||
"""
|
||||
This extension indicates the reason that a certificate was revoked.
|
||||
|
||||
:return:
|
||||
None or a CRLReason object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._crl_reason_value
|
||||
|
||||
@property
|
||||
def invalidity_date_value(self):
|
||||
"""
|
||||
This extension indicates the suspected date/time the private key was
|
||||
compromised or the certificate became invalid. This would usually be
|
||||
before the revocation date, which is when the CA processed the
|
||||
revocation.
|
||||
|
||||
:return:
|
||||
None or a GeneralizedTime object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._invalidity_date_value
|
||||
|
||||
@property
|
||||
def certificate_issuer_value(self):
|
||||
"""
|
||||
This extension indicates the issuer of the certificate in question.
|
||||
|
||||
:return:
|
||||
None or an x509.GeneralNames object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._certificate_issuer_value
|
||||
|
||||
|
||||
class Responses(SequenceOf):
|
||||
_child_spec = SingleResponse
|
||||
|
||||
|
||||
class ResponseDataExtensionId(ObjectIdentifier):
|
||||
_map = {
|
||||
'1.3.6.1.5.5.7.48.1.2': 'nonce',
|
||||
'1.3.6.1.5.5.7.48.1.9': 'extended_revoke',
|
||||
}
|
||||
|
||||
|
||||
class ResponseDataExtension(Sequence):
|
||||
_fields = [
|
||||
('extn_id', ResponseDataExtensionId),
|
||||
('critical', Boolean, {'default': False}),
|
||||
('extn_value', ParsableOctetString),
|
||||
]
|
||||
|
||||
_oid_pair = ('extn_id', 'extn_value')
|
||||
_oid_specs = {
|
||||
'nonce': OctetString,
|
||||
'extended_revoke': Null,
|
||||
}
|
||||
|
||||
|
||||
class ResponseDataExtensions(SequenceOf):
|
||||
_child_spec = ResponseDataExtension
|
||||
|
||||
|
||||
class ResponseData(Sequence):
|
||||
_fields = [
|
||||
('version', Version, {'explicit': 0, 'default': 'v1'}),
|
||||
('responder_id', ResponderId),
|
||||
('produced_at', GeneralizedTime),
|
||||
('responses', Responses),
|
||||
('response_extensions', ResponseDataExtensions, {'explicit': 1, 'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class BasicOCSPResponse(Sequence):
|
||||
_fields = [
|
||||
('tbs_response_data', ResponseData),
|
||||
('signature_algorithm', SignedDigestAlgorithm),
|
||||
('signature', OctetBitString),
|
||||
('certs', Certificates, {'explicit': 0, 'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class ResponseBytes(Sequence):
|
||||
_fields = [
|
||||
('response_type', ResponseType),
|
||||
('response', ParsableOctetString),
|
||||
]
|
||||
|
||||
_oid_pair = ('response_type', 'response')
|
||||
_oid_specs = {
|
||||
'basic_ocsp_response': BasicOCSPResponse,
|
||||
}
|
||||
|
||||
|
||||
class OCSPResponse(Sequence):
|
||||
_fields = [
|
||||
('response_status', OCSPResponseStatus),
|
||||
('response_bytes', ResponseBytes, {'explicit': 0, 'optional': True}),
|
||||
]
|
||||
|
||||
_processed_extensions = False
|
||||
_critical_extensions = None
|
||||
_nonce_value = None
|
||||
_extended_revoke_value = None
|
||||
|
||||
def _set_extensions(self):
|
||||
"""
|
||||
Sets common named extensions to private attributes and creates a list
|
||||
of critical extensions
|
||||
"""
|
||||
|
||||
self._critical_extensions = set()
|
||||
|
||||
for extension in self['response_bytes']['response'].parsed['tbs_response_data']['response_extensions']:
|
||||
name = extension['extn_id'].native
|
||||
attribute_name = '_%s_value' % name
|
||||
if hasattr(self, attribute_name):
|
||||
setattr(self, attribute_name, extension['extn_value'].parsed)
|
||||
if extension['critical'].native:
|
||||
self._critical_extensions.add(name)
|
||||
|
||||
self._processed_extensions = True
|
||||
|
||||
@property
|
||||
def critical_extensions(self):
|
||||
"""
|
||||
Returns a set of the names (or OID if not a known extension) of the
|
||||
extensions marked as critical
|
||||
|
||||
:return:
|
||||
A set of unicode strings
|
||||
"""
|
||||
|
||||
if not self._processed_extensions:
|
||||
self._set_extensions()
|
||||
return self._critical_extensions
|
||||
|
||||
@property
|
||||
def nonce_value(self):
|
||||
"""
|
||||
This extension is used to prevent replay attacks on the request/response
|
||||
exchange
|
||||
|
||||
:return:
|
||||
None or an OctetString object
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._nonce_value
|
||||
|
||||
@property
|
||||
def extended_revoke_value(self):
|
||||
"""
|
||||
This extension is used to signal that the responder will return a
|
||||
"revoked" status for non-issued certificates.
|
||||
|
||||
:return:
|
||||
None or a Null object (if present)
|
||||
"""
|
||||
|
||||
if self._processed_extensions is False:
|
||||
self._set_extensions()
|
||||
return self._extended_revoke_value
|
||||
|
||||
@property
|
||||
def basic_ocsp_response(self):
|
||||
"""
|
||||
A shortcut into the BasicOCSPResponse sequence
|
||||
|
||||
:return:
|
||||
None or an asn1crypto.ocsp.BasicOCSPResponse object
|
||||
"""
|
||||
|
||||
return self['response_bytes']['response'].parsed
|
||||
|
||||
@property
|
||||
def response_data(self):
|
||||
"""
|
||||
A shortcut into the parsed, ResponseData sequence
|
||||
|
||||
:return:
|
||||
None or an asn1crypto.ocsp.ResponseData object
|
||||
"""
|
||||
|
||||
return self['response_bytes']['response'].parsed['tbs_response_data']
|
||||
292
jc/parsers/asn1crypto/parser.py
Normal file
292
jc/parsers/asn1crypto/parser.py
Normal file
@@ -0,0 +1,292 @@
|
||||
# coding: utf-8
|
||||
|
||||
"""
|
||||
Functions for parsing and dumping using the ASN.1 DER encoding. Exports the
|
||||
following items:
|
||||
|
||||
- emit()
|
||||
- parse()
|
||||
- peek()
|
||||
|
||||
Other type classes are defined that help compose the types listed above.
|
||||
"""
|
||||
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
import sys
|
||||
|
||||
from ._types import byte_cls, chr_cls, type_name
|
||||
from .util import int_from_bytes, int_to_bytes
|
||||
|
||||
_PY2 = sys.version_info <= (3,)
|
||||
_INSUFFICIENT_DATA_MESSAGE = 'Insufficient data - %s bytes requested but only %s available'
|
||||
_MAX_DEPTH = 10
|
||||
|
||||
|
||||
def emit(class_, method, tag, contents):
|
||||
"""
|
||||
Constructs a byte string of an ASN.1 DER-encoded value
|
||||
|
||||
This is typically not useful. Instead, use one of the standard classes from
|
||||
asn1crypto.core, or construct a new class with specific fields, and call the
|
||||
.dump() method.
|
||||
|
||||
:param class_:
|
||||
An integer ASN.1 class value: 0 (universal), 1 (application),
|
||||
2 (context), 3 (private)
|
||||
|
||||
:param method:
|
||||
An integer ASN.1 method value: 0 (primitive), 1 (constructed)
|
||||
|
||||
:param tag:
|
||||
An integer ASN.1 tag value
|
||||
|
||||
:param contents:
|
||||
A byte string of the encoded byte contents
|
||||
|
||||
:return:
|
||||
A byte string of the ASN.1 DER value (header and contents)
|
||||
"""
|
||||
|
||||
if not isinstance(class_, int):
|
||||
raise TypeError('class_ must be an integer, not %s' % type_name(class_))
|
||||
|
||||
if class_ < 0 or class_ > 3:
|
||||
raise ValueError('class_ must be one of 0, 1, 2 or 3, not %s' % class_)
|
||||
|
||||
if not isinstance(method, int):
|
||||
raise TypeError('method must be an integer, not %s' % type_name(method))
|
||||
|
||||
if method < 0 or method > 1:
|
||||
raise ValueError('method must be 0 or 1, not %s' % method)
|
||||
|
||||
if not isinstance(tag, int):
|
||||
raise TypeError('tag must be an integer, not %s' % type_name(tag))
|
||||
|
||||
if tag < 0:
|
||||
raise ValueError('tag must be greater than zero, not %s' % tag)
|
||||
|
||||
if not isinstance(contents, byte_cls):
|
||||
raise TypeError('contents must be a byte string, not %s' % type_name(contents))
|
||||
|
||||
return _dump_header(class_, method, tag, contents) + contents
|
||||
|
||||
|
||||
def parse(contents, strict=False):
|
||||
"""
|
||||
Parses a byte string of ASN.1 BER/DER-encoded data.
|
||||
|
||||
This is typically not useful. Instead, use one of the standard classes from
|
||||
asn1crypto.core, or construct a new class with specific fields, and call the
|
||||
.load() class method.
|
||||
|
||||
:param contents:
|
||||
A byte string of BER/DER-encoded data
|
||||
|
||||
:param strict:
|
||||
A boolean indicating if trailing data should be forbidden - if so, a
|
||||
ValueError will be raised when trailing data exists
|
||||
|
||||
:raises:
|
||||
ValueError - when the contents do not contain an ASN.1 header or are truncated in some way
|
||||
TypeError - when contents is not a byte string
|
||||
|
||||
:return:
|
||||
A 6-element tuple:
|
||||
- 0: integer class (0 to 3)
|
||||
- 1: integer method
|
||||
- 2: integer tag
|
||||
- 3: byte string header
|
||||
- 4: byte string content
|
||||
- 5: byte string trailer
|
||||
"""
|
||||
|
||||
if not isinstance(contents, byte_cls):
|
||||
raise TypeError('contents must be a byte string, not %s' % type_name(contents))
|
||||
|
||||
contents_len = len(contents)
|
||||
info, consumed = _parse(contents, contents_len)
|
||||
if strict and consumed != contents_len:
|
||||
raise ValueError('Extra data - %d bytes of trailing data were provided' % (contents_len - consumed))
|
||||
return info
|
||||
|
||||
|
||||
def peek(contents):
|
||||
"""
|
||||
Parses a byte string of ASN.1 BER/DER-encoded data to find the length
|
||||
|
||||
This is typically used to look into an encoded value to see how long the
|
||||
next chunk of ASN.1-encoded data is. Primarily it is useful when a
|
||||
value is a concatenation of multiple values.
|
||||
|
||||
:param contents:
|
||||
A byte string of BER/DER-encoded data
|
||||
|
||||
:raises:
|
||||
ValueError - when the contents do not contain an ASN.1 header or are truncated in some way
|
||||
TypeError - when contents is not a byte string
|
||||
|
||||
:return:
|
||||
An integer with the number of bytes occupied by the ASN.1 value
|
||||
"""
|
||||
|
||||
if not isinstance(contents, byte_cls):
|
||||
raise TypeError('contents must be a byte string, not %s' % type_name(contents))
|
||||
|
||||
info, consumed = _parse(contents, len(contents))
|
||||
return consumed
|
||||
|
||||
|
||||
def _parse(encoded_data, data_len, pointer=0, lengths_only=False, depth=0):
|
||||
"""
|
||||
Parses a byte string into component parts
|
||||
|
||||
:param encoded_data:
|
||||
A byte string that contains BER-encoded data
|
||||
|
||||
:param data_len:
|
||||
The integer length of the encoded data
|
||||
|
||||
:param pointer:
|
||||
The index in the byte string to parse from
|
||||
|
||||
:param lengths_only:
|
||||
A boolean to cause the call to return a 2-element tuple of the integer
|
||||
number of bytes in the header and the integer number of bytes in the
|
||||
contents. Internal use only.
|
||||
|
||||
:param depth:
|
||||
The recursion depth when evaluating indefinite-length encoding.
|
||||
|
||||
:return:
|
||||
A 2-element tuple:
|
||||
- 0: A tuple of (class_, method, tag, header, content, trailer)
|
||||
- 1: An integer indicating how many bytes were consumed
|
||||
"""
|
||||
|
||||
if depth > _MAX_DEPTH:
|
||||
raise ValueError('Indefinite-length recursion limit exceeded')
|
||||
|
||||
start = pointer
|
||||
|
||||
if data_len < pointer + 1:
|
||||
raise ValueError(_INSUFFICIENT_DATA_MESSAGE % (1, data_len - pointer))
|
||||
first_octet = ord(encoded_data[pointer]) if _PY2 else encoded_data[pointer]
|
||||
|
||||
pointer += 1
|
||||
|
||||
tag = first_octet & 31
|
||||
constructed = (first_octet >> 5) & 1
|
||||
# Base 128 length using 8th bit as continuation indicator
|
||||
if tag == 31:
|
||||
tag = 0
|
||||
while True:
|
||||
if data_len < pointer + 1:
|
||||
raise ValueError(_INSUFFICIENT_DATA_MESSAGE % (1, data_len - pointer))
|
||||
num = ord(encoded_data[pointer]) if _PY2 else encoded_data[pointer]
|
||||
pointer += 1
|
||||
if num == 0x80 and tag == 0:
|
||||
raise ValueError('Non-minimal tag encoding')
|
||||
tag *= 128
|
||||
tag += num & 127
|
||||
if num >> 7 == 0:
|
||||
break
|
||||
if tag < 31:
|
||||
raise ValueError('Non-minimal tag encoding')
|
||||
|
||||
if data_len < pointer + 1:
|
||||
raise ValueError(_INSUFFICIENT_DATA_MESSAGE % (1, data_len - pointer))
|
||||
length_octet = ord(encoded_data[pointer]) if _PY2 else encoded_data[pointer]
|
||||
pointer += 1
|
||||
trailer = b''
|
||||
|
||||
if length_octet >> 7 == 0:
|
||||
contents_end = pointer + (length_octet & 127)
|
||||
|
||||
else:
|
||||
length_octets = length_octet & 127
|
||||
if length_octets:
|
||||
if data_len < pointer + length_octets:
|
||||
raise ValueError(_INSUFFICIENT_DATA_MESSAGE % (length_octets, data_len - pointer))
|
||||
pointer += length_octets
|
||||
contents_end = pointer + int_from_bytes(encoded_data[pointer - length_octets:pointer], signed=False)
|
||||
|
||||
else:
|
||||
# To properly parse indefinite length values, we need to scan forward
|
||||
# parsing headers until we find a value with a length of zero. If we
|
||||
# just scanned looking for \x00\x00, nested indefinite length values
|
||||
# would not work.
|
||||
if not constructed:
|
||||
raise ValueError('Indefinite-length element must be constructed')
|
||||
contents_end = pointer
|
||||
while data_len < contents_end + 2 or encoded_data[contents_end:contents_end+2] != b'\x00\x00':
|
||||
_, contents_end = _parse(encoded_data, data_len, contents_end, lengths_only=True, depth=depth+1)
|
||||
contents_end += 2
|
||||
trailer = b'\x00\x00'
|
||||
|
||||
if contents_end > data_len:
|
||||
raise ValueError(_INSUFFICIENT_DATA_MESSAGE % (contents_end - pointer, data_len - pointer))
|
||||
|
||||
if lengths_only:
|
||||
return (pointer, contents_end)
|
||||
|
||||
return (
|
||||
(
|
||||
first_octet >> 6,
|
||||
constructed,
|
||||
tag,
|
||||
encoded_data[start:pointer],
|
||||
encoded_data[pointer:contents_end-len(trailer)],
|
||||
trailer
|
||||
),
|
||||
contents_end
|
||||
)
|
||||
|
||||
|
||||
def _dump_header(class_, method, tag, contents):
|
||||
"""
|
||||
Constructs the header bytes for an ASN.1 object
|
||||
|
||||
:param class_:
|
||||
An integer ASN.1 class value: 0 (universal), 1 (application),
|
||||
2 (context), 3 (private)
|
||||
|
||||
:param method:
|
||||
An integer ASN.1 method value: 0 (primitive), 1 (constructed)
|
||||
|
||||
:param tag:
|
||||
An integer ASN.1 tag value
|
||||
|
||||
:param contents:
|
||||
A byte string of the encoded byte contents
|
||||
|
||||
:return:
|
||||
A byte string of the ASN.1 DER header
|
||||
"""
|
||||
|
||||
header = b''
|
||||
|
||||
id_num = 0
|
||||
id_num |= class_ << 6
|
||||
id_num |= method << 5
|
||||
|
||||
if tag >= 31:
|
||||
cont_bit = 0
|
||||
while tag > 0:
|
||||
header = chr_cls(cont_bit | (tag & 0x7f)) + header
|
||||
if not cont_bit:
|
||||
cont_bit = 0x80
|
||||
tag = tag >> 7
|
||||
header = chr_cls(id_num | 31) + header
|
||||
else:
|
||||
header += chr_cls(id_num | tag)
|
||||
|
||||
length = len(contents)
|
||||
if length <= 127:
|
||||
header += chr_cls(length)
|
||||
else:
|
||||
length_bytes = int_to_bytes(length)
|
||||
header += chr_cls(0x80 | len(length_bytes))
|
||||
header += length_bytes
|
||||
|
||||
return header
|
||||
84
jc/parsers/asn1crypto/pdf.py
Normal file
84
jc/parsers/asn1crypto/pdf.py
Normal file
@@ -0,0 +1,84 @@
|
||||
# coding: utf-8
|
||||
|
||||
"""
|
||||
ASN.1 type classes for PDF signature structures. Adds extra oid mapping and
|
||||
value parsing to asn1crypto.x509.Extension() and asn1crypto.xms.CMSAttribute().
|
||||
"""
|
||||
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
from .cms import CMSAttributeType, CMSAttribute
|
||||
from .core import (
|
||||
Boolean,
|
||||
Integer,
|
||||
Null,
|
||||
ObjectIdentifier,
|
||||
OctetString,
|
||||
Sequence,
|
||||
SequenceOf,
|
||||
SetOf,
|
||||
)
|
||||
from .crl import CertificateList
|
||||
from .ocsp import OCSPResponse
|
||||
from .x509 import (
|
||||
Extension,
|
||||
ExtensionId,
|
||||
GeneralName,
|
||||
KeyPurposeId,
|
||||
)
|
||||
|
||||
|
||||
class AdobeArchiveRevInfo(Sequence):
|
||||
_fields = [
|
||||
('version', Integer)
|
||||
]
|
||||
|
||||
|
||||
class AdobeTimestamp(Sequence):
|
||||
_fields = [
|
||||
('version', Integer),
|
||||
('location', GeneralName),
|
||||
('requires_auth', Boolean, {'optional': True, 'default': False}),
|
||||
]
|
||||
|
||||
|
||||
class OtherRevInfo(Sequence):
|
||||
_fields = [
|
||||
('type', ObjectIdentifier),
|
||||
('value', OctetString),
|
||||
]
|
||||
|
||||
|
||||
class SequenceOfCertificateList(SequenceOf):
|
||||
_child_spec = CertificateList
|
||||
|
||||
|
||||
class SequenceOfOCSPResponse(SequenceOf):
|
||||
_child_spec = OCSPResponse
|
||||
|
||||
|
||||
class SequenceOfOtherRevInfo(SequenceOf):
|
||||
_child_spec = OtherRevInfo
|
||||
|
||||
|
||||
class RevocationInfoArchival(Sequence):
|
||||
_fields = [
|
||||
('crl', SequenceOfCertificateList, {'explicit': 0, 'optional': True}),
|
||||
('ocsp', SequenceOfOCSPResponse, {'explicit': 1, 'optional': True}),
|
||||
('other_rev_info', SequenceOfOtherRevInfo, {'explicit': 2, 'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class SetOfRevocationInfoArchival(SetOf):
|
||||
_child_spec = RevocationInfoArchival
|
||||
|
||||
|
||||
ExtensionId._map['1.2.840.113583.1.1.9.2'] = 'adobe_archive_rev_info'
|
||||
ExtensionId._map['1.2.840.113583.1.1.9.1'] = 'adobe_timestamp'
|
||||
ExtensionId._map['1.2.840.113583.1.1.10'] = 'adobe_ppklite_credential'
|
||||
Extension._oid_specs['adobe_archive_rev_info'] = AdobeArchiveRevInfo
|
||||
Extension._oid_specs['adobe_timestamp'] = AdobeTimestamp
|
||||
Extension._oid_specs['adobe_ppklite_credential'] = Null
|
||||
KeyPurposeId._map['1.2.840.113583.1.1.5'] = 'pdf_signing'
|
||||
CMSAttributeType._map['1.2.840.113583.1.1.8'] = 'adobe_revocation_info_archival'
|
||||
CMSAttribute._oid_specs['adobe_revocation_info_archival'] = SetOfRevocationInfoArchival
|
||||
222
jc/parsers/asn1crypto/pem.py
Normal file
222
jc/parsers/asn1crypto/pem.py
Normal file
@@ -0,0 +1,222 @@
|
||||
# coding: utf-8
|
||||
|
||||
"""
|
||||
Encoding DER to PEM and decoding PEM to DER. Exports the following items:
|
||||
|
||||
- armor()
|
||||
- detect()
|
||||
- unarmor()
|
||||
|
||||
"""
|
||||
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
import base64
|
||||
import re
|
||||
import sys
|
||||
|
||||
from ._errors import unwrap
|
||||
from ._types import type_name as _type_name, str_cls, byte_cls
|
||||
|
||||
if sys.version_info < (3,):
|
||||
from cStringIO import StringIO as BytesIO
|
||||
else:
|
||||
from io import BytesIO
|
||||
|
||||
|
||||
def detect(byte_string):
|
||||
"""
|
||||
Detect if a byte string seems to contain a PEM-encoded block
|
||||
|
||||
:param byte_string:
|
||||
A byte string to look through
|
||||
|
||||
:return:
|
||||
A boolean, indicating if a PEM-encoded block is contained in the byte
|
||||
string
|
||||
"""
|
||||
|
||||
if not isinstance(byte_string, byte_cls):
|
||||
raise TypeError(unwrap(
|
||||
'''
|
||||
byte_string must be a byte string, not %s
|
||||
''',
|
||||
_type_name(byte_string)
|
||||
))
|
||||
|
||||
return byte_string.find(b'-----BEGIN') != -1 or byte_string.find(b'---- BEGIN') != -1
|
||||
|
||||
|
||||
def armor(type_name, der_bytes, headers=None):
|
||||
"""
|
||||
Armors a DER-encoded byte string in PEM
|
||||
|
||||
:param type_name:
|
||||
A unicode string that will be capitalized and placed in the header
|
||||
and footer of the block. E.g. "CERTIFICATE", "PRIVATE KEY", etc. This
|
||||
will appear as "-----BEGIN CERTIFICATE-----" and
|
||||
"-----END CERTIFICATE-----".
|
||||
|
||||
:param der_bytes:
|
||||
A byte string to be armored
|
||||
|
||||
:param headers:
|
||||
An OrderedDict of the header lines to write after the BEGIN line
|
||||
|
||||
:return:
|
||||
A byte string of the PEM block
|
||||
"""
|
||||
|
||||
if not isinstance(der_bytes, byte_cls):
|
||||
raise TypeError(unwrap(
|
||||
'''
|
||||
der_bytes must be a byte string, not %s
|
||||
''' % _type_name(der_bytes)
|
||||
))
|
||||
|
||||
if not isinstance(type_name, str_cls):
|
||||
raise TypeError(unwrap(
|
||||
'''
|
||||
type_name must be a unicode string, not %s
|
||||
''',
|
||||
_type_name(type_name)
|
||||
))
|
||||
|
||||
type_name = type_name.upper().encode('ascii')
|
||||
|
||||
output = BytesIO()
|
||||
output.write(b'-----BEGIN ')
|
||||
output.write(type_name)
|
||||
output.write(b'-----\n')
|
||||
if headers:
|
||||
for key in headers:
|
||||
output.write(key.encode('ascii'))
|
||||
output.write(b': ')
|
||||
output.write(headers[key].encode('ascii'))
|
||||
output.write(b'\n')
|
||||
output.write(b'\n')
|
||||
b64_bytes = base64.b64encode(der_bytes)
|
||||
b64_len = len(b64_bytes)
|
||||
i = 0
|
||||
while i < b64_len:
|
||||
output.write(b64_bytes[i:i + 64])
|
||||
output.write(b'\n')
|
||||
i += 64
|
||||
output.write(b'-----END ')
|
||||
output.write(type_name)
|
||||
output.write(b'-----\n')
|
||||
|
||||
return output.getvalue()
|
||||
|
||||
|
||||
def _unarmor(pem_bytes):
|
||||
"""
|
||||
Convert a PEM-encoded byte string into one or more DER-encoded byte strings
|
||||
|
||||
:param pem_bytes:
|
||||
A byte string of the PEM-encoded data
|
||||
|
||||
:raises:
|
||||
ValueError - when the pem_bytes do not appear to be PEM-encoded bytes
|
||||
|
||||
:return:
|
||||
A generator of 3-element tuples in the format: (object_type, headers,
|
||||
der_bytes). The object_type is a unicode string of what is between
|
||||
"-----BEGIN " and "-----". Examples include: "CERTIFICATE",
|
||||
"PUBLIC KEY", "PRIVATE KEY". The headers is a dict containing any lines
|
||||
in the form "Name: Value" that are right after the begin line.
|
||||
"""
|
||||
|
||||
if not isinstance(pem_bytes, byte_cls):
|
||||
raise TypeError(unwrap(
|
||||
'''
|
||||
pem_bytes must be a byte string, not %s
|
||||
''',
|
||||
_type_name(pem_bytes)
|
||||
))
|
||||
|
||||
# Valid states include: "trash", "headers", "body"
|
||||
state = 'trash'
|
||||
headers = {}
|
||||
base64_data = b''
|
||||
object_type = None
|
||||
|
||||
found_start = False
|
||||
found_end = False
|
||||
|
||||
for line in pem_bytes.splitlines(False):
|
||||
if line == b'':
|
||||
continue
|
||||
|
||||
if state == "trash":
|
||||
# Look for a starting line since some CA cert bundle show the cert
|
||||
# into in a parsed format above each PEM block
|
||||
type_name_match = re.match(b'^(?:---- |-----)BEGIN ([A-Z0-9 ]+)(?: ----|-----)', line)
|
||||
if not type_name_match:
|
||||
continue
|
||||
object_type = type_name_match.group(1).decode('ascii')
|
||||
|
||||
found_start = True
|
||||
state = 'headers'
|
||||
continue
|
||||
|
||||
if state == 'headers':
|
||||
if line.find(b':') == -1:
|
||||
state = 'body'
|
||||
else:
|
||||
decoded_line = line.decode('ascii')
|
||||
name, value = decoded_line.split(':', 1)
|
||||
headers[name] = value.strip()
|
||||
continue
|
||||
|
||||
if state == 'body':
|
||||
if line[0:5] in (b'-----', b'---- '):
|
||||
der_bytes = base64.b64decode(base64_data)
|
||||
|
||||
yield (object_type, headers, der_bytes)
|
||||
|
||||
state = 'trash'
|
||||
headers = {}
|
||||
base64_data = b''
|
||||
object_type = None
|
||||
found_end = True
|
||||
continue
|
||||
|
||||
base64_data += line
|
||||
|
||||
if not found_start or not found_end:
|
||||
raise ValueError(unwrap(
|
||||
'''
|
||||
pem_bytes does not appear to contain PEM-encoded data - no
|
||||
BEGIN/END combination found
|
||||
'''
|
||||
))
|
||||
|
||||
|
||||
def unarmor(pem_bytes, multiple=False):
|
||||
"""
|
||||
Convert a PEM-encoded byte string into a DER-encoded byte string
|
||||
|
||||
:param pem_bytes:
|
||||
A byte string of the PEM-encoded data
|
||||
|
||||
:param multiple:
|
||||
If True, function will return a generator
|
||||
|
||||
:raises:
|
||||
ValueError - when the pem_bytes do not appear to be PEM-encoded bytes
|
||||
|
||||
:return:
|
||||
A 3-element tuple (object_name, headers, der_bytes). The object_name is
|
||||
a unicode string of what is between "-----BEGIN " and "-----". Examples
|
||||
include: "CERTIFICATE", "PUBLIC KEY", "PRIVATE KEY". The headers is a
|
||||
dict containing any lines in the form "Name: Value" that are right
|
||||
after the begin line.
|
||||
"""
|
||||
|
||||
generator = _unarmor(pem_bytes)
|
||||
|
||||
if not multiple:
|
||||
return next(generator)
|
||||
|
||||
return generator
|
||||
193
jc/parsers/asn1crypto/pkcs12.py
Normal file
193
jc/parsers/asn1crypto/pkcs12.py
Normal file
@@ -0,0 +1,193 @@
|
||||
# coding: utf-8
|
||||
|
||||
"""
|
||||
ASN.1 type classes for PKCS#12 files. Exports the following items:
|
||||
|
||||
- CertBag()
|
||||
- CrlBag()
|
||||
- Pfx()
|
||||
- SafeBag()
|
||||
- SecretBag()
|
||||
|
||||
Other type classes are defined that help compose the types listed above.
|
||||
"""
|
||||
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
from .algos import DigestInfo
|
||||
from .cms import ContentInfo, SignedData
|
||||
from .core import (
|
||||
Any,
|
||||
BMPString,
|
||||
Integer,
|
||||
ObjectIdentifier,
|
||||
OctetString,
|
||||
ParsableOctetString,
|
||||
Sequence,
|
||||
SequenceOf,
|
||||
SetOf,
|
||||
)
|
||||
from .keys import PrivateKeyInfo, EncryptedPrivateKeyInfo
|
||||
from .x509 import Certificate, KeyPurposeId
|
||||
|
||||
|
||||
# The structures in this file are taken from https://tools.ietf.org/html/rfc7292
|
||||
|
||||
class MacData(Sequence):
|
||||
_fields = [
|
||||
('mac', DigestInfo),
|
||||
('mac_salt', OctetString),
|
||||
('iterations', Integer, {'default': 1}),
|
||||
]
|
||||
|
||||
|
||||
class Version(Integer):
|
||||
_map = {
|
||||
3: 'v3'
|
||||
}
|
||||
|
||||
|
||||
class AttributeType(ObjectIdentifier):
|
||||
_map = {
|
||||
# https://tools.ietf.org/html/rfc2985#page-18
|
||||
'1.2.840.113549.1.9.20': 'friendly_name',
|
||||
'1.2.840.113549.1.9.21': 'local_key_id',
|
||||
# https://support.microsoft.com/en-us/kb/287547
|
||||
'1.3.6.1.4.1.311.17.1': 'microsoft_local_machine_keyset',
|
||||
# https://github.com/frohoff/jdk8u-dev-jdk/blob/master/src/share/classes/sun/security/pkcs12/PKCS12KeyStore.java
|
||||
# this is a set of OIDs, representing key usage, the usual value is a SET of one element OID 2.5.29.37.0
|
||||
'2.16.840.1.113894.746875.1.1': 'trusted_key_usage',
|
||||
}
|
||||
|
||||
|
||||
class SetOfAny(SetOf):
|
||||
_child_spec = Any
|
||||
|
||||
|
||||
class SetOfBMPString(SetOf):
|
||||
_child_spec = BMPString
|
||||
|
||||
|
||||
class SetOfOctetString(SetOf):
|
||||
_child_spec = OctetString
|
||||
|
||||
|
||||
class SetOfKeyPurposeId(SetOf):
|
||||
_child_spec = KeyPurposeId
|
||||
|
||||
|
||||
class Attribute(Sequence):
|
||||
_fields = [
|
||||
('type', AttributeType),
|
||||
('values', None),
|
||||
]
|
||||
|
||||
_oid_specs = {
|
||||
'friendly_name': SetOfBMPString,
|
||||
'local_key_id': SetOfOctetString,
|
||||
'microsoft_csp_name': SetOfBMPString,
|
||||
'trusted_key_usage': SetOfKeyPurposeId,
|
||||
}
|
||||
|
||||
def _values_spec(self):
|
||||
return self._oid_specs.get(self['type'].native, SetOfAny)
|
||||
|
||||
_spec_callbacks = {
|
||||
'values': _values_spec
|
||||
}
|
||||
|
||||
|
||||
class Attributes(SetOf):
|
||||
_child_spec = Attribute
|
||||
|
||||
|
||||
class Pfx(Sequence):
|
||||
_fields = [
|
||||
('version', Version),
|
||||
('auth_safe', ContentInfo),
|
||||
('mac_data', MacData, {'optional': True})
|
||||
]
|
||||
|
||||
_authenticated_safe = None
|
||||
|
||||
@property
|
||||
def authenticated_safe(self):
|
||||
if self._authenticated_safe is None:
|
||||
content = self['auth_safe']['content']
|
||||
if isinstance(content, SignedData):
|
||||
content = content['content_info']['content']
|
||||
self._authenticated_safe = AuthenticatedSafe.load(content.native)
|
||||
return self._authenticated_safe
|
||||
|
||||
|
||||
class AuthenticatedSafe(SequenceOf):
|
||||
_child_spec = ContentInfo
|
||||
|
||||
|
||||
class BagId(ObjectIdentifier):
|
||||
_map = {
|
||||
'1.2.840.113549.1.12.10.1.1': 'key_bag',
|
||||
'1.2.840.113549.1.12.10.1.2': 'pkcs8_shrouded_key_bag',
|
||||
'1.2.840.113549.1.12.10.1.3': 'cert_bag',
|
||||
'1.2.840.113549.1.12.10.1.4': 'crl_bag',
|
||||
'1.2.840.113549.1.12.10.1.5': 'secret_bag',
|
||||
'1.2.840.113549.1.12.10.1.6': 'safe_contents',
|
||||
}
|
||||
|
||||
|
||||
class CertId(ObjectIdentifier):
|
||||
_map = {
|
||||
'1.2.840.113549.1.9.22.1': 'x509',
|
||||
'1.2.840.113549.1.9.22.2': 'sdsi',
|
||||
}
|
||||
|
||||
|
||||
class CertBag(Sequence):
|
||||
_fields = [
|
||||
('cert_id', CertId),
|
||||
('cert_value', ParsableOctetString, {'explicit': 0}),
|
||||
]
|
||||
|
||||
_oid_pair = ('cert_id', 'cert_value')
|
||||
_oid_specs = {
|
||||
'x509': Certificate,
|
||||
}
|
||||
|
||||
|
||||
class CrlBag(Sequence):
|
||||
_fields = [
|
||||
('crl_id', ObjectIdentifier),
|
||||
('crl_value', OctetString, {'explicit': 0}),
|
||||
]
|
||||
|
||||
|
||||
class SecretBag(Sequence):
|
||||
_fields = [
|
||||
('secret_type_id', ObjectIdentifier),
|
||||
('secret_value', OctetString, {'explicit': 0}),
|
||||
]
|
||||
|
||||
|
||||
class SafeContents(SequenceOf):
|
||||
pass
|
||||
|
||||
|
||||
class SafeBag(Sequence):
|
||||
_fields = [
|
||||
('bag_id', BagId),
|
||||
('bag_value', Any, {'explicit': 0}),
|
||||
('bag_attributes', Attributes, {'optional': True}),
|
||||
]
|
||||
|
||||
_oid_pair = ('bag_id', 'bag_value')
|
||||
_oid_specs = {
|
||||
'key_bag': PrivateKeyInfo,
|
||||
'pkcs8_shrouded_key_bag': EncryptedPrivateKeyInfo,
|
||||
'cert_bag': CertBag,
|
||||
'crl_bag': CrlBag,
|
||||
'secret_bag': SecretBag,
|
||||
'safe_contents': SafeContents
|
||||
}
|
||||
|
||||
|
||||
SafeContents._child_spec = SafeBag
|
||||
310
jc/parsers/asn1crypto/tsp.py
Normal file
310
jc/parsers/asn1crypto/tsp.py
Normal file
@@ -0,0 +1,310 @@
|
||||
# coding: utf-8
|
||||
|
||||
"""
|
||||
ASN.1 type classes for the time stamp protocol (TSP). Exports the following
|
||||
items:
|
||||
|
||||
- TimeStampReq()
|
||||
- TimeStampResp()
|
||||
|
||||
Also adds TimeStampedData() support to asn1crypto.cms.ContentInfo(),
|
||||
TimeStampedData() and TSTInfo() support to
|
||||
asn1crypto.cms.EncapsulatedContentInfo() and some oids and value parsers to
|
||||
asn1crypto.cms.CMSAttribute().
|
||||
|
||||
Other type classes are defined that help compose the types listed above.
|
||||
"""
|
||||
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
from .algos import DigestAlgorithm
|
||||
from .cms import (
|
||||
CMSAttribute,
|
||||
CMSAttributeType,
|
||||
ContentInfo,
|
||||
ContentType,
|
||||
EncapsulatedContentInfo,
|
||||
)
|
||||
from .core import (
|
||||
Any,
|
||||
BitString,
|
||||
Boolean,
|
||||
Choice,
|
||||
GeneralizedTime,
|
||||
IA5String,
|
||||
Integer,
|
||||
ObjectIdentifier,
|
||||
OctetString,
|
||||
Sequence,
|
||||
SequenceOf,
|
||||
SetOf,
|
||||
UTF8String,
|
||||
)
|
||||
from .crl import CertificateList
|
||||
from .x509 import (
|
||||
Attributes,
|
||||
CertificatePolicies,
|
||||
GeneralName,
|
||||
GeneralNames,
|
||||
)
|
||||
|
||||
|
||||
# The structures in this file are based on https://tools.ietf.org/html/rfc3161,
|
||||
# https://tools.ietf.org/html/rfc4998, https://tools.ietf.org/html/rfc5544,
|
||||
# https://tools.ietf.org/html/rfc5035, https://tools.ietf.org/html/rfc2634
|
||||
|
||||
class Version(Integer):
|
||||
_map = {
|
||||
0: 'v0',
|
||||
1: 'v1',
|
||||
2: 'v2',
|
||||
3: 'v3',
|
||||
4: 'v4',
|
||||
5: 'v5',
|
||||
}
|
||||
|
||||
|
||||
class MessageImprint(Sequence):
|
||||
_fields = [
|
||||
('hash_algorithm', DigestAlgorithm),
|
||||
('hashed_message', OctetString),
|
||||
]
|
||||
|
||||
|
||||
class Accuracy(Sequence):
|
||||
_fields = [
|
||||
('seconds', Integer, {'optional': True}),
|
||||
('millis', Integer, {'implicit': 0, 'optional': True}),
|
||||
('micros', Integer, {'implicit': 1, 'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class Extension(Sequence):
|
||||
_fields = [
|
||||
('extn_id', ObjectIdentifier),
|
||||
('critical', Boolean, {'default': False}),
|
||||
('extn_value', OctetString),
|
||||
]
|
||||
|
||||
|
||||
class Extensions(SequenceOf):
|
||||
_child_spec = Extension
|
||||
|
||||
|
||||
class TSTInfo(Sequence):
|
||||
_fields = [
|
||||
('version', Version),
|
||||
('policy', ObjectIdentifier),
|
||||
('message_imprint', MessageImprint),
|
||||
('serial_number', Integer),
|
||||
('gen_time', GeneralizedTime),
|
||||
('accuracy', Accuracy, {'optional': True}),
|
||||
('ordering', Boolean, {'default': False}),
|
||||
('nonce', Integer, {'optional': True}),
|
||||
('tsa', GeneralName, {'explicit': 0, 'optional': True}),
|
||||
('extensions', Extensions, {'implicit': 1, 'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class TimeStampReq(Sequence):
|
||||
_fields = [
|
||||
('version', Version),
|
||||
('message_imprint', MessageImprint),
|
||||
('req_policy', ObjectIdentifier, {'optional': True}),
|
||||
('nonce', Integer, {'optional': True}),
|
||||
('cert_req', Boolean, {'default': False}),
|
||||
('extensions', Extensions, {'implicit': 0, 'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class PKIStatus(Integer):
|
||||
_map = {
|
||||
0: 'granted',
|
||||
1: 'granted_with_mods',
|
||||
2: 'rejection',
|
||||
3: 'waiting',
|
||||
4: 'revocation_warning',
|
||||
5: 'revocation_notification',
|
||||
}
|
||||
|
||||
|
||||
class PKIFreeText(SequenceOf):
|
||||
_child_spec = UTF8String
|
||||
|
||||
|
||||
class PKIFailureInfo(BitString):
|
||||
_map = {
|
||||
0: 'bad_alg',
|
||||
2: 'bad_request',
|
||||
5: 'bad_data_format',
|
||||
14: 'time_not_available',
|
||||
15: 'unaccepted_policy',
|
||||
16: 'unaccepted_extensions',
|
||||
17: 'add_info_not_available',
|
||||
25: 'system_failure',
|
||||
}
|
||||
|
||||
|
||||
class PKIStatusInfo(Sequence):
|
||||
_fields = [
|
||||
('status', PKIStatus),
|
||||
('status_string', PKIFreeText, {'optional': True}),
|
||||
('fail_info', PKIFailureInfo, {'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class TimeStampResp(Sequence):
|
||||
_fields = [
|
||||
('status', PKIStatusInfo),
|
||||
('time_stamp_token', ContentInfo),
|
||||
]
|
||||
|
||||
|
||||
class MetaData(Sequence):
|
||||
_fields = [
|
||||
('hash_protected', Boolean),
|
||||
('file_name', UTF8String, {'optional': True}),
|
||||
('media_type', IA5String, {'optional': True}),
|
||||
('other_meta_data', Attributes, {'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class TimeStampAndCRL(Sequence):
|
||||
_fields = [
|
||||
('time_stamp', EncapsulatedContentInfo),
|
||||
('crl', CertificateList, {'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class TimeStampTokenEvidence(SequenceOf):
|
||||
_child_spec = TimeStampAndCRL
|
||||
|
||||
|
||||
class DigestAlgorithms(SequenceOf):
|
||||
_child_spec = DigestAlgorithm
|
||||
|
||||
|
||||
class EncryptionInfo(Sequence):
|
||||
_fields = [
|
||||
('encryption_info_type', ObjectIdentifier),
|
||||
('encryption_info_value', Any),
|
||||
]
|
||||
|
||||
|
||||
class PartialHashtree(SequenceOf):
|
||||
_child_spec = OctetString
|
||||
|
||||
|
||||
class PartialHashtrees(SequenceOf):
|
||||
_child_spec = PartialHashtree
|
||||
|
||||
|
||||
class ArchiveTimeStamp(Sequence):
|
||||
_fields = [
|
||||
('digest_algorithm', DigestAlgorithm, {'implicit': 0, 'optional': True}),
|
||||
('attributes', Attributes, {'implicit': 1, 'optional': True}),
|
||||
('reduced_hashtree', PartialHashtrees, {'implicit': 2, 'optional': True}),
|
||||
('time_stamp', ContentInfo),
|
||||
]
|
||||
|
||||
|
||||
class ArchiveTimeStampSequence(SequenceOf):
|
||||
_child_spec = ArchiveTimeStamp
|
||||
|
||||
|
||||
class EvidenceRecord(Sequence):
|
||||
_fields = [
|
||||
('version', Version),
|
||||
('digest_algorithms', DigestAlgorithms),
|
||||
('crypto_infos', Attributes, {'implicit': 0, 'optional': True}),
|
||||
('encryption_info', EncryptionInfo, {'implicit': 1, 'optional': True}),
|
||||
('archive_time_stamp_sequence', ArchiveTimeStampSequence),
|
||||
]
|
||||
|
||||
|
||||
class OtherEvidence(Sequence):
|
||||
_fields = [
|
||||
('oe_type', ObjectIdentifier),
|
||||
('oe_value', Any),
|
||||
]
|
||||
|
||||
|
||||
class Evidence(Choice):
|
||||
_alternatives = [
|
||||
('tst_evidence', TimeStampTokenEvidence, {'implicit': 0}),
|
||||
('ers_evidence', EvidenceRecord, {'implicit': 1}),
|
||||
('other_evidence', OtherEvidence, {'implicit': 2}),
|
||||
]
|
||||
|
||||
|
||||
class TimeStampedData(Sequence):
|
||||
_fields = [
|
||||
('version', Version),
|
||||
('data_uri', IA5String, {'optional': True}),
|
||||
('meta_data', MetaData, {'optional': True}),
|
||||
('content', OctetString, {'optional': True}),
|
||||
('temporal_evidence', Evidence),
|
||||
]
|
||||
|
||||
|
||||
class IssuerSerial(Sequence):
|
||||
_fields = [
|
||||
('issuer', GeneralNames),
|
||||
('serial_number', Integer),
|
||||
]
|
||||
|
||||
|
||||
class ESSCertID(Sequence):
|
||||
_fields = [
|
||||
('cert_hash', OctetString),
|
||||
('issuer_serial', IssuerSerial, {'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class ESSCertIDs(SequenceOf):
|
||||
_child_spec = ESSCertID
|
||||
|
||||
|
||||
class SigningCertificate(Sequence):
|
||||
_fields = [
|
||||
('certs', ESSCertIDs),
|
||||
('policies', CertificatePolicies, {'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class SetOfSigningCertificates(SetOf):
|
||||
_child_spec = SigningCertificate
|
||||
|
||||
|
||||
class ESSCertIDv2(Sequence):
|
||||
_fields = [
|
||||
('hash_algorithm', DigestAlgorithm, {'default': {'algorithm': 'sha256'}}),
|
||||
('cert_hash', OctetString),
|
||||
('issuer_serial', IssuerSerial, {'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class ESSCertIDv2s(SequenceOf):
|
||||
_child_spec = ESSCertIDv2
|
||||
|
||||
|
||||
class SigningCertificateV2(Sequence):
|
||||
_fields = [
|
||||
('certs', ESSCertIDv2s),
|
||||
('policies', CertificatePolicies, {'optional': True}),
|
||||
]
|
||||
|
||||
|
||||
class SetOfSigningCertificatesV2(SetOf):
|
||||
_child_spec = SigningCertificateV2
|
||||
|
||||
|
||||
EncapsulatedContentInfo._oid_specs['tst_info'] = TSTInfo
|
||||
EncapsulatedContentInfo._oid_specs['timestamped_data'] = TimeStampedData
|
||||
ContentInfo._oid_specs['timestamped_data'] = TimeStampedData
|
||||
ContentType._map['1.2.840.113549.1.9.16.1.4'] = 'tst_info'
|
||||
ContentType._map['1.2.840.113549.1.9.16.1.31'] = 'timestamped_data'
|
||||
CMSAttributeType._map['1.2.840.113549.1.9.16.2.12'] = 'signing_certificate'
|
||||
CMSAttribute._oid_specs['signing_certificate'] = SetOfSigningCertificates
|
||||
CMSAttributeType._map['1.2.840.113549.1.9.16.2.47'] = 'signing_certificate_v2'
|
||||
CMSAttribute._oid_specs['signing_certificate_v2'] = SetOfSigningCertificatesV2
|
||||
878
jc/parsers/asn1crypto/util.py
Normal file
878
jc/parsers/asn1crypto/util.py
Normal file
@@ -0,0 +1,878 @@
|
||||
# coding: utf-8
|
||||
|
||||
"""
|
||||
Miscellaneous data helpers, including functions for converting integers to and
|
||||
from bytes and UTC timezone. Exports the following items:
|
||||
|
||||
- OrderedDict()
|
||||
- int_from_bytes()
|
||||
- int_to_bytes()
|
||||
- timezone.utc
|
||||
- utc_with_dst
|
||||
- create_timezone()
|
||||
- inet_ntop()
|
||||
- inet_pton()
|
||||
- uri_to_iri()
|
||||
- iri_to_uri()
|
||||
"""
|
||||
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
import math
|
||||
import sys
|
||||
from datetime import datetime, date, timedelta, tzinfo
|
||||
|
||||
from ._errors import unwrap
|
||||
from ._iri import iri_to_uri, uri_to_iri # noqa
|
||||
from ._ordereddict import OrderedDict # noqa
|
||||
from ._types import type_name
|
||||
|
||||
if sys.platform == 'win32':
|
||||
from ._inet import inet_ntop, inet_pton
|
||||
else:
|
||||
from socket import inet_ntop, inet_pton # noqa
|
||||
|
||||
|
||||
# Python 2
|
||||
if sys.version_info <= (3,):
|
||||
|
||||
def int_to_bytes(value, signed=False, width=None):
|
||||
"""
|
||||
Converts an integer to a byte string
|
||||
|
||||
:param value:
|
||||
The integer to convert
|
||||
|
||||
:param signed:
|
||||
If the byte string should be encoded using two's complement
|
||||
|
||||
:param width:
|
||||
If None, the minimal possible size (but at least 1),
|
||||
otherwise an integer of the byte width for the return value
|
||||
|
||||
:return:
|
||||
A byte string
|
||||
"""
|
||||
|
||||
if value == 0 and width == 0:
|
||||
return b''
|
||||
|
||||
# Handle negatives in two's complement
|
||||
is_neg = False
|
||||
if signed and value < 0:
|
||||
is_neg = True
|
||||
bits = int(math.ceil(len('%x' % abs(value)) / 2.0) * 8)
|
||||
value = (value + (1 << bits)) % (1 << bits)
|
||||
|
||||
hex_str = '%x' % value
|
||||
if len(hex_str) & 1:
|
||||
hex_str = '0' + hex_str
|
||||
|
||||
output = hex_str.decode('hex')
|
||||
|
||||
if signed and not is_neg and ord(output[0:1]) & 0x80:
|
||||
output = b'\x00' + output
|
||||
|
||||
if width is not None:
|
||||
if len(output) > width:
|
||||
raise OverflowError('int too big to convert')
|
||||
if is_neg:
|
||||
pad_char = b'\xFF'
|
||||
else:
|
||||
pad_char = b'\x00'
|
||||
output = (pad_char * (width - len(output))) + output
|
||||
elif is_neg and ord(output[0:1]) & 0x80 == 0:
|
||||
output = b'\xFF' + output
|
||||
|
||||
return output
|
||||
|
||||
def int_from_bytes(value, signed=False):
|
||||
"""
|
||||
Converts a byte string to an integer
|
||||
|
||||
:param value:
|
||||
The byte string to convert
|
||||
|
||||
:param signed:
|
||||
If the byte string should be interpreted using two's complement
|
||||
|
||||
:return:
|
||||
An integer
|
||||
"""
|
||||
|
||||
if value == b'':
|
||||
return 0
|
||||
|
||||
num = long(value.encode("hex"), 16) # noqa
|
||||
|
||||
if not signed:
|
||||
return num
|
||||
|
||||
# Check for sign bit and handle two's complement
|
||||
if ord(value[0:1]) & 0x80:
|
||||
bit_len = len(value) * 8
|
||||
return num - (1 << bit_len)
|
||||
|
||||
return num
|
||||
|
||||
class timezone(tzinfo): # noqa
|
||||
"""
|
||||
Implements datetime.timezone for py2.
|
||||
Only full minute offsets are supported.
|
||||
DST is not supported.
|
||||
"""
|
||||
|
||||
def __init__(self, offset, name=None):
|
||||
"""
|
||||
:param offset:
|
||||
A timedelta with this timezone's offset from UTC
|
||||
|
||||
:param name:
|
||||
Name of the timezone; if None, generate one.
|
||||
"""
|
||||
|
||||
if not timedelta(hours=-24) < offset < timedelta(hours=24):
|
||||
raise ValueError('Offset must be in [-23:59, 23:59]')
|
||||
|
||||
if offset.seconds % 60 or offset.microseconds:
|
||||
raise ValueError('Offset must be full minutes')
|
||||
|
||||
self._offset = offset
|
||||
|
||||
if name is not None:
|
||||
self._name = name
|
||||
elif not offset:
|
||||
self._name = 'UTC'
|
||||
else:
|
||||
self._name = 'UTC' + _format_offset(offset)
|
||||
|
||||
def __eq__(self, other):
|
||||
"""
|
||||
Compare two timezones
|
||||
|
||||
:param other:
|
||||
The other timezone to compare to
|
||||
|
||||
:return:
|
||||
A boolean
|
||||
"""
|
||||
|
||||
if type(other) != timezone:
|
||||
return False
|
||||
return self._offset == other._offset
|
||||
|
||||
def __getinitargs__(self):
|
||||
"""
|
||||
Called by tzinfo.__reduce__ to support pickle and copy.
|
||||
|
||||
:return:
|
||||
offset and name, to be used for __init__
|
||||
"""
|
||||
|
||||
return self._offset, self._name
|
||||
|
||||
def tzname(self, dt):
|
||||
"""
|
||||
:param dt:
|
||||
A datetime object; ignored.
|
||||
|
||||
:return:
|
||||
Name of this timezone
|
||||
"""
|
||||
|
||||
return self._name
|
||||
|
||||
def utcoffset(self, dt):
|
||||
"""
|
||||
:param dt:
|
||||
A datetime object; ignored.
|
||||
|
||||
:return:
|
||||
A timedelta object with the offset from UTC
|
||||
"""
|
||||
|
||||
return self._offset
|
||||
|
||||
def dst(self, dt):
|
||||
"""
|
||||
:param dt:
|
||||
A datetime object; ignored.
|
||||
|
||||
:return:
|
||||
Zero timedelta
|
||||
"""
|
||||
|
||||
return timedelta(0)
|
||||
|
||||
timezone.utc = timezone(timedelta(0))
|
||||
|
||||
# Python 3
|
||||
else:
|
||||
|
||||
from datetime import timezone # noqa
|
||||
|
||||
def int_to_bytes(value, signed=False, width=None):
|
||||
"""
|
||||
Converts an integer to a byte string
|
||||
|
||||
:param value:
|
||||
The integer to convert
|
||||
|
||||
:param signed:
|
||||
If the byte string should be encoded using two's complement
|
||||
|
||||
:param width:
|
||||
If None, the minimal possible size (but at least 1),
|
||||
otherwise an integer of the byte width for the return value
|
||||
|
||||
:return:
|
||||
A byte string
|
||||
"""
|
||||
|
||||
if width is None:
|
||||
if signed:
|
||||
if value < 0:
|
||||
bits_required = abs(value + 1).bit_length()
|
||||
else:
|
||||
bits_required = value.bit_length()
|
||||
if bits_required % 8 == 0:
|
||||
bits_required += 1
|
||||
else:
|
||||
bits_required = value.bit_length()
|
||||
width = math.ceil(bits_required / 8) or 1
|
||||
return value.to_bytes(width, byteorder='big', signed=signed)
|
||||
|
||||
def int_from_bytes(value, signed=False):
|
||||
"""
|
||||
Converts a byte string to an integer
|
||||
|
||||
:param value:
|
||||
The byte string to convert
|
||||
|
||||
:param signed:
|
||||
If the byte string should be interpreted using two's complement
|
||||
|
||||
:return:
|
||||
An integer
|
||||
"""
|
||||
|
||||
return int.from_bytes(value, 'big', signed=signed)
|
||||
|
||||
|
||||
def _format_offset(off):
|
||||
"""
|
||||
Format a timedelta into "[+-]HH:MM" format or "" for None
|
||||
"""
|
||||
|
||||
if off is None:
|
||||
return ''
|
||||
mins = off.days * 24 * 60 + off.seconds // 60
|
||||
sign = '-' if mins < 0 else '+'
|
||||
return sign + '%02d:%02d' % divmod(abs(mins), 60)
|
||||
|
||||
|
||||
class _UtcWithDst(tzinfo):
|
||||
"""
|
||||
Utc class where dst does not return None; required for astimezone
|
||||
"""
|
||||
|
||||
def tzname(self, dt):
|
||||
return 'UTC'
|
||||
|
||||
def utcoffset(self, dt):
|
||||
return timedelta(0)
|
||||
|
||||
def dst(self, dt):
|
||||
return timedelta(0)
|
||||
|
||||
|
||||
utc_with_dst = _UtcWithDst()
|
||||
|
||||
_timezone_cache = {}
|
||||
|
||||
|
||||
def create_timezone(offset):
|
||||
"""
|
||||
Returns a new datetime.timezone object with the given offset.
|
||||
Uses cached objects if possible.
|
||||
|
||||
:param offset:
|
||||
A datetime.timedelta object; It needs to be in full minutes and between -23:59 and +23:59.
|
||||
|
||||
:return:
|
||||
A datetime.timezone object
|
||||
"""
|
||||
|
||||
try:
|
||||
tz = _timezone_cache[offset]
|
||||
except KeyError:
|
||||
tz = _timezone_cache[offset] = timezone(offset)
|
||||
return tz
|
||||
|
||||
|
||||
class extended_date(object):
|
||||
"""
|
||||
A datetime.datetime-like object that represents the year 0. This is just
|
||||
to handle 0000-01-01 found in some certificates. Python's datetime does
|
||||
not support year 0.
|
||||
|
||||
The proleptic gregorian calendar repeats itself every 400 years. Therefore,
|
||||
the simplest way to format is to substitute year 2000.
|
||||
"""
|
||||
|
||||
def __init__(self, year, month, day):
|
||||
"""
|
||||
:param year:
|
||||
The integer 0
|
||||
|
||||
:param month:
|
||||
An integer from 1 to 12
|
||||
|
||||
:param day:
|
||||
An integer from 1 to 31
|
||||
"""
|
||||
|
||||
if year != 0:
|
||||
raise ValueError('year must be 0')
|
||||
|
||||
self._y2k = date(2000, month, day)
|
||||
|
||||
@property
|
||||
def year(self):
|
||||
"""
|
||||
:return:
|
||||
The integer 0
|
||||
"""
|
||||
|
||||
return 0
|
||||
|
||||
@property
|
||||
def month(self):
|
||||
"""
|
||||
:return:
|
||||
An integer from 1 to 12
|
||||
"""
|
||||
|
||||
return self._y2k.month
|
||||
|
||||
@property
|
||||
def day(self):
|
||||
"""
|
||||
:return:
|
||||
An integer from 1 to 31
|
||||
"""
|
||||
|
||||
return self._y2k.day
|
||||
|
||||
def strftime(self, format):
|
||||
"""
|
||||
Formats the date using strftime()
|
||||
|
||||
:param format:
|
||||
A strftime() format string
|
||||
|
||||
:return:
|
||||
A str, the formatted date as a unicode string
|
||||
in Python 3 and a byte string in Python 2
|
||||
"""
|
||||
|
||||
# Format the date twice, once with year 2000, once with year 4000.
|
||||
# The only differences in the result will be in the millennium. Find them and replace by zeros.
|
||||
y2k = self._y2k.strftime(format)
|
||||
y4k = self._y2k.replace(year=4000).strftime(format)
|
||||
return ''.join('0' if (c2, c4) == ('2', '4') else c2 for c2, c4 in zip(y2k, y4k))
|
||||
|
||||
def isoformat(self):
|
||||
"""
|
||||
Formats the date as %Y-%m-%d
|
||||
|
||||
:return:
|
||||
The date formatted to %Y-%m-%d as a unicode string in Python 3
|
||||
and a byte string in Python 2
|
||||
"""
|
||||
|
||||
return self.strftime('0000-%m-%d')
|
||||
|
||||
def replace(self, year=None, month=None, day=None):
|
||||
"""
|
||||
Returns a new datetime.date or asn1crypto.util.extended_date
|
||||
object with the specified components replaced
|
||||
|
||||
:return:
|
||||
A datetime.date or asn1crypto.util.extended_date object
|
||||
"""
|
||||
|
||||
if year is None:
|
||||
year = self.year
|
||||
if month is None:
|
||||
month = self.month
|
||||
if day is None:
|
||||
day = self.day
|
||||
|
||||
if year > 0:
|
||||
cls = date
|
||||
else:
|
||||
cls = extended_date
|
||||
|
||||
return cls(
|
||||
year,
|
||||
month,
|
||||
day
|
||||
)
|
||||
|
||||
def __str__(self):
|
||||
"""
|
||||
:return:
|
||||
A str representing this extended_date, e.g. "0000-01-01"
|
||||
"""
|
||||
|
||||
return self.strftime('%Y-%m-%d')
|
||||
|
||||
def __eq__(self, other):
|
||||
"""
|
||||
Compare two extended_date objects
|
||||
|
||||
:param other:
|
||||
The other extended_date to compare to
|
||||
|
||||
:return:
|
||||
A boolean
|
||||
"""
|
||||
|
||||
# datetime.date object wouldn't compare equal because it can't be year 0
|
||||
if not isinstance(other, self.__class__):
|
||||
return False
|
||||
return self.__cmp__(other) == 0
|
||||
|
||||
def __ne__(self, other):
|
||||
"""
|
||||
Compare two extended_date objects
|
||||
|
||||
:param other:
|
||||
The other extended_date to compare to
|
||||
|
||||
:return:
|
||||
A boolean
|
||||
"""
|
||||
|
||||
return not self.__eq__(other)
|
||||
|
||||
def _comparison_error(self, other):
|
||||
raise TypeError(unwrap(
|
||||
'''
|
||||
An asn1crypto.util.extended_date object can only be compared to
|
||||
an asn1crypto.util.extended_date or datetime.date object, not %s
|
||||
''',
|
||||
type_name(other)
|
||||
))
|
||||
|
||||
def __cmp__(self, other):
|
||||
"""
|
||||
Compare two extended_date or datetime.date objects
|
||||
|
||||
:param other:
|
||||
The other extended_date object to compare to
|
||||
|
||||
:return:
|
||||
An integer smaller than, equal to, or larger than 0
|
||||
"""
|
||||
|
||||
# self is year 0, other is >= year 1
|
||||
if isinstance(other, date):
|
||||
return -1
|
||||
|
||||
if not isinstance(other, self.__class__):
|
||||
self._comparison_error(other)
|
||||
|
||||
if self._y2k < other._y2k:
|
||||
return -1
|
||||
if self._y2k > other._y2k:
|
||||
return 1
|
||||
return 0
|
||||
|
||||
def __lt__(self, other):
|
||||
return self.__cmp__(other) < 0
|
||||
|
||||
def __le__(self, other):
|
||||
return self.__cmp__(other) <= 0
|
||||
|
||||
def __gt__(self, other):
|
||||
return self.__cmp__(other) > 0
|
||||
|
||||
def __ge__(self, other):
|
||||
return self.__cmp__(other) >= 0
|
||||
|
||||
|
||||
class extended_datetime(object):
|
||||
"""
|
||||
A datetime.datetime-like object that represents the year 0. This is just
|
||||
to handle 0000-01-01 found in some certificates. Python's datetime does
|
||||
not support year 0.
|
||||
|
||||
The proleptic gregorian calendar repeats itself every 400 years. Therefore,
|
||||
the simplest way to format is to substitute year 2000.
|
||||
"""
|
||||
|
||||
# There are 97 leap days during 400 years.
|
||||
DAYS_IN_400_YEARS = 400 * 365 + 97
|
||||
DAYS_IN_2000_YEARS = 5 * DAYS_IN_400_YEARS
|
||||
|
||||
def __init__(self, year, *args, **kwargs):
|
||||
"""
|
||||
:param year:
|
||||
The integer 0
|
||||
|
||||
:param args:
|
||||
Other positional arguments; see datetime.datetime.
|
||||
|
||||
:param kwargs:
|
||||
Other keyword arguments; see datetime.datetime.
|
||||
"""
|
||||
|
||||
if year != 0:
|
||||
raise ValueError('year must be 0')
|
||||
|
||||
self._y2k = datetime(2000, *args, **kwargs)
|
||||
|
||||
@property
|
||||
def year(self):
|
||||
"""
|
||||
:return:
|
||||
The integer 0
|
||||
"""
|
||||
|
||||
return 0
|
||||
|
||||
@property
|
||||
def month(self):
|
||||
"""
|
||||
:return:
|
||||
An integer from 1 to 12
|
||||
"""
|
||||
|
||||
return self._y2k.month
|
||||
|
||||
@property
|
||||
def day(self):
|
||||
"""
|
||||
:return:
|
||||
An integer from 1 to 31
|
||||
"""
|
||||
|
||||
return self._y2k.day
|
||||
|
||||
@property
|
||||
def hour(self):
|
||||
"""
|
||||
:return:
|
||||
An integer from 1 to 24
|
||||
"""
|
||||
|
||||
return self._y2k.hour
|
||||
|
||||
@property
|
||||
def minute(self):
|
||||
"""
|
||||
:return:
|
||||
An integer from 1 to 60
|
||||
"""
|
||||
|
||||
return self._y2k.minute
|
||||
|
||||
@property
|
||||
def second(self):
|
||||
"""
|
||||
:return:
|
||||
An integer from 1 to 60
|
||||
"""
|
||||
|
||||
return self._y2k.second
|
||||
|
||||
@property
|
||||
def microsecond(self):
|
||||
"""
|
||||
:return:
|
||||
An integer from 0 to 999999
|
||||
"""
|
||||
|
||||
return self._y2k.microsecond
|
||||
|
||||
@property
|
||||
def tzinfo(self):
|
||||
"""
|
||||
:return:
|
||||
If object is timezone aware, a datetime.tzinfo object, else None.
|
||||
"""
|
||||
|
||||
return self._y2k.tzinfo
|
||||
|
||||
def utcoffset(self):
|
||||
"""
|
||||
:return:
|
||||
If object is timezone aware, a datetime.timedelta object, else None.
|
||||
"""
|
||||
|
||||
return self._y2k.utcoffset()
|
||||
|
||||
def time(self):
|
||||
"""
|
||||
:return:
|
||||
A datetime.time object
|
||||
"""
|
||||
|
||||
return self._y2k.time()
|
||||
|
||||
def date(self):
|
||||
"""
|
||||
:return:
|
||||
An asn1crypto.util.extended_date of the date
|
||||
"""
|
||||
|
||||
return extended_date(0, self.month, self.day)
|
||||
|
||||
def strftime(self, format):
|
||||
"""
|
||||
Performs strftime(), always returning a str
|
||||
|
||||
:param format:
|
||||
A strftime() format string
|
||||
|
||||
:return:
|
||||
A str of the formatted datetime
|
||||
"""
|
||||
|
||||
# Format the datetime twice, once with year 2000, once with year 4000.
|
||||
# The only differences in the result will be in the millennium. Find them and replace by zeros.
|
||||
y2k = self._y2k.strftime(format)
|
||||
y4k = self._y2k.replace(year=4000).strftime(format)
|
||||
return ''.join('0' if (c2, c4) == ('2', '4') else c2 for c2, c4 in zip(y2k, y4k))
|
||||
|
||||
def isoformat(self, sep='T'):
|
||||
"""
|
||||
Formats the date as "%Y-%m-%d %H:%M:%S" with the sep param between the
|
||||
date and time portions
|
||||
|
||||
:param set:
|
||||
A single character of the separator to place between the date and
|
||||
time
|
||||
|
||||
:return:
|
||||
The formatted datetime as a unicode string in Python 3 and a byte
|
||||
string in Python 2
|
||||
"""
|
||||
|
||||
s = '0000-%02d-%02d%c%02d:%02d:%02d' % (self.month, self.day, sep, self.hour, self.minute, self.second)
|
||||
if self.microsecond:
|
||||
s += '.%06d' % self.microsecond
|
||||
return s + _format_offset(self.utcoffset())
|
||||
|
||||
def replace(self, year=None, *args, **kwargs):
|
||||
"""
|
||||
Returns a new datetime.datetime or asn1crypto.util.extended_datetime
|
||||
object with the specified components replaced
|
||||
|
||||
:param year:
|
||||
The new year to substitute. None to keep it.
|
||||
|
||||
:param args:
|
||||
Other positional arguments; see datetime.datetime.replace.
|
||||
|
||||
:param kwargs:
|
||||
Other keyword arguments; see datetime.datetime.replace.
|
||||
|
||||
:return:
|
||||
A datetime.datetime or asn1crypto.util.extended_datetime object
|
||||
"""
|
||||
|
||||
if year:
|
||||
return self._y2k.replace(year, *args, **kwargs)
|
||||
|
||||
return extended_datetime.from_y2k(self._y2k.replace(2000, *args, **kwargs))
|
||||
|
||||
def astimezone(self, tz):
|
||||
"""
|
||||
Convert this extended_datetime to another timezone.
|
||||
|
||||
:param tz:
|
||||
A datetime.tzinfo object.
|
||||
|
||||
:return:
|
||||
A new extended_datetime or datetime.datetime object
|
||||
"""
|
||||
|
||||
return extended_datetime.from_y2k(self._y2k.astimezone(tz))
|
||||
|
||||
def timestamp(self):
|
||||
"""
|
||||
Return POSIX timestamp. Only supported in python >= 3.3
|
||||
|
||||
:return:
|
||||
A float representing the seconds since 1970-01-01 UTC. This will be a negative value.
|
||||
"""
|
||||
|
||||
return self._y2k.timestamp() - self.DAYS_IN_2000_YEARS * 86400
|
||||
|
||||
def __str__(self):
|
||||
"""
|
||||
:return:
|
||||
A str representing this extended_datetime, e.g. "0000-01-01 00:00:00.000001-10:00"
|
||||
"""
|
||||
|
||||
return self.isoformat(sep=' ')
|
||||
|
||||
def __eq__(self, other):
|
||||
"""
|
||||
Compare two extended_datetime objects
|
||||
|
||||
:param other:
|
||||
The other extended_datetime to compare to
|
||||
|
||||
:return:
|
||||
A boolean
|
||||
"""
|
||||
|
||||
# Only compare against other datetime or extended_datetime objects
|
||||
if not isinstance(other, (self.__class__, datetime)):
|
||||
return False
|
||||
|
||||
# Offset-naive and offset-aware datetimes are never the same
|
||||
if (self.tzinfo is None) != (other.tzinfo is None):
|
||||
return False
|
||||
|
||||
return self.__cmp__(other) == 0
|
||||
|
||||
def __ne__(self, other):
|
||||
"""
|
||||
Compare two extended_datetime objects
|
||||
|
||||
:param other:
|
||||
The other extended_datetime to compare to
|
||||
|
||||
:return:
|
||||
A boolean
|
||||
"""
|
||||
|
||||
return not self.__eq__(other)
|
||||
|
||||
def _comparison_error(self, other):
|
||||
"""
|
||||
Raises a TypeError about the other object not being suitable for
|
||||
comparison
|
||||
|
||||
:param other:
|
||||
The object being compared to
|
||||
"""
|
||||
|
||||
raise TypeError(unwrap(
|
||||
'''
|
||||
An asn1crypto.util.extended_datetime object can only be compared to
|
||||
an asn1crypto.util.extended_datetime or datetime.datetime object,
|
||||
not %s
|
||||
''',
|
||||
type_name(other)
|
||||
))
|
||||
|
||||
def __cmp__(self, other):
|
||||
"""
|
||||
Compare two extended_datetime or datetime.datetime objects
|
||||
|
||||
:param other:
|
||||
The other extended_datetime or datetime.datetime object to compare to
|
||||
|
||||
:return:
|
||||
An integer smaller than, equal to, or larger than 0
|
||||
"""
|
||||
|
||||
if not isinstance(other, (self.__class__, datetime)):
|
||||
self._comparison_error(other)
|
||||
|
||||
if (self.tzinfo is None) != (other.tzinfo is None):
|
||||
raise TypeError("can't compare offset-naive and offset-aware datetimes")
|
||||
|
||||
diff = self - other
|
||||
zero = timedelta(0)
|
||||
if diff < zero:
|
||||
return -1
|
||||
if diff > zero:
|
||||
return 1
|
||||
return 0
|
||||
|
||||
def __lt__(self, other):
|
||||
return self.__cmp__(other) < 0
|
||||
|
||||
def __le__(self, other):
|
||||
return self.__cmp__(other) <= 0
|
||||
|
||||
def __gt__(self, other):
|
||||
return self.__cmp__(other) > 0
|
||||
|
||||
def __ge__(self, other):
|
||||
return self.__cmp__(other) >= 0
|
||||
|
||||
def __add__(self, other):
|
||||
"""
|
||||
Adds a timedelta
|
||||
|
||||
:param other:
|
||||
A datetime.timedelta object to add.
|
||||
|
||||
:return:
|
||||
A new extended_datetime or datetime.datetime object.
|
||||
"""
|
||||
|
||||
return extended_datetime.from_y2k(self._y2k + other)
|
||||
|
||||
def __sub__(self, other):
|
||||
"""
|
||||
Subtracts a timedelta or another datetime.
|
||||
|
||||
:param other:
|
||||
A datetime.timedelta or datetime.datetime or extended_datetime object to subtract.
|
||||
|
||||
:return:
|
||||
If a timedelta is passed, a new extended_datetime or datetime.datetime object.
|
||||
Else a datetime.timedelta object.
|
||||
"""
|
||||
|
||||
if isinstance(other, timedelta):
|
||||
return extended_datetime.from_y2k(self._y2k - other)
|
||||
|
||||
if isinstance(other, extended_datetime):
|
||||
return self._y2k - other._y2k
|
||||
|
||||
if isinstance(other, datetime):
|
||||
return self._y2k - other - timedelta(days=self.DAYS_IN_2000_YEARS)
|
||||
|
||||
return NotImplemented
|
||||
|
||||
def __rsub__(self, other):
|
||||
return -(self - other)
|
||||
|
||||
@classmethod
|
||||
def from_y2k(cls, value):
|
||||
"""
|
||||
Revert substitution of year 2000.
|
||||
|
||||
:param value:
|
||||
A datetime.datetime object which is 2000 years in the future.
|
||||
:return:
|
||||
A new extended_datetime or datetime.datetime object.
|
||||
"""
|
||||
|
||||
year = value.year - 2000
|
||||
|
||||
if year > 0:
|
||||
new_cls = datetime
|
||||
else:
|
||||
new_cls = cls
|
||||
|
||||
return new_cls(
|
||||
year,
|
||||
value.month,
|
||||
value.day,
|
||||
value.hour,
|
||||
value.minute,
|
||||
value.second,
|
||||
value.microsecond,
|
||||
value.tzinfo
|
||||
)
|
||||
6
jc/parsers/asn1crypto/version.py
Normal file
6
jc/parsers/asn1crypto/version.py
Normal file
@@ -0,0 +1,6 @@
|
||||
# coding: utf-8
|
||||
from __future__ import unicode_literals, division, absolute_import, print_function
|
||||
|
||||
|
||||
__version__ = '1.5.1'
|
||||
__version_info__ = (1, 5, 1)
|
||||
3036
jc/parsers/asn1crypto/x509.py
Normal file
3036
jc/parsers/asn1crypto/x509.py
Normal file
File diff suppressed because it is too large
Load Diff
151
jc/parsers/chage.py
Normal file
151
jc/parsers/chage.py
Normal file
@@ -0,0 +1,151 @@
|
||||
"""jc - JSON Convert `chage --list` command output parser
|
||||
|
||||
Supports `chage -l <username>` or `chage --list <username>`
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ chage -l johndoe | jc --chage
|
||||
|
||||
or
|
||||
|
||||
$ jc chage -l johndoe
|
||||
|
||||
Usage (module):
|
||||
|
||||
import jc
|
||||
result = jc.parse('chage', chage_command_output)
|
||||
|
||||
Schema:
|
||||
|
||||
{
|
||||
"password_last_changed": string,
|
||||
"password_expires": string,
|
||||
"password_inactive": string,
|
||||
"account_expires": string,
|
||||
"min_days_between_password_change": integer,
|
||||
"max_days_between_password_change": integer,
|
||||
"warning_days_before_password_expires": integer
|
||||
}
|
||||
|
||||
Examples:
|
||||
|
||||
$ chage --list joeuser | jc --chage -p
|
||||
{
|
||||
"password_last_changed": "never",
|
||||
"password_expires": "never",
|
||||
"password_inactive": "never",
|
||||
"account_expires": "never",
|
||||
"min_days_between_password_change": 0,
|
||||
"max_days_between_password_change": 99999,
|
||||
"warning_days_before_password_expires": 7
|
||||
}
|
||||
|
||||
$ chage --list joeuser | jc --chage -p -r
|
||||
{
|
||||
"password_last_changed": "never",
|
||||
"password_expires": "never",
|
||||
"password_inactive": "never",
|
||||
"account_expires": "never",
|
||||
"min_days_between_password_change": "0",
|
||||
"max_days_between_password_change": "99999",
|
||||
"warning_days_before_password_expires": "7"
|
||||
}
|
||||
"""
|
||||
from typing import List, Dict
|
||||
import jc.utils
|
||||
|
||||
|
||||
class info():
|
||||
"""Provides parser metadata (version, author, etc.)"""
|
||||
version = '1.0'
|
||||
description = '`chage --list` command parser'
|
||||
author = 'Kelly Brazil'
|
||||
author_email = 'kellyjonbrazil@gmail.com'
|
||||
compatible = ['linux']
|
||||
magic_commands = ['chage --list', 'chage -l']
|
||||
|
||||
|
||||
__version__ = info.version
|
||||
|
||||
|
||||
def _process(proc_data: Dict) -> Dict:
|
||||
"""
|
||||
Final processing to conform to the schema.
|
||||
|
||||
Parameters:
|
||||
|
||||
proc_data: (Dictionary) raw structured data to process
|
||||
|
||||
Returns:
|
||||
|
||||
Dictionary. Structured to conform to the schema.
|
||||
"""
|
||||
int_list = ['min_days_between_password_change', 'max_days_between_password_change',
|
||||
'warning_days_before_password_expires']
|
||||
|
||||
for key in proc_data:
|
||||
if key in int_list:
|
||||
proc_data[key] = jc.utils.convert_to_int(proc_data[key])
|
||||
|
||||
return proc_data
|
||||
|
||||
|
||||
def parse(
|
||||
data: str,
|
||||
raw: bool = False,
|
||||
quiet: bool = False
|
||||
) -> Dict:
|
||||
"""
|
||||
Main text parsing function
|
||||
|
||||
Parameters:
|
||||
|
||||
data: (string) text data to parse
|
||||
raw: (boolean) unprocessed output if True
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
|
||||
Returns:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
"""
|
||||
jc.utils.compatibility(__name__, info.compatible, quiet)
|
||||
jc.utils.input_type_check(data)
|
||||
|
||||
raw_output: Dict = {}
|
||||
|
||||
if jc.utils.has_data(data):
|
||||
|
||||
for line in filter(None, data.splitlines()):
|
||||
key, val = line.split(':', maxsplit=1)
|
||||
key = key.strip()
|
||||
val = val.strip()
|
||||
|
||||
if key == 'Last password change':
|
||||
raw_output['password_last_changed'] = val
|
||||
continue
|
||||
|
||||
if key == 'Password expires':
|
||||
raw_output['password_expires'] = val
|
||||
continue
|
||||
|
||||
if key == 'Password inactive':
|
||||
raw_output['password_inactive'] = val
|
||||
continue
|
||||
|
||||
if key == 'Account expires':
|
||||
raw_output['account_expires'] = val
|
||||
continue
|
||||
|
||||
if key == 'Minimum number of days between password change':
|
||||
raw_output['min_days_between_password_change'] = val
|
||||
continue
|
||||
|
||||
if key == 'Maximum number of days between password change':
|
||||
raw_output['max_days_between_password_change'] = val
|
||||
continue
|
||||
|
||||
if key == 'Number of days of warning before password expires':
|
||||
raw_output['warning_days_before_password_expires'] = val
|
||||
continue
|
||||
|
||||
return raw_output if raw else _process(raw_output)
|
||||
@@ -1,14 +1,14 @@
|
||||
"""jc - JSON Convert `csv` file streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns a Generator
|
||||
iterator of Dictionaries (module)
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
The `csv` streaming parser will attempt to automatically detect the
|
||||
delimiter character. If the delimiter cannot be detected it will default
|
||||
to comma. The first row of the file must be a header row.
|
||||
|
||||
Note: The first 100 rows are read into memory to enable delimiter detection,
|
||||
then the rest of the rows are loaded lazily.
|
||||
> Note: The first 100 rows are read into memory to enable delimiter
|
||||
> detection, then the rest of the rows are loaded lazily.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
@@ -95,7 +95,7 @@ def _process(proc_data):
|
||||
@add_jc_meta
|
||||
def parse(data, raw=False, quiet=False, ignore_exceptions=False):
|
||||
"""
|
||||
Main text parsing generator function. Returns an iterator object.
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
@@ -106,13 +106,9 @@ def parse(data, raw=False, quiet=False, ignore_exceptions=False):
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
Yields:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
|
||||
Returns:
|
||||
|
||||
Iterator object (generator)
|
||||
Iterable of Dictionaries
|
||||
"""
|
||||
jc.utils.compatibility(__name__, info.compatible, quiet)
|
||||
streaming_input_type_check(data)
|
||||
|
||||
@@ -99,7 +99,7 @@ import jc.parsers.universal
|
||||
|
||||
class info():
|
||||
"""Provides parser metadata (version, author, etc.)"""
|
||||
version = '1.9'
|
||||
version = '1.10'
|
||||
description = '`df` command parser'
|
||||
author = 'Kelly Brazil'
|
||||
author_email = 'kellyjonbrazil@gmail.com'
|
||||
@@ -208,7 +208,9 @@ def parse(data, raw=False, quiet=False):
|
||||
jc.utils.compatibility(__name__, info.compatible, quiet)
|
||||
jc.utils.input_type_check(data)
|
||||
|
||||
cleandata = data.splitlines()
|
||||
# remove blank lines
|
||||
cleandata = list(filter(None, data.splitlines()))
|
||||
|
||||
fix_data = []
|
||||
raw_output = []
|
||||
filesystem_map = {}
|
||||
|
||||
@@ -14,7 +14,7 @@ time of the system the parser is run on)
|
||||
|
||||
Usage (cli):
|
||||
|
||||
C:> dir | jc --dir
|
||||
C:\> dir | jc --dir
|
||||
|
||||
Usage (module):
|
||||
|
||||
@@ -37,7 +37,7 @@ Schema:
|
||||
|
||||
Examples:
|
||||
|
||||
C:> dir | jc --dir -p
|
||||
C:\> dir | jc --dir -p
|
||||
[
|
||||
{
|
||||
"date": "03/24/2021",
|
||||
@@ -78,7 +78,7 @@ Examples:
|
||||
...
|
||||
]
|
||||
|
||||
C:> dir | jc --dir -p -r
|
||||
C:\> dir | jc --dir -p -r
|
||||
[
|
||||
{
|
||||
"date": "03/24/2021",
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""jc - JSON Convert `foo` command output streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns a Generator
|
||||
iterator of Dictionaries (module)
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
<<Short foo description and caveats>>
|
||||
|
||||
@@ -40,7 +40,7 @@ Examples:
|
||||
{example output}
|
||||
...
|
||||
"""
|
||||
from typing import Dict, Iterable, Generator, Union
|
||||
from typing import Dict, Iterable, Union
|
||||
import jc.utils
|
||||
from jc.streaming import (
|
||||
add_jc_meta, streaming_input_type_check, streaming_line_input_type_check, raise_or_yield
|
||||
@@ -90,9 +90,9 @@ def parse(
|
||||
raw: bool = False,
|
||||
quiet: bool = False,
|
||||
ignore_exceptions: bool = False
|
||||
) -> Union[Generator[Dict, None, None], tuple]:
|
||||
) -> Union[Iterable[Dict], tuple]:
|
||||
"""
|
||||
Main text parsing generator function. Returns an iterator object.
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
@@ -103,13 +103,10 @@ def parse(
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
Yields:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
|
||||
Returns:
|
||||
|
||||
Iterator object (generator)
|
||||
Iterable of Dictionaries
|
||||
"""
|
||||
jc.utils.compatibility(__name__, info.compatible, quiet)
|
||||
streaming_input_type_check(data)
|
||||
|
||||
@@ -38,8 +38,8 @@ Schema:
|
||||
"author": string,
|
||||
"author_email": string,
|
||||
"date": string,
|
||||
"epoch": integer, [0]
|
||||
"epoch_utc": integer, [1]
|
||||
"epoch": integer, # [0]
|
||||
"epoch_utc": integer, # [1]
|
||||
"commit_by": string,
|
||||
"commit_by_email": string,
|
||||
"commit_by_date": string,
|
||||
@@ -148,12 +148,12 @@ import re
|
||||
from typing import List, Dict
|
||||
import jc.utils
|
||||
|
||||
hash_pattern = re.compile(r'([0-9]|[a-f])+')
|
||||
hash_pattern = re.compile(r'(?:[0-9]|[a-f]){40}')
|
||||
changes_pattern = re.compile(r'\s(?P<files>\d+)\s+(files? changed),\s+(?P<insertions>\d+)\s(insertions?\(\+\))?(,\s+)?(?P<deletions>\d+)?(\s+deletions?\(\-\))?')
|
||||
|
||||
class info():
|
||||
"""Provides parser metadata (version, author, etc.)"""
|
||||
version = '1.0'
|
||||
version = '1.1'
|
||||
description = '`git log` command parser'
|
||||
author = 'Kelly Brazil'
|
||||
author_email = 'kellyjonbrazil@gmail.com'
|
||||
@@ -234,7 +234,7 @@ def parse(
|
||||
line_list = line.split(maxsplit=1)
|
||||
|
||||
# oneline style
|
||||
if line_list and _is_commit_hash(line_list[0]):
|
||||
if not line.startswith(' ') and line_list and _is_commit_hash(line_list[0]):
|
||||
if output_line:
|
||||
if file_list:
|
||||
output_line['stats']['files'] = file_list
|
||||
|
||||
282
jc/parsers/git_log_s.py
Normal file
282
jc/parsers/git_log_s.py
Normal file
@@ -0,0 +1,282 @@
|
||||
"""jc - JSON Convert `git log` command output streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
Can be used with the following format options:
|
||||
- `oneline`
|
||||
- `short`
|
||||
- `medium`
|
||||
- `full`
|
||||
- `fuller`
|
||||
|
||||
Additional options supported:
|
||||
- `--stat`
|
||||
- `--shortstat`
|
||||
|
||||
The `epoch` calculated timestamp field is naive. (i.e. based on the
|
||||
local time of the system the parser is run on)
|
||||
|
||||
The `epoch_utc` calculated timestamp field is timezone-aware and is
|
||||
only available if the timezone field is UTC.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ git log | jc --git-log-s
|
||||
|
||||
Usage (module):
|
||||
|
||||
import jc
|
||||
|
||||
result = jc.parse('git_log_s', git_log_command_output.splitlines())
|
||||
for item in result:
|
||||
# do something
|
||||
|
||||
Schema:
|
||||
|
||||
{
|
||||
"commit": string,
|
||||
"author": string,
|
||||
"author_email": string,
|
||||
"date": string,
|
||||
"epoch": integer, # [0]
|
||||
"epoch_utc": integer, # [1]
|
||||
"commit_by": string,
|
||||
"commit_by_email": string,
|
||||
"commit_by_date": string,
|
||||
"message": string,
|
||||
"stats" : {
|
||||
"files_changed": integer,
|
||||
"insertions": integer,
|
||||
"deletions": integer,
|
||||
"files": [
|
||||
string
|
||||
]
|
||||
}
|
||||
|
||||
# below object only exists if using -qq or ignore_exceptions=True
|
||||
"_jc_meta": {
|
||||
"success": boolean, # false if error parsing
|
||||
"error": string, # exists if "success" is false
|
||||
"line": string # exists if "success" is false
|
||||
}
|
||||
}
|
||||
|
||||
[0] naive timestamp if "date" field is parsable, else null
|
||||
[1] timezone aware timestamp availabe for UTC, else null
|
||||
|
||||
Examples:
|
||||
|
||||
$ git log | jc --git-log-s
|
||||
{"commit":"a730ae18c8e81c5261db132df73cd74f272a0a26","author":"Kelly...}
|
||||
{"commit":"930bf439c06c48a952baec05a9896c8d92b7693e","author":"Kelly...}
|
||||
...
|
||||
"""
|
||||
import re
|
||||
from typing import List, Dict, Iterable, Union
|
||||
import jc.utils
|
||||
from jc.streaming import (
|
||||
add_jc_meta, streaming_input_type_check, streaming_line_input_type_check, raise_or_yield
|
||||
)
|
||||
from jc.exceptions import ParseError
|
||||
|
||||
|
||||
hash_pattern = re.compile(r'(?:[0-9]|[a-f]){40}')
|
||||
changes_pattern = re.compile(r'\s(?P<files>\d+)\s+(files? changed),\s+(?P<insertions>\d+)\s(insertions?\(\+\))?(,\s+)?(?P<deletions>\d+)?(\s+deletions?\(\-\))?')
|
||||
|
||||
|
||||
class info():
|
||||
"""Provides parser metadata (version, author, etc.)"""
|
||||
version = '1.1'
|
||||
description = '`git log` command streaming parser'
|
||||
author = 'Kelly Brazil'
|
||||
author_email = 'kellyjonbrazil@gmail.com'
|
||||
compatible = ['linux', 'darwin', 'cygwin', 'win32', 'aix', 'freebsd']
|
||||
streaming = True
|
||||
|
||||
|
||||
__version__ = info.version
|
||||
|
||||
|
||||
def _process(proc_data: Dict) -> Dict:
|
||||
"""
|
||||
Final processing to conform to the schema.
|
||||
|
||||
Parameters:
|
||||
|
||||
proc_data: (Dictionary) raw structured data to process
|
||||
|
||||
Returns:
|
||||
|
||||
Dictionary. Structured data to conform to the schema.
|
||||
"""
|
||||
int_list = ['files_changed', 'insertions', 'deletions']
|
||||
|
||||
if 'date' in proc_data:
|
||||
ts = jc.utils.timestamp(proc_data['date'], format_hint=(1100,))
|
||||
proc_data['epoch'] = ts.naive
|
||||
proc_data['epoch_utc'] = ts.utc
|
||||
|
||||
if 'stats' in proc_data:
|
||||
for key in proc_data['stats']:
|
||||
if key in int_list:
|
||||
proc_data['stats'][key] = jc.utils.convert_to_int(proc_data['stats'][key])
|
||||
|
||||
return proc_data
|
||||
|
||||
|
||||
def _is_commit_hash(hash_string: str) -> bool:
|
||||
# 0c55240e9da30ac4293dc324f1094de2abd3da91
|
||||
if len(hash_string) != 40:
|
||||
return False
|
||||
|
||||
if hash_pattern.match(hash_string):
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
|
||||
@add_jc_meta
|
||||
def parse(
|
||||
data: Iterable[str],
|
||||
raw: bool = False,
|
||||
quiet: bool = False,
|
||||
ignore_exceptions: bool = False
|
||||
) -> Union[Iterable[Dict], tuple]:
|
||||
"""
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
data: (iterable) line-based text data to parse
|
||||
(e.g. sys.stdin or str.splitlines())
|
||||
|
||||
raw: (boolean) unprocessed output if True
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
|
||||
Returns:
|
||||
|
||||
Iterable of Dictionaries
|
||||
"""
|
||||
jc.utils.compatibility(__name__, info.compatible, quiet)
|
||||
streaming_input_type_check(data)
|
||||
|
||||
output_line: Dict = {}
|
||||
message_lines: List[str] = []
|
||||
file_list: List[str] = []
|
||||
|
||||
for line in data:
|
||||
try:
|
||||
streaming_line_input_type_check(line)
|
||||
|
||||
if line == '' or line == '\n':
|
||||
continue
|
||||
|
||||
line_list = line.rstrip().split(maxsplit=1)
|
||||
|
||||
# oneline style
|
||||
if not line.startswith(' ') and line_list and _is_commit_hash(line_list[0]):
|
||||
if output_line:
|
||||
if file_list:
|
||||
output_line['stats']['files'] = file_list
|
||||
|
||||
yield output_line if raw else _process(output_line)
|
||||
|
||||
output_line = {}
|
||||
message_lines = []
|
||||
file_list = []
|
||||
output_line = {
|
||||
'commit': line_list[0],
|
||||
'message': line_list[1]
|
||||
}
|
||||
continue
|
||||
|
||||
# all other styles
|
||||
if line.startswith('commit '):
|
||||
if output_line:
|
||||
if message_lines:
|
||||
output_line['message'] = '\n'.join(message_lines)
|
||||
|
||||
if file_list:
|
||||
output_line['stats']['files'] = file_list
|
||||
|
||||
yield output_line if raw else _process(output_line)
|
||||
|
||||
output_line = {}
|
||||
message_lines = []
|
||||
file_list = []
|
||||
output_line['commit'] = line_list[1]
|
||||
continue
|
||||
|
||||
if line.startswith('Merge: '):
|
||||
output_line['merge'] = line_list[1]
|
||||
continue
|
||||
|
||||
if line.startswith('Author: '):
|
||||
values = line_list[1].rsplit(maxsplit=1)
|
||||
output_line['author'] = values[0]
|
||||
output_line['author_email'] = values[1].strip('<').strip('>')
|
||||
continue
|
||||
|
||||
if line.startswith('Date: '):
|
||||
output_line['date'] = line_list[1]
|
||||
continue
|
||||
|
||||
if line.startswith('AuthorDate: '):
|
||||
output_line['date'] = line_list[1]
|
||||
continue
|
||||
|
||||
if line.startswith('CommitDate: '):
|
||||
output_line['commit_by_date'] = line_list[1]
|
||||
continue
|
||||
|
||||
if line.startswith('Commit: '):
|
||||
values = line_list[1].rsplit(maxsplit=1)
|
||||
output_line['commit_by'] = values[0]
|
||||
output_line['commit_by_email'] = values[1].strip('<').strip('>')
|
||||
continue
|
||||
|
||||
if line.startswith(' '):
|
||||
message_lines.append(line.strip())
|
||||
continue
|
||||
|
||||
if line.startswith(' ') and 'changed, ' not in line:
|
||||
# this is a file name
|
||||
file_name = line.split('|')[0].strip()
|
||||
file_list.append(file_name)
|
||||
continue
|
||||
|
||||
if line.startswith(' ') and 'changed, ' in line:
|
||||
# this is the stat summary
|
||||
changes = changes_pattern.match(line)
|
||||
if changes:
|
||||
files = changes['files']
|
||||
insertions = changes['insertions']
|
||||
deletions = changes['deletions']
|
||||
|
||||
output_line['stats'] = {
|
||||
'files_changed': files or '0',
|
||||
'insertions': insertions or '0',
|
||||
'deletions': deletions or '0'
|
||||
}
|
||||
continue
|
||||
|
||||
raise ParseError('Not git_log_s data')
|
||||
|
||||
except Exception as e:
|
||||
yield raise_or_yield(ignore_exceptions, e, line)
|
||||
|
||||
try:
|
||||
if output_line:
|
||||
if message_lines:
|
||||
output_line['message'] = '\n'.join(message_lines)
|
||||
|
||||
if file_list:
|
||||
output_line['stats']['files'] = file_list
|
||||
|
||||
yield output_line if raw else _process(output_line)
|
||||
|
||||
except Exception as e:
|
||||
yield raise_or_yield(ignore_exceptions, e, line)
|
||||
337
jc/parsers/gpg.py
Normal file
337
jc/parsers/gpg.py
Normal file
@@ -0,0 +1,337 @@
|
||||
"""jc - JSON Convert `gpg --with-colons` command output parser
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ gpg --with-colons --show-keys file.gpg | jc --gpg
|
||||
|
||||
or
|
||||
|
||||
$ jc gpg --with-colons --show-keys file.gpg
|
||||
|
||||
Usage (module):
|
||||
|
||||
import jc
|
||||
result = jc.parse('gpg', gpg_command_output)
|
||||
|
||||
Schema:
|
||||
|
||||
Field definitions from https://git.gnupg.org/cgi-bin/gitweb.cgi?p=gnupg.git;a=blob_plain;f=doc/DETAILS
|
||||
|
||||
> Note: Number values are not converted to integers because many field
|
||||
> specifications are overloaded and future augmentations are implied in the
|
||||
> documentation.
|
||||
|
||||
[
|
||||
{
|
||||
"type": string,
|
||||
"validity": string,
|
||||
"key_length": string,
|
||||
"pub_key_alg": string,
|
||||
"key_id": string,
|
||||
"creation_date": string,
|
||||
"expiration_date": string,
|
||||
"certsn_uidhash_trustinfo": string,
|
||||
"owner_trust": string,
|
||||
"user_id": string,
|
||||
"signature_class": string,
|
||||
"key_capabilities": string,
|
||||
"cert_fingerprint_other": string,
|
||||
"flag": string,
|
||||
"token_sn": string,
|
||||
"hash_alg": string,
|
||||
"curve_name": string,
|
||||
"compliance_flags": string,
|
||||
"last_update_date": string,
|
||||
"origin": string,
|
||||
"comment": string,
|
||||
"index": string, # [0]
|
||||
"bits": string, # [0]
|
||||
"value": string, # [0]
|
||||
"version": string, # [1], [4]
|
||||
"signature_count": string, # [1]
|
||||
"encryption_count": string, # [1]
|
||||
"policy": string, # [1]
|
||||
"signature_first_seen": string, # [1]
|
||||
"signature_most_recent_seen": string, # [1]
|
||||
"encryption_first_done": string, # [1]
|
||||
"encryption_most_recent_done": string, # [1]
|
||||
"staleness_reason": string, # [2]
|
||||
"trust_model": string, # [2]
|
||||
"trust_db_created": string, # [2]
|
||||
"trust_db_expires": string, # [2]
|
||||
"marginally_trusted_users": string, # [2]
|
||||
"completely_trusted_users": string, # [2]
|
||||
"cert_chain_max_depth": string, # [2]
|
||||
"subpacket_number": string, # [3]
|
||||
"hex_flags": string, # [3]
|
||||
"subpacket_length": string, # [3]
|
||||
"subpacket_data": string, # [3]
|
||||
"pubkey": string, # [4]
|
||||
"cipher": string, # [4]
|
||||
"digest": string, # [4]
|
||||
"compress": string, # [4]
|
||||
"group": string, # [4]
|
||||
"members": string, # [4]
|
||||
"curve_names": string, # [4]
|
||||
}
|
||||
]
|
||||
|
||||
All blank values are converted to null/None.
|
||||
|
||||
[0] for 'pkd' type
|
||||
[1] for 'tfs' type
|
||||
[2] for 'tru' type
|
||||
[3] for 'skp' type
|
||||
[4] for 'cfg' type
|
||||
|
||||
Examples:
|
||||
|
||||
$ gpg --with-colons --show-keys file.gpg | jc --gpg -p
|
||||
[
|
||||
{
|
||||
"type": "pub",
|
||||
"validity": "f",
|
||||
"key_length": "1024",
|
||||
"pub_key_alg": "17",
|
||||
"key_id": "6C7EE1B8621CC013",
|
||||
"creation_date": "899817715",
|
||||
"expiration_date": "1055898235",
|
||||
"certsn_uidhash_trustinfo": null,
|
||||
"owner_trust": "m",
|
||||
"user_id": null,
|
||||
"signature_class": null,
|
||||
"key_capabilities": "scESC",
|
||||
"cert_fingerprint_other": null,
|
||||
"flag": null,
|
||||
"token_sn": null,
|
||||
"hash_alg": null,
|
||||
"curve_name": null,
|
||||
"compliance_flags": null,
|
||||
"last_update_date": null,
|
||||
"origin": null,
|
||||
"comment": null
|
||||
},
|
||||
...
|
||||
]
|
||||
"""
|
||||
from typing import List, Dict, Optional
|
||||
import jc.utils
|
||||
|
||||
|
||||
class info():
|
||||
"""Provides parser metadata (version, author, etc.)"""
|
||||
version = '1.0'
|
||||
description = '`gpg --with-colons` command parser'
|
||||
author = 'Kelly Brazil'
|
||||
author_email = 'kellyjonbrazil@gmail.com'
|
||||
compatible = ['linux']
|
||||
magic_commands = ['gpg --with-colons']
|
||||
|
||||
|
||||
__version__ = info.version
|
||||
|
||||
|
||||
def _process(proc_data: List[Dict]) -> List[Dict]:
|
||||
"""
|
||||
Final processing to conform to the schema.
|
||||
|
||||
Parameters:
|
||||
|
||||
proc_data: (List of Dictionaries) raw structured data to process
|
||||
|
||||
Returns:
|
||||
|
||||
List of Dictionaries. Structured to conform to the schema.
|
||||
"""
|
||||
return proc_data
|
||||
|
||||
|
||||
def _list_get(my_list: List, index: int, default_val=None) -> Optional[str]:
|
||||
"""get a list value or return None/default value if out of range."""
|
||||
if index <= len(my_list) - 1:
|
||||
return my_list[index] or None
|
||||
|
||||
return default_val
|
||||
|
||||
|
||||
def parse(
|
||||
data: str,
|
||||
raw: bool = False,
|
||||
quiet: bool = False
|
||||
) -> List[Dict]:
|
||||
"""
|
||||
Main text parsing function
|
||||
|
||||
Parameters:
|
||||
|
||||
data: (string) text data to parse
|
||||
raw: (boolean) unprocessed output if True
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
|
||||
Returns:
|
||||
|
||||
List of Dictionaries. Raw or processed structured data.
|
||||
"""
|
||||
jc.utils.compatibility(__name__, info.compatible, quiet)
|
||||
jc.utils.input_type_check(data)
|
||||
|
||||
raw_output: List = []
|
||||
|
||||
if jc.utils.has_data(data):
|
||||
|
||||
for line in filter(None, data.splitlines()):
|
||||
values = line.split(':')
|
||||
temp_obj = {
|
||||
'type': _list_get(values, 0),
|
||||
'validity': _list_get(values, 1),
|
||||
'key_length': _list_get(values, 2),
|
||||
'pub_key_alg': _list_get(values, 3),
|
||||
'key_id': _list_get(values, 4),
|
||||
'creation_date': _list_get(values, 5),
|
||||
'expiration_date': _list_get(values, 6),
|
||||
'certsn_uidhash_trustinfo': _list_get(values, 7),
|
||||
'owner_trust': _list_get(values, 8),
|
||||
'user_id': _list_get(values, 9),
|
||||
'signature_class': _list_get(values, 10),
|
||||
'key_capabilities': _list_get(values, 11),
|
||||
'cert_fingerprint_other': _list_get(values, 12),
|
||||
'flag': _list_get(values, 13),
|
||||
'token_sn': _list_get(values, 14),
|
||||
'hash_alg': _list_get(values, 15),
|
||||
'curve_name': _list_get(values, 16),
|
||||
'compliance_flags': _list_get(values, 17),
|
||||
'last_update_date': _list_get(values, 18),
|
||||
'origin': _list_get(values, 19),
|
||||
'comment': _list_get(values, 20)
|
||||
}
|
||||
|
||||
# field mappings change for special types: pkd, tfs, tru, skp, cfg
|
||||
|
||||
if temp_obj['type'] == 'pkd':
|
||||
# pkd:0:1024:B665B1435F4C2 .... FF26ABB:
|
||||
# ! ! !-- the value
|
||||
# ! !------ for information number of bits in the value
|
||||
# !--------- index (eg. DSA goes from 0 to 3: p,q,g,y)
|
||||
line_obj = {
|
||||
'type': temp_obj['type'],
|
||||
'index': temp_obj['validity'],
|
||||
'bits': temp_obj['key_length'],
|
||||
'value': temp_obj['pub_key_alg']
|
||||
}
|
||||
|
||||
elif temp_obj['type'] == 'tfs':
|
||||
# - Field 2 :: tfs record version (must be 1)
|
||||
# - Field 3 :: validity - A number with validity code.
|
||||
# - Field 4 :: signcount - The number of signatures seen.
|
||||
# - Field 5 :: encrcount - The number of encryptions done.
|
||||
# - Field 6 :: policy - A string with the policy
|
||||
# - Field 7 :: signture-first-seen - a timestamp or 0 if not known.
|
||||
# - Field 8 :: signature-most-recent-seen - a timestamp or 0 if not known.
|
||||
# - Field 9 :: encryption-first-done - a timestamp or 0 if not known.
|
||||
# - Field 10 :: encryption-most-recent-done - a timestamp or 0 if not known.
|
||||
line_obj = {
|
||||
'type': temp_obj['type'],
|
||||
'version': temp_obj['validity'],
|
||||
'validity': temp_obj['key_length'],
|
||||
'signature_count': temp_obj['pub_key_alg'],
|
||||
'encryption_count': temp_obj['key_id'],
|
||||
'policy': temp_obj['creation_date'],
|
||||
'signature_first_seen': temp_obj['expiration_date'],
|
||||
'signature_most_recent_seen': temp_obj['certsn_uidhash_trustinfo'],
|
||||
'encryption_first_done': temp_obj['owner_trust'],
|
||||
'encryption_most_recent_done': temp_obj['user_id']
|
||||
}
|
||||
|
||||
elif temp_obj['type'] == 'tru':
|
||||
# tru:o:0:1166697654:1:3:1:5
|
||||
# - Field 2 :: Reason for staleness of trust.
|
||||
# - Field 3 :: Trust model
|
||||
# - Field 4 :: Date trustdb was created in seconds since Epoch.
|
||||
# - Field 5 :: Date trustdb will expire in seconds since Epoch.
|
||||
# - Field 6 :: Number of marginally trusted users to introduce a new key signer.
|
||||
# - Field 7 :: Number of completely trusted users to introduce a new key signer.
|
||||
# - Field 8 :: Maximum depth of a certification chain.
|
||||
line_obj = {
|
||||
'type': temp_obj['type'],
|
||||
'staleness_reason': temp_obj['validity'],
|
||||
'trust_model': temp_obj['key_length'],
|
||||
'trust_db_created': temp_obj['pub_key_alg'],
|
||||
'trust_db_expires': temp_obj['key_id'],
|
||||
'marginally_trusted_users': temp_obj['creation_date'],
|
||||
'completely_trusted_users': temp_obj['expiration_date'],
|
||||
'cert_chain_max_depth': temp_obj['certsn_uidhash_trustinfo']
|
||||
}
|
||||
|
||||
elif temp_obj['type'] == 'skp':
|
||||
# - Field 2 :: Subpacket number as per RFC-4880 and later.
|
||||
# - Field 3 :: Flags in hex.
|
||||
# - Field 4 :: Length of the subpacket.
|
||||
# - Field 5 :: The subpacket data.
|
||||
line_obj = {
|
||||
'type': temp_obj['type'],
|
||||
'subpacket_number': temp_obj['validity'],
|
||||
'hex_flags': temp_obj['key_length'],
|
||||
'subpacket_length': temp_obj['pub_key_alg'],
|
||||
'subpacket_data': temp_obj['key_id']
|
||||
}
|
||||
|
||||
elif temp_obj['type'] == 'cfg':
|
||||
|
||||
# there are several 'cfg' formats
|
||||
|
||||
if temp_obj['validity'] == 'version':
|
||||
# cfg:version:1.3.5
|
||||
line_obj = {
|
||||
'type': temp_obj['type'],
|
||||
'version': temp_obj['key_length']
|
||||
}
|
||||
|
||||
elif temp_obj['validity'] == 'pubkey':
|
||||
# cfg:pubkey:1;2;3;16;17
|
||||
line_obj = {
|
||||
'type': temp_obj['type'],
|
||||
'pubkey': temp_obj['key_length']
|
||||
}
|
||||
|
||||
elif temp_obj['validity'] == 'cipher':
|
||||
# cfg:cipher:2;3;4;7;8;9;10
|
||||
line_obj = {
|
||||
'type': temp_obj['type'],
|
||||
'cipher': temp_obj['key_length']
|
||||
}
|
||||
|
||||
elif temp_obj['validity'] == 'digest':
|
||||
# cfg:digest:1;2;3;8;9;10
|
||||
line_obj = {
|
||||
'type': temp_obj['type'],
|
||||
'digest': temp_obj['key_length']
|
||||
}
|
||||
|
||||
elif temp_obj['validity'] == 'compress':
|
||||
# cfg:compress:0;1;2;3
|
||||
line_obj = {
|
||||
'type': temp_obj['type'],
|
||||
'compress': temp_obj['key_length']
|
||||
}
|
||||
|
||||
elif temp_obj['validity'] == 'group':
|
||||
# cfg:group:mynames:patti;joe;0x12345678;paige
|
||||
line_obj = {
|
||||
'type': temp_obj['type'],
|
||||
'group': temp_obj['key_length'],
|
||||
'members': temp_obj['pub_key_alg']
|
||||
}
|
||||
|
||||
elif temp_obj['validity'] == 'curve':
|
||||
# cfg:curve:ed25519;nistp256;nistp384;nistp521
|
||||
line_obj = {
|
||||
'type': temp_obj['type'],
|
||||
'curve_names': temp_obj['key_length']
|
||||
}
|
||||
|
||||
else:
|
||||
line_obj = temp_obj
|
||||
|
||||
raw_output.append(line_obj)
|
||||
|
||||
return raw_output if raw else _process(raw_output)
|
||||
@@ -105,7 +105,7 @@ import jc.utils
|
||||
|
||||
class info():
|
||||
"""Provides parser metadata (version, author, etc.)"""
|
||||
version = '1.4'
|
||||
version = '1.5'
|
||||
description = '`id` command parser'
|
||||
author = 'Kelly Brazil'
|
||||
author_email = 'kellyjonbrazil@gmail.com'
|
||||
@@ -144,6 +144,13 @@ def _process(proc_data):
|
||||
return proc_data
|
||||
|
||||
|
||||
def _get_item(my_list, index, default=None):
|
||||
if index < len(my_list):
|
||||
return my_list[index]
|
||||
|
||||
return default
|
||||
|
||||
|
||||
def parse(data, raw=False, quiet=False):
|
||||
"""
|
||||
Main text parsing function
|
||||
@@ -174,14 +181,14 @@ def parse(data, raw=False, quiet=False):
|
||||
uid_parsed = uid_parsed.split('=')
|
||||
raw_output['uid'] = {}
|
||||
raw_output['uid']['id'] = uid_parsed[1]
|
||||
raw_output['uid']['name'] = uid_parsed[2]
|
||||
raw_output['uid']['name'] = _get_item(uid_parsed, 2)
|
||||
|
||||
if section.startswith('gid'):
|
||||
gid_parsed = section.replace('(', '=').replace(')', '=')
|
||||
gid_parsed = gid_parsed.split('=')
|
||||
raw_output['gid'] = {}
|
||||
raw_output['gid']['id'] = gid_parsed[1]
|
||||
raw_output['gid']['name'] = gid_parsed[2]
|
||||
raw_output['gid']['name'] = _get_item(gid_parsed, 2)
|
||||
|
||||
if section.startswith('groups'):
|
||||
groups_parsed = section.replace('(', '=').replace(')', '=')
|
||||
@@ -193,7 +200,7 @@ def parse(data, raw=False, quiet=False):
|
||||
group_dict = {}
|
||||
grp_parsed = group.split('=')
|
||||
group_dict['id'] = grp_parsed[0]
|
||||
group_dict['name'] = grp_parsed[1]
|
||||
group_dict['name'] = _get_item(grp_parsed, 1)
|
||||
raw_output['groups'].append(group_dict)
|
||||
|
||||
if section.startswith('context'):
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""jc - JSON Convert `ifconfig` command output parser
|
||||
|
||||
Note: No `ifconfig` options are supported.
|
||||
> Note: No `ifconfig` options are supported.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
|
||||
@@ -6,9 +6,10 @@ Parses standard `INI` files and files containing simple key/value pairs.
|
||||
- Comment prefix can be `#` or `;`. Comments must be on their own line.
|
||||
- If duplicate keys are found, only the last value will be used.
|
||||
|
||||
> Note: Values starting and ending with quotation marks will have the marks
|
||||
removed. If you would like to keep the quotation marks, use the `-r`
|
||||
command-line argument or the `raw=True` argument in `parse()`.
|
||||
> Note: Values starting and ending with double or single quotation marks
|
||||
> will have the marks removed. If you would like to keep the quotation
|
||||
> marks, use the `-r` command-line argument or the `raw=True` argument in
|
||||
> `parse()`.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
@@ -21,8 +22,8 @@ Usage (module):
|
||||
|
||||
Schema:
|
||||
|
||||
ini or key/value document converted to a dictionary - see the
|
||||
configparser standard library documentation for more details.
|
||||
ini or key/value document converted to a dictionary - see the configparser
|
||||
standard library documentation for more details.
|
||||
|
||||
{
|
||||
"key1": string,
|
||||
@@ -69,7 +70,7 @@ import configparser
|
||||
|
||||
class info():
|
||||
"""Provides parser metadata (version, author, etc.)"""
|
||||
version = '1.6'
|
||||
version = '1.7'
|
||||
description = 'INI file parser'
|
||||
author = 'Kelly Brazil'
|
||||
author_email = 'kellyjonbrazil@gmail.com'
|
||||
@@ -99,17 +100,21 @@ def _process(proc_data):
|
||||
for key, value in proc_data[heading].items():
|
||||
if value is not None and value.startswith('"') and value.endswith('"'):
|
||||
proc_data[heading][key] = value.lstrip('"').rstrip('"')
|
||||
|
||||
elif value is not None and value.startswith("'") and value.endswith("'"):
|
||||
proc_data[heading][key] = value.lstrip("'").rstrip("'")
|
||||
|
||||
elif value is None:
|
||||
proc_data[heading][key] = ''
|
||||
|
||||
# simple key/value files with no headers
|
||||
else:
|
||||
if (proc_data[heading] is not None and
|
||||
proc_data[heading].startswith('"') and
|
||||
proc_data[heading].endswith('"')):
|
||||
|
||||
if proc_data[heading] is not None and proc_data[heading].startswith('"') and proc_data[heading].endswith('"'):
|
||||
proc_data[heading] = proc_data[heading].lstrip('"').rstrip('"')
|
||||
|
||||
elif proc_data[heading] is not None and proc_data[heading].startswith("'") and proc_data[heading].endswith("'"):
|
||||
proc_data[heading] = proc_data[heading].lstrip("'").rstrip("'")
|
||||
|
||||
elif proc_data[heading] is None:
|
||||
proc_data[heading] = ''
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""jc - JSON Convert `iostat` command output parser
|
||||
|
||||
Note: `iostat` version 11 and higher include a JSON output option
|
||||
> Note: `iostat` version 11 and higher include a JSON output option
|
||||
|
||||
Usage (cli):
|
||||
|
||||
|
||||
@@ -1,20 +1,20 @@
|
||||
"""jc - JSON Convert `iostat` command output streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns a Generator
|
||||
iterator of Dictionaries (module)
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
Note: `iostat` version 11 and higher include a JSON output option
|
||||
> Note: `iostat` version 11 and higher include a JSON output option
|
||||
|
||||
Usage (cli):
|
||||
|
||||
$ iostat | jc --iostat-s
|
||||
|
||||
> Note: When piping `jc` converted `iostat` output to other processes it may
|
||||
appear the output is hanging due to the OS pipe buffers. This is because
|
||||
`iostat` output is too small to quickly fill up the buffer. Use the `-u`
|
||||
option to unbuffer the `jc` output if you would like immediate output. See
|
||||
the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
|
||||
for more information.
|
||||
> appear the output is hanging due to the OS pipe buffers. This is because
|
||||
> `iostat` output is too small to quickly fill up the buffer. Use the `-u`
|
||||
> option to unbuffer the `jc` output if you would like immediate output. See
|
||||
> the [readme](https://github.com/kellyjonbrazil/jc/tree/master#unbuffering-output)
|
||||
> for more information.
|
||||
|
||||
Usage (module):
|
||||
|
||||
@@ -163,7 +163,7 @@ def _create_obj_list(section_list, section_name):
|
||||
@add_jc_meta
|
||||
def parse(data, raw=False, quiet=False, ignore_exceptions=False):
|
||||
"""
|
||||
Main text parsing generator function. Returns an iterator object.
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
@@ -174,13 +174,9 @@ def parse(data, raw=False, quiet=False, ignore_exceptions=False):
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
Yields:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
|
||||
Returns:
|
||||
|
||||
Iterator object (generator)
|
||||
Iterable of Dictionaries
|
||||
"""
|
||||
jc.utils.compatibility(__name__, info.compatible, quiet)
|
||||
streaming_input_type_check(data)
|
||||
|
||||
@@ -163,7 +163,7 @@ import jc.utils
|
||||
|
||||
class info():
|
||||
"""Provides parser metadata (version, author, etc.)"""
|
||||
version = '1.7'
|
||||
version = '1.8'
|
||||
description = '`iptables` command parser'
|
||||
author = 'Kelly Brazil'
|
||||
author_email = 'kellyjonbrazil@gmail.com'
|
||||
@@ -264,7 +264,6 @@ def parse(data, raw=False, quiet=False):
|
||||
continue
|
||||
|
||||
elif line.startswith('target') or line.find('pkts') == 1 or line.startswith('num'):
|
||||
headers = []
|
||||
headers = [h for h in ' '.join(line.lower().strip().split()).split() if h]
|
||||
headers.append("options")
|
||||
|
||||
|
||||
@@ -7,8 +7,8 @@ Supports files containing simple key/value pairs.
|
||||
- If duplicate keys are found, only the last value will be used.
|
||||
|
||||
> Note: Values starting and ending with quotation marks will have the marks
|
||||
removed. If you would like to keep the quotation marks, use the `-r`
|
||||
command-line argument or the `raw=True` argument in `parse()`.
|
||||
> removed. If you would like to keep the quotation marks, use the `-r`
|
||||
> command-line argument or the `raw=True` argument in `parse()`.
|
||||
|
||||
Usage (cli):
|
||||
|
||||
@@ -21,8 +21,8 @@ Usage (module):
|
||||
|
||||
Schema:
|
||||
|
||||
key/value document converted to a dictionary - see the
|
||||
configparser standard library documentation for more details.
|
||||
key/value document converted to a dictionary - see the configparser standard
|
||||
library documentation for more details.
|
||||
|
||||
{
|
||||
"key1": string,
|
||||
|
||||
@@ -4,11 +4,12 @@ Options supported:
|
||||
- `lbaR1`
|
||||
- `--time-style=full-iso`
|
||||
|
||||
Note: The `-1`, `-l`, or `-b` option of `ls` should be used to correctly
|
||||
parse filenames that include newline characters. Since `ls` does not encode
|
||||
newlines in filenames when outputting to a pipe it will cause `jc` to see
|
||||
multiple files instead of a single file if `-1`, `-l`, or `-b` is not used.
|
||||
Alternatively, `vdir` can be used, which is the same as running `ls -lb`.
|
||||
> Note: The `-1`, `-l`, or `-b` option of `ls` should be used to correctly
|
||||
> parse filenames that include newline characters. Since `ls` does not
|
||||
> encode newlines in filenames when outputting to a pipe it will cause `jc`
|
||||
> to see multiple files instead of a single file if `-1`, `-l`, or `-b` is
|
||||
> not used. Alternatively, `vdir` can be used, which is the same as running
|
||||
> `ls -lb`.
|
||||
|
||||
The `epoch` calculated timestamp field is naive. (i.e. based on the local
|
||||
time of the system the parser is run on)
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
"""jc - JSON Convert `ls` and `vdir` command output streaming parser
|
||||
|
||||
> This streaming parser outputs JSON Lines (cli) or returns a Generator
|
||||
iterator of Dictionaries (module)
|
||||
> This streaming parser outputs JSON Lines (cli) or returns an Iterable of
|
||||
> Dictionaries (module)
|
||||
|
||||
Requires the `-l` option to be used on `ls`. If there are newline characters
|
||||
in the filename, then make sure to use the `-b` option on `ls`.
|
||||
@@ -118,7 +118,7 @@ def _process(proc_data):
|
||||
@add_jc_meta
|
||||
def parse(data, raw=False, quiet=False, ignore_exceptions=False):
|
||||
"""
|
||||
Main text parsing generator function. Returns an iterator object.
|
||||
Main text parsing generator function. Returns an iterable object.
|
||||
|
||||
Parameters:
|
||||
|
||||
@@ -129,13 +129,9 @@ def parse(data, raw=False, quiet=False, ignore_exceptions=False):
|
||||
quiet: (boolean) suppress warning messages if True
|
||||
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
||||
|
||||
Yields:
|
||||
|
||||
Dictionary. Raw or processed structured data.
|
||||
|
||||
Returns:
|
||||
|
||||
Iterator object (generator)
|
||||
Iterable of Dictionaries
|
||||
"""
|
||||
jc.utils.compatibility(__name__, info.compatible, quiet)
|
||||
streaming_input_type_check(data)
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user