1
0
mirror of https://github.com/kellyjonbrazil/jc.git synced 2025-06-17 00:07:37 +02:00
Files
jc/README.md

953 lines
31 KiB
Markdown
Raw Normal View History

2020-04-03 14:47:08 -07:00
![Tests](https://github.com/kellyjonbrazil/jc/workflows/Tests/badge.svg?branch=master)
2020-04-07 08:44:15 -07:00
![Pypi](https://img.shields.io/pypi/v/jc.svg)
2020-04-02 10:55:32 -07:00
2020-08-26 15:21:45 -07:00
> Try the new `jc` [web demo](https://jc-web-demo.herokuapp.com/)!
2020-08-26 14:41:01 -07:00
2020-08-30 11:58:42 -07:00
> JC is [now available](https://galaxy.ansible.com/community/general) as an Ansible filter plugin in the `community.general` collection! See this [blog post](https://blog.kellybrazil.com/2020/08/30/parsing-command-output-in-ansible-with-jc/) for an example.
2020-08-20 10:35:49 -07:00
2019-10-15 15:06:09 -07:00
# JC
2019-10-17 13:04:34 -07:00
JSON CLI output utility
2019-10-15 15:06:09 -07:00
2020-05-21 11:10:00 -07:00
`jc` JSONifies the output of many CLI tools and file-types for easier parsing in scripts. See the [**Parsers**](#parsers) section for supported commands and file-types.
2019-10-15 15:06:09 -07:00
2020-05-20 16:14:03 -07:00
This allows further command-line processing of output with tools like `jq` by piping commands:
2020-06-25 21:11:21 -07:00
```bash
ls -l /usr/bin | jc --ls | jq '.[] | select(.size > 50000000)'
```
2020-06-25 12:28:23 -07:00
```json
2019-10-18 09:57:10 -07:00
{
2019-11-12 11:46:52 -08:00
"filename": "docker",
"flags": "-rwxr-xr-x",
"links": 1,
2019-10-18 09:57:10 -07:00
"owner": "root",
2019-11-12 11:46:52 -08:00
"group": "root",
"size": 68677120,
"date": "Aug 14 19:41"
2019-10-18 09:57:10 -07:00
}
```
2020-02-13 17:24:10 -08:00
or using the alternative "magic" syntax:
2020-06-25 21:11:21 -07:00
```bash
jc ls -l /usr/bin | jq '.[] | select(.size > 50000000)'
2020-06-25 21:12:30 -07:00
```
2020-06-25 12:28:23 -07:00
```json
2020-02-13 17:20:00 -08:00
{
"filename": "docker",
"flags": "-rwxr-xr-x",
"links": 1,
"owner": "root",
"group": "root",
"size": 68677120,
"date": "Aug 14 19:41"
}
```
2020-02-24 13:05:35 -08:00
The `jc` parsers can also be used as python modules. In this case the output will be a python dictionary, or list of dictionaries, instead of JSON:
2020-06-25 12:28:23 -07:00
```python
2019-10-22 07:40:42 -07:00
>>> import jc.parsers.ls
2019-10-25 15:39:48 -07:00
>>>
2019-10-22 07:40:42 -07:00
>>> data='''-rwxr-xr-x 1 root wheel 23648 May 3 22:26 cat
... -rwxr-xr-x 1 root wheel 30016 May 3 22:26 chmod
... -rwxr-xr-x 1 root wheel 29024 May 3 22:26 cp
... -rwxr-xr-x 1 root wheel 375824 May 3 22:26 csh
... -rwxr-xr-x 1 root wheel 28608 May 3 22:26 date
... -rwxr-xr-x 1 root wheel 32000 May 3 22:26 dd
... -rwxr-xr-x 1 root wheel 23392 May 3 22:26 df
... -rwxr-xr-x 1 root wheel 18128 May 3 22:26 echo'''
2019-11-12 11:46:52 -08:00
>>>
2019-10-22 07:40:42 -07:00
>>> jc.parsers.ls.parse(data)
2019-11-12 11:49:14 -08:00
[{'filename': 'cat', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'size': 23648,
'date': 'May 3 22:26'}, {'filename': 'chmod', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root',
'group': 'wheel', 'size': 30016, 'date': 'May 3 22:26'}, {'filename': 'cp', 'flags': '-rwxr-xr-x',
'links': 1, 'owner': 'root', 'group': 'wheel', 'size': 29024, 'date': 'May 3 22:26'}, {'filename': 'csh',
'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'size': 375824, 'date': 'May 3
22:26'}, {'filename': 'date', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel',
'size': 28608, 'date': 'May 3 22:26'}, {'filename': 'dd', 'flags': '-rwxr-xr-x', 'links': 1, 'owner':
'root', 'group': 'wheel', 'size': 32000, 'date': 'May 3 22:26'}, {'filename': 'df', 'flags':
'-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'size': 23392, 'date': 'May 3 22:26'},
{'filename': 'echo', 'flags': '-rwxr-xr-x', 'links': 1, 'owner': 'root', 'group': 'wheel', 'size': 18128,
'date': 'May 3 22:26'}]
2019-10-25 15:39:48 -07:00
```
2019-12-17 09:38:50 -08:00
Two representations of the data are possible. The default representation uses a strict schema per parser and converts known numbers to int/float JSON values. Certain known values of `None` are converted to JSON `null`, known boolean values are converted, and, in some cases, additional semantic context fields are added.
2019-11-12 07:18:27 -08:00
> Note: Some parsers have calculated epoch timestamp fields added to the output. Unless a timestamp field name has a `_utc` suffix it is considered naive. (i.e. based on the local timezone of the system the `jc` parser was run on).
2021-03-25 15:10:07 -07:00
>
2021-03-25 15:34:54 -07:00
> If a UTC timezone can be detected in the text of the command output, the timestamp will be timezone aware and have a `_utc` suffix on the key name. (e.g. `epoch_utc`) No other timezones are supported for aware timestamps.
2019-11-14 13:43:07 -08:00
To access the raw, pre-processed JSON, use the `-r` cli option or the `raw=True` function parameter in `parse()`.
2019-11-12 07:18:27 -08:00
2021-04-05 12:03:35 -07:00
Schemas for each parser can be found at the documentation link beside each parser below.
2019-10-21 14:13:31 -07:00
2020-03-03 11:37:59 -08:00
Release notes can be found [here](https://blog.kellybrazil.com/category/jc-news/).
2020-06-26 09:53:57 -07:00
## Why Would Anyone Do This!?
2020-05-27 09:52:13 -07:00
For more information on the motivations for this project, please see my [blog post](https://blog.kellybrazil.com/2019/11/26/bringing-the-unix-philosophy-to-the-21st-century/).
2020-02-14 09:43:02 -08:00
2020-06-26 09:53:57 -07:00
See also:
- [libxo on FreeBSD](http://juniper.github.io/libxo/libxo-manual.html)
- [powershell](https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.utility/convertto-json?view=powershell-7)
- [blog: linux apps should have a json flag](https://thomashunter.name/posts/2012-06-06-linux-cli-apps-should-have-a-json-flag)
2019-10-17 15:03:32 -07:00
## Installation
2020-08-05 11:04:07 -07:00
There are several ways to get `jc`. You can install via `pip`; other OS package repositories like `apt-get`, `dnf`, `zypper`, `pacman`, `nix-env`, `guix`, `brew`, or `portsnap`; via DEB/RPM packages; or by downloading the correct binary for your architecture and running it anywhere on your filesystem.
2020-04-15 21:22:43 -07:00
2020-04-16 14:03:31 -07:00
### Pip (macOS, linux, unix, Windows)
2020-06-25 12:28:23 -07:00
```bash
pip3 install jc
2019-10-17 15:03:32 -07:00
```
2020-05-27 14:58:16 -07:00
### OS Package Repositories
2020-05-27 17:08:54 -07:00
2020-06-27 18:53:19 -07:00
| OS | Command |
|-----------------------|-------------------------------------------------------------------------------|
2020-11-19 07:07:30 -08:00
| Debian/Ubuntu linux | `apt-get install jc` |
| Fedora linux | `dnf install jc` |
2020-06-27 18:53:19 -07:00
| openSUSE linux | `zypper install jc` |
2020-07-01 13:28:58 -07:00
| Arch linux | `pacman -S jc` |
2021-03-26 09:32:49 -07:00
| NixOS linux | `nix-env -iA nixpkgs.jc` or `nix-env -iA nixos.jc` |
2020-06-27 18:53:19 -07:00
| Guix System linux | `guix install jc` |
| MacOS | `brew install jc` |
| FreeBSD | `portsnap fetch update && cd /usr/ports/textproc/py-jc && make install clean` |
2020-08-20 10:39:34 -07:00
| Ansible filter plugin | `ansible-galaxy collection install community.general` |
2020-05-27 14:58:16 -07:00
2020-06-27 18:53:19 -07:00
> For more packages and binaries, see https://kellyjonbrazil.github.io/jc-packaging/.
2020-04-15 21:22:43 -07:00
2019-10-15 15:06:09 -07:00
## Usage
2020-02-13 18:19:19 -08:00
`jc` accepts piped input from `STDIN` and outputs a JSON representation of the previous command's output to `STDOUT`.
2020-06-25 12:28:23 -07:00
```bash
2020-02-13 17:20:00 -08:00
COMMAND | jc PARSER [OPTIONS]
```
2021-01-08 08:23:29 -08:00
Alternatively, the "magic" syntax can be used by prepending `jc` to the command to be converted. Options can be passed to `jc` immediately before the command is given. (Note: command aliases and shell builtins are not supported)
2020-06-25 12:28:23 -07:00
```bash
2020-02-13 17:20:00 -08:00
jc [OPTIONS] COMMAND
2019-10-21 13:47:22 -07:00
```
2020-02-13 18:19:19 -08:00
The JSON output can be compact (default) or pretty formatted with the `-p` option.
2019-10-15 15:06:09 -07:00
2021-02-11 11:27:54 -08:00
> Note: For best results set the `LANG` locale environment variable to `C`. For example, either by setting directly on the command-line: `$ LANG=C date | jc --date`, or by exporting to the environment before running commands: `$ export LANG=C`.
2021-02-11 11:27:01 -08:00
2019-10-22 11:15:44 -07:00
### Parsers
2021-04-05 11:49:30 -07:00
- `--acpi` enables the `acpi` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/acpi))
- `--airport` enables the `airport -I` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/airport))
- `--airport-s` enables the `airport -s` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/airport_s))
- `--arp` enables the `arp` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/arp))
- `--blkid` enables the `blkid` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/blkid))
- `--cksum` enables the `cksum` and `sum` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/cksum))
- `--crontab` enables the `crontab` command and file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab))
- `--crontab-u` enables the `crontab` file parser with user support ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/crontab_u))
- `--csv` enables the CSV file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/csv))
- `--date` enables the `date` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/date))
- `--df` enables the `df` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/df))
- `--dig` enables the `dig` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/dig))
- `--dir` enables the `dir` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/dir))
- `--dmidecode` enables the `dmidecode` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/dmidecode))
- `--dpkg-l` enables the `dpkg -l` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/dpkg_l))
- `--du` enables the `du` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/du))
- `--env` enables the `env` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/env))
- `--file` enables the `file` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/file))
- `--free` enables the `free` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/free))
- `--fstab` enables the `/etc/fstab` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/fstab))
- `--group` enables the `/etc/group` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/group))
- `--gshadow` enables the `/etc/gshadow` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/gshadow))
- `--hash` enables the `hash` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/hash))
- `--hashsum` enables the hashsum command parser (`md5sum`, `shasum`, etc.) ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/hashsum))
- `--hciconfig` enables the `hciconfig` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/hciconfig))
- `--history` enables the `history` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/history))
- `--hosts` enables the `/etc/hosts` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/hosts))
- `--id` enables the `id` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/id))
- `--ifconfig` enables the `ifconfig` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ifconfig))
- `--ini` enables the INI file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ini))
- `--iptables` enables the `iptables` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/iptables))
- `--iw-scan` enables the `iw dev [device] scan` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/iw_scan))
- `--jobs` enables the `jobs` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/jobs))
- `--kv` enables the Key/Value file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/kv))
- `--last` enables the `last` and `lastb` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/last))
- `--ls` enables the `ls` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ls))
- `--lsblk` enables the `lsblk` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/lsblk))
- `--lsmod` enables the `lsmod` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/lsmod))
- `--lsof` enables the `lsof` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/lsof))
- `--mount` enables the `mount` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/mount))
- `--netstat` enables the `netstat` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/netstat))
- `--ntpq` enables the `ntpq -p` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ntpq))
- `--passwd` enables the `/etc/passwd` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/passwd))
- `--ping` enables the `ping` and `ping6` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ping))
- `--pip-list` enables the `pip list` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_list))
- `--pip-show` enables the `pip show` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/pip_show))
- `--ps` enables the `ps` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ps))
- `--route` enables the `route` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/route))
- `--shadow` enables the `/etc/shadow` file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/shadow))
- `--ss` enables the `ss` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/ss))
- `--stat` enables the `stat` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/stat))
- `--sysctl` enables the `sysctl` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/sysctl))
- `--systemctl` enables the `systemctl` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl))
- `--systemctl-lj` enables the `systemctl list-jobs` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_lj))
- `--systemctl-ls` enables the `systemctl list-sockets` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_ls))
- `--systemctl-luf` enables the `systemctl list-unit-files` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/systemctl_luf))
- `--time` enables the `/usr/bin/time` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/time))
- `--timedatectl` enables the `timedatectl status` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/timedatectl))
- `--tracepath` enables the `tracepath` and `tracepath6` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/tracepath))
- `--traceroute` enables the `traceroute` and `traceroute6` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/traceroute))
- `--uname` enables the `uname -a` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/uname))
- `--upower` enables the `upower` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/upower))
- `--uptime` enables the `uptime` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/uptime))
- `--w` enables the `w` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/w))
- `--wc` enables the `wc` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/wc))
- `--who` enables the `who` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/who))
- `--xml` enables the XML file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/xml))
- `--yaml` enables the YAML file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/yaml))
2019-10-15 15:06:09 -07:00
2019-10-22 11:15:44 -07:00
### Options
2019-12-14 23:27:56 -08:00
- `-a` about `jc`. Prints information about `jc` and the parsers (in JSON, of course!)
2021-01-08 08:23:29 -08:00
- `-d` debug mode. Prints trace messages if parsing issues are encountered (use `-dd` for verbose debugging)
- `-h` `jc` help
2020-04-02 17:40:06 -07:00
- `-m` monochrome JSON output
2019-11-04 12:50:37 -08:00
- `-p` pretty format the JSON output
2021-01-07 12:09:41 -08:00
- `-q` quiet mode. Suppresses parser warning messages
2020-07-28 11:03:41 -07:00
- `-r` raw output. Provides a more literal JSON output with all values as strings and no additional semantic processing
2021-03-30 16:43:53 -07:00
- `-v` version information
2019-10-15 15:06:09 -07:00
2020-04-12 13:10:57 -07:00
### Setting Custom Colors via Environment Variable
2020-04-12 13:13:28 -07:00
You can specify custom colors via the `JC_COLORS` environment variable. The `JC_COLORS` environment variable takes four comma separated string values in the following format:
2020-06-25 12:28:23 -07:00
```bash
2020-04-12 13:23:58 -07:00
JC_COLORS=<keyname_color>,<keyword_color>,<number_color>,<string_color>
2020-04-12 13:10:57 -07:00
```
Where colors are: `black`, `red`, `green`, `yellow`, `blue`, `magenta`, `cyan`, `gray`, `brightblack`, `brightred`, `brightgreen`, `brightyellow`, `brightblue`, `brightmagenta`, `brightcyan`, `white`, or `default`
For example, to set to the default colors:
2020-06-25 12:28:23 -07:00
```bash
2020-04-12 13:10:57 -07:00
JC_COLORS=blue,brightblack,magenta,green
```
or
2020-06-25 12:28:23 -07:00
```bash
2020-04-12 13:10:57 -07:00
JC_COLORS=default,default,default,default
```
2020-06-08 10:44:09 -07:00
### Custom Parsers
Custom local parser plugins may be placed in a `jc/jcparsers` folder in your local **"App data directory"**:
- Linux/unix: `$HOME/.local/share/jc/jcparsers`
- macOS: `$HOME/Library/Application Support/jc/jcparsers`
2020-06-08 10:48:58 -07:00
- Windows: `$LOCALAPPDATA\jc\jc\jcparsers`
Local parser plugins are standard python module files. Use the [`jc/parsers/foo.py`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py) parser as a template and simply place a `.py` file in the `jcparsers` subfolder.
Local plugin filenames must be valid python module names, therefore must consist entirely of alphanumerics and start with a letter. Local plugins may override default plugins.
> Note: The application data directory follows the [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html)
2020-02-27 09:36:57 -08:00
## Compatibility
Some parsers like `ls`, `ps`, `dig`, etc. will work on any platform. Other parsers that are platform-specific will generate a warning message if they are used on an unsupported platform. To see all parser information, including compatibility, run `jc -ap`.
2021-04-04 15:57:44 -07:00
You may still use a parser on an unsupported platform - for example, you may want to parse a file with linux `lsof` output on an macOS laptop. In that case you can suppress the warning message with the `-q` cli option or the `quiet=True` function parameter in `parse()`:
2020-02-27 09:36:57 -08:00
2020-06-25 12:28:23 -07:00
```bash
cat lsof.out | jc --lsof -q
2020-02-27 09:36:57 -08:00
```
Tested on:
- Centos 7.7
- Ubuntu 18.04
- Ubuntu 20.04
2020-05-21 09:07:01 -07:00
- Fedora32
2021-04-04 15:57:44 -07:00
- macOS 10.11.6
- macOS 10.14.6
- NixOS
- FreeBSD12
2021-04-02 12:14:26 -07:00
- Windows 10
2020-02-27 09:36:57 -08:00
2020-05-21 09:07:01 -07:00
## Contributions
Feel free to add/improve code or parsers! You can use the [`jc/parsers/foo.py`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py) parser as a template and submit your parser with a pull request.
2021-04-02 13:19:47 -07:00
Please see the [Contributing Guidelines](https://github.com/kellyjonbrazil/jc/blob/master/CONTRIBUTING.md) for more information.
2020-02-27 09:36:57 -08:00
## Acknowledgments
2020-06-07 12:23:28 -07:00
- Local parser plugin feature contributed by [Dean Serenevy](https://github.com/duelafn)
2020-06-07 12:29:10 -07:00
- CI automation and code optimizations by [philippeitis](https://github.com/philippeitis)
- [`ifconfig-parser`](https://github.com/KnightWhoSayNi/ifconfig-parser) module by KnightWhoSayNi
- [`xmltodict`](https://github.com/martinblech/xmltodict) module by Martín Blech
- [`ruamel.yaml`](https://pypi.org/project/ruamel.yaml) module by Anthon van der Neut
2020-07-27 10:33:15 -07:00
- [`trparse`](https://github.com/lbenitez000/trparse) module by Luis Benitez
2020-06-07 12:29:10 -07:00
- Parsing [code](https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501) from Conor Heine adapted for some parsers
- Excellent constructive feedback from [Ilya Sher](https://github.com/ilyash-b)
2020-02-27 09:36:57 -08:00
2019-10-15 15:06:09 -07:00
## Examples
2020-07-29 11:40:47 -07:00
Here are some examples of `jc` output. For more examples, see [EXAMPLES.md](https://github.com/kellyjonbrazil/jc/blob/master/EXAMPLES.md) or the [parser documentation](https://github.com/kellyjonbrazil/jc/tree/master/docs/parsers).
2019-10-30 13:22:12 -07:00
### arp
2020-06-25 21:11:21 -07:00
```bash
arp | jc --arp -p # or: jc -p arp
```
2020-06-25 12:56:06 -07:00
```json
2019-10-30 13:22:12 -07:00
[
{
"address": "gateway",
"hwtype": "ether",
"hwaddress": "00:50:56:f7:4a:fc",
"flags_mask": "C",
"iface": "ens33"
},
2019-10-30 13:52:31 -07:00
{
"address": "192.168.71.1",
"hwtype": "ether",
"hwaddress": "00:50:56:c0:00:08",
"flags_mask": "C",
"iface": "ens33"
},
2019-10-30 13:22:12 -07:00
{
"address": "192.168.71.254",
"hwtype": "ether",
"hwaddress": "00:50:56:fe:7a:b4",
"flags_mask": "C",
"iface": "ens33"
}
]
```
2020-03-02 14:07:29 -08:00
### CSV files
2020-06-25 21:11:21 -07:00
```bash
cat homes.csv
```
2020-06-25 13:01:10 -07:00
```
2020-03-02 14:07:29 -08:00
"Sell", "List", "Living", "Rooms", "Beds", "Baths", "Age", "Acres", "Taxes"
142, 160, 28, 10, 5, 3, 60, 0.28, 3167
175, 180, 18, 8, 4, 1, 12, 0.43, 4033
129, 132, 13, 6, 3, 1, 41, 0.33, 1471
...
2020-06-25 13:01:10 -07:00
```
2020-06-25 21:11:21 -07:00
```bash
cat homes.csv | jc --csv -p
```
2020-06-25 13:01:10 -07:00
```json
2020-03-02 14:07:29 -08:00
[
{
"Sell": "142",
"List": "160",
"Living": "28",
"Rooms": "10",
"Beds": "5",
"Baths": "3",
"Age": "60",
"Acres": "0.28",
"Taxes": "3167"
},
{
"Sell": "175",
"List": "180",
"Living": "18",
"Rooms": "8",
"Beds": "4",
"Baths": "1",
"Age": "12",
"Acres": "0.43",
"Taxes": "4033"
},
{
"Sell": "129",
"List": "132",
"Living": "13",
"Rooms": "6",
"Beds": "3",
"Baths": "1",
"Age": "41",
"Acres": "0.33",
"Taxes": "1471"
2020-06-25 12:56:06 -07:00
}
2020-03-02 14:07:29 -08:00
]
```
2019-10-31 07:27:12 -07:00
### dig
2020-06-25 21:11:21 -07:00
```bash
2020-07-29 14:36:20 -07:00
dig cnn.com @205.251.194.64 | jc --dig -p # or: jc -p dig cnn.com @205.251.194.64
2020-06-25 21:11:21 -07:00
```
2020-06-25 12:56:06 -07:00
```json
2019-10-31 07:27:12 -07:00
[
{
"id": 52172,
2019-10-31 07:27:12 -07:00
"opcode": "QUERY",
"status": "NOERROR",
2019-11-12 07:18:27 -08:00
"flags": [
"qr",
"rd",
"ra"
],
"query_num": 1,
2020-07-29 14:36:20 -07:00
"answer_num": 1,
2019-11-12 07:18:27 -08:00
"authority_num": 0,
"additional_num": 1,
2019-10-31 07:27:12 -07:00
"question": {
"name": "cnn.com.",
"class": "IN",
"type": "A"
},
"answer": [
{
"name": "cnn.com.",
"class": "IN",
"type": "A",
"ttl": 27,
"data": "151.101.65.67"
2019-10-31 07:27:12 -07:00
}
],
"query_time": 38,
2019-11-12 07:18:27 -08:00
"server": "2600",
"when": "Tue Mar 30 20:07:59 PDT 2021",
"rcvd": 100,
"when_epoch": 1617160079,
"when_epoch_utc": null
2019-10-31 07:27:12 -07:00
}
]
```
2020-02-04 21:46:52 -08:00
### /etc/hosts file
2020-06-25 21:11:21 -07:00
```bash
cat /etc/hosts | jc --hosts -p
```
2020-06-25 12:56:06 -07:00
```json
2019-11-14 21:59:06 -08:00
[
{
"ip": "127.0.0.1",
"hostname": [
"localhost"
]
},
{
"ip": "::1",
"hostname": [
"ip6-localhost",
"ip6-loopback"
]
},
{
"ip": "fe00::0",
"hostname": [
"ip6-localnet"
]
}
]
```
2020-07-29 11:40:47 -07:00
### ifconfig
2020-06-25 21:11:21 -07:00
```bash
2020-07-29 11:40:47 -07:00
ifconfig | jc --ifconfig -p # or: jc -p ifconfig
2020-06-25 21:11:21 -07:00
```
2020-06-25 12:56:06 -07:00
```json
2019-10-15 15:32:23 -07:00
[
{
"name": "ens33",
2019-11-12 13:05:19 -08:00
"flags": 4163,
"state": [
"UP",
"BROADCAST",
"RUNNING",
"MULTICAST"
],
2019-11-12 13:05:19 -08:00
"mtu": 1500,
"ipv4_addr": "192.168.71.137",
2019-10-15 15:32:23 -07:00
"ipv4_mask": "255.255.255.0",
"ipv4_bcast": "192.168.71.255",
"ipv6_addr": "fe80::c1cb:715d:bc3e:b8a0",
2019-11-12 13:05:19 -08:00
"ipv6_mask": 64,
"ipv6_scope": "0x20",
2019-10-15 15:32:23 -07:00
"mac_addr": "00:0c:29:3b:58:0e",
"type": "Ethernet",
"rx_packets": 8061,
"rx_bytes": 1514413,
2019-11-12 13:05:19 -08:00
"rx_errors": 0,
"rx_dropped": 0,
"rx_overruns": 0,
"rx_frame": 0,
"tx_packets": 4502,
"tx_bytes": 866622,
2019-11-12 13:05:19 -08:00
"tx_errors": 0,
"tx_dropped": 0,
"tx_overruns": 0,
"tx_carrier": 0,
"tx_collisions": 0,
2019-10-15 15:32:23 -07:00
"metric": null
}
]
2019-10-17 15:03:32 -07:00
```
2020-07-29 11:49:00 -07:00
### INI files
2020-06-25 21:11:21 -07:00
```bash
cat example.ini
```
2020-02-03 21:38:08 -08:00
```
[DEFAULT]
ServerAliveInterval = 45
Compression = yes
CompressionLevel = 9
ForwardX11 = yes
[bitbucket.org]
User = hg
[topsecret.server.com]
Port = 50022
ForwardX11 = no
2020-06-25 12:56:06 -07:00
```
2020-06-25 21:11:21 -07:00
```bash
cat example.ini | jc --ini -p
```
2020-06-25 12:56:06 -07:00
```json
2020-02-03 21:38:08 -08:00
{
"bitbucket.org": {
"serveraliveinterval": "45",
"compression": "yes",
"compressionlevel": "9",
"forwardx11": "yes",
"user": "hg"
},
"topsecret.server.com": {
"serveraliveinterval": "45",
"compression": "yes",
"compressionlevel": "9",
"forwardx11": "no",
"port": "50022"
}
}
```
2019-10-21 17:26:00 -07:00
### ls
2020-06-25 21:11:21 -07:00
```bash
2020-02-13 17:20:00 -08:00
$ ls -l /usr/bin | jc --ls -p # or: jc -p ls -l /usr/bin
2020-06-25 21:11:21 -07:00
```
```json
2019-10-21 17:26:00 -07:00
[
{
2019-10-25 15:39:48 -07:00
"filename": "apropos",
"link_to": "whatis",
"flags": "lrwxrwxrwx.",
2019-11-12 13:05:19 -08:00
"links": 1,
2019-10-21 17:26:00 -07:00
"owner": "root",
2019-10-25 15:39:48 -07:00
"group": "root",
2019-11-12 13:05:19 -08:00
"size": 6,
2019-10-25 15:39:48 -07:00
"date": "Aug 15 10:53"
2019-10-21 17:26:00 -07:00
},
{
2019-11-12 13:05:19 -08:00
"filename": "ar",
2019-10-25 15:39:48 -07:00
"flags": "-rwxr-xr-x.",
2019-11-12 13:05:19 -08:00
"links": 1,
2019-10-21 17:26:00 -07:00
"owner": "root",
2019-10-25 15:39:48 -07:00
"group": "root",
2019-11-12 13:05:19 -08:00
"size": 62744,
"date": "Aug 8 16:14"
2019-10-25 15:39:48 -07:00
},
{
2019-11-12 13:05:19 -08:00
"filename": "arch",
2019-10-25 15:39:48 -07:00
"flags": "-rwxr-xr-x.",
2019-11-12 13:05:19 -08:00
"links": 1,
2019-10-25 15:39:48 -07:00
"owner": "root",
"group": "root",
2019-11-12 13:05:19 -08:00
"size": 33080,
2019-10-25 15:39:48 -07:00
"date": "Aug 19 23:25"
2020-06-25 12:56:06 -07:00
}
2019-10-21 17:26:00 -07:00
]
```
2019-10-18 09:57:10 -07:00
### netstat
2020-06-25 21:11:21 -07:00
```bash
netstat -apee | jc --netstat -p # or: jc -p netstat -apee
```
2020-06-25 12:56:06 -07:00
```json
2019-10-21 13:47:22 -07:00
[
{
2019-11-12 13:05:19 -08:00
"proto": "tcp",
"recv_q": 0,
"send_q": 0,
"local_address": "localhost",
"foreign_address": "0.0.0.0",
"state": "LISTEN",
"user": "systemd-resolve",
"inode": 26958,
"program_name": "systemd-resolve",
"kind": "network",
"pid": 887,
"local_port": "domain",
"foreign_port": "*",
2019-10-21 18:22:51 -07:00
"transport_protocol": "tcp",
2019-11-12 13:05:19 -08:00
"network_protocol": "ipv4"
2019-10-21 13:47:22 -07:00
},
2019-11-12 13:05:19 -08:00
{
"proto": "tcp6",
"recv_q": 0,
"send_q": 0,
"local_address": "[::]",
"foreign_address": "[::]",
2019-10-21 13:47:22 -07:00
"state": "LISTEN",
2019-11-12 13:05:19 -08:00
"user": "root",
"inode": 30510,
2019-10-21 13:47:22 -07:00
"program_name": "sshd",
2019-11-12 13:05:19 -08:00
"kind": "network",
"pid": 1186,
"local_port": "ssh",
"foreign_port": "*",
"transport_protocol": "tcp",
"network_protocol": "ipv6"
2019-10-21 13:47:22 -07:00
},
{
2019-11-12 13:05:19 -08:00
"proto": "udp",
"recv_q": 0,
"send_q": 0,
"local_address": "localhost",
"foreign_address": "0.0.0.0",
"state": null,
"user": "systemd-resolve",
"inode": 26957,
"program_name": "systemd-resolve",
"kind": "network",
"pid": 887,
"local_port": "domain",
2019-10-21 13:47:22 -07:00
"foreign_port": "*",
2019-11-12 13:05:19 -08:00
"transport_protocol": "udp",
"network_protocol": "ipv4"
2019-10-21 13:47:22 -07:00
},
{
2019-11-12 13:05:19 -08:00
"proto": "raw6",
"recv_q": 0,
"send_q": 0,
"local_address": "[::]",
"foreign_address": "[::]",
"state": "7",
"user": "systemd-network",
"inode": 27001,
"program_name": "systemd-network",
"kind": "network",
"pid": 867,
"local_port": "ipv6-icmp",
2019-10-21 13:47:22 -07:00
"foreign_port": "*",
2019-11-12 13:05:19 -08:00
"transport_protocol": null,
"network_protocol": "ipv6"
},
{
"proto": "unix",
"refcnt": 2,
"flags": null,
"type": "DGRAM",
"state": null,
"inode": 33322,
"program_name": "systemd",
"path": "/run/user/1000/systemd/notify",
"kind": "socket",
"pid": 1607
2020-06-25 12:56:06 -07:00
}
2019-10-21 13:47:22 -07:00
]
2020-06-25 21:11:21 -07:00
```
### /etc/passwd file
2020-06-25 21:11:21 -07:00
```bash
cat /etc/passwd | jc --passwd -p
```
2020-06-25 12:56:06 -07:00
```json
[
{
"username": "root",
"password": "*",
"uid": 0,
"gid": 0,
"comment": "System Administrator",
"home": "/var/root",
"shell": "/bin/sh"
},
{
"username": "daemon",
"password": "*",
"uid": 1,
"gid": 1,
"comment": "System Services",
"home": "/var/root",
"shell": "/usr/bin/false"
2020-06-25 12:56:06 -07:00
}
]
```
2020-07-27 09:29:30 -07:00
### ping
```bash
ping 8.8.8.8 -c 3 | jc --ping -p # or: jc -p ping 8.8.8.8 -c 3
```
```json
{
"destination_ip": "8.8.8.8",
"data_bytes": 56,
"pattern": null,
"destination": "8.8.8.8",
"packets_transmitted": 3,
"packets_received": 3,
"packet_loss_percent": 0.0,
"duplicates": 0,
"time_ms": 2005.0,
"round_trip_ms_min": 23.835,
"round_trip_ms_avg": 30.46,
"round_trip_ms_max": 34.838,
"round_trip_ms_stddev": 4.766,
"responses": [
{
"type": "reply",
"timestamp": null,
"bytes": 64,
"response_ip": "8.8.8.8",
"icmp_seq": 1,
"ttl": 118,
"time_ms": 23.8,
"duplicate": false
},
{
"type": "reply",
"timestamp": null,
"bytes": 64,
"response_ip": "8.8.8.8",
"icmp_seq": 2,
"ttl": 118,
"time_ms": 34.8,
"duplicate": false
},
{
"type": "reply",
"timestamp": null,
"bytes": 64,
"response_ip": "8.8.8.8",
"icmp_seq": 3,
"ttl": 118,
"time_ms": 32.7,
"duplicate": false
}
]
}
```
2019-10-18 13:34:28 -07:00
### ps
2020-06-25 21:11:21 -07:00
```bash
ps axu | jc --ps -p # or: jc -p ps axu
```
2020-06-25 12:56:06 -07:00
```json
2019-11-12 13:05:19 -08:00
[
2019-10-18 13:34:28 -07:00
{
2019-11-12 13:05:19 -08:00
"user": "root",
"pid": 1,
2019-11-12 15:05:21 -08:00
"cpu_percent": 0.0,
"mem_percent": 0.1,
"vsz": 128072,
"rss": 6784,
2019-11-12 13:05:19 -08:00
"tty": null,
"stat": "Ss",
"start": "Nov09",
2019-11-12 15:05:21 -08:00
"time": "0:08",
2019-11-12 13:05:19 -08:00
"command": "/usr/lib/systemd/systemd --switched-root --system --deserialize 22"
2019-10-25 14:58:15 -07:00
},
{
2019-11-12 13:05:19 -08:00
"user": "root",
"pid": 2,
2019-11-12 15:05:21 -08:00
"cpu_percent": 0.0,
"mem_percent": 0.0,
"vsz": 0,
"rss": 0,
2019-11-12 13:05:19 -08:00
"tty": null,
"stat": "S",
"start": "Nov09",
"time": "0:00",
"command": "[kthreadd]"
},
{
"user": "root",
"pid": 4,
2019-11-12 15:05:21 -08:00
"cpu_percent": 0.0,
"mem_percent": 0.0,
"vsz": 0,
"rss": 0,
2019-11-12 13:05:19 -08:00
"tty": null,
"stat": "S<",
"start": "Nov09",
"time": "0:00",
"command": "[kworker/0:0H]"
2020-06-25 12:56:06 -07:00
}
2019-10-18 13:34:28 -07:00
]
```
2020-07-27 09:29:30 -07:00
### traceroute
```bash
2020-07-29 14:36:20 -07:00
traceroute -m 2 8.8.8.8 | jc --traceroute -p # or: jc -p traceroute -m 2 8.8.8.8
2020-07-27 09:29:30 -07:00
```
```json
{
"destination_ip": "8.8.8.8",
"destination_name": "8.8.8.8",
"hops": [
{
"hop": 1,
"probes": [
{
"annotation": null,
"asn": null,
"ip": "192.168.1.254",
"name": "dsldevice.local.net",
"rtt": 6.616
},
{
"annotation": null,
"asn": null,
"ip": "192.168.1.254",
"name": "dsldevice.local.net",
"rtt": 6.413
},
{
"annotation": null,
"asn": null,
"ip": "192.168.1.254",
"name": "dsldevice.local.net",
"rtt": 6.308
}
]
},
{
"hop": 2,
"probes": [
{
"annotation": null,
"asn": null,
"ip": "76.220.24.1",
"name": "76-220-24-1.lightspeed.sntcca.sbcglobal.net",
"rtt": 29.367
},
{
"annotation": null,
"asn": null,
"ip": "76.220.24.1",
"name": "76-220-24-1.lightspeed.sntcca.sbcglobal.net",
"rtt": 40.197
},
{
"annotation": null,
"asn": null,
"ip": "76.220.24.1",
"name": "76-220-24-1.lightspeed.sntcca.sbcglobal.net",
"rtt": 29.162
}
]
}
]
}
```
2019-10-24 17:09:32 -07:00
### uptime
2020-06-25 21:11:21 -07:00
```bash
uptime | jc --uptime -p # or: jc -p uptime
```
2020-06-25 12:56:06 -07:00
```json
2019-10-24 17:09:32 -07:00
{
2021-03-25 11:43:30 -07:00
"time": "11:35",
"uptime": "3 days, 4:03",
"users": 5,
"load_1m": 1.88,
"load_5m": 2.0,
"load_15m": 1.94,
"time_hour": 11,
"time_minute": 35,
"time_second": null,
"uptime_days": 3,
"uptime_hours": 4,
"uptime_minutes": 3,
"uptime_total_seconds": 273780
2019-10-24 17:09:32 -07:00
}
```
2020-02-04 21:46:52 -08:00
### XML files
2020-06-25 21:11:21 -07:00
```bash
cat cd_catalog.xml
2020-02-03 22:13:06 -08:00
```
2020-06-25 21:11:21 -07:00
```xml
2020-02-03 22:13:06 -08:00
<?xml version="1.0" encoding="UTF-8"?>
<CATALOG>
<CD>
<TITLE>Empire Burlesque</TITLE>
<ARTIST>Bob Dylan</ARTIST>
<COUNTRY>USA</COUNTRY>
<COMPANY>Columbia</COMPANY>
<PRICE>10.90</PRICE>
<YEAR>1985</YEAR>
</CD>
<CD>
<TITLE>Hide your heart</TITLE>
<ARTIST>Bonnie Tyler</ARTIST>
<COUNTRY>UK</COUNTRY>
<COMPANY>CBS Records</COMPANY>
<PRICE>9.90</PRICE>
<YEAR>1988</YEAR>
</CD>
...
2020-06-25 12:56:06 -07:00
```
2020-06-25 21:11:21 -07:00
```bash
cat cd_catalog.xml | jc --xml -p
```
2020-06-25 12:56:06 -07:00
```json
2020-02-03 22:13:06 -08:00
{
"CATALOG": {
"CD": [
{
"TITLE": "Empire Burlesque",
"ARTIST": "Bob Dylan",
"COUNTRY": "USA",
"COMPANY": "Columbia",
"PRICE": "10.90",
"YEAR": "1985"
},
{
"TITLE": "Hide your heart",
"ARTIST": "Bonnie Tyler",
"COUNTRY": "UK",
"COMPANY": "CBS Records",
"PRICE": "9.90",
"YEAR": "1988"
2020-06-25 12:56:06 -07:00
}
]
}
2020-02-03 22:25:30 -08:00
}
2020-02-03 22:13:06 -08:00
```
2020-02-04 21:46:52 -08:00
### YAML files
2020-06-25 21:11:21 -07:00
```bash
cat istio.yaml
2020-02-03 21:38:08 -08:00
```
2020-06-25 21:11:21 -07:00
```yaml
2020-02-03 21:38:08 -08:00
apiVersion: "authentication.istio.io/v1alpha1"
kind: "Policy"
metadata:
name: "default"
namespace: "default"
spec:
peers:
- mtls: {}
---
apiVersion: "networking.istio.io/v1alpha3"
kind: "DestinationRule"
metadata:
name: "default"
namespace: "default"
spec:
host: "*.default.svc.cluster.local"
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
2020-06-25 12:56:06 -07:00
```
2020-06-25 21:11:21 -07:00
```bash
cat istio.yaml | jc --yaml -p
```
2020-06-25 12:56:06 -07:00
```json
2020-02-03 21:38:08 -08:00
[
{
"apiVersion": "authentication.istio.io/v1alpha1",
"kind": "Policy",
"metadata": {
"name": "default",
"namespace": "default"
},
"spec": {
"peers": [
{
"mtls": {}
}
]
}
},
{
"apiVersion": "networking.istio.io/v1alpha3",
"kind": "DestinationRule",
"metadata": {
"name": "default",
"namespace": "default"
},
"spec": {
"host": "*.default.svc.cluster.local",
"trafficPolicy": {
"tls": {
"mode": "ISTIO_MUTUAL"
}
}
}
}
]
```
© 2019-2021 Kelly Brazil