Merge pull request #29 from kellyjonbrazil/dev

Dev v1.7.5

MANIFEST.in

@@ -0,0 +1 @@
+graft tests/fixtures

README.md

@@ -1,10 +1,9 @@
 # JC
 JSON CLI output utility
 
-`jc` is used to JSONify the output of many standard linux cli tools and file types for easier parsing in scripts. See the **Parsers** section for supported commands.
+`jc` is used to JSONify the output of many standard linux cli tools and file types for easier parsing in scripts. See the [**Parsers**](#parsers) section for supported commands and file types.
 
 This allows further command line processing of output with tools like `jq` simply by piping commands:
-
 ```
 $ ls -l /usr/bin | jc --ls | jq '.[] | select(.size > 50000000)'
 {
@@ -17,9 +16,20 @@ $ ls -l /usr/bin | jc --ls | jq '.[] | select(.size > 50000000)'
   "date": "Aug 14 19:41"
 }
 ```
-For more information on the motivations for this project, please see my blog post at https://blog.kellybrazil.com/2019/11/26/bringing-the-unix-philosophy-to-the-21st-century/.
-
-The `jc` parsers can also be used as python modules. In this case the output will be a python dictionary instead of JSON:
+or using the alternative "magic" syntax:
+```
+$ jc ls -l /usr/bin | jq '.[] | select(.size > 50000000)'
+{
+  "filename": "docker",
+  "flags": "-rwxr-xr-x",
+  "links": 1,
+  "owner": "root",
+  "group": "root",
+  "size": 68677120,
+  "date": "Aug 14 19:41"
+}
+```
+The `jc` parsers can also be used as python modules. In this case the output will be a python dictionary, or list of dictionaries, instead of JSON:
 ```
 >>> import jc.parsers.ls
 >>> 
@@ -49,7 +59,9 @@ Two representations of the data are possible. The default representation uses a
 
 To access the raw, pre-processed JSON, use the `-r` cli option or the `raw=True` function parameter in `parse()`.
 
-Schemas for each parser can be found in the `docs/parsers` folder.
+Schemas for each parser can be found in the [`docs/parsers`](https://github.com/kellyjonbrazil/jc/tree/master/docs/parsers) folder. 
+
+For more information on the motivations for this project, please see my [blog post](https://blog.kellybrazil.com/2019/11/26/bringing-the-unix-philosophy-to-the-21st-century/).
 
 ## Installation
 ```
@@ -57,48 +69,56 @@ $ pip3 install --upgrade jc
 ```
 
 ## Usage
+`jc` accepts piped input from `STDIN` and outputs a JSON representation of the previous command's output to `STDOUT`.
 ``` 
-jc PARSER [OPTIONS]
+COMMAND | jc PARSER [OPTIONS]
 ```
-
-`jc` accepts piped input from `STDIN` and outputs a JSON representation of the previous command's output to `STDOUT`. The JSON output can be compact or pretty formatted.
+or
+```
+COMMAND | jc [OPTIONS] PARSER
+```
+Alternatively, the "magic" syntax can be used by prepending `jc` to the command to be converted. Options can be passed to `jc` immediately before the command is given. (Note: command aliases are not supported)
+```
+jc [OPTIONS] COMMAND
+```
+The JSON output can be compact (default) or pretty formatted with the `-p` option.
 
 ### Parsers
-- `--arp` enables the `arp` parser
-- `--crontab` enables the `crontab` file parser
+- `--arp` enables the `arp` command parser
+- `--crontab` enables the `crontab` command and file parser
 - `--crontab-u` enables the `crontab` file parser with user support
-- `--df` enables the `df` parser
-- `--dig` enables the `dig` parser
-- `--du` enables the `du` parser
-- `--env` enables the `env` parser
-- `--free` enables the `free` parser
+- `--df` enables the `df` command parser
+- `--dig` enables the `dig` command parser
+- `--du` enables the `du` command parser
+- `--env` enables the `env` command parser
+- `--free` enables the `free` command parser
 - `--fstab` enables the `/etc/fstab` file parser
-- `--history` enables the `history` parser
+- `--history` enables the `history` command parser
 - `--hosts` enables the `/etc/hosts` file parser
-- `--id` enables the `id` parser
-- `--ifconfig` enables the `ifconfig` parser
+- `--id` enables the `id` command parser
+- `--ifconfig` enables the `ifconfig` command parser
 - `--ini` enables the `INI` file parser
-- `--iptables` enables the `iptables` parser
-- `--jobs` enables the `jobs` parser
-- `--ls` enables the `ls` parser
-- `--lsblk` enables the `lsblk` parser
-- `--lsmod` enables the `lsmod` parser
-- `--lsof` enables the `lsof` parser
-- `--mount` enables the `mount` parser
-- `--netstat` enables the `netstat` parser
-- `--pip-list` enables the `pip list` parser
-- `--pip-show` enables the `pip show` parser
-- `--ps` enables the `ps` parser
-- `--route` enables the `route` parser
-- `--ss` enables the `ss` parser
-- `--stat` enables the `stat` parser
-- `--systemctl` enables the `systemctl` parser
-- `--systemctl-lj` enables the `systemctl list-jobs` parser
-- `--systemctl-ls` enables the `systemctl list-sockets` parser
-- `--systemctl-luf` enables the `systemctl list-unit-files` parser
-- `--uname` enables the `uname -a` parser
-- `--uptime` enables the `uptime` parser
-- `--w` enables the `w` parser
+- `--iptables` enables the `iptables` command parser
+- `--jobs` enables the `jobs` command parser
+- `--ls` enables the `ls` command parser
+- `--lsblk` enables the `lsblk` command parser
+- `--lsmod` enables the `lsmod` command parser
+- `--lsof` enables the `lsof` command parser
+- `--mount` enables the `mount` command parser
+- `--netstat` enables the `netstat` command parser
+- `--pip-list` enables the `pip list` command parser
+- `--pip-show` enables the `pip show` command parser
+- `--ps` enables the `ps` command parser
+- `--route` enables the `route` command parser
+- `--ss` enables the `ss` command parser
+- `--stat` enables the `stat` command parser
+- `--systemctl` enables the `systemctl` command parser
+- `--systemctl-lj` enables the `systemctl list-jobs` command parser
+- `--systemctl-ls` enables the `systemctl list-sockets` command parser
+- `--systemctl-luf` enables the `systemctl list-unit-files` command parser
+- `--uname` enables the `uname -a` command parser
+- `--uptime` enables the `uptime` command parser
+- `--w` enables the `w` command parser
 - `--xml` enables the `XML` file parser
 - `--yaml` enables the `YAML` file parser
 
@@ -109,10 +129,36 @@ jc PARSER [OPTIONS]
 - `-q` quiet mode. Suppresses warning messages
 - `-r` raw output. Provides a more literal JSON output with all values as text and no additional sematic processing
 
+## Contributions
+Feel free to add/improve code or parsers! You can use the [`jc/parsers/foo.py`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py) parser as a template and submit your parser with a pull request.
+
+## Compatibility
+Some parsers like `ls`, `ps`, `dig`, etc. will work on any platform. Other parsers that are platform-specific will generate a warning message if they are used on an unsupported platform. To see all parser information, including compatibility, run `jc -ap`.
+
+
+You may still use a parser on an unsupported platform - for example, you may want to parse a file with linux `lsof` output on an OSX laptop. In that case you can suppress the warning message with the `-q` cli option or the `quiet=True` function parameter in `parse()`:
+
+```
+$ cat lsof.out | jc --lsof -q
+```
+
+Tested on:
+- Centos 7.7
+- Ubuntu 18.4
+- OSX 10.11.6
+- OSX 10.14.6
+
+## Acknowledgments
+- `ifconfig-parser` module from https://github.com/KnightWhoSayNi/ifconfig-parser
+- `xmltodict` module from https://github.com/martinblech/xmltodict by Martín Blech
+- `ruamel.yaml` library from https://pypi.org/project/ruamel.yaml by  Anthon van der Neut
+- Parsing code from Conor Heine at https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501 adapted for some parsers
+- Excellent constructive feedback from Ilya Sher (https://github.com/ilyash-b)
+
 ## Examples
 ### arp
 ```
-$ arp | jc --arp -p
+$ arp | jc --arp -p          # or:  jc -p arp
 [
   {
     "address": "gateway",
@@ -138,7 +184,7 @@ $ arp | jc --arp -p
 ]
 ```
 ```
-$ arp -a | jc --arp -p
+$ arp -a | jc --arp -p          # or:  jc -p arp -a
 [
   {
     "name": null,
@@ -165,7 +211,7 @@ $ arp -a | jc --arp -p
 ```
 ### crontab
 ```
-$ cat /etc/crontab | jc --crontab -p
+$ cat /etc/crontab | jc --crontab -p          # or:  jc -p crontab -l
 {
   "variables": [
     {
@@ -312,7 +358,7 @@ $ cat /etc/crontab | jc --crontab-u -p
 ```
 ### df
 ```
-$ df | jc --df -p
+$ df | jc --df -p          # or:  jc -p df
 [
   {
     "filesystem": "devtmpfs",
@@ -335,7 +381,7 @@ $ df | jc --df -p
 ```
 ### dig
 ```
-$ dig cnn.com www.cnn.com @205.251.194.64 | jc --dig -p
+$ dig cnn.com www.cnn.com @205.251.194.64 | jc --dig -p          # or:  jc -p dig cnn.com www.cnn.com @205.251.194.64
 [
   {
     "id": 5509,
@@ -455,7 +501,7 @@ $ dig cnn.com www.cnn.com @205.251.194.64 | jc --dig -p
 ]
 ```
 ```
-$ dig -x 1.1.1.1 | jc --dig -p
+$ dig -x 1.1.1.1 | jc --dig -p          # or:  jc -p dig -x 1.1.1.1
 [
   {
     "id": 50324,
@@ -493,7 +539,7 @@ $ dig -x 1.1.1.1 | jc --dig -p
 ```
 ### du
 ```
-$ du /usr | jc --du -p
+$ du /usr | jc --du -p          # or:  jc -p du /usr
 [
   {
     "size": 104608,
@@ -524,7 +570,7 @@ $ du /usr | jc --du -p
 ```
 ### env
 ```
-$ env | jc --env -p
+$ env | jc --env -p          # or:  jc -p env
 [
   {
     "name": "XDG_SESSION_ID",
@@ -551,7 +597,7 @@ $ env | jc --env -p
 ```
 ### free
 ```
-$ free | jc --free -p
+$ free | jc --free -p          # or:  jc -p free
 [
   {
     "type": "Mem",
@@ -674,7 +720,7 @@ $ cat /etc/hosts | jc --hosts -p
 ```
 ### id
 ```
-$ id | jc --id -p
+$ id | jc --id -p          # or:  jc -p id
 {
   "uid": {
     "id": 1000,
@@ -704,7 +750,7 @@ $ id | jc --id -p
 ```
 ### ifconfig
 ```
-$ ifconfig | jc --ifconfig -p
+$ ifconfig | jc --ifconfig -p          # or:  jc -p ifconfig
 [
   {
     "name": "ens33",
@@ -809,7 +855,7 @@ $ cat example.ini | jc --ini -p
 ```
 ### iptables
 ```
-$ sudo iptables --line-numbers -v -L -t nat | jc --iptables -p
+$ sudo iptables --line-numbers -v -L -t nat | jc --iptables -p          # or:  sudo jc -p iptables --line-numbers -v -L -t nat
 [
   {
     "chain": "PREROUTING",
@@ -870,7 +916,7 @@ $ sudo iptables --line-numbers -v -L -t nat | jc --iptables -p
 ```
 ### jobs
 ```
-$ jobs -l | jc --jobs -p
+$ jobs -l | jc --jobs -p          # or:  jc -p jobs
 [
   {
     "job_number": 1,
@@ -902,7 +948,7 @@ $ jobs -l | jc --jobs -p
 ```
 ### ls
 ```
-$ ls -l /usr/bin | jc --ls -p
+$ ls -l /usr/bin | jc --ls -p          # or:  jc -p ls -l /usr/bin
 [
   {
     "filename": "apropos",
@@ -937,7 +983,7 @@ $ ls -l /usr/bin | jc --ls -p
 ```
 ### lsblk
 ```
-$ lsblk | jc --lsblk -p
+$ lsblk | jc --lsblk -p          # or:  jc -p lsblk
 [
   {
     "name": "sda",
@@ -962,7 +1008,7 @@ $ lsblk | jc --lsblk -p
 ```
 ### lsmod
 ```
-$ lsmod | jc --lsmod -p
+$ lsmod | jc --lsmod -p          # or:  jc -p lsmod
 [
   ...
   {
@@ -1009,7 +1055,7 @@ $ lsmod | jc --lsmod -p
 ```
 ### lsof
 ```
-$ sudo lsof | jc --lsof -p
+$ sudo lsof | jc --lsof -p          # or:  sudo jc -p lsof
 [
   {
     "command": "systemd",
@@ -1052,7 +1098,7 @@ $ sudo lsof | jc --lsof -p
 ```
 ### mount
 ```
-$ mount | jc --mount -p
+$ mount | jc --mount -p          # or:  jc -p mount
 [
   {
     "filesystem": "sysfs",
@@ -1096,7 +1142,7 @@ $ mount | jc --mount -p
 ```
 ### netstat
 ```
-$ sudo netstat -apee | jc --netstat -p
+$ sudo netstat -apee | jc --netstat -p          # or:  sudo jc -p netstat -apee
 [
   {
     "proto": "tcp",
@@ -1248,7 +1294,7 @@ $ sudo netstat -apee | jc --netstat -p
 ```
 ### pip list
 ```
-$ pip list | jc --pip-list -p
+$ pip list | jc --pip-list -p          # or:  jc -p pip list          # or:  jc -p pip3 list
 [
   {
     "package": "ansible",
@@ -1268,7 +1314,7 @@ $ pip list | jc --pip-list -p
 ```
 ### pip show
 ```
-$ pip show wrapt wheel | jc --pip-show -p
+$ pip show wrapt wheel | jc --pip-show -p          # or:  jc -p pip show wrapt wheel          # or:  jc -p pip3 show wrapt wheel
 [
   {
     "name": "wrapt",
@@ -1298,7 +1344,7 @@ $ pip show wrapt wheel | jc --pip-show -p
 ```
 ### ps
 ```
-$ ps -ef | jc --ps -p
+$ ps -ef | jc --ps -p          # or:  jc -p ps -ef
 [
   {
     "uid": "root",
@@ -1334,7 +1380,7 @@ $ ps -ef | jc --ps -p
 ]
 ```
 ```
-$ ps axu | jc --ps -p
+$ ps axu | jc --ps -p          # or:  jc -p ps axu
 [
   {
     "user": "root",
@@ -1380,7 +1426,7 @@ $ ps axu | jc --ps -p
 ```
 ### route
 ```
-$ route -ee | jc --route -p
+$ route -ee | jc --route -p          # or:  jc -p route -ee
 [
   {
     "destination": "default",
@@ -1425,7 +1471,7 @@ $ route -ee | jc --route -p
 ```
 ### ss
 ```
-$ sudo ss -a | jc --ss -p
+$ sudo ss -a | jc --ss -p          # or:  sudo jc -p ss -a
 [
   {
     "netid": "nl",
@@ -1544,7 +1590,7 @@ $ sudo ss -a | jc --ss -p
 ```
 ### stat
 ```
-$ stat /bin/* | jc --stat -p
+$ stat /bin/* | jc --stat -p          # or:  jc -p stat /bin/*
 [
   {
     "file": "/bin/bash",
@@ -1591,7 +1637,7 @@ $ stat /bin/* | jc --stat -p
 ```
 ### systemctl
 ```
-$ systemctl -a | jc --systemctl -p
+$ systemctl -a | jc --systemctl -p          # or:  jc -p systemctl -a
 [
   {
     "unit": "proc-sys-fs-binfmt_misc.automount",
@@ -1619,7 +1665,7 @@ $ systemctl -a | jc --systemctl -p
 ```
 ### systemctl list-jobs
 ```
-$ systemctl list-jobs| jc --systemctl-lj -p
+$ systemctl list-jobs | jc --systemctl-lj -p          # or:  jc -p systemctl list-jobs
 [
   {
     "job": 3543,
@@ -1643,7 +1689,7 @@ $ systemctl list-jobs| jc --systemctl-lj -p
 ```
 ### systemctl list-sockets
 ```
-$ systemctl list-sockets | jc --systemctl-ls -p
+$ systemctl list-sockets | jc --systemctl-ls -p          # or:  jc -p systemctl list-sockets
 [
   {
     "listen": "/dev/log",
@@ -1665,7 +1711,7 @@ $ systemctl list-sockets | jc --systemctl-ls -p
 ```
 ### systemctl list-unit-files
 ```
-$ systemctl list-unit-files | jc --systemctl-luf -p
+$ systemctl list-unit-files | jc --systemctl-luf -p          # or:  jc -p systemctl list-unit-files
 [
   {
     "unit_file": "proc-sys-fs-binfmt_misc.automount",
@@ -1684,7 +1730,7 @@ $ systemctl list-unit-files | jc --systemctl-luf -p
 ```
 ### uname -a
 ```
-$ uname -a | jc --uname -p
+$ uname -a | jc --uname -p          # or:  jc -p uname -a
 {
   "kernel_name": "Linux",
   "node_name": "user-ubuntu",
@@ -1698,7 +1744,7 @@ $ uname -a | jc --uname -p
 ```
 ### uptime
 ```
-$ uptime | jc --uptime -p
+$ uptime | jc --uptime -p          # or:  jc -p uptime
 {
   "time": "11:30:44",
   "uptime": "1 day, 21:17",
@@ -1710,7 +1756,7 @@ $ uptime | jc --uptime -p
 ```
 ### w
 ```
-$ w | jc --w -p
+$ w | jc --w -p          # or:  jc -p w
 [
   {
     "user": "root",
@@ -1792,7 +1838,7 @@ $ cat cd_catalog.xml | jc --xml -p
 ```
 ### YAML files
 ```
-$ cat istio-mtls-permissive.yaml 
+$ cat istio.yaml 
 apiVersion: "authentication.istio.io/v1alpha1"
 kind: "Policy"
 metadata:
@@ -1813,7 +1859,7 @@ spec:
     tls:
       mode: ISTIO_MUTUAL
 
-$ cat istio-mtls-permissive.yaml | jc --yaml -p
+$ cat istio.yaml | jc --yaml -p
 [
   {
     "apiVersion": "authentication.istio.io/v1alpha1",
@@ -1847,34 +1893,4 @@ $ cat istio-mtls-permissive.yaml | jc --yaml -p
     }
   }
 ]
-```
-## TODO
-Future parsers:
-- /proc files
-- /sys files
-
-## Contributions
-Feel free to add/improve code or parsers! You can use the `jc/parsers/foo.py` parser as a template and submit your parser with a pull request.
-
-## Compatibility
-Some parsers like `ls`, `ps`, `dig`, etc. will work on any platform. Other parsers that are platform-specific will generate a warning message if they are used on an unsupported platform. To see all parser information, including compatibility, run `jc -a -p`.
-
-
-You may still use a parser on an unsupported platform - for example, you may want to parse a file with linux `lsof` output on an OSX laptop. In that case you can suppress the warning message with the `-q` cli option or the `quiet=True` function parameter in `parse()`:
-
-```
-$ cat lsof.out | jc --lsof -q
-```
-
-Tested on:
-- Centos 7.7
-- Ubuntu 18.4
-- OSX 10.11.6
-- OSX 10.14.6
-
-## Acknowledgments
-- `ifconfig-parser` module from https://github.com/KnightWhoSayNi/ifconfig-parser
-- `xmltodict` module from https://github.com/martinblech/xmltodict by Martín Blech
-- `ruamel.yaml` library from https://pypi.org/project/ruamel.yaml by  Anthon van der Neut
-- Parsing code from Conor Heine at https://gist.github.com/cahna/43a1a3ff4d075bcd71f9d7120037a501 adapted for some parsers
-- Excellent constructive feedback from Ilya Sher (https://github.com/ilyash-b)
+```

changelog.txt

@@ -1,5 +1,18 @@
 jc changelog
 
+20200227 v1.7.5
+- Updated ls parser to support filenames with newline characters
+
+20200219 v1.7.4
+- Updated ls parser to support multiple directories, globbing, and -R (recursive)
+
+20200211 v1.7.3
+- Add alternative 'magic' syntax: e.g. `jc ls -al`
+- Options can now be condensed (e.g. -prq is equivalant to -p -r -q)
+
+20200208 v1.7.2
+- Include test fixtures in wheel and sdist
+
 20200205 v1.7.1
 - Add YAML file parser
 - Add INI file parser

docs/parsers/crontab.md

@@ -1,9 +1,9 @@
 # jc.parsers.crontab
-jc - JSON CLI output utility crontab file Parser
+jc - JSON CLI output utility crontab command and file Parser
 
 Usage:
 
-    specify --crontab as the first argument if the piped input is coming from a crontab file
+    specify --crontab as the first argument if the piped input is coming from crontab -l or a crontab file
 
 Compatibility:
 
@@ -11,7 +11,7 @@ Compatibility:
 
 Examples:
 
-    $ cat /etc/crontab | jc --crontab -p
+    $ crontab -l | jc --crontab -p
     {
       "variables": [
         {

docs/parsers/crontab_u.md

@@ -11,7 +11,7 @@ Compatibility:
 
 Examples:
 
-    $ cat /etc/crontab | jc --crontab -p
+    $ cat /etc/crontab | jc --crontab-u -p
     {
       "variables": [
         {

docs/parsers/ls.md

@@ -1,13 +1,17 @@
 # jc.parsers.ls
 jc - JSON CLI output utility ls Parser
 
+Note: The -l option of ls should be used to correctly parse filenames that include newline characters.
+      Since ls does not encode newlines in filenames when outputting to a pipe it will cause jc to see
+      multiple files instead of a single file if -l is not used.
+
 Usage:
 
     specify --ls as the first argument if the piped input is coming from ls
 
     ls options supported:
-    - None
-    - la
+    - laR
+    --time-style=full-iso
     - h       file sizes will be available in text form with -r but larger file sizes
               with human readable suffixes will be converted to Null in default view
               since the parser attempts to convert this field to an integer.
@@ -165,6 +169,7 @@ Returns:
         "filename": string,
         "flags":    string,
         "links":    integer,
+        "parent":   string,
         "owner":    string,
         "group":    string,
         "size":     integer,

jc/cli.py

@@ -3,12 +3,24 @@
 JC cli module
 """
 import sys
+import os
+import shlex
 import importlib
 import textwrap
 import signal
 import json
 import jc.utils
 
+
+class info():
+    version = '1.7.5'
+    description = 'jc cli output JSON conversion tool'
+    author = 'Kelly Brazil'
+    author_email = 'kellyjonbrazil@gmail.com'
+
+
+__version__ = info.version
+
 parsers = [
     'arp',
     'crontab',
@@ -50,6 +62,11 @@ parsers = [
 ]
 
 
+def ctrlc(signum, frame):
+    """exit with error on SIGINT"""
+    sys.exit(1)
+
+
 def parser_shortname(parser_argument):
     """short name of the parser with dashes and no -- prefix"""
     return parser_argument[2:]
@@ -66,26 +83,13 @@ def parser_mod_shortname(parser):
 
 
 def parser_module(parser):
-    """import the module just in time and present the module object"""
+    """import the module just in time and return the module object"""
     importlib.import_module('jc.parsers.' + parser_mod_shortname(parser))
     return getattr(jc.parsers, parser_mod_shortname(parser))
 
 
-class info():
-    version = '1.7.1'
-    description = 'jc cli output JSON conversion tool'
-    author = 'Kelly Brazil'
-    author_email = 'kellyjonbrazil@gmail.com'
-
-
-__version__ = info.version
-
-
-def ctrlc(signum, frame):
-    sys.exit(1)
-
-
 def parsers_text(indent=0, pad=0):
+    """return the argument and description information from each parser"""
     ptext = ''
     for parser in parsers:
         parser_arg = parser_argument(parser)
@@ -103,6 +107,7 @@ def parsers_text(indent=0, pad=0):
 
 
 def about_jc():
+    """return jc info and the contents of each parser.info as a dictionary"""
     parser_list = []
 
     for parser in parsers:
@@ -132,12 +137,21 @@ def about_jc():
 
 
 def helptext(message):
+    """return the help text with the list of parsers"""
     parsers_string = parsers_text(indent=12, pad=17)
 
     helptext_string = f'''
     jc:     {message}
 
-    Usage:  jc PARSER [OPTIONS]
+    Usage:  COMMAND | jc PARSER [OPTIONS]
+
+            or
+
+            COMMAND | jc [OPTIONS] PARSER
+
+            or magic syntax:
+
+            jc [OPTIONS] COMMAND
 
     Parsers:
 {parsers_string}
@@ -150,6 +164,10 @@ def helptext(message):
 
     Example:
             ls -al | jc --ls -p
+
+            or using the magic syntax:
+
+            jc -p ls -al
     '''
     print(textwrap.dedent(helptext_string), file=sys.stderr)
 
@@ -161,28 +179,102 @@ def json_out(data, pretty=False):
         print(json.dumps(data))
 
 
+def magic():
+    """Parse with magic syntax: jc -p ls -al"""
+    if len(sys.argv) > 1 and not sys.argv[1].startswith('--'):
+        parser_info = about_jc()['parsers']
+        # correctly parse escape characters and spaces with shlex
+        args_given = " ".join(map(shlex.quote, sys.argv[1:])).split()
+        options = []
+        found_parser = None
+
+        # find the options
+        if args_given[0].startswith('-'):
+            p = 0
+            for i, arg in list(enumerate(args_given)):
+                # parser found - use standard syntax
+                if arg.startswith('--'):
+                    return
+                # option found - populate option list
+                elif arg.startswith('-'):
+                    options.append(args_given.pop(i - p)[1:])
+                    p = p + 1
+                # command found if iterator didn't already stop - stop iterating
+                else:
+                    break
+
+        # find the command and parser
+        for parser in parser_info:
+            if 'magic_commands' in parser:
+                # first pass for two word commands: e.g. 'pip list'
+                for magic_command in parser['magic_commands']:
+                    try:
+                        if ' '.join(args_given[0:2]) == magic_command:
+                            found_parser = parser['argument']
+                            break
+                    # No command found - go to next loop (for cases like 'jc -a')
+                    except Exception:
+                        break
+
+                # second pass for one word commands: e.g. 'ls'
+                if not found_parser:
+                    for magic_command in parser['magic_commands']:
+                        try:
+                            if args_given[0] == magic_command:
+                                found_parser = parser['argument']
+                                break
+                        # No command found - use standard syntax (for cases like 'jc -a')
+                        except Exception:
+                            return
+
+        # construct a new command line using the standard syntax: COMMAND | jc --PARSER -OPTIONS
+        run_command = ' '.join(args_given)
+        if found_parser:
+            if options:
+                cmd_options = '-' + ''.join(options)
+            else:
+                cmd_options = ''
+            whole_command = ' '.join([run_command, '|', 'jc', found_parser, cmd_options])
+
+            os.system(whole_command)
+            exit()
+        else:
+            helptext(f'parser not found for "{run_command}"')
+            sys.exit(1)
+
+
 def main():
+    # break on ctrl-c keyboard interrupt
     signal.signal(signal.SIGINT, ctrlc)
 
+    # try magic syntax first: e.g. jc -p ls -al
+    magic()
+
+    options = []
     debug = False
     pretty = False
     quiet = False
     raw = False
 
     # options
-    if '-d' in sys.argv:
+    for opt in sys.argv:
+        if opt.startswith('-') and not opt.startswith('--'):
+            for flag in opt[1:]:
+                options.append(flag)
+
+    if 'd' in options:
         debug = True
 
-    if '-p' in sys.argv:
+    if 'p' in options:
         pretty = True
 
-    if '-q' in sys.argv:
+    if 'q' in options:
         quiet = True
 
-    if '-r' in sys.argv:
+    if 'r' in options:
         raw = True
 
-    if '-a' in sys.argv:
+    if 'a' in options:
         json_out(about_jc(), pretty=pretty)
         exit()
 
@@ -215,7 +307,7 @@ def main():
                     result = parser.parse(data, raw=raw, quiet=quiet)
                     found = True
                     break
-                except:
+                except Exception:
                     jc.utils.error_message(f'{parser_name} parser could not parse the input data. Did you use the correct parser?\n         For details use the -d option.')
                     sys.exit(1)
 

jc/parsers/arp.py

@@ -92,12 +92,13 @@ import jc.parsers.universal
 
 class info():
     version = '1.1'
-    description = 'arp parser'
+    description = 'arp command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'aix', 'freebsd', 'darwin']
+    magic_commands = ['arp']
 
 
 __version__ = info.version

jc/parsers/crontab.py

@@ -1,8 +1,8 @@
-"""jc - JSON CLI output utility crontab file Parser
+"""jc - JSON CLI output utility crontab command and file Parser
 
 Usage:
 
-    specify --crontab as the first argument if the piped input is coming from a crontab file
+    specify --crontab as the first argument if the piped input is coming from crontab -l or a crontab file
 
 Compatibility:
 
@@ -10,7 +10,7 @@ Compatibility:
 
 Examples:
 
-    $ cat /etc/crontab | jc --crontab -p
+    $ crontab -l | jc --crontab -p
     {
       "variables": [
         {
@@ -133,13 +133,14 @@ import jc.parsers.universal
 
 class info():
     version = '1.1'
-    description = 'crontab file parser'
+    description = 'crontab command and file parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
     # details = 'enter any other details here'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin', 'aix', 'freebsd']
+    magic_commands = ['crontab']
 
 
 __version__ = info.version

jc/parsers/crontab_u.py

@@ -10,7 +10,7 @@ Compatibility:
 
 Examples:
 
-    $ cat /etc/crontab | jc --crontab -p
+    $ cat /etc/crontab | jc --crontab-u -p
     {
       "variables": [
         {

jc/parsers/df.py

@@ -74,12 +74,13 @@ import jc.parsers.universal
 
 class info():
     version = '1.1'
-    description = 'df parser'
+    description = 'df command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin']
+    magic_commands = ['df']
 
 
 __version__ = info.version

jc/parsers/dig.py

@@ -325,12 +325,13 @@ import jc.utils
 
 class info():
     version = '1.0'
-    description = 'dig parser'
+    description = 'dig command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'aix', 'freebsd', 'darwin']
+    magic_commands = ['dig']
 
 
 __version__ = info.version

jc/parsers/du.py

@@ -74,13 +74,14 @@ import jc.parsers.universal
 
 class info():
     version = '1.1'
-    description = 'du parser'
+    description = 'du command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
     # details = 'enter any other details here'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin', 'aix', 'freebsd']
+    magic_commands = ['du']
 
 
 __version__ = info.version

jc/parsers/env.py

@@ -53,12 +53,13 @@ import jc.utils
 
 class info():
     version = '1.1'
-    description = 'env parser'
+    description = 'env command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin', 'cygwin', 'win32', 'aix', 'freebsd']
+    magic_commands = ['env']
 
 
 __version__ = info.version

jc/parsers/foo.py

@@ -21,13 +21,14 @@ import jc.utils
 
 class info():
     version = '1.0'
-    description = 'foo parser'
+    description = 'foo command parser'
     author = 'John Doe'
     author_email = 'johndoe@gmail.com'
     # details = 'enter any other details here'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin', 'cygwin', 'win32', 'aix', 'freebsd']
+    magic_commands = ['foo']
 
 
 __version__ = info.version

jc/parsers/free.py

@@ -54,12 +54,13 @@ import jc.parsers.universal
 
 class info():
     version = '1.0'
-    description = 'free parser'
+    description = 'free command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux']
+    magic_commands = ['free']
 
 
 __version__ = info.version

jc/parsers/fstab.py

@@ -71,7 +71,7 @@ import jc.utils
 
 class info():
     version = '1.0'
-    description = '/etc/fstab file parser'
+    description = 'fstab file parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 

jc/parsers/history.py

@@ -45,7 +45,7 @@ import jc
 
 class info():
     version = '1.1'
-    description = 'history parser'
+    description = 'history command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 

jc/parsers/id.py

@@ -71,13 +71,14 @@ import jc.utils
 
 class info():
     version = '1.0'
-    description = 'id parser'
+    description = 'id command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
     # details = 'enter any other details here'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin', 'aix', 'freebsd']
+    magic_commands = ['id']
 
 
 __version__ = info.version

jc/parsers/ifconfig.py

@@ -147,13 +147,14 @@ from ifconfigparser import IfconfigParser
 
 class info():
     version = '1.5'
-    description = 'ifconfig parser'
+    description = 'ifconfig command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
     details = 'Using ifconfig-parser package from https://github.com/KnightWhoSayNi/ifconfig-parser'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'aix', 'freebsd', 'darwin']
+    magic_commands = ['ifconfig']
 
 
 __version__ = info.version

jc/parsers/iptables.py

@@ -135,12 +135,13 @@ import jc.utils
 
 class info():
     version = '1.1'
-    description = 'iptables parser'
+    description = 'iptables command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux']
+    magic_commands = ['iptables']
 
 
 __version__ = info.version

jc/parsers/jobs.py

@@ -78,12 +78,13 @@ import jc.utils
 
 class info():
     version = '1.0'
-    description = 'jobs parser'
+    description = 'jobs command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin', 'cygwin', 'aix', 'freebsd']
+    magic_commands = ['jobs']
 
 
 __version__ = info.version

jc/parsers/ls.py

@@ -1,12 +1,16 @@
 """jc - JSON CLI output utility ls Parser
 
+Note: The -l option of ls should be used to correctly parse filenames that include newline characters.
+      Since ls does not encode newlines in filenames when outputting to a pipe it will cause jc to see
+      multiple files instead of a single file if -l is not used.
+
 Usage:
 
     specify --ls as the first argument if the piped input is coming from ls
 
     ls options supported:
-    - None
-    - la
+    - laR
+    --time-style=full-iso
     - h       file sizes will be available in text form with -r but larger file sizes
               with human readable suffixes will be converted to Null in default view
               since the parser attempts to convert this field to an integer.
@@ -144,13 +148,14 @@ import jc.utils
 
 
 class info():
-    version = '1.0'
-    description = 'ls parser'
+    version = '1.2'
+    description = 'ls command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin', 'cygwin', 'aix', 'freebsd']
+    magic_commands = ['ls']
 
 
 __version__ = info.version
@@ -173,6 +178,7 @@ def process(proc_data):
             "filename": string,
             "flags":    string,
             "links":    integer,
+            "parent":   string,
             "owner":    string,
             "group":    string,
             "size":     integer,
@@ -212,27 +218,58 @@ def parse(data, raw=False, quiet=False):
         jc.utils.compatibility(__name__, info.compatible)
 
     raw_output = []
+    warned = False
+    parent = ''
+    next_is_parent = False
+    new_section = False
 
     linedata = data.splitlines()
 
-    # Delete first line if it starts with 'total'
+    # Delete first line if it starts with 'total 1234'
     if linedata:
-        if linedata[0].find('total') == 0:
+        if re.match('^total [0-9]+', linedata[0]):
             linedata.pop(0)
 
-    # Clear any blank lines
-    cleandata = list(filter(None, linedata))
+    # Look for parent line if glob or -R is used
+    if not re.match('^[-dclpsbDCMnP?]([-r][-w][-xsS]){2}([-r][-w][-xtT])[+]?', linedata[0]) \
+       and linedata[0].endswith(':'):
+        parent = linedata.pop(0)[:-1]
+        # Pop following total line
+        linedata.pop(0)
 
-    if cleandata:
+    if linedata:
         # Check if -l was used to parse extra data
-        if re.match('^[-dclpsbDCMnP?]([-r][-w][-xsS]){2}([-r][-w][-xtT])[+]?', cleandata[0]):
-            for entry in cleandata:
+        if re.match('^[-dclpsbDCMnP?]([-r][-w][-xsS]){2}([-r][-w][-xtT])[+]?', linedata[0]):
+            for entry in linedata:
                 output_line = {}
 
                 parsed_line = entry.split(maxsplit=8)
 
+                if not re.match('^[-dclpsbDCMnP?]([-r][-w][-xsS]){2}([-r][-w][-xtT])[+]?', entry) \
+                   and entry.endswith(':'):
+                    parent = entry[:-1]
+                    new_section = True
+
+                    # fixup to remove trailing \n in previous entry
+                    raw_output[-1]['filename'] = raw_output[-1]['filename'][:-1]
+                    continue
+
+                if re.match('^total [0-9]+', entry):
+                    new_section = False
+                    continue
+
+                # fixup for filenames with newlines
+                if not new_section \
+                   and not re.match('^[-dclpsbDCMnP?]([-r][-w][-xsS]){2}([-r][-w][-xtT])[+]?', entry):
+                    raw_output[-1]['filename'] = raw_output[-1]['filename'] + '\n' + entry
+                    continue
+
                 # split filenames and links
-                filename_field = parsed_line[8].split(' -> ')
+                if len(parsed_line) == 9:
+                    filename_field = parsed_line[8].split(' -> ')
+                else:
+                    # in case of filenames starting with a newline character
+                    filename_field = ['']
 
                 # create list of dictionaries
                 output_line['filename'] = filename_field[0]
@@ -240,6 +277,9 @@ def parse(data, raw=False, quiet=False):
                 if len(filename_field) > 1:
                     output_line['link_to'] = filename_field[1]
 
+                if parent:
+                    output_line['parent'] = parent
+
                 output_line['flags'] = parsed_line[0]
                 output_line['links'] = parsed_line[1]
                 output_line['owner'] = parsed_line[2]
@@ -248,9 +288,27 @@ def parse(data, raw=False, quiet=False):
                 output_line['date'] = ' '.join(parsed_line[5:8])
                 raw_output.append(output_line)
         else:
-            for entry in cleandata:
+            for entry in linedata:
                 output_line = {}
+
+                if entry == '':
+                    next_is_parent = True
+                    continue
+
+                if next_is_parent and entry.endswith(':'):
+                    parent = entry[:-1]
+                    next_is_parent = False
+                    continue
+
+                if not quiet and next_is_parent and not entry.endswith(':') and not warned:
+                    jc.utils.warning_message('Newline characters detected. Filenames probably corrupted. Use ls -l instead.')
+                    warned = True
+
                 output_line['filename'] = entry
+
+                if parent:
+                    output_line['parent'] = parent
+
                 raw_output.append(output_line)
 
     if raw:

jc/parsers/lsblk.py

@@ -217,12 +217,13 @@ import jc.parsers.universal
 
 class info():
     version = '1.3'
-    description = 'lsblk parser'
+    description = 'lsblk command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux']
+    magic_commands = ['lsblk']
 
 
 __version__ = info.version

jc/parsers/lsmod.py

@@ -108,12 +108,13 @@ import jc.parsers.universal
 
 class info():
     version = '1.1'
-    description = 'lsmod parser'
+    description = 'lsmod command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux']
+    magic_commands = ['lsmod']
 
 
 __version__ = info.version

jc/parsers/lsof.py

@@ -98,12 +98,13 @@ import jc.parsers.universal
 
 class info():
     version = '1.0'
-    description = 'lsof parser'
+    description = 'lsof command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux']
+    magic_commands = ['lsof']
 
 
 __version__ = info.version

jc/parsers/mount.py

@@ -57,12 +57,13 @@ import jc.utils
 
 class info():
     version = '1.1'
-    description = 'mount parser'
+    description = 'mount command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin']
+    magic_commands = ['mount']
 
 
 __version__ = info.version

jc/parsers/netstat.py

@@ -314,12 +314,13 @@ import jc.utils
 
 class info():
     version = '1.2'
-    description = 'netstat parser'
+    description = 'netstat command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux']
+    magic_commands = ['netstat']
 
 
 __version__ = info.version

jc/parsers/pip_list.py

@@ -33,12 +33,13 @@ import jc.parsers.universal
 
 class info():
     version = '1.0'
-    description = 'pip list parser'
+    description = 'pip list command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin', 'cygwin', 'win32', 'aix', 'freebsd']
+    magic_commands = ['pip list', 'pip3 list']
 
 
 __version__ = info.version

jc/parsers/pip_show.py

@@ -43,12 +43,13 @@ import jc.utils
 
 class info():
     version = '1.0'
-    description = 'pip show parser'
+    description = 'pip show command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin', 'cygwin', 'win32', 'aix', 'freebsd']
+    magic_commands = ['pip show', 'pip3 show']
 
 
 __version__ = info.version

jc/parsers/ps.py

@@ -178,12 +178,13 @@ import jc.parsers.universal
 
 class info():
     version = '1.1'
-    description = 'ps parser'
+    description = 'ps command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin', 'cygwin', 'aix', 'freebsd']
+    magic_commands = ['ps']
 
 
 __version__ = info.version

jc/parsers/route.py

@@ -102,12 +102,13 @@ import jc.parsers.universal
 
 class info():
     version = '1.0'
-    description = 'route parser'
+    description = 'route command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux']
+    magic_commands = ['route']
 
 
 __version__ = info.version

jc/parsers/ss.py

@@ -252,12 +252,13 @@ import jc.utils
 
 class info():
     version = '1.0'
-    description = 'ss parser'
+    description = 'ss command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux']
+    magic_commands = ['ss']
 
 
 __version__ = info.version

jc/parsers/stat.py

@@ -105,12 +105,13 @@ import jc.utils
 
 class info():
     version = '1.0'
-    description = 'stat parser'
+    description = 'stat command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux']
+    magic_commands = ['stat']
 
 
 __version__ = info.version

jc/parsers/systemctl.py

@@ -41,12 +41,13 @@ import jc.utils
 
 class info():
     version = '1.0'
-    description = 'systemctl parser'
+    description = 'systemctl command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux']
+    magic_commands = ['systemctl']
 
 
 __version__ = info.version

jc/parsers/systemctl_lj.py

@@ -60,12 +60,13 @@ import jc.utils
 
 class info():
     version = '1.0'
-    description = 'systemctl list-jobs parser'
+    description = 'systemctl list-jobs command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux']
+    magic_commands = ['systemctl list-jobs']
 
 
 __version__ = info.version

jc/parsers/systemctl_ls.py

@@ -35,12 +35,13 @@ import jc.utils
 
 class info():
     version = '1.0'
-    description = 'systemctl list-sockets parser'
+    description = 'systemctl list-sockets command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux']
+    magic_commands = ['systemctl list-sockets']
 
 
 __version__ = info.version

jc/parsers/systemctl_luf.py

@@ -32,12 +32,13 @@ import jc.utils
 
 class info():
     version = '1.0'
-    description = 'systemctl list-unit-files parser'
+    description = 'systemctl list-unit-files command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux']
+    magic_commands = ['systemctl list-unit-files']
 
 
 __version__ = info.version

jc/parsers/uname.py

@@ -31,12 +31,13 @@ import jc.utils
 
 class info():
     version = '1.1'
-    description = 'uname -a parser'
+    description = 'uname -a command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin']
+    magic_commands = ['uname']
 
 
 __version__ = info.version

jc/parsers/uptime.py

@@ -35,12 +35,13 @@ import jc.utils
 
 class info():
     version = '1.0'
-    description = 'uptime parser'
+    description = 'uptime command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin', 'cygwin', 'aix', 'freebsd']
+    magic_commands = ['uptime']
 
 
 __version__ = info.version

jc/parsers/w.py

@@ -84,12 +84,13 @@ import jc.utils
 
 class info():
     version = '1.0'
-    description = 'w parser'
+    description = 'w command parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
 
     # compatible options: linux, darwin, cygwin, win32, aix, freebsd
     compatible = ['linux', 'darwin', 'cygwin', 'aix', 'freebsd']
+    magic_commands = ['w']
 
 
 __version__ = info.version

setup.py

@@ -5,7 +5,7 @@ with open('README.md', 'r') as f:
 
 setuptools.setup(
     name='jc',
-    version='1.7.1',
+    version='1.7.5',
     author='Kelly Brazil',
     author_email='kellyjonbrazil@gmail.com',
     description='This tool serializes the output of popular command line tools and filetypes to structured JSON output.',
@@ -20,6 +20,7 @@ setuptools.setup(
     python_requires='>=3.6',
     url='https://github.com/kellyjonbrazil/jc',
     packages=setuptools.find_packages(),
+    include_package_data=True,
     entry_points={
         'console_scripts': [
             'jc=jc.cli:main'

tests/fixtures/centos-7.7/ls-R-newlines.json

@@ -0,0 +1 @@
+[{"filename": "systemd-private-016de60725a3426792b93fc9f120b8f0-chronyd.service-oZqq4u", "parent": "."}, {"filename": "systemd-private-a30a5a178daa4042b42dfaf5ff9e5f68-chronyd.service-a1tpxv", "parent": "."}, {"filename": "systemd-private-af69d7360f3e40cfa947358c0fb5a6f8-chronyd.service-T3MQ4j", "parent": "."}, {"filename": "tmp.CvALl2jE6u", "parent": "."}, {"filename": "tmp.e7AlxSxY5a", "parent": "."}, {"filename": "tmp.uXm9yegjwj", "parent": "."}, {"filename": "a regular filename", "parent": "./lstest"}, {"filename": "this file has", "parent": "./lstest"}, {"filename": "a combination", "parent": "./lstest"}, {"filename": "of everything", "parent": "./lstest"}, {"filename": "this file has", "parent": "./lstest"}, {"filename": "a newline inside", "parent": "./lstest"}, {"filename": "this file has", "parent": "./lstest"}, {"filename": "four contiguous newlines inside", "parent": "./lstest"}, {"filename": "this file", "parent": "./lstest"}, {"filename": "has", "parent": "./lstest"}, {"filename": "six", "parent": "./lstest"}, {"filename": "newlines", "parent": "./lstest"}, {"filename": "within", "parent": "./lstest"}, {"filename": "this file starts with four newlines", "parent": "./lstest"}, {"filename": "this file starts with one newline", "parent": "./lstest"}]

tests/fixtures/centos-7.7/ls-R-newlines.out

@@ -0,0 +1,43 @@
+.:
+lstest
+systemd-private-016de60725a3426792b93fc9f120b8f0-chronyd.service-oZqq4u
+systemd-private-a30a5a178daa4042b42dfaf5ff9e5f68-chronyd.service-a1tpxv
+systemd-private-af69d7360f3e40cfa947358c0fb5a6f8-chronyd.service-T3MQ4j
+tmp.CvALl2jE6u
+tmp.e7AlxSxY5a
+tmp.uXm9yegjwj
+
+./lstest:
+a regular filename
+
+
+this file has
+a combination
+
+
+of everything
+
+
+
+
+this file has
+a newline inside
+this file has
+
+
+
+four contiguous newlines inside
+this file
+has
+six
+
+newlines
+
+within
+
+
+
+
+this file starts with four newlines
+
+this file starts with one newline

tests/fixtures/centos-7.7/ls-l-newlines.json

@@ -0,0 +1 @@
+[{"filename": "a regular filename", "flags": "-rw-rw-r--.", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 22 00:12"}, {"filename": "\n\nthis file has\na combination\n\n\nof everything\n\n\n\n", "flags": "-rw-rw-r--.", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 22 00:17"}, {"filename": "this file has\na newline inside", "flags": "-rw-rw-r--.", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 22 00:14"}, {"filename": "this file has\n\n\n\nfour contiguous newlines inside", "flags": "-rw-rw-r--.", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 22 00:14"}, {"filename": "this file\nhas\nsix\n\nnewlines\n\nwithin", "flags": "-rw-rw-r--.", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 22 00:16"}, {"filename": "\n\n\n\nthis file starts with four newlines", "flags": "-rw-rw-r--.", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 22 00:40"}, {"filename": "\nthis file starts with one newline", "flags": "-rw-rw-r--.", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 22 00:13"}]

tests/fixtures/centos-7.7/ls-l-newlines.out

@@ -0,0 +1,34 @@
+total 0
+-rw-rw-r--. 1 kbrazil kbrazil 0 Feb 22 00:12 a regular filename
+-rw-rw-r--. 1 kbrazil kbrazil 0 Feb 22 00:17 
+
+this file has
+a combination
+
+
+of everything
+
+
+
+
+-rw-rw-r--. 1 kbrazil kbrazil 0 Feb 22 00:14 this file has
+a newline inside
+-rw-rw-r--. 1 kbrazil kbrazil 0 Feb 22 00:14 this file has
+
+
+
+four contiguous newlines inside
+-rw-rw-r--. 1 kbrazil kbrazil 0 Feb 22 00:16 this file
+has
+six
+
+newlines
+
+within
+-rw-rw-r--. 1 kbrazil kbrazil 0 Feb 22 00:40 
+
+
+
+this file starts with four newlines
+-rw-rw-r--. 1 kbrazil kbrazil 0 Feb 22 00:13 
+this file starts with one newline

tests/fixtures/centos-7.7/ls-lR-newlines.json

@@ -0,0 +1 @@
+[{"filename": "lstest", "parent": ".", "flags": "drwxrwxr-x.", "links": 2, "owner": "kbrazil", "group": "kbrazil", "size": 4096, "date": "Feb 22 00:40"}, {"filename": "systemd-private-016de60725a3426792b93fc9f120b8f0-chronyd.service-oZqq4u", "parent": ".", "flags": "drwx------.", "links": 3, "owner": "root", "group": "root", "size": 17, "date": "Feb 13 16:44"}, {"filename": "systemd-private-a30a5a178daa4042b42dfaf5ff9e5f68-chronyd.service-a1tpxv", "parent": ".", "flags": "drwx------.", "links": 3, "owner": "root", "group": "root", "size": 17, "date": "Feb 17 17:48"}, {"filename": "systemd-private-af69d7360f3e40cfa947358c0fb5a6f8-chronyd.service-T3MQ4j", "parent": ".", "flags": "drwx------.", "links": 3, "owner": "root", "group": "root", "size": 17, "date": "Feb 20 14:41"}, {"filename": "tmp.CvALl2jE6u", "parent": ".", "flags": "drwx------.", "links": 2, "owner": "root", "group": "root", "size": 6, "date": "Feb 13 16:44"}, {"filename": "tmp.e7AlxSxY5a", "parent": ".", "flags": "drwx------.", "links": 2, "owner": "root", "group": "root", "size": 6, "date": "Feb 20 14:41"}, {"filename": "tmp.uXm9yegjwj", "parent": ".", "flags": "drwx------.", "links": 2, "owner": "root", "group": "root", "size": 6, "date": "Feb 17 17:48"}, {"filename": "a regular filename", "parent": "./lstest", "flags": "-rw-rw-r--.", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 22 00:12"}, {"filename": "\n\nthis file has\na combination\n\n\nof everything\n\n\n\n", "parent": "./lstest", "flags": "-rw-rw-r--.", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 22 00:17"}, {"filename": "this file has\na newline inside", "parent": "./lstest", "flags": "-rw-rw-r--.", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 22 00:14"}, {"filename": "this file has\n\n\n\nfour contiguous newlines inside", "parent": "./lstest", "flags": "-rw-rw-r--.", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 22 00:14"}, {"filename": "this file\nhas\nsix\n\nnewlines\n\nwithin", "parent": "./lstest", "flags": "-rw-rw-r--.", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 22 00:16"}, {"filename": "\n\n\n\nthis file starts with four newlines", "parent": "./lstest", "flags": "-rw-rw-r--.", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 22 00:40"}, {"filename": "\nthis file starts with one newline", "parent": "./lstest", "flags": "-rw-rw-r--.", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 22 00:13"}]

tests/fixtures/centos-7.7/ls-lR-newlines.out

@@ -0,0 +1,45 @@
+.:
+total 4
+drwxrwxr-x. 2 kbrazil kbrazil 4096 Feb 22 00:40 lstest
+drwx------. 3 root    root      17 Feb 13 16:44 systemd-private-016de60725a3426792b93fc9f120b8f0-chronyd.service-oZqq4u
+drwx------. 3 root    root      17 Feb 17 17:48 systemd-private-a30a5a178daa4042b42dfaf5ff9e5f68-chronyd.service-a1tpxv
+drwx------. 3 root    root      17 Feb 20 14:41 systemd-private-af69d7360f3e40cfa947358c0fb5a6f8-chronyd.service-T3MQ4j
+drwx------. 2 root    root       6 Feb 13 16:44 tmp.CvALl2jE6u
+drwx------. 2 root    root       6 Feb 20 14:41 tmp.e7AlxSxY5a
+drwx------. 2 root    root       6 Feb 17 17:48 tmp.uXm9yegjwj
+
+./lstest:
+total 0
+-rw-rw-r--. 1 kbrazil kbrazil 0 Feb 22 00:12 a regular filename
+-rw-rw-r--. 1 kbrazil kbrazil 0 Feb 22 00:17 
+
+this file has
+a combination
+
+
+of everything
+
+
+
+
+-rw-rw-r--. 1 kbrazil kbrazil 0 Feb 22 00:14 this file has
+a newline inside
+-rw-rw-r--. 1 kbrazil kbrazil 0 Feb 22 00:14 this file has
+
+
+
+four contiguous newlines inside
+-rw-rw-r--. 1 kbrazil kbrazil 0 Feb 22 00:16 this file
+has
+six
+
+newlines
+
+within
+-rw-rw-r--. 1 kbrazil kbrazil 0 Feb 22 00:40 
+
+
+
+this file starts with four newlines
+-rw-rw-r--. 1 kbrazil kbrazil 0 Feb 22 00:13 
+this file starts with one newline

tests/fixtures/centos-7.7/ls-newlines.json

@@ -0,0 +1 @@
+[{"filename": "a regular filename"}, {"filename": "this file has"}, {"filename": "a combination"}, {"filename": "of everything"}, {"filename": "this file has"}, {"filename": "a newline inside"}, {"filename": "this file has"}, {"filename": "four contiguous newlines inside"}, {"filename": "this file"}, {"filename": "has"}, {"filename": "six"}, {"filename": "newlines"}, {"filename": "within"}, {"filename": "this file starts with four newlines"}, {"filename": "this file starts with one newline"}]

tests/fixtures/centos-7.7/ls-newlines.out

@@ -0,0 +1,33 @@
+a regular filename
+
+
+this file has
+a combination
+
+
+of everything
+
+
+
+
+this file has
+a newline inside
+this file has
+
+
+
+four contiguous newlines inside
+this file
+has
+six
+
+newlines
+
+within
+
+
+
+
+this file starts with four newlines
+
+this file starts with one newline

tests/fixtures/create_fixtures.sh

@@ -35,6 +35,26 @@ jobs       > jobs.out
 ls /       > ls.out
 ls -al /   > ls-al.out
 ls -alh /  > ls-alh.out
+
+ls -R /usr     > ls-R.out
+ls -alR /usr   > ls-alR.out
+ls /usr/*      > ls-glob.out
+
+cd /tmp/lstest
+touch 'a regular filename'
+touch $'\nthis file starts with one newline'
+touch $'\n\n\n\nthis file starts with four newlines'
+touch $'this file has\na newline inside'
+touch $'this file has\n\n\n\nfour contiguous newlines inside'
+touch $'this file\nhas\nsix\n\nnewlines\n\nwithin'
+touch $'\n\nthis file has\na combination\n\n\nof everything\n\n\n\n'
+cd /tmp
+ls -R > ~/utils/ls-R-newlines.out
+ls -lR > ~/utils/ls-lR-newlines.out
+cd lstest
+ls > ~/utils/ls-newlines.out
+ls -l > ~/utils/ls-l-newlines.out
+
 lsblk      > lsblk.out
 lsblk -o +KNAME,FSTYPE,LABEL,UUID,PARTLABEL,PARTUUID,RA,MODEL,SERIAL,STATE,OWNER,GROUP,MODE,ALIGNMENT,MIN-IO,OPT-IO,PHY-SEC,LOG-SEC,ROTA,SCHED,RQ-SIZE,DISC-ALN,DISC-GRAN,DISC-MAX,DISC-ZERO,WSAME,WWN,RAND,PKNAME,HCTL,TRAN,REV,VENDOR > lsblk-allcols.out
 lsmod      > lsmod.out

tests/fixtures/osx-10.14.6/ls-R-newlines.json

@@ -0,0 +1 @@
+[{"filename": "lstest"}, {"filename": "this file starts with four newlines", "parent": "./lstest"}, {"filename": "this file has", "parent": "./lstest"}, {"filename": "a combination", "parent": "./lstest"}, {"filename": "of everything", "parent": "./lstest"}, {"filename": "this file starts with one newline", "parent": "./lstest"}, {"filename": "a regular filename", "parent": "./lstest"}, {"filename": "this file", "parent": "./lstest"}, {"filename": "has", "parent": "./lstest"}, {"filename": "six", "parent": "./lstest"}, {"filename": "newlines", "parent": "./lstest"}, {"filename": "within", "parent": "./lstest"}, {"filename": "this file has", "parent": "./lstest"}, {"filename": "four contiguous newlines inside", "parent": "./lstest"}, {"filename": "this file has", "parent": "./lstest"}, {"filename": "a newline inside", "parent": "./lstest"}]

tests/fixtures/osx-10.14.6/ls-R-newlines.out

@@ -0,0 +1,36 @@
+lstest
+
+./lstest:
+
+
+
+
+this file starts with four newlines
+
+
+this file has
+a combination
+
+
+of everything
+
+
+
+
+
+this file starts with one newline
+a regular filename
+this file
+has
+six
+
+newlines
+
+within
+this file has
+
+
+
+four contiguous newlines inside
+this file has
+a newline inside

tests/fixtures/osx-10.14.6/ls-l-newlines.json

@@ -0,0 +1 @@
+[{"filename": "\n\n\n\nthis file starts with four newlines", "flags": "-rw-r--r--", "links": 1, "owner": "kbrazil", "group": "staff", "size": 0, "date": "Feb 27 10:19"}, {"filename": "\n\nthis file has\na combination\n\n\nof everything\n\n\n\n", "flags": "-rw-r--r--", "links": 1, "owner": "kbrazil", "group": "staff", "size": 0, "date": "Feb 27 10:19"}, {"filename": "\nthis file starts with one newline", "flags": "-rw-r--r--", "links": 1, "owner": "kbrazil", "group": "staff", "size": 0, "date": "Feb 27 10:19"}, {"filename": "a regular filename", "flags": "-rw-r--r--", "links": 1, "owner": "kbrazil", "group": "staff", "size": 0, "date": "Feb 27 10:19"}, {"filename": "this file\nhas\nsix\n\nnewlines\n\nwithin", "flags": "-rw-r--r--", "links": 1, "owner": "kbrazil", "group": "staff", "size": 0, "date": "Feb 27 10:19"}, {"filename": "this file has\n\n\n\nfour contiguous newlines inside", "flags": "-rw-r--r--", "links": 1, "owner": "kbrazil", "group": "staff", "size": 0, "date": "Feb 27 10:19"}, {"filename": "this file has\na newline inside", "flags": "-rw-r--r--", "links": 1, "owner": "kbrazil", "group": "staff", "size": 0, "date": "Feb 27 10:19"}]

tests/fixtures/osx-10.14.6/ls-l-newlines.out

@@ -0,0 +1,34 @@
+total 0
+-rw-r--r--  1 kbrazil  staff  0 Feb 27 10:19 
+
+
+
+this file starts with four newlines
+-rw-r--r--  1 kbrazil  staff  0 Feb 27 10:19 
+
+this file has
+a combination
+
+
+of everything
+
+
+
+
+-rw-r--r--  1 kbrazil  staff  0 Feb 27 10:19 
+this file starts with one newline
+-rw-r--r--  1 kbrazil  staff  0 Feb 27 10:19 a regular filename
+-rw-r--r--  1 kbrazil  staff  0 Feb 27 10:19 this file
+has
+six
+
+newlines
+
+within
+-rw-r--r--  1 kbrazil  staff  0 Feb 27 10:19 this file has
+
+
+
+four contiguous newlines inside
+-rw-r--r--  1 kbrazil  staff  0 Feb 27 10:19 this file has
+a newline inside

tests/fixtures/osx-10.14.6/ls-lR-newlines.json

@@ -0,0 +1 @@
+[{"filename": "lstest", "flags": "drwxr-xr-x", "links": 9, "owner": "kbrazil", "group": "staff", "size": 288, "date": "Feb 27 10:19"}, {"filename": "\n\n\n\nthis file starts with four newlines", "parent": "./lstest", "flags": "-rw-r--r--", "links": 1, "owner": "kbrazil", "group": "staff", "size": 0, "date": "Feb 27 10:19"}, {"filename": "\n\nthis file has\na combination\n\n\nof everything\n\n\n\n", "parent": "./lstest", "flags": "-rw-r--r--", "links": 1, "owner": "kbrazil", "group": "staff", "size": 0, "date": "Feb 27 10:19"}, {"filename": "\nthis file starts with one newline", "parent": "./lstest", "flags": "-rw-r--r--", "links": 1, "owner": "kbrazil", "group": "staff", "size": 0, "date": "Feb 27 10:19"}, {"filename": "a regular filename", "parent": "./lstest", "flags": "-rw-r--r--", "links": 1, "owner": "kbrazil", "group": "staff", "size": 0, "date": "Feb 27 10:19"}, {"filename": "this file\nhas\nsix\n\nnewlines\n\nwithin", "parent": "./lstest", "flags": "-rw-r--r--", "links": 1, "owner": "kbrazil", "group": "staff", "size": 0, "date": "Feb 27 10:19"}, {"filename": "this file has\n\n\n\nfour contiguous newlines inside", "parent": "./lstest", "flags": "-rw-r--r--", "links": 1, "owner": "kbrazil", "group": "staff", "size": 0, "date": "Feb 27 10:19"}, {"filename": "this file has\na newline inside", "parent": "./lstest", "flags": "-rw-r--r--", "links": 1, "owner": "kbrazil", "group": "staff", "size": 0, "date": "Feb 27 10:19"}]

tests/fixtures/osx-10.14.6/ls-lR-newlines.out

@@ -0,0 +1,38 @@
+total 0
+drwxr-xr-x  9 kbrazil  staff  288 Feb 27 10:19 lstest
+
+./lstest:
+total 0
+-rw-r--r--  1 kbrazil  staff  0 Feb 27 10:19 
+
+
+
+this file starts with four newlines
+-rw-r--r--  1 kbrazil  staff  0 Feb 27 10:19 
+
+this file has
+a combination
+
+
+of everything
+
+
+
+
+-rw-r--r--  1 kbrazil  staff  0 Feb 27 10:19 
+this file starts with one newline
+-rw-r--r--  1 kbrazil  staff  0 Feb 27 10:19 a regular filename
+-rw-r--r--  1 kbrazil  staff  0 Feb 27 10:19 this file
+has
+six
+
+newlines
+
+within
+-rw-r--r--  1 kbrazil  staff  0 Feb 27 10:19 this file has
+
+
+
+four contiguous newlines inside
+-rw-r--r--  1 kbrazil  staff  0 Feb 27 10:19 this file has
+a newline inside

tests/fixtures/osx-10.14.6/ls-newlines.json

@@ -0,0 +1 @@
+[{"filename": "this file starts with four newlines"}, {"filename": "this file has"}, {"filename": "a combination"}, {"filename": "of everything"}, {"filename": "this file starts with one newline"}, {"filename": "a regular filename"}, {"filename": "this file"}, {"filename": "has"}, {"filename": "six"}, {"filename": "newlines"}, {"filename": "within"}, {"filename": "this file has"}, {"filename": "four contiguous newlines inside"}, {"filename": "this file has"}, {"filename": "a newline inside"}]

tests/fixtures/osx-10.14.6/ls-newlines.out

@@ -0,0 +1,33 @@
+
+
+
+
+this file starts with four newlines
+
+
+this file has
+a combination
+
+
+of everything
+
+
+
+
+
+this file starts with one newline
+a regular filename
+this file
+has
+six
+
+newlines
+
+within
+this file has
+
+
+
+four contiguous newlines inside
+this file has
+a newline inside

tests/fixtures/ubuntu-18.04/ls-R-newlines.json

@@ -0,0 +1 @@
+[{"filename": "systemd-private-65c1089f1d4c4cf5bc50ca55478abfde-systemd-resolved.service-La9AqY", "parent": "."}, {"filename": "systemd-private-65c1089f1d4c4cf5bc50ca55478abfde-systemd-timesyncd.service-L7q4cJ", "parent": "."}, {"filename": "vmware-root_670-2722828838", "parent": "."}, {"filename": "a regular filename", "parent": "./lstest"}, {"filename": "this file has", "parent": "./lstest"}, {"filename": "a combination", "parent": "./lstest"}, {"filename": "of everything", "parent": "./lstest"}, {"filename": "this file has", "parent": "./lstest"}, {"filename": "a newline inside", "parent": "./lstest"}, {"filename": "this file has", "parent": "./lstest"}, {"filename": "four contiguous newlines inside", "parent": "./lstest"}, {"filename": "this file", "parent": "./lstest"}, {"filename": "has", "parent": "./lstest"}, {"filename": "six", "parent": "./lstest"}, {"filename": "newlines", "parent": "./lstest"}, {"filename": "within", "parent": "./lstest"}, {"filename": "this file starts with four newlines", "parent": "./lstest"}, {"filename": "this file starts with one newline", "parent": "./lstest"}]

tests/fixtures/ubuntu-18.04/ls-R-newlines.out

@@ -0,0 +1,40 @@
+.:
+lstest
+systemd-private-65c1089f1d4c4cf5bc50ca55478abfde-systemd-resolved.service-La9AqY
+systemd-private-65c1089f1d4c4cf5bc50ca55478abfde-systemd-timesyncd.service-L7q4cJ
+vmware-root_670-2722828838
+
+./lstest:
+a regular filename
+
+
+this file has
+a combination
+
+
+of everything
+
+
+
+
+this file has
+a newline inside
+this file has
+
+
+
+four contiguous newlines inside
+this file
+has
+six
+
+newlines
+
+within
+
+
+
+
+this file starts with four newlines
+
+this file starts with one newline

tests/fixtures/ubuntu-18.04/ls-l-newlines.json

@@ -0,0 +1 @@
+[{"filename": "a regular filename", "flags": "-rw-rw-r--", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 27 18:08"}, {"filename": "\n\nthis file has\na combination\n\n\nof everything\n\n\n\n", "flags": "-rw-rw-r--", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 27 18:08"}, {"filename": "this file has\na newline inside", "flags": "-rw-rw-r--", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 27 18:08"}, {"filename": "this file has\n\n\n\nfour contiguous newlines inside", "flags": "-rw-rw-r--", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 27 18:08"}, {"filename": "this file\nhas\nsix\n\nnewlines\n\nwithin", "flags": "-rw-rw-r--", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 27 18:08"}, {"filename": "\n\n\n\nthis file starts with four newlines", "flags": "-rw-rw-r--", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 27 18:08"}, {"filename": "\nthis file starts with one newline", "flags": "-rw-rw-r--", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 27 18:08"}]

tests/fixtures/ubuntu-18.04/ls-l-newlines.out

@@ -0,0 +1,34 @@
+total 0
+-rw-rw-r-- 1 kbrazil kbrazil 0 Feb 27 18:08 a regular filename
+-rw-rw-r-- 1 kbrazil kbrazil 0 Feb 27 18:08 
+
+this file has
+a combination
+
+
+of everything
+
+
+
+
+-rw-rw-r-- 1 kbrazil kbrazil 0 Feb 27 18:08 this file has
+a newline inside
+-rw-rw-r-- 1 kbrazil kbrazil 0 Feb 27 18:08 this file has
+
+
+
+four contiguous newlines inside
+-rw-rw-r-- 1 kbrazil kbrazil 0 Feb 27 18:08 this file
+has
+six
+
+newlines
+
+within
+-rw-rw-r-- 1 kbrazil kbrazil 0 Feb 27 18:08 
+
+
+
+this file starts with four newlines
+-rw-rw-r-- 1 kbrazil kbrazil 0 Feb 27 18:08 
+this file starts with one newline

tests/fixtures/ubuntu-18.04/ls-lR-newlines.json

@@ -0,0 +1 @@
+[{"filename": "lstest", "parent": ".", "flags": "drwxrwxr-x", "links": 2, "owner": "kbrazil", "group": "kbrazil", "size": 4096, "date": "Feb 27 18:08"}, {"filename": "systemd-private-65c1089f1d4c4cf5bc50ca55478abfde-systemd-resolved.service-La9AqY", "parent": ".", "flags": "drwx------", "links": 3, "owner": "root", "group": "root", "size": 4096, "date": "Feb 27 18:07"}, {"filename": "systemd-private-65c1089f1d4c4cf5bc50ca55478abfde-systemd-timesyncd.service-L7q4cJ", "parent": ".", "flags": "drwx------", "links": 3, "owner": "root", "group": "root", "size": 4096, "date": "Feb 27 18:07"}, {"filename": "vmware-root_670-2722828838", "parent": ".", "flags": "drwx------", "links": 2, "owner": "root", "group": "root", "size": 4096, "date": "Feb 27 18:07"}, {"filename": "a regular filename", "parent": "./lstest", "flags": "-rw-rw-r--", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 27 18:08"}, {"filename": "\n\nthis file has\na combination\n\n\nof everything\n\n\n\n", "parent": "./lstest", "flags": "-rw-rw-r--", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 27 18:08"}, {"filename": "this file has\na newline inside", "parent": "./lstest", "flags": "-rw-rw-r--", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 27 18:08"}, {"filename": "this file has\n\n\n\nfour contiguous newlines inside", "parent": "./lstest", "flags": "-rw-rw-r--", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 27 18:08"}, {"filename": "this file\nhas\nsix\n\nnewlines\n\nwithin", "parent": "./lstest", "flags": "-rw-rw-r--", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 27 18:08"}, {"filename": "\n\n\n\nthis file starts with four newlines", "parent": "./lstest", "flags": "-rw-rw-r--", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 27 18:08"}, {"filename": "\nthis file starts with one newline", "parent": "./lstest", "flags": "-rw-rw-r--", "links": 1, "owner": "kbrazil", "group": "kbrazil", "size": 0, "date": "Feb 27 18:08"}]

tests/fixtures/ubuntu-18.04/ls-lR-newlines.out

@@ -0,0 +1,42 @@
+.:
+total 16
+drwxrwxr-x 2 kbrazil kbrazil 4096 Feb 27 18:08 lstest
+drwx------ 3 root    root    4096 Feb 27 18:07 systemd-private-65c1089f1d4c4cf5bc50ca55478abfde-systemd-resolved.service-La9AqY
+drwx------ 3 root    root    4096 Feb 27 18:07 systemd-private-65c1089f1d4c4cf5bc50ca55478abfde-systemd-timesyncd.service-L7q4cJ
+drwx------ 2 root    root    4096 Feb 27 18:07 vmware-root_670-2722828838
+
+./lstest:
+total 0
+-rw-rw-r-- 1 kbrazil kbrazil 0 Feb 27 18:08 a regular filename
+-rw-rw-r-- 1 kbrazil kbrazil 0 Feb 27 18:08 
+
+this file has
+a combination
+
+
+of everything
+
+
+
+
+-rw-rw-r-- 1 kbrazil kbrazil 0 Feb 27 18:08 this file has
+a newline inside
+-rw-rw-r-- 1 kbrazil kbrazil 0 Feb 27 18:08 this file has
+
+
+
+four contiguous newlines inside
+-rw-rw-r-- 1 kbrazil kbrazil 0 Feb 27 18:08 this file
+has
+six
+
+newlines
+
+within
+-rw-rw-r-- 1 kbrazil kbrazil 0 Feb 27 18:08 
+
+
+
+this file starts with four newlines
+-rw-rw-r-- 1 kbrazil kbrazil 0 Feb 27 18:08 
+this file starts with one newline

tests/fixtures/ubuntu-18.04/ls-newlines.json

@@ -0,0 +1 @@
+[{"filename": "a regular filename"}, {"filename": "this file has"}, {"filename": "a combination"}, {"filename": "of everything"}, {"filename": "this file has"}, {"filename": "a newline inside"}, {"filename": "this file has"}, {"filename": "four contiguous newlines inside"}, {"filename": "this file"}, {"filename": "has"}, {"filename": "six"}, {"filename": "newlines"}, {"filename": "within"}, {"filename": "this file starts with four newlines"}, {"filename": "this file starts with one newline"}]

tests/fixtures/ubuntu-18.04/ls-newlines.out

@@ -0,0 +1,33 @@
+a regular filename
+
+
+this file has
+a combination
+
+
+of everything
+
+
+
+
+this file has
+a newline inside
+this file has
+
+
+
+four contiguous newlines inside
+this file
+has
+six
+
+newlines
+
+within
+
+
+
+
+this file starts with four newlines
+
+this file starts with one newline

tests/test_ls.py

@@ -46,6 +46,69 @@ class MyTests(unittest.TestCase):
         with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-alh.out'), 'r') as f:
             self.osx_10_14_6_ls_alh = f.read()
 
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls-R.out'), 'r') as f:
+            self.centos_7_7_ls_R = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/ubuntu-18.04/ls-R.out'), 'r') as f:
+            self.ubuntu_18_4_ls_R = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-R.out'), 'r') as f:
+            self.osx_10_14_6_ls_R = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls-alR.out'), 'r') as f:
+            self.centos_7_7_ls_alR = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/ubuntu-18.04/ls-alR.out'), 'r') as f:
+            self.ubuntu_18_4_ls_alR = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-alR.out'), 'r') as f:
+            self.osx_10_14_6_ls_alR = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls-glob.out'), 'r') as f:
+            self.centos_7_7_ls_glob = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/ubuntu-18.04/ls-glob.out'), 'r') as f:
+            self.ubuntu_18_4_ls_glob = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-glob.out'), 'r') as f:
+            self.osx_10_14_6_ls_glob = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls-R-newlines.out'), 'r') as f:
+            self.centos_7_7_ls_R_newlines = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/ubuntu-18.04/ls-R-newlines.out'), 'r') as f:
+            self.ubuntu_18_4_ls_R_newlines = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-R-newlines.out'), 'r') as f:
+            self.osx_10_14_6_ls_R_newlines = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls-l-newlines.out'), 'r') as f:
+            self.centos_7_7_ls_l_newlines = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/ubuntu-18.04/ls-l-newlines.out'), 'r') as f:
+            self.ubuntu_18_4_ls_l_newlines = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-l-newlines.out'), 'r') as f:
+            self.osx_10_14_6_ls_l_newlines = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls-lR-newlines.out'), 'r') as f:
+            self.centos_7_7_ls_lR_newlines = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/ubuntu-18.04/ls-lR-newlines.out'), 'r') as f:
+            self.ubuntu_18_4_ls_lR_newlines = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-lR-newlines.out'), 'r') as f:
+            self.osx_10_14_6_ls_lR_newlines = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls-newlines.out'), 'r') as f:
+            self.centos_7_7_ls_newlines = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/ubuntu-18.04/ls-newlines.out'), 'r') as f:
+            self.ubuntu_18_4_ls_newlines = f.read()
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-newlines.out'), 'r') as f:
+            self.osx_10_14_6_ls_newlines = f.read()
+
         # output
         with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls.json'), 'r') as f:
             self.centos_7_7_ls_json = json.loads(f.read())
@@ -83,6 +146,69 @@ class MyTests(unittest.TestCase):
         with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-alh.json'), 'r') as f:
             self.osx_10_14_6_ls_alh_json = json.loads(f.read())
 
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls-R.json'), 'r') as f:
+            self.centos_7_7_ls_R_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/ubuntu-18.04/ls-R.json'), 'r') as f:
+            self.ubuntu_18_4_ls_R_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-R.json'), 'r') as f:
+            self.osx_10_14_6_ls_R_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls-alR.json'), 'r') as f:
+            self.centos_7_7_ls_alR_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/ubuntu-18.04/ls-alR.json'), 'r') as f:
+            self.ubuntu_18_4_ls_alR_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-alR.json'), 'r') as f:
+            self.osx_10_14_6_ls_alR_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls-glob.json'), 'r') as f:
+            self.centos_7_7_ls_glob_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/ubuntu-18.04/ls-glob.json'), 'r') as f:
+            self.ubuntu_18_4_ls_glob_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-glob.json'), 'r') as f:
+            self.osx_10_14_6_ls_glob_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls-R-newlines.json'), 'r') as f:
+            self.centos_7_7_ls_R_newlines_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/ubuntu-18.04/ls-R-newlines.json'), 'r') as f:
+            self.ubuntu_18_4_ls_R_newlines_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-R-newlines.json'), 'r') as f:
+            self.osx_10_14_6_ls_R_newlines_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls-l-newlines.json'), 'r') as f:
+            self.centos_7_7_ls_l_newlines_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/ubuntu-18.04/ls-l-newlines.json'), 'r') as f:
+            self.ubuntu_18_4_ls_l_newlines_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-l-newlines.json'), 'r') as f:
+            self.osx_10_14_6_ls_l_newlines_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls-lR-newlines.json'), 'r') as f:
+            self.centos_7_7_ls_lR_newlines_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/ubuntu-18.04/ls-lR-newlines.json'), 'r') as f:
+            self.ubuntu_18_4_ls_lR_newlines_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-lR-newlines.json'), 'r') as f:
+            self.osx_10_14_6_ls_lR_newlines_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/centos-7.7/ls-newlines.json'), 'r') as f:
+            self.centos_7_7_ls_newlines_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/ubuntu-18.04/ls-newlines.json'), 'r') as f:
+            self.ubuntu_18_4_ls_newlines_json = json.loads(f.read())
+
+        with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/osx-10.14.6/ls-newlines.json'), 'r') as f:
+            self.osx_10_14_6_ls_newlines_json = json.loads(f.read())
+
     def test_ls_centos_7_7(self):
         """
         Test plain 'ls /' on Centos 7.7
@@ -155,6 +281,132 @@ class MyTests(unittest.TestCase):
         """
         self.assertEqual(jc.parsers.ls.parse(self.osx_10_14_6_ls_alh, quiet=True), self.osx_10_14_6_ls_alh_json)
 
+    def test_ls_R_centos_7_7(self):
+        """
+        Test 'ls -R /usr' on Centos 7.7
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.centos_7_7_ls_R, quiet=True), self.centos_7_7_ls_R_json)
+
+    def test_ls_R_ubuntu_18_4(self):
+        """
+        Test 'ls -R /usr' on Ubuntu 18.4
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.ubuntu_18_4_ls_R, quiet=True), self.ubuntu_18_4_ls_R_json)
+
+    def test_ls_R_osx_10_14_6(self):
+        """
+        Test 'ls -R /usr' on OSX 10.14.6
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.osx_10_14_6_ls_R, quiet=True), self.osx_10_14_6_ls_R_json)
+
+    def test_ls_alR_centos_7_7(self):
+        """
+        Test 'ls -alR /usr' on Centos 7.7
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.centos_7_7_ls_alR, quiet=True), self.centos_7_7_ls_alR_json)
+
+    def test_ls_alR_ubuntu_18_4(self):
+        """
+        Test 'ls -alR /usr' on Ubuntu 18.4
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.ubuntu_18_4_ls_alR, quiet=True), self.ubuntu_18_4_ls_alR_json)
+
+    def test_ls_alR_osx_10_14_6(self):
+        """
+        Test 'ls -alR /usr' on OSX 10.14.6
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.osx_10_14_6_ls_alR, quiet=True), self.osx_10_14_6_ls_alR_json)
+
+    def test_ls_glob_centos_7_7(self):
+        """
+        Test 'ls /usr/*' on Centos 7.7
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.centos_7_7_ls_glob, quiet=True), self.centos_7_7_ls_glob_json)
+
+    def test_ls_glob_ubuntu_18_4(self):
+        """
+        Test 'ls /usr/*' on Ubuntu 18.4
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.ubuntu_18_4_ls_glob, quiet=True), self.ubuntu_18_4_ls_glob_json)
+
+    def test_ls_glob_osx_10_14_6(self):
+        """
+        Test 'ls /usr/*' on OSX 10.14.6
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.osx_10_14_6_ls_glob, quiet=True), self.osx_10_14_6_ls_glob_json)
+
+    def test_ls_R_newlines_centos_7_7(self):
+        """
+        Test 'ls -R' for filenames with newline characters on Centos 7.7
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.centos_7_7_ls_R_newlines, quiet=True), self.centos_7_7_ls_R_newlines_json)
+
+    def test_ls_R_newlines_ubuntu_18_4(self):
+        """
+        Test 'ls -R' for filenames with newline characters on Ubuntu 18.4
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.ubuntu_18_4_ls_R_newlines, quiet=True), self.ubuntu_18_4_ls_R_newlines_json)
+
+    def test_ls_R_newlines_osx_10_14_6(self):
+        """
+        Test 'ls -R' for filenames with newline characters on OSX 10.14.6
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.osx_10_14_6_ls_R_newlines, quiet=True), self.osx_10_14_6_ls_R_newlines_json)
+
+    def test_ls_l_newlines_centos_7_7(self):
+        """
+        Test 'ls -l' for filenames with newline characters on Centos 7.7
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.centos_7_7_ls_l_newlines, quiet=True), self.centos_7_7_ls_l_newlines_json)
+
+    def test_ls_l_newlines_ubuntu_18_4(self):
+        """
+        Test 'ls -l' for filenames with newline characters on Ubuntu 18.4
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.ubuntu_18_4_ls_l_newlines, quiet=True), self.ubuntu_18_4_ls_l_newlines_json)
+
+    def test_ls_l_newlines_osx_10_14_6(self):
+        """
+        Test 'ls -l' for filenames with newline characters on OSX 10.14.6
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.osx_10_14_6_ls_l_newlines, quiet=True), self.osx_10_14_6_ls_l_newlines_json)
+
+    def test_ls_lR_newlines_centos_7_7(self):
+        """
+        Test 'ls -lR' for filenames with newline characters on Centos 7.7
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.centos_7_7_ls_lR_newlines, quiet=True), self.centos_7_7_ls_lR_newlines_json)
+
+    def test_ls_lR_newlines_ubuntu_18_4(self):
+        """
+        Test 'ls -lR' for filenames with newline characters on Ubuntu 18.4
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.ubuntu_18_4_ls_lR_newlines, quiet=True), self.ubuntu_18_4_ls_lR_newlines_json)
+
+    def test_ls_lR_newlines_osx_10_14_6(self):
+        """
+        Test 'ls -lR' for filenames with newline characters on OSX 10.14.6
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.osx_10_14_6_ls_lR_newlines, quiet=True), self.osx_10_14_6_ls_lR_newlines_json)
+
+    def test_ls_newlines_centos_7_7(self):
+        """
+        Test 'ls' for filenames with newline characters on Centos 7.7
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.centos_7_7_ls_newlines, quiet=True), self.centos_7_7_ls_newlines_json)
+
+    def test_ls_newlines_ubuntu_18_4(self):
+        """
+        Test 'ls' for filenames with newline characters on Ubuntu 18.4
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.ubuntu_18_4_ls_newlines, quiet=True), self.ubuntu_18_4_ls_newlines_json)
+
+    def test_ls_newlines_osx_10_14_6(self):
+        """
+        Test 'ls' for filenames with newline characters on OSX 10.14.6
+        """
+        self.assertEqual(jc.parsers.ls.parse(self.osx_10_14_6_ls_newlines, quiet=True), self.osx_10_14_6_ls_newlines_json)
+
 
 if __name__ == '__main__':
     unittest.main()