Merge pull request #141 from kellyjonbrazil/dev

Dev v1.15.5

CHANGELOG

@@ -1,8 +1,12 @@
 jc changelog
 
+20210520 v1.15.5
+- Fix issue where help and about information would not display if a 3rd party parser library was missing. (e.g. xmltodict)
+- Add more error message detail when encountering ParseError and LibraryNotFound exceptions
+
 20210517 v1.15.4
 - Update ping parser to support error responses in OSX and BSD
-- Update ping parser to be more resillient against parsing errors for unknown error types
+- Update ping parser to be more resilient against parsing errors for unknown error types
 - Update dig parser to support `+noall +answer` use case
 - Update dig parser compatibility to all platforms
 - Fix colors in Windows terminals (cmd.exe and PowerShell)

README.md

@@ -83,7 +83,7 @@ Use Cases:
 - [Nornir command output parsing](https://blog.kellybrazil.com/2020/12/09/parsing-command-output-in-nornir-with-jc/)
 
 ## Installation
-There are several ways to get `jc`. You can install via `pip`; other OS package repositories like `apt-get`, `dnf`, `zypper`, `pacman`, `nix-env`, `guix`, `brew`, or `portsnap`; via DEB/RPM packaged binaries; or by downloading the correct binary for your architecture and running it anywhere on your filesystem.
+There are several ways to get `jc`. You can install via `pip`; other OS package repositories like `apt-get`, `dnf`, `zypper`, `pacman`, `nix-env`, `guix`, `brew`, or `portsnap`; via DEB, RPM, and MSI packaged binaries for linux and Windows; or by downloading the correct binary for your architecture and running it anywhere on your filesystem.
 
 ### Pip (macOS, linux, unix, Windows)
 ```bash
@@ -248,7 +248,7 @@ Local plugin filenames must be valid python module names, therefore must consist
 ### Caveats
 **Locale:**
 
-For best results set the `LANG` locale environment variable to `C`. For example, either by setting directly on the command-line:
+For best results set the `LANG` locale environment variable to `C` or `en_US.UTF-8`. For example, either by setting directly on the command-line:
 ```
 $ LANG=C date | jc --date
 ```
@@ -266,11 +266,16 @@ If a UTC timezone can be detected in the text of the command output, the timesta
 ## Compatibility
 Some parsers like `dig`, `xml`, `csv`, etc. will work on any platform. Other parsers that convert platform-specific output will generate a warning message if they are run on an unsupported platform. To see all parser information, including compatibility, run `jc -ap`.
 
-You may still use a parser on an unsupported platform - for example, you may want to parse a file with linux `lsof` output on an macOS laptop. In that case you can suppress the warning message with the `-q` cli option or the `quiet=True` function parameter in `parse()`:
+You may still use a parser on an unsupported platform - for example, you may want to parse a file with linux `lsof` output on an macOS or Windows laptop. In that case you can suppress the warning message with the `-q` cli option or the `quiet=True` function parameter in `parse()`:
 
+macOS:
 ```bash
 cat lsof.out | jc --lsof -q
 ```
+or Windows:
+```bash
+type lsof.out | jc --lsof -q
+```
 
 Tested on:
 - Centos 7.7
@@ -282,6 +287,8 @@ Tested on:
 - NixOS
 - FreeBSD12
 - Windows 10
+- Windows 2016 Server
+- Windows 2019 Server
 
 ## Contributions
 Feel free to add/improve code or parsers! You can use the [`jc/parsers/foo.py`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py) parser as a template and submit your parser with a pull request.

docs/parsers/xml.md

@@ -95,4 +95,4 @@ Returns:
 ## Parser Information
 Compatibility:  linux, darwin, cygwin, win32, aix, freebsd
 
-Version 1.4 by Kelly Brazil (kellyjonbrazil@gmail.com)
+Version 1.5 by Kelly Brazil (kellyjonbrazil@gmail.com)

docs/parsers/yaml.md

@@ -109,4 +109,4 @@ Returns:
 ## Parser Information
 Compatibility:  linux, darwin, cygwin, win32, aix, freebsd
 
-Version 1.4 by Kelly Brazil (kellyjonbrazil@gmail.com)
+Version 1.5 by Kelly Brazil (kellyjonbrazil@gmail.com)

jc/__init__.py

@@ -86,4 +86,4 @@ Module Example:
 """
 
 name = 'jc'
-__version__ = '1.15.4'
+__version__ = '1.15.5'

jc/cli.py

@@ -16,6 +16,7 @@ import jc
 import jc.appdirs as appdirs
 import jc.utils
 import jc.tracebackplus
+from jc.exceptions import LibraryNotInstalled, ParseError
 
 # make pygments import optional
 try:
@@ -603,6 +604,16 @@ def main():
     try:
         result = parser.parse(data, raw=raw, quiet=quiet)
 
+    except (ParseError, LibraryNotInstalled) as e:
+        if debug:
+            raise
+        else:
+            jc.utils.error_message(
+                f'Parser issue with {parser_name}:\n'
+                f'             {e}\n'
+                '             For details use the -d or -dd option. Use "jc -h" for help.')
+            sys.exit(combined_exit_code(magic_exit_code, JC_ERROR_EXIT))
+
     except Exception:
         if debug:
             raise

jc/exceptions.py

@@ -3,3 +3,7 @@
 
 class ParseError(Exception):
     pass
+
+
+class LibraryNotInstalled(Exception):
+    pass

jc/parsers/xml.py

@@ -65,19 +65,13 @@ Examples:
       ...
     }
 """
-import sys
 import jc.utils
-# check if xml library is installed and fail gracefully if it is not
-try:
-    import xmltodict
-except Exception:
-    jc.utils.error_message('The xmltodict library is not installed.')
-    sys.exit(1)
+from jc.exceptions import LibraryNotInstalled
 
 
 class info():
     """Provides parser metadata (version, author, etc.)"""
-    version = '1.4'
+    version = '1.5'
     description = 'XML file parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
@@ -121,6 +115,12 @@ def parse(data, raw=False, quiet=False):
 
         Dictionary. Raw or processed structured data.
     """
+    # check if xml library is installed and fail gracefully if it is not
+    try:
+        import xmltodict
+    except Exception:
+        raise LibraryNotInstalled('The xmltodict library is not installed.')
+
     if not quiet:
         jc.utils.compatibility(__name__, info.compatible)
 

jc/parsers/yaml.py

@@ -79,19 +79,13 @@ Examples:
       }
     ]
 """
-import sys
 import jc.utils
-# check if yaml library is installed and fail gracefully if it is not
-try:
-    from ruamel.yaml import YAML
-except Exception:
-    jc.utils.error_message('The ruamel.yaml library is not installed.')
-    sys.exit(1)
+from jc.exceptions import LibraryNotInstalled
 
 
 class info():
     """Provides parser metadata (version, author, etc.)"""
-    version = '1.4'
+    version = '1.5'
     description = 'YAML file parser'
     author = 'Kelly Brazil'
     author_email = 'kellyjonbrazil@gmail.com'
@@ -135,6 +129,12 @@ def parse(data, raw=False, quiet=False):
 
         List of Dictionaries representing the YAML documents.
     """
+    # check if yaml library is installed and fail gracefully if it is not
+    try:
+        from ruamel.yaml import YAML
+    except Exception:
+        raise LibraryNotInstalled('The ruamel.yaml library is not installed.')
+
     if not quiet:
         jc.utils.compatibility(__name__, info.compatible)
 

setup.py

@@ -5,7 +5,7 @@ with open('README.md', 'r') as f:
 
 setuptools.setup(
     name='jc',
-    version='1.15.4',
+    version='1.15.5',
     author='Kelly Brazil',
     author_email='kellyjonbrazil@gmail.com',
     description='Converts the output of popular command-line tools and file-types to JSON.',

templates/manpage_template

@@ -4,12 +4,12 @@ jc \- JSONifies the output of many CLI tools and file-types
 .SH SYNOPSIS
 COMMAND | jc PARSER [OPTIONS]
 
-or magic syntax:
+or "Magic" syntax:
 
 jc [OPTIONS] COMMAND
 
 .SH DESCRIPTION
-jc JSONifies the output of many CLI tools and file-types for easier parsing in scripts. jc accepts piped input from \fBSTDIN\fP and outputs a JSON representation of the previous command's output to \fBSTDOUT\fP. Alternatively, the "magic" syntax can be used by prepending jc to the command to be converted. Options can be passed to jc immediately before the command is given. (Note: command aliases are not supported).
+jc JSONifies the output of many CLI tools and file-types for easier parsing in scripts. jc accepts piped input from \fBSTDIN\fP and outputs a JSON representation of the previous command's output to \fBSTDOUT\fP. Alternatively, the "Magic" syntax can be used by prepending jc to the command to be converted. Options can be passed to jc immediately before the command is given. (Note: "Magic" syntax does not support shell builtins or command aliases)
 
 .SH OPTIONS
 .B
@@ -63,7 +63,7 @@ raw JSON output
 version information
 
 .SH EXIT CODES
-Any fatal errors within jc will generate an exit code of \fB100\fP, otherwise the exit code will be \fB0\fP. When using the "magic" syntax (e.g. \fBjc ifconfig eth0\fP), jc will store the exit code of the program being parsed and add it to the  jc exit code. This way it is easier to determine if an error was from the parsed program or jc.
+Any fatal errors within jc will generate an exit code of \fB100\fP, otherwise the exit code will be \fB0\fP. When using the "Magic" syntax (e.g. \fBjc ifconfig eth0\fP), jc will store the exit code of the program being parsed and add it to the  jc exit code. This way it is easier to determine if an error was from the parsed program or jc.
 
 Consider the following examples using `ifconfig`:
 
@@ -112,7 +112,7 @@ Local plugin filenames must be valid python module names, therefore must consist
 Note: The application data directory follows the XDG Base Directory Specification
 
 .SH CAVEATS
-\fBLocale:\fP For best results set the \fBLANG\fP locale environment variable to \fBC\fP. For example, either by setting directly on the command-line:
+\fBLocale:\fP For best results set the \fBLANG\fP locale environment variable to \fBC\fP or \fBen_US.UTF-8\fP. For example, either by setting directly on the command-line:
 
 \fB$ LANG=C date | jc --date\fP
 

templates/readme_template

@@ -83,7 +83,7 @@ Use Cases:
 - [Nornir command output parsing](https://blog.kellybrazil.com/2020/12/09/parsing-command-output-in-nornir-with-jc/)
 
 ## Installation
-There are several ways to get `jc`. You can install via `pip`; other OS package repositories like `apt-get`, `dnf`, `zypper`, `pacman`, `nix-env`, `guix`, `brew`, or `portsnap`; via DEB/RPM packaged binaries; or by downloading the correct binary for your architecture and running it anywhere on your filesystem.
+There are several ways to get `jc`. You can install via `pip`; other OS package repositories like `apt-get`, `dnf`, `zypper`, `pacman`, `nix-env`, `guix`, `brew`, or `portsnap`; via DEB, RPM, and MSI packaged binaries for linux and Windows; or by downloading the correct binary for your architecture and running it anywhere on your filesystem.
 
 ### Pip (macOS, linux, unix, Windows)
 ```bash
@@ -176,7 +176,7 @@ Local plugin filenames must be valid python module names, therefore must consist
 ### Caveats
 **Locale:**
 
-For best results set the `LANG` locale environment variable to `C`. For example, either by setting directly on the command-line:
+For best results set the `LANG` locale environment variable to `C` or `en_US.UTF-8`. For example, either by setting directly on the command-line:
 ```
 $ LANG=C date | jc --date
 ```
@@ -194,11 +194,16 @@ If a UTC timezone can be detected in the text of the command output, the timesta
 ## Compatibility
 Some parsers like `dig`, `xml`, `csv`, etc. will work on any platform. Other parsers that convert platform-specific output will generate a warning message if they are run on an unsupported platform. To see all parser information, including compatibility, run `jc -ap`.
 
-You may still use a parser on an unsupported platform - for example, you may want to parse a file with linux `lsof` output on an macOS laptop. In that case you can suppress the warning message with the `-q` cli option or the `quiet=True` function parameter in `parse()`:
+You may still use a parser on an unsupported platform - for example, you may want to parse a file with linux `lsof` output on an macOS or Windows laptop. In that case you can suppress the warning message with the `-q` cli option or the `quiet=True` function parameter in `parse()`:
 
+macOS:
 ```bash
 cat lsof.out | jc --lsof -q
 ```
+or Windows:
+```bash
+type lsof.out | jc --lsof -q
+```
 
 Tested on:
 - Centos 7.7
@@ -210,6 +215,8 @@ Tested on:
 - NixOS
 - FreeBSD12
 - Windows 10
+- Windows 2016 Server
+- Windows 2019 Server
 
 ## Contributions
 Feel free to add/improve code or parsers! You can use the [`jc/parsers/foo.py`](https://github.com/kellyjonbrazil/jc/blob/master/jc/parsers/foo.py) parser as a template and submit your parser with a pull request.

tests/test_utils.py

@@ -18,6 +18,8 @@ class MyTests(unittest.TestCase):
             '3/22/2021, 1:15:51 PM (UTC-0600)': {'string': '3/22/2021, 1:15:51 PM (UTC-0600)', 'format': 1700, 'naive': 1616444151, 'utc': None},
             # Windows english format with UTC tz (found in systeminfo cli output)
             '3/22/2021, 1:15:51 PM (UTC)': {'string': '3/22/2021, 1:15:51 PM (UTC)', 'format': 1705, 'naive': 1616444151, 'utc': 1616418951},
+            # Windows english format with UTC tz in long-form (found in systeminfo cli output)
+            '3/22/2021, 1:15:51 PM (Coordinated Universal Time)': {'string': '3/22/2021, 1:15:51 PM (Coordinated Universal Time)', 'format': 1705, 'naive': 1616444151, 'utc': 1616418951},
             # Windows english format with UTC tz (found in systeminfo cli output)
             '3/22/2021, 1:15:51 PM (UTC+0000)': {'string': '3/22/2021, 1:15:51 PM (UTC+0000)', 'format': 1710, 'naive': 1616444151, 'utc': 1616418951},
             # en_US.UTF-8 local format (found in upower cli output)