python/jc
1
0
Fork 0
mirror of https://github.com/kellyjonbrazil/jc.git synced 2026-06-19 22:28:17 +02:00
Code Issues Releases Activity

refine streaming parsers

Browse Source
This commit is contained in:
Kelly Brazil
2022-02-07 06:29:17 -08:00
parent d1e0ee6123
commit 2f3f78e8d3
16 changed files with 194 additions and 185 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/parsers/rsync_s.md
+1 -1
View File
@@ -122,4 +122,4 @@ Returns:
### Parser Information
Compatibility: linux, darwin, freebsd
Version 1.1 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.0 by Kelly Brazil (kellyjonbrazil@gmail.com)
docs/readme.md
+2
View File
@@ -14,6 +14,8 @@ and file-types to dictionaries and lists of dictionaries.
>>> help('jc')
>>> help('jc.lib')
>>> help('jc.utils')
>>> help('jc.streaming')
>>> help('jc.parsers.universal')
>>> jc.get_help('parser_module_name')
## Online Documentation
docs/streaming.md
+114
View File
@@ -0,0 +1,114 @@
# Table of Contents
* [jc.streaming](#jc.streaming)
* [streaming\_input\_type\_check](#jc.streaming.streaming_input_type_check)
* [streaming\_line\_input\_type\_check](#jc.streaming.streaming_line_input_type_check)
* [stream\_success](#jc.streaming.stream_success)
* [stream\_error](#jc.streaming.stream_error)
* [add\_jc\_meta](#jc.streaming.add_jc_meta)
* [raise\_or\_yield](#jc.streaming.raise_or_yield)
<a id="jc.streaming"></a>
# jc.streaming
jc - JSON CLI output utility streaming utils
<a id="jc.streaming.streaming_input_type_check"></a>
### streaming\_input\_type\_check
```python
def streaming_input_type_check(data: Iterable) -> None
```
Ensure input data is an iterable, but not a string or bytes. Raises
`TypeError` if not.
<a id="jc.streaming.streaming_line_input_type_check"></a>
### streaming\_line\_input\_type\_check
```python
def streaming_line_input_type_check(line: str) -> None
```
Ensure each line is a string. Raises `TypeError` if not.
<a id="jc.streaming.stream_success"></a>
### stream\_success
```python
def stream_success(output_line: Dict, ignore_exceptions: bool) -> Dict
```
Add `_jc_meta` object to output line if `ignore_exceptions=True`
<a id="jc.streaming.stream_error"></a>
### stream\_error
```python
def stream_error(e: BaseException, line: str) -> Dict
```
Return an error `_jc_meta` field.
<a id="jc.streaming.add_jc_meta"></a>
### add\_jc\_meta
```python
def add_jc_meta(func)
```
Decorator for streaming parsers to add stream_success and stream_error
objects. This simplifies the yield lines in the streaming parsers.
With the decorator on parse():
# successfully parsed line:
yield output_line if raw else _process(output_line)
# unsuccessfully parsed line:
except Exception as e:
yield raise_or_yield(ignore_exceptions, e, line)
Without the decorator on parse():
# successfully parsed line:
if raw:
yield stream_success(output_line, ignore_exceptions)
else:
stream_success(_process(output_line), ignore_exceptions)
# unsuccessfully parsed line:
except Exception as e:
yield stream_error(raise_or_yield(ignore_exceptions, e, line))
In all cases above:
output_line: (Dict) successfully parsed line yielded as a dict
e: (BaseException) exception object as the first value
of the tuple if the line was not successfully parsed.
line: (str) string of the original line that did not
successfully parse.
ignore_exceptions: (bool) continue processing lines and ignore
exceptions if True.
<a id="jc.streaming.raise_or_yield"></a>
### raise\_or\_yield
```python
def raise_or_yield(ignore_exceptions: bool, e: BaseException, line: str) -> tuple
```
Return the exception object and line string if ignore_exceptions is
True. Otherwise, re-raise the exception from the exception object with
an annotation.
docs/utils.md
-93
View File
@@ -8,12 +8,7 @@
* [convert\_to\_int](#jc.utils.convert_to_int)
* [convert\_to\_float](#jc.utils.convert_to_float)
* [convert\_to\_bool](#jc.utils.convert_to_bool)
* [stream\_success](#jc.utils.stream_success)
* [stream\_error](#jc.utils.stream_error)
* [add\_jc\_meta](#jc.utils.add_jc_meta)
* [input\_type\_check](#jc.utils.input_type_check)
* [streaming\_input\_type\_check](#jc.utils.streaming_input_type_check)
* [streaming\_line\_input\_type\_check](#jc.utils.streaming_line_input_type_check)
* [timestamp](#jc.utils.timestamp)
* [\_\_init\_\_](#jc.utils.timestamp.__init__)
@@ -166,73 +161,6 @@ Returns:
True/False False unless a 'truthy' number or string is found
('y', 'yes', 'true', '1', 1, -1, etc.)
<a id="jc.utils.stream_success"></a>
### stream\_success
```python
def stream_success(output_line: Dict, ignore_exceptions: bool) -> Dict
```
Add `_jc_meta` object to output line if `ignore_exceptions=True`
<a id="jc.utils.stream_error"></a>
### stream\_error
```python
def stream_error(e: BaseException, line: str) -> Dict
```
Return an error `_jc_meta` field.
<a id="jc.utils.add_jc_meta"></a>
### add\_jc\_meta
```python
def add_jc_meta(func)
```
Decorator for streaming parsers to add stream_success and stream_error
objects. This simplifies the yield lines in the streaming parsers.
With the decorator on parse():
# successfully parsed line:
yield output_line if raw else _process(output_line)
# unsuccessfully parsed line:
except Exception as e:
if not ignore_exceptions:
e.args = (str(e) + ignore_exceptions_msg,)
raise e
yield e, line
Without the decorator on parse():
# successfully parsed line:
yield stream_success(output_line, ignore_exceptions) if raw else stream_success(_process(output_line), ignore_exceptions)
# unsuccessfully parsed line:
except Exception as e:
if not ignore_exceptions:
e.args = (str(e) + ignore_exceptions_msg,)
raise e
yield stream_error(e, line)
In all cases above:
output_line: (Dict): successfully parsed line yielded as a dict
e: (BaseException): exception object as the first value
of the tuple if the line was not successfully parsed.
line: (str): string of the original line that did not
successfully parse.
<a id="jc.utils.input_type_check"></a>
### input\_type\_check
@@ -243,27 +171,6 @@ def input_type_check(data: str) -> None
Ensure input data is a string. Raises `TypeError` if not.
<a id="jc.utils.streaming_input_type_check"></a>
### streaming\_input\_type\_check
```python
def streaming_input_type_check(data: Iterable) -> None
```
Ensure input data is an iterable, but not a string or bytes. Raises
`TypeError` if not.
<a id="jc.utils.streaming_line_input_type_check"></a>
### streaming\_line\_input\_type\_check
```python
def streaming_line_input_type_check(line: str) -> None
```
Ensure each line is a string. Raises `TypeError` if not.
<a id="jc.utils.timestamp"></a>
### timestamp Objects