1
0
mirror of https://github.com/kellyjonbrazil/jc.git synced 2026-04-03 17:44:07 +02:00

Compare commits

...

26 Commits

Author SHA1 Message Date
Kelly Brazil
0ee244756b spelling 2022-01-03 09:22:35 -08:00
Kelly Brazil
6d5ac9abe6 update docs 2022-01-03 09:19:40 -08:00
Kelly Brazil
44f6d9e132 changelog update 2022-01-03 09:18:43 -08:00
Kelly Brazil
360154559c Merge pull request #193 from kellyjonbrazil/dev
Dev add csv doublequote fix
2022-01-03 09:12:50 -08:00
Kelly Brazil
78672bd7ad Merge pull request #190 from shaikustin/csv-doubleqouted
fix doubleqoute in csv
2022-01-03 09:08:57 -08:00
Kelly Brazil
65d96e26b5 add streaming tests 2022-01-03 09:06:00 -08:00
Kelly Brazil
241d53af9a Merge pull request #192 from kellyjonbrazil/dev
Dev v1.17.6
2022-01-03 08:49:15 -08:00
Kelly Brazil
5563829df2 make dialect sniff behavior match non-streaming parser 2022-01-03 08:48:23 -08:00
Kelly Brazil
3a4a27e1f9 version bump 2022-01-02 11:44:25 -08:00
Kelly Brazil
9c887a36a8 update csv_s parser with csv changes 2022-01-02 11:44:18 -08:00
Kelly Brazil
bc7973af36 update copyright 2022-01-02 11:07:15 -08:00
Kelly Brazil
32972d8fdb doc update 2022-01-02 11:00:52 -08:00
Kelly Brazil
b128d9109c add MANIFEST.MF tests 2022-01-02 10:51:02 -08:00
Kelly Brazil
929d7273a4 doc updates 2022-01-02 10:29:56 -08:00
shaik
2a40f84274 fix doubleqoute in csv 2022-01-02 17:30:25 +02:00
Kelly Brazil
9ff6fa818f add jar-manifest 2022-01-01 10:08:40 -08:00
Kelly Brazil
dac73a4bfe Merge pull request #189 from kellyjonbrazil/master
use github releases instead of packaging site
2022-01-01 10:06:46 -08:00
Kelly Brazil
d7895547f7 Merge pull request #188 from listuser/new_branch
Create key value pairs from a MANIFEST.MF file, to include key multiline value pairs.
2022-01-01 10:04:47 -08:00
listuser
b7d439cb87 Renamed jar-manifest.py to jar_manifest.py, added multi manifest outputs to tests... 2021-12-31 12:15:50 -08:00
listuser
7cc903a5f5 Removed inflating from examples in jar-manifest.py 2021-12-30 15:10:53 -08:00
listuser
c495a8291b Updated test output in MANIFEST.MF.json 2021-12-30 15:06:06 -08:00
listuser
5e1d7d777c Renamed metamf.py to jar-manifest.py, plus other changes 2021-12-30 15:02:48 -08:00
listuser
7edad3f676 Create key value pairs from a MANIFEST.MF file, to include key multiline value pairs. 2021-12-29 12:15:15 -08:00
Kelly Brazil
f7331001d4 use github releases instead of packaging site 2021-12-23 11:35:39 -08:00
Kelly Brazil
f8e09ae2ff Merge pull request #187 from kellyjonbrazil/master
sync to dev
2021-12-21 15:46:39 -06:00
Kelly Brazil
433c7cc0f0 formatting 2021-12-21 13:42:24 -08:00
25 changed files with 2940 additions and 16 deletions

View File

@@ -1,5 +1,9 @@
jc changelog
20220103 v1.17.6
- Add jar-manifest file parser (for MANIFEST.MF files)
- Fix CSV parsers for some files that include doublequotes
20211221 v1.17.5
- Add zipinfo parser tested on linux and macOS

View File

@@ -91,7 +91,10 @@ pip3 install jc
| FreeBSD | `portsnap fetch update && cd /usr/ports/textproc/py-jc && make install clean` |
| Ansible filter plugin | `ansible-galaxy collection install community.general` |
> For more packages and binaries, see the [jc packaging](https://kellyjonbrazil.github.io/jc-packaging/) site.
> For more OS Packages, see https://repology.org/project/jc/versions.
### Binaries and Packages
For precompiled binaries and packages, see [Releases](https://github.com/kellyjonbrazil/jc/releases) on Github.
## Usage
`jc` accepts piped input from `STDIN` and outputs a JSON representation of the previous command's output to `STDOUT`.
@@ -142,6 +145,7 @@ The JSON output can be compact (default) or pretty formatted with the `-p` optio
- `--iostat-s` enables the `iostat` command streaming parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/iostat_s))
- `--iptables` enables the `iptables` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/iptables))
- `--iw-scan` enables the `iw dev [device] scan` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/iw_scan))
- `--jar-manifest` enables the MANIFEST.MF file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/jar_manifest))
- `--jobs` enables the `jobs` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/jobs))
- `--kv` enables the Key/Value file parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/kv))
- `--last` enables the `last` and `lastb` command parser ([documentation](https://kellyjonbrazil.github.io/jc/docs/parsers/last))
@@ -1006,4 +1010,4 @@ cat istio.yaml | jc --yaml -p
]
```
© 2019-2021 Kelly Brazil
© 2019-2022 Kelly Brazil

View File

@@ -99,4 +99,4 @@ Returns:
## Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.3 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.4 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -82,4 +82,4 @@ Returns:
## Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 1.1 by Kelly Brazil (kellyjonbrazil@gmail.com)
Version 1.2 by Kelly Brazil (kellyjonbrazil@gmail.com)

View File

@@ -0,0 +1,104 @@
[Home](https://kellyjonbrazil.github.io/jc/)
# jc.parsers.jar_manifest
jc - JSON CLI output utility `MANIFEST.MF` file parser
Usage (cli):
$ cat MANIFEST.MF | jc --jar-manifest
Usage (module):
import jc.parsers.jar_manifest
result = jc.parsers.jar_manifest.parse(jar_manifest_file_output)
Schema:
[
{
"key1": string,
"key2": string
}
]
Examples:
$ cat MANIFEST.MF | jc --jar-manifest -p
$ unzip -c apache-log4j-2.16.0-bin/log4j-core-2.16.0.jar META-INF/MANIFEST.MF | jc --jar-manifest -p
$ unzip -c 'apache-log4j-2.16.0-bin/*.jar' META-INF/MANIFEST.MF | jc --jar-manifest -p
$ cat MANIFEST.MF | jc --jar-manifest -p
[
{
"Import_Package": "com.conversantmedia.util.concurrent;resolution:=optional,com.fasterxml.jackson.annotation;version="[2.12,3)";resolution:=optional,com.fasterxml.jackson.core;version="[2.12,3)";resolution:=optional,com.fasterxml.jackson.core.type;version="[2.12,3)";resolution:=optional,com.fasterxml.jackson.cor...",
"Export_Package": "org.apache.logging.log4j.core;uses:="org.apache.logging.log4j,org.apache.logging.log4j.core.config,org.apache.logging.log4j.core.impl,org.apache.logging.log4j.core.layout,org.apache.logging.log4j.core.time,org.apache.logging.log4j.message,org.apache.logging.log4j.spi,org.apache.logging.log4j.status...",
"Manifest_Version": "1.0",
"Bundle_License": "https://www.apache.org/licenses/LICENSE-2.0.txt",
"Bundle_SymbolicName": "org.apache.logging.log4j.core",
"Built_By": "matt",
"Bnd_LastModified": "1639373735804",
"Implementation_Vendor_Id": "org.apache.logging.log4j",
"Specification_Title": "Apache Log4j Core",
"Log4jReleaseManager": "Matt Sicker",
...
}
]
$ unzip -c 'apache-log4j-2.16.0-bin/*.jar' META-INF/MANIFEST.MF | jc --jar-manifest -p
[
...
{
"Archive": "apache-log4j-2.16.0-bin/log4j-spring-boot-2.16.0-sources.jar",
"Manifest_Version": "1.0",
"Built_By": "matt",
"Created_By": "Apache Maven 3.8.4",
"Build_Jdk": "1.8.0_312"
},
{
"Archive": "apache-log4j-2.16.0-bin/log4j-spring-boot-2.16.0-javadoc.jar",
"Manifest_Version": "1.0",
"Built_By": "matt",
"Created_By": "Apache Maven 3.8.4",
"Build_Jdk": "1.8.0_312"
},
{
"Bundle_SymbolicName": "org.apache.logging.log4j.spring-cloud-config-client.logging.log4j.core.util;version="[2.16,3)",org.springframework.boot.autoconfigure.condition,org.springframework.cloud.context.environment,org.springframework.context,org.springframework.stereotype",
"Export_Package": "org.apache.logging.log4j.spring.cloud.config.controller;version="2.16.0"ient",
"Archive": "apache-log4j-2.16.0-bin/log4j-spring-cloud-config-client-2.16.0.jar",
"Manifest_Version": "1.0",
"Bundle_License": "https://www.apache.org/licenses/LICENSE-2.0.txt",
...
}
...
]
## info
```python
info()
```
Provides parser metadata (version, author, etc.)
## parse
```python
parse(data, raw=False, quiet=False)
```
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
## Parser Information
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
Version 0.01 by Matt J (https://github.com/listuser)

View File

@@ -22,6 +22,7 @@ Usage (module):
result = jc.parsers.zipinfo.parse(zipinfo_command_output)
Schema:
[
{
"archive": string,

View File

@@ -73,4 +73,4 @@ Module Example:
"""
name = 'jc'
__version__ = '1.17.5'
__version__ = '1.17.6'

View File

@@ -37,7 +37,7 @@ class info():
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
website = 'https://github.com/kellyjonbrazil/jc'
copyright = '© 2019-2021 Kelly Brazil'
copyright = '© 2019-2022 Kelly Brazil'
license = 'MIT License'
@@ -80,6 +80,7 @@ parsers = [
'iostat-s',
'iptables',
'iw-scan',
'jar-manifest',
'jobs',
'kv',
'last',

View File

@@ -75,7 +75,7 @@ import csv
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.3'
version = '1.4'
description = 'CSV file parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -130,9 +130,11 @@ def parse(data, raw=False, quiet=False):
if jc.utils.has_data(data):
dialect = None
dialect = 'excel' # default in csv module
try:
dialect = csv.Sniffer().sniff(data[:1024])
if '""' in data:
dialect.doublequote = True
except Exception:
pass

View File

@@ -56,7 +56,7 @@ from jc.exceptions import ParseError
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '1.1'
version = '1.2'
description = 'CSV file streaming parser'
author = 'Kelly Brazil'
author_email = 'kellyjonbrazil@gmail.com'
@@ -113,18 +113,20 @@ def parse(data, raw=False, quiet=False, ignore_exceptions=False):
# first, load the first 100 lines into a list to detect the CSV dialect
for line in itertools.islice(data, 100):
temp_list.append(line)
temp_list.append(line.rstrip())
# check for Python bug that does not split on `\r` newlines from sys.stdin correctly
# https://bugs.python.org/issue45617
if len(temp_list) == 1:
raise ParseError('Unable to detect line endings. Please try the non-streaming CSV parser instead.')
sniffdata = '\n'.join(temp_list)
sniffdata = '\n'.join(temp_list)[:1024]
dialect = 'excel' # default in csv module
dialect = None
try:
dialect = csv.Sniffer().sniff(sniffdata)
if '""' in sniffdata:
dialect.doublequote = True
except Exception:
pass

215
jc/parsers/jar_manifest.py Normal file
View File

@@ -0,0 +1,215 @@
"""jc - JSON CLI output utility `MANIFEST.MF` file parser
Usage (cli):
$ cat MANIFEST.MF | jc --jar-manifest
Usage (module):
import jc.parsers.jar_manifest
result = jc.parsers.jar_manifest.parse(jar_manifest_file_output)
Schema:
[
{
"key1": string,
"key2": string
}
]
Examples:
$ cat MANIFEST.MF | jc --jar-manifest -p
$ unzip -c apache-log4j-2.16.0-bin/log4j-core-2.16.0.jar META-INF/MANIFEST.MF | jc --jar-manifest -p
$ unzip -c 'apache-log4j-2.16.0-bin/*.jar' META-INF/MANIFEST.MF | jc --jar-manifest -p
$ cat MANIFEST.MF | jc --jar-manifest -p
[
{
"Import_Package": "com.conversantmedia.util.concurrent;resolution:=optional,com.fasterxml.jackson.annotation;version=\"[2.12,3)\";resolution:=optional,com.fasterxml.jackson.core;version=\"[2.12,3)\";resolution:=optional,com.fasterxml.jackson.core.type;version=\"[2.12,3)\";resolution:=optional,com.fasterxml.jackson.cor...",
"Export_Package": "org.apache.logging.log4j.core;uses:=\"org.apache.logging.log4j,org.apache.logging.log4j.core.config,org.apache.logging.log4j.core.impl,org.apache.logging.log4j.core.layout,org.apache.logging.log4j.core.time,org.apache.logging.log4j.message,org.apache.logging.log4j.spi,org.apache.logging.log4j.status...",
"Manifest_Version": "1.0",
"Bundle_License": "https://www.apache.org/licenses/LICENSE-2.0.txt",
"Bundle_SymbolicName": "org.apache.logging.log4j.core",
"Built_By": "matt",
"Bnd_LastModified": "1639373735804",
"Implementation_Vendor_Id": "org.apache.logging.log4j",
"Specification_Title": "Apache Log4j Core",
"Log4jReleaseManager": "Matt Sicker",
...
}
]
$ unzip -c 'apache-log4j-2.16.0-bin/*.jar' META-INF/MANIFEST.MF | jc --jar-manifest -p
[
...
{
"Archive": "apache-log4j-2.16.0-bin/log4j-spring-boot-2.16.0-sources.jar",
"Manifest_Version": "1.0",
"Built_By": "matt",
"Created_By": "Apache Maven 3.8.4",
"Build_Jdk": "1.8.0_312"
},
{
"Archive": "apache-log4j-2.16.0-bin/log4j-spring-boot-2.16.0-javadoc.jar",
"Manifest_Version": "1.0",
"Built_By": "matt",
"Created_By": "Apache Maven 3.8.4",
"Build_Jdk": "1.8.0_312"
},
{
"Bundle_SymbolicName": "org.apache.logging.log4j.spring-cloud-config-client.logging.log4j.core.util;version=\"[2.16,3)\",org.springframework.boot.autoconfigure.condition,org.springframework.cloud.context.environment,org.springframework.context,org.springframework.stereotype",
"Export_Package": "org.apache.logging.log4j.spring.cloud.config.controller;version=\"2.16.0\"ient",
"Archive": "apache-log4j-2.16.0-bin/log4j-spring-cloud-config-client-2.16.0.jar",
"Manifest_Version": "1.0",
"Bundle_License": "https://www.apache.org/licenses/LICENSE-2.0.txt",
...
}
...
]
"""
import jc.utils
import re
class info():
"""Provides parser metadata (version, author, etc.)"""
version = '0.01'
description = 'MANIFEST.MF file parser'
author = 'Matt J'
author_email = 'https://github.com/listuser'
compatible = ['linux', 'darwin', 'cygwin', 'win32', 'aix', 'freebsd']
__version__ = info.version
def _process(proc_data):
"""
Final processing to conform to the schema.
Parameters:
proc_data: (List of Dictionaries) raw structured data to process
Returns:
List of Dictionaries. Structured data to conform to the schema.
"""
return proc_data
def parse(data, raw=False, quiet=False):
"""
Main text parsing function
Parameters:
data: (string) text data to parse
raw: (boolean) output preprocessed JSON if True
quiet: (boolean) suppress warning messages if True
Returns:
List of Dictionaries. Raw or processed structured data.
"""
jc.utils.compatibility(__name__, info.compatible, quiet)
jc.utils.input_type_check(data)
raw_output = []
archives = []
if jc.utils.has_data(data):
datalines = data.splitlines()
# remove last line of multi-archive output since it is not needed
if datalines[-1].endswith('archives were successfully processed.'):
datalines.pop(-1)
# extract each archive into its own list of lines.
# archives are separated by a blank line
this_archive = []
for row in datalines:
if row == '':
archives.append(this_archive)
this_archive = []
continue
this_archive.append(row)
if this_archive:
archives.append(this_archive)
# iterate through list of archives and parse
for archive_item in archives:
manifests = []
this_manifest = {}
plines = []
for i, line in enumerate(archive_item):
last = archive_item[-1]
# remove line since it is not needed and starts with "space"
if (re.match(r'^\s+inflating\s*:\s*META-INF/MANIFEST.MF', line, re.IGNORECASE)):
archive_item.pop(i)
continue
# if line starts with "space"
# begin key multiline value pair concatenation
if (re.match(r'\s', line)):
# expectation is this "if" sets a key once
if (not this_manifest):
# previous line contains a key
k, v = archive_item[i - 1].split(":", maxsplit=1)
v = v + line
v = re.sub(r'\s', '', v)
this_manifest = {k: v}
plines.append(i - 1)
plines.append(i)
# continue key multiline value pair concatenation
else:
plines.append(i)
linecmp = line
for k, v in this_manifest.items():
line = v + line
line = re.sub(r'\s', '', line)
this_manifest.update({k:line})
if linecmp is not last:
nextline = archive_item[i + 1]
# if next line starts with not "space",
# end key multiline value pair concatenation
if (re.match(r'\S', nextline)):
manifests.append(this_manifest)
this_manifest = False
else:
manifests.append(this_manifest)
# pop key multiline value pair lines
if plines:
for p in reversed(plines):
archive_item.pop(p)
# all other key value pairs
for i, line in enumerate(archive_item):
k, v = line.split(":", maxsplit=1)
v = v.strip()
manifests.append({k: v})
if manifests:
this_manifest = {}
for d in manifests:
for k, v in d.items():
k = re.sub(r'\s', '', k)
k = re.sub(r'-', '_', k)
this_manifest.update({k: v})
raw_output.append(this_manifest)
return raw_output if raw else _process(raw_output)

View File

@@ -19,6 +19,7 @@ Usage (module):
result = jc.parsers.zipinfo.parse(zipinfo_command_output)
Schema:
[
{
"archive": string,

View File

@@ -1,4 +1,4 @@
.TH jc 1 2021-12-21 1.17.5 "JSON CLI output utility"
.TH jc 1 2022-01-03 1.17.6 "JSON CLI output utility"
.SH NAME
jc \- JSONifies the output of many CLI tools and file-types
.SH SYNOPSIS
@@ -197,6 +197,11 @@ INI file parser
\fB--iw-scan\fP
`iw dev [device] scan` command parser
.TP
.B
\fB--jar-manifest\fP
MANIFEST.MF file parser
.TP
.B
\fB--jobs\fP
@@ -639,6 +644,6 @@ Kelly Brazil (kellyjonbrazil@gmail.com)
https://github.com/kellyjonbrazil/jc
.SH COPYRIGHT
Copyright (c) 2019-2021 Kelly Brazil
Copyright (c) 2019-2022 Kelly Brazil
License: MIT License

View File

@@ -5,7 +5,7 @@ with open('README.md', 'r') as f:
setuptools.setup(
name='jc',
version='1.17.5',
version='1.17.6',
author='Kelly Brazil',
author_email='kellyjonbrazil@gmail.com',
description='Converts the output of popular command-line tools and file-types to JSON.',

View File

@@ -91,7 +91,10 @@ pip3 install jc
| FreeBSD | `portsnap fetch update && cd /usr/ports/textproc/py-jc && make install clean` |
| Ansible filter plugin | `ansible-galaxy collection install community.general` |
> For more packages and binaries, see the [jc packaging](https://kellyjonbrazil.github.io/jc-packaging/) site.
> For more OS Packages, see https://repology.org/project/jc/versions.
### Binaries and Packages
For precompiled binaries and packages, see [Releases](https://github.com/kellyjonbrazil/jc/releases) on Github.
## Usage
`jc` accepts piped input from `STDIN` and outputs a JSON representation of the previous command's output to `STDOUT`.

View File

@@ -0,0 +1 @@
[{"A":"1","B":"this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this"},{"A":"2","B":"this is a field with \" in it\""}]

View File

@@ -0,0 +1,3 @@
A,B
1,"this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this"
2,"this is a field with "" in it"
1 A B
2 1 this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this
3 2 this is a field with " in it

View File

@@ -0,0 +1,4 @@
[
{"A": "1", "B": "this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this is 1024 bytes long field this"},
{"A": "2", "B": "this is a field with \" in it"}
]

File diff suppressed because one or more lines are too long

File diff suppressed because it is too large Load Diff

File diff suppressed because one or more lines are too long

292
tests/fixtures/rhel-8/MANIFEST.MF.out vendored Normal file
View File

@@ -0,0 +1,292 @@
Manifest-Version: 1.0
Bundle-License: https://www.apache.org/licenses/LICENSE-2.0.txt
Bundle-SymbolicName: org.apache.logging.log4j.core
Built-By: rgoers
Bnd-LastModified: 1639792304782
Implementation-Vendor-Id: org.apache.logging.log4j
Specification-Title: Apache Log4j Core
Log4jReleaseManager: Ralph Goers
Bundle-DocURL: https://www.apache.org/
Import-Package: com.conversantmedia.util.concurrent;resolution:=option
al,com.fasterxml.jackson.annotation;version="[2.12,3)";resolution:=op
tional,com.fasterxml.jackson.core;version="[2.12,3)";resolution:=opti
onal,com.fasterxml.jackson.core.type;version="[2.12,3)";resolution:=o
ptional,com.fasterxml.jackson.core.util;version="[2.12,3)";resolution
:=optional,com.fasterxml.jackson.databind;version="[2.12,3)";resoluti
on:=optional,com.fasterxml.jackson.databind.annotation;version="[2.12
,3)";resolution:=optional,com.fasterxml.jackson.databind.deser.std;ve
rsion="[2.12,3)";resolution:=optional,com.fasterxml.jackson.databind.
module;version="[2.12,3)";resolution:=optional,com.fasterxml.jackson.
databind.node;version="[2.12,3)";resolution:=optional,com.fasterxml.j
ackson.databind.ser;version="[2.12,3)";resolution:=optional,com.faste
rxml.jackson.databind.ser.impl;version="[2.12,3)";resolution:=optiona
l,com.fasterxml.jackson.databind.ser.std;version="[2.12,3)";resolutio
n:=optional,com.fasterxml.jackson.dataformat.xml;version="[2.12,3)";r
esolution:=optional,com.fasterxml.jackson.dataformat.xml.annotation;v
ersion="[2.12,3)";resolution:=optional,com.fasterxml.jackson.dataform
at.xml.util;version="[2.12,3)";resolution:=optional,com.fasterxml.jac
kson.dataformat.yaml;version="[2.12,3)";resolution:=optional,com.lmax
.disruptor;version="[3.4,4)";resolution:=optional,com.lmax.disruptor.
dsl;version="[3.4,4)";resolution:=optional,javax.activation;version="
[1.2,2)";resolution:=optional,javax.annotation.processing,javax.crypt
o,javax.jms;version="[1.1,2)";resolution:=optional,javax.lang.model,j
avax.lang.model.element,javax.lang.model.util,javax.mail;version="[1.
6,2)";resolution:=optional,javax.mail.internet;version="[1.6,2)";reso
lution:=optional,javax.mail.util;version="[1.6,2)";resolution:=option
al,javax.management,javax.naming,javax.net,javax.net.ssl,javax.script
,javax.sql,javax.tools,javax.xml.parsers,javax.xml.stream,javax.xml.t
ransform,javax.xml.transform.stream,javax.xml.validation,org.apache.c
ommons.compress.compressors;version="[1.21,2)";resolution:=optional,o
rg.apache.commons.compress.utils;version="[1.21,2)";resolution:=optio
nal,org.apache.commons.csv;version="[1.9,2)";resolution:=optional,org
.apache.kafka.clients.producer;resolution:=optional,org.apache.loggin
g.log4j;version="[2.17,3)",org.apache.logging.log4j.core,org.apache.l
ogging.log4j.core.appender,org.apache.logging.log4j.core.appender.db,
org.apache.logging.log4j.core.appender.rewrite,org.apache.logging.log
4j.core.appender.rolling,org.apache.logging.log4j.core.appender.rolli
ng.action,org.apache.logging.log4j.core.async,org.apache.logging.log4
j.core.config,org.apache.logging.log4j.core.config.arbiters,org.apach
e.logging.log4j.core.config.builder.api,org.apache.logging.log4j.core
.config.builder.impl,org.apache.logging.log4j.core.config.composite,o
rg.apache.logging.log4j.core.config.json,org.apache.logging.log4j.cor
e.config.plugins,org.apache.logging.log4j.core.config.plugins.convert
,org.apache.logging.log4j.core.config.plugins.processor,org.apache.lo
gging.log4j.core.config.plugins.util,org.apache.logging.log4j.core.co
nfig.plugins.validation,org.apache.logging.log4j.core.config.plugins.
validation.constraints,org.apache.logging.log4j.core.config.plugins.v
alidation.validators,org.apache.logging.log4j.core.config.plugins.vis
itors,org.apache.logging.log4j.core.config.status,org.apache.logging.
log4j.core.filter,org.apache.logging.log4j.core.impl,org.apache.loggi
ng.log4j.core.jackson,org.apache.logging.log4j.core.jmx,org.apache.lo
gging.log4j.core.layout,org.apache.logging.log4j.core.layout.internal
,org.apache.logging.log4j.core.lookup,org.apache.logging.log4j.core.n
et,org.apache.logging.log4j.core.net.ssl,org.apache.logging.log4j.cor
e.pattern,org.apache.logging.log4j.core.script,org.apache.logging.log
4j.core.selector,org.apache.logging.log4j.core.time,org.apache.loggin
g.log4j.core.tools.picocli,org.apache.logging.log4j.core.util,org.apa
che.logging.log4j.core.util.datetime,org.apache.logging.log4j.message
;version="[2.17,3)",org.apache.logging.log4j.spi;version="[2.17,3)",o
rg.apache.logging.log4j.status;version="[2.17,3)",org.apache.logging.
log4j.util;version="[2.17,3)",org.codehaus.stax2;version="[4.2,5)";re
solution:=optional,org.fusesource.jansi;version="[2.3,3)";resolution:
=optional,org.jctools.queues;resolution:=optional,org.osgi.framework;
version="[1.6,2)",org.osgi.framework.wiring;version="[1.0,2)",org.w3c
.dom,org.xml.sax,org.zeromq;version="[0.4,1)";resolution:=optional,su
n.reflect;resolution:=optional
Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.8))"
Export-Package: org.apache.logging.log4j.core;uses:="org.apache.loggin
g.log4j,org.apache.logging.log4j.core.config,org.apache.logging.log4j
.core.impl,org.apache.logging.log4j.core.layout,org.apache.logging.lo
g4j.core.time,org.apache.logging.log4j.message,org.apache.logging.log
4j.spi,org.apache.logging.log4j.status,org.apache.logging.log4j.util"
;version="2.17.0",org.apache.logging.log4j.core.appender;uses:="org.a
pache.logging.log4j,org.apache.logging.log4j.core,org.apache.logging.
log4j.core.appender.rolling,org.apache.logging.log4j.core.async,org.a
pache.logging.log4j.core.config,org.apache.logging.log4j.core.config.
plugins,org.apache.logging.log4j.core.config.plugins.validation.const
raints,org.apache.logging.log4j.core.filter,org.apache.logging.log4j.
core.impl,org.apache.logging.log4j.core.layout,org.apache.logging.log
4j.core.net,org.apache.logging.log4j.core.net.ssl,org.apache.logging.
log4j.core.script,org.apache.logging.log4j.core.util,org.apache.loggi
ng.log4j.status";version="2.17.0",org.apache.logging.log4j.core.appen
der.db;uses:="org.apache.logging.log4j.core,org.apache.logging.log4j.
core.appender,org.apache.logging.log4j.core.config,org.apache.logging
.log4j.core.config.plugins,org.apache.logging.log4j.core.util";versio
n="2.17.0",org.apache.logging.log4j.core.appender.db.jdbc;uses:="org.
apache.logging.log4j,org.apache.logging.log4j.core,org.apache.logging
.log4j.core.appender.db,org.apache.logging.log4j.core.config,org.apac
he.logging.log4j.core.config.plugins,org.apache.logging.log4j.core.co
nfig.plugins.validation.constraints,org.apache.logging.log4j.core.lay
out,org.apache.logging.log4j.core.util";version="2.17.0",org.apache.l
ogging.log4j.core.appender.mom;uses:="javax.jms,org.apache.logging.lo
g4j.core,org.apache.logging.log4j.core.appender,org.apache.logging.lo
g4j.core.config,org.apache.logging.log4j.core.config.plugins,org.apac
he.logging.log4j.core.net,org.apache.logging.log4j.core.util";version
="2.17.0",org.apache.logging.log4j.core.appender.mom.jeromq;uses:="or
g.apache.logging.log4j.core,org.apache.logging.log4j.core.appender,or
g.apache.logging.log4j.core.config,org.apache.logging.log4j.core.conf
ig.plugins,org.apache.logging.log4j.core.config.plugins.validation.co
nstraints,org.zeromq";version="2.17.0",org.apache.logging.log4j.core.
appender.mom.kafka;uses:="org.apache.kafka.clients.producer,org.apach
e.logging.log4j.core,org.apache.logging.log4j.core.appender,org.apach
e.logging.log4j.core.config,org.apache.logging.log4j.core.config.plug
ins,org.apache.logging.log4j.core.util";version="2.17.0",org.apache.l
ogging.log4j.core.appender.nosql;uses:="org.apache.logging.log4j.core
,org.apache.logging.log4j.core.appender,org.apache.logging.log4j.core
.appender.db,org.apache.logging.log4j.core.config.plugins,org.apache.
logging.log4j.core.util";version="2.17.0",org.apache.logging.log4j.co
re.appender.rewrite;uses:="org.apache.logging.log4j,org.apache.loggin
g.log4j.core,org.apache.logging.log4j.core.appender,org.apache.loggin
g.log4j.core.config,org.apache.logging.log4j.core.config.plugins,org.
apache.logging.log4j.core.util";version="2.17.0",org.apache.logging.l
og4j.core.appender.rolling;uses:="org.apache.logging.log4j,org.apache
.logging.log4j.core,org.apache.logging.log4j.core.appender,org.apache
.logging.log4j.core.appender.rolling.action,org.apache.logging.log4j.
core.config,org.apache.logging.log4j.core.config.plugins,org.apache.l
ogging.log4j.core.lookup,org.apache.logging.log4j.core.util";version=
"2.17.0",org.apache.logging.log4j.core.appender.rolling.action;uses:=
"org.apache.logging.log4j,org.apache.logging.log4j.core.config,org.ap
ache.logging.log4j.core.config.plugins,org.apache.logging.log4j.core.
lookup,org.apache.logging.log4j.core.script,org.apache.logging.log4j.
core.util";version="2.17.0",org.apache.logging.log4j.core.appender.ro
uting;uses:="org.apache.logging.log4j.core,org.apache.logging.log4j.c
ore.appender,org.apache.logging.log4j.core.appender.rewrite,org.apach
e.logging.log4j.core.config,org.apache.logging.log4j.core.config.plug
ins,org.apache.logging.log4j.core.script,org.apache.logging.log4j.cor
e.util";version="2.17.0",org.apache.logging.log4j.core.async;uses:="c
om.conversantmedia.util.concurrent,com.lmax.disruptor,org.apache.logg
ing.log4j,org.apache.logging.log4j.core,org.apache.logging.log4j.core
.appender,org.apache.logging.log4j.core.config,org.apache.logging.log
4j.core.config.plugins,org.apache.logging.log4j.core.config.plugins.v
alidation.constraints,org.apache.logging.log4j.core.impl,org.apache.l
ogging.log4j.core.jmx,org.apache.logging.log4j.core.selector,org.apac
he.logging.log4j.core.time,org.apache.logging.log4j.core.util,org.apa
che.logging.log4j.message,org.apache.logging.log4j.util";version="2.1
7.0",org.apache.logging.log4j.core.config;uses:="org.apache.logging.l
og4j,org.apache.logging.log4j.core,org.apache.logging.log4j.core.asyn
c,org.apache.logging.log4j.core.config.builder.api,org.apache.logging
.log4j.core.config.plugins,org.apache.logging.log4j.core.config.plugi
ns.util,org.apache.logging.log4j.core.config.plugins.validation.const
raints,org.apache.logging.log4j.core.filter,org.apache.logging.log4j.
core.impl,org.apache.logging.log4j.core.lookup,org.apache.logging.log
4j.core.net,org.apache.logging.log4j.core.script,org.apache.logging.l
og4j.core.util,org.apache.logging.log4j.message,org.apache.logging.lo
g4j.util";version="2.17.0",org.apache.logging.log4j.core.config.arbit
ers;uses:="org.apache.logging.log4j.core.config,org.apache.logging.lo
g4j.core.config.plugins,org.apache.logging.log4j.core.util";version="
2.17.0",org.apache.logging.log4j.core.config.builder.api;uses:="org.a
pache.logging.log4j,org.apache.logging.log4j.core,org.apache.logging.
log4j.core.config,org.apache.logging.log4j.core.config.builder.impl,o
rg.apache.logging.log4j.core.util";version="2.17.0",org.apache.loggin
g.log4j.core.config.builder.impl;uses:="javax.xml.transform,org.apach
e.logging.log4j,org.apache.logging.log4j.core,org.apache.logging.log4
j.core.config,org.apache.logging.log4j.core.config.builder.api,org.ap
ache.logging.log4j.core.config.plugins.util,org.apache.logging.log4j.
core.config.status";version="2.17.0",org.apache.logging.log4j.core.co
nfig.composite;uses:="org.apache.logging.log4j.core.config,org.apache
.logging.log4j.core.config.plugins.util";version="2.17.0",org.apache.
logging.log4j.core.config.json;uses:="com.fasterxml.jackson.databind,
org.apache.logging.log4j.core,org.apache.logging.log4j.core.config,or
g.apache.logging.log4j.core.config.plugins";version="2.17.0",org.apac
he.logging.log4j.core.config.plugins;uses:="org.apache.logging.log4j.
core.config.plugins.visitors";version="2.17.0",org.apache.logging.log
4j.core.config.plugins.convert;uses:="org.apache.logging.log4j,org.ap
ache.logging.log4j.core.appender.rolling.action,org.apache.logging.lo
g4j.core.config.plugins,org.apache.logging.log4j.core.util";version="
2.17.0",org.apache.logging.log4j.core.config.plugins.processor;uses:=
"javax.annotation.processing,javax.lang.model,javax.lang.model.elemen
t";version="2.17.0",org.apache.logging.log4j.core.config.plugins.util
;uses:="org.apache.logging.log4j.core,org.apache.logging.log4j.core.c
onfig,org.apache.logging.log4j.core.config.plugins.processor,org.apac
he.logging.log4j.core.util";version="2.17.0",org.apache.logging.log4j
.core.config.plugins.validation;version="2.17.0",org.apache.logging.l
og4j.core.config.plugins.validation.constraints;uses:="org.apache.log
ging.log4j.core.config.plugins.validation,org.apache.logging.log4j.co
re.config.plugins.validation.validators";version="2.17.0",org.apache.
logging.log4j.core.config.plugins.validation.validators;uses:="org.ap
ache.logging.log4j.core.config.plugins.validation,org.apache.logging.
log4j.core.config.plugins.validation.constraints";version="2.17.0",or
g.apache.logging.log4j.core.config.plugins.visitors;uses:="org.apache
.logging.log4j,org.apache.logging.log4j.core,org.apache.logging.log4j
.core.config,org.apache.logging.log4j.core.config.plugins,org.apache.
logging.log4j.core.lookup";version="2.17.0",org.apache.logging.log4j.
core.config.properties;uses:="org.apache.logging.log4j.core,org.apach
e.logging.log4j.core.config,org.apache.logging.log4j.core.config.buil
der.api,org.apache.logging.log4j.core.config.builder.impl,org.apache.
logging.log4j.core.config.plugins,org.apache.logging.log4j.core.util"
;version="2.17.0",org.apache.logging.log4j.core.config.status;uses:="
org.apache.logging.log4j";version="2.17.0",org.apache.logging.log4j.c
ore.config.xml;uses:="org.apache.logging.log4j.core,org.apache.loggin
g.log4j.core.config,org.apache.logging.log4j.core.config.plugins";ver
sion="2.17.0",org.apache.logging.log4j.core.config.yaml;uses:="com.fa
sterxml.jackson.databind,org.apache.logging.log4j.core,org.apache.log
ging.log4j.core.config,org.apache.logging.log4j.core.config.json,org.
apache.logging.log4j.core.config.plugins";version="2.17.0",org.apache
.logging.log4j.core.filter;uses:="org.apache.logging.log4j,org.apache
.logging.log4j.core,org.apache.logging.log4j.core.config,org.apache.l
ogging.log4j.core.config.plugins,org.apache.logging.log4j.core.script
,org.apache.logging.log4j.core.util,org.apache.logging.log4j.message,
org.apache.logging.log4j.util";version="2.17.0",org.apache.logging.lo
g4j.core.impl;uses:="org.apache.logging.log4j,org.apache.logging.log4
j.core,org.apache.logging.log4j.core.config,org.apache.logging.log4j.
core.pattern,org.apache.logging.log4j.core.selector,org.apache.loggin
g.log4j.core.time,org.apache.logging.log4j.core.util,org.apache.loggi
ng.log4j.message,org.apache.logging.log4j.spi,org.apache.logging.log4
j.util";version="2.17.0",org.apache.logging.log4j.core.jackson;uses:=
"com.fasterxml.jackson.core,com.fasterxml.jackson.databind,com.faster
xml.jackson.databind.deser.std,com.fasterxml.jackson.databind.ser.std
,com.fasterxml.jackson.dataformat.xml,com.fasterxml.jackson.dataforma
t.yaml,org.apache.logging.log4j.message,org.apache.logging.log4j.util
";version="2.17.0",org.apache.logging.log4j.core.jmx;uses:="com.lmax.
disruptor,javax.management,org.apache.logging.log4j,org.apache.loggin
g.log4j.core,org.apache.logging.log4j.core.appender,org.apache.loggin
g.log4j.core.config,org.apache.logging.log4j.core.selector,org.apache
.logging.log4j.status";version="2.17.0",org.apache.logging.log4j.core
.layout;uses:="com.fasterxml.jackson.annotation,com.fasterxml.jackson
.core,com.fasterxml.jackson.databind,com.fasterxml.jackson.dataformat
.xml.annotation,org.apache.commons.csv,org.apache.logging.log4j,org.a
pache.logging.log4j.core,org.apache.logging.log4j.core.config,org.apa
che.logging.log4j.core.config.plugins,org.apache.logging.log4j.core.i
mpl,org.apache.logging.log4j.core.net,org.apache.logging.log4j.core.p
attern,org.apache.logging.log4j.core.script,org.apache.logging.log4j.
core.util,org.apache.logging.log4j.message";version="2.17.0",org.apac
he.logging.log4j.core.layout.internal;version="2.17.0",org.apache.log
ging.log4j.core.lookup;uses:="org.apache.logging.log4j.core,org.apach
e.logging.log4j.core.config,org.apache.logging.log4j.core.config.plug
ins";version="2.17.0",org.apache.logging.log4j.core.message;uses:="or
g.apache.logging.log4j.message";version="2.17.0",org.apache.logging.l
og4j.core.net;uses:="javax.mail,javax.mail.internet,javax.naming,org.
apache.logging.log4j,org.apache.logging.log4j.core,org.apache.logging
.log4j.core.appender,org.apache.logging.log4j.core.config,org.apache.
logging.log4j.core.config.plugins,org.apache.logging.log4j.core.net.s
sl,org.apache.logging.log4j.core.util";version="2.17.0",org.apache.lo
gging.log4j.core.net.ssl;uses:="javax.net.ssl,org.apache.logging.log4
j.core.config.plugins,org.apache.logging.log4j.status";version="2.17.
0",org.apache.logging.log4j.core.osgi;uses:="org.apache.logging.log4j
.core,org.apache.logging.log4j.core.selector,org.osgi.framework";vers
ion="2.17.0",org.apache.logging.log4j.core.parser;uses:="org.apache.l
ogging.log4j.core";version="2.17.0",org.apache.logging.log4j.core.pat
tern;uses:="org.apache.logging.log4j,org.apache.logging.log4j.core,or
g.apache.logging.log4j.core.config,org.apache.logging.log4j.core.conf
ig.plugins,org.apache.logging.log4j.core.impl,org.apache.logging.log4
j.core.time,org.apache.logging.log4j.message,org.fusesource.jansi";ve
rsion="2.17.0",org.apache.logging.log4j.core.script;uses:="javax.scri
pt,org.apache.logging.log4j,org.apache.logging.log4j.core.config,org.
apache.logging.log4j.core.config.plugins,org.apache.logging.log4j.cor
e.util";version="2.17.0",org.apache.logging.log4j.core.selector;uses:
="org.apache.logging.log4j.core,org.apache.logging.log4j.spi,org.apac
he.logging.log4j.status";version="2.17.0",org.apache.logging.log4j.co
re.time;uses:="org.apache.logging.log4j.core.util,org.apache.logging.
log4j.util";version="2.17.0",org.apache.logging.log4j.core.time.inter
nal;uses:="org.apache.logging.log4j.core.time";version="2.17.0",org.a
pache.logging.log4j.core.tools;version="2.17.0",org.apache.logging.lo
g4j.core.tools.picocli;version="2.17.0",org.apache.logging.log4j.core
.util;uses:="javax.crypto,javax.naming,org.apache.logging.log4j,org.a
pache.logging.log4j.core,org.apache.logging.log4j.core.config,org.apa
che.logging.log4j.core.config.plugins,org.apache.logging.log4j.util";
version="2.17.0",org.apache.logging.log4j.core.util.datetime;uses:="o
rg.apache.logging.log4j.core.time";version="2.17.0"
Bundle-Name: Apache Log4j Core
Log4jReleaseVersionJava6: 2.3
Multi-Release: true
Bundle-Activator: org.apache.logging.log4j.core.osgi.Activator
Log4jReleaseVersionJava7: 2.12.2
Log4jReleaseVersion: 2.17.0
Implementation-Title: Apache Log4j Core
Bundle-Description: The Apache Log4j Implementation
Automatic-Module-Name: org.apache.logging.log4j.core
Implementation-Version: 2.17.0
Specification-Vendor: The Apache Software Foundation
Bundle-ManifestVersion: 2
Bundle-Vendor: The Apache Software Foundation
Tool: Bnd-3.5.0.201709291849
Implementation-Vendor: The Apache Software Foundation
Bundle-Version: 2.17.0
X-Compile-Target-JDK: 1.8
X-Compile-Source-JDK: 1.8
Created-By: Apache Maven Bundle Plugin
Build-Jdk: 1.8.0_144
Specification-Version: 2.17.0
Implementation-URL: https://logging.apache.org/log4j/2.x/log4j-core/
Log4jReleaseKey: B3D8E1BA

View File

@@ -37,6 +37,9 @@ class MyTests(unittest.TestCase):
with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/generic/csv-insurance.csv'), 'r', encoding='utf-8') as f:
self.generic_csv_insurance = f.read()
with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/generic/csv-doubleqouted.csv'), 'r', encoding='utf-8') as f:
self.generic_csv_doubleqouted = f.read()
# output
with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/generic/csv-biostats.json'), 'r', encoding='utf-8') as f:
self.generic_csv_biostats_json = json.loads(f.read())
@@ -65,6 +68,9 @@ class MyTests(unittest.TestCase):
with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/generic/csv-insurance.json'), 'r', encoding='utf-8') as f:
self.generic_csv_insurance_json = json.loads(f.read())
with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/generic/csv-doubleqouted.json'), 'r', encoding='utf-8') as f:
self.generic_csv_doubleqouted_json = json.loads(f.read())
def test_csv_nodata(self):
"""
Test with no data
@@ -125,6 +131,12 @@ class MyTests(unittest.TestCase):
"""
self.assertEqual(jc.parsers.csv.parse(self.generic_csv_insurance, quiet=True), self.generic_csv_insurance_json)
def test_doubleqouted(self):
"""
Test 'csv-doubleqouted.csv' file
"""
self.assertEqual(jc.parsers.csv.parse(self.generic_csv_doubleqouted, quiet=True), self.generic_csv_doubleqouted_json)
if __name__ == '__main__':
unittest.main()

View File

@@ -42,6 +42,9 @@ class MyTests(unittest.TestCase):
with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/generic/csv-10k-sales-records.csv'), 'r', encoding='utf-8') as f:
self.generic_csv_10k_sales_records = f.read()
with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/generic/csv-doubleqouted.csv'), 'r', encoding='utf-8') as f:
self.generic_csv_doubleqouted = f.read()
# output
with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/generic/csv-biostats-streaming.json'), 'r', encoding='utf-8') as f:
self.generic_csv_biostats_streaming_json = json.loads(f.read())
@@ -70,6 +73,9 @@ class MyTests(unittest.TestCase):
with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/generic/csv-10k-sales-records-streaming.json'), 'r', encoding='utf-8') as f:
self.generic_csv_10k_sales_records_streaming_json = json.loads(f.read())
with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/generic/csv-doubleqouted-streaming.json'), 'r', encoding='utf-8') as f:
self.generic_csv_doublequoted_streaming_json = json.loads(f.read())
def test_csv_s_nodata(self):
"""
Test CSV parser with no data
@@ -141,6 +147,12 @@ class MyTests(unittest.TestCase):
"""
self.assertEqual(list(jc.parsers.csv_s.parse(self.generic_csv_10k_sales_records.splitlines(), quiet=True)), self.generic_csv_10k_sales_records_streaming_json)
def test_csv_s_doublequoted(self):
"""
Test 'doublequoted.csv' file
"""
self.assertEqual(list(jc.parsers.csv_s.parse(self.generic_csv_doubleqouted.splitlines(), quiet=True)), self.generic_csv_doublequoted_streaming_json)
if __name__ == '__main__':
unittest.main()

View File

@@ -0,0 +1,46 @@
import os
import unittest
import json
import jc.parsers.jar_manifest
THIS_DIR = os.path.dirname(os.path.abspath(__file__))
class MyTests(unittest.TestCase):
def setUp(self):
# input
with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/rhel-8/MANIFEST.MF.out'), 'r', encoding='utf-8') as f:
self.rhel_8_manifest_mf = f.read()
with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/rhel-8/MANIFEST.MF.MULTI.out'), 'r', encoding='utf-8') as f:
self.rhel_8_manifest_mf_multi = f.read()
# output
with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/rhel-8/MANIFEST.MF.json'), 'r', encoding='utf-8') as f:
self.rhel_8_manifest_mf_json = json.loads(f.read())
with open(os.path.join(THIS_DIR, os.pardir, 'tests/fixtures/rhel-8/MANIFEST.MF.MULTI.json'), 'r', encoding='utf-8') as f:
self.rhel_8_manifest_mf_multi_json = json.loads(f.read())
def test_jar_manifest_nodata(self):
"""
Test 'jar_manifest' parser with no data
"""
self.assertEqual(jc.parsers.jar_manifest.parse('', quiet=True), [])
def test_jar_manifest_rhel_8(self):
"""
Test 'cat MANIFEST.MF | jc --jar_manifest'
"""
self.assertEqual(jc.parsers.jar_manifest.parse(self.rhel_8_manifest_mf, quiet=True), self.rhel_8_manifest_mf_json)
def test_jar_manifest_multi_rhel_8(self):
"""
Test 'unzip -c apache-log4j-2.16.0-bin/log4j-core-2.16.0.jar META-INF/MANIFEST.MF | jc --jar_manifest'
"""
self.assertEqual(jc.parsers.jar_manifest.parse(self.rhel_8_manifest_mf_multi, quiet=True), self.rhel_8_manifest_mf_multi_json)
if __name__ == '__main__':
unittest.main()