2021-10-24 13:10:47 -07:00
|
|
|
[Home](https://kellyjonbrazil.github.io/jc/)
|
2022-01-25 17:07:47 -08:00
|
|
|
<a id="jc.parsers.csv_s"></a>
|
|
|
|
|
|
|
|
# jc.parsers.csv\_s
|
2021-10-24 13:10:47 -07:00
|
|
|
|
2022-03-04 13:27:39 -08:00
|
|
|
jc - JSON Convert `csv` file streaming parser
|
2021-10-24 13:10:47 -07:00
|
|
|
|
2022-03-10 13:32:26 -08:00
|
|
|
> This streaming parser outputs JSON Lines (cli) or returns a Generator
|
|
|
|
iterator of Dictionaries (module)
|
2021-10-30 13:57:36 -07:00
|
|
|
|
2022-01-19 17:30:14 -08:00
|
|
|
The `csv` streaming parser will attempt to automatically detect the
|
|
|
|
delimiter character. If the delimiter cannot be detected it will default
|
|
|
|
to comma. The first row of the file must be a header row.
|
2021-10-24 13:10:47 -07:00
|
|
|
|
2022-01-19 17:30:14 -08:00
|
|
|
Note: The first 100 rows are read into memory to enable delimiter detection,
|
|
|
|
then the rest of the rows are loaded lazily.
|
2021-10-24 13:10:47 -07:00
|
|
|
|
|
|
|
Usage (cli):
|
|
|
|
|
2022-01-25 18:03:34 -08:00
|
|
|
$ cat file.csv | jc --csv-s
|
2021-10-24 13:10:47 -07:00
|
|
|
|
|
|
|
Usage (module):
|
|
|
|
|
2022-01-25 18:03:34 -08:00
|
|
|
import jc
|
2022-01-18 14:18:12 -08:00
|
|
|
|
2022-03-10 13:32:26 -08:00
|
|
|
result = jc.parse('csv_s', csv_output.splitlines())
|
2022-01-25 18:03:34 -08:00
|
|
|
for item in result:
|
|
|
|
# do something
|
2021-10-24 13:10:47 -07:00
|
|
|
|
|
|
|
Schema:
|
|
|
|
|
2022-01-25 18:03:34 -08:00
|
|
|
csv file converted to a Dictionary:
|
|
|
|
https://docs.python.org/3/library/csv.html
|
|
|
|
|
|
|
|
{
|
|
|
|
"column_name1": string,
|
|
|
|
"column_name2": string,
|
|
|
|
|
|
|
|
# below object only exists if using -qq or ignore_exceptions=True
|
2022-03-10 10:10:57 -08:00
|
|
|
"_jc_meta": {
|
|
|
|
"success": boolean, # false if error parsing
|
|
|
|
"error": string, # exists if "success" is false
|
|
|
|
"line": string # exists if "success" is false
|
|
|
|
}
|
2022-01-25 18:03:34 -08:00
|
|
|
}
|
|
|
|
|
|
|
|
Examples:
|
|
|
|
|
|
|
|
$ cat homes.csv
|
|
|
|
"Sell", "List", "Living", "Rooms", "Beds", "Baths", "Age", "Acres"...
|
|
|
|
142, 160, 28, 10, 5, 3, 60, 0.28, 3167
|
|
|
|
175, 180, 18, 8, 4, 1, 12, 0.43, 4033
|
|
|
|
129, 132, 13, 6, 3, 1, 41, 0.33, 1471
|
|
|
|
...
|
|
|
|
|
|
|
|
$ cat homes.csv | jc --csv-s
|
|
|
|
{"Sell":"142","List":"160","Living":"28","Rooms":"10","Beds":"5"...}
|
|
|
|
{"Sell":"175","List":"180","Living":"18","Rooms":"8","Beds":"4"...}
|
|
|
|
{"Sell":"129","List":"132","Living":"13","Rooms":"6","Beds":"3"...}
|
|
|
|
...
|
2021-10-24 13:10:47 -07:00
|
|
|
|
2022-01-25 17:07:47 -08:00
|
|
|
<a id="jc.parsers.csv_s.parse"></a>
|
|
|
|
|
2022-03-05 12:15:14 -08:00
|
|
|
### parse
|
2022-01-25 17:07:47 -08:00
|
|
|
|
2021-10-24 13:10:47 -07:00
|
|
|
```python
|
2022-02-03 15:47:46 -08:00
|
|
|
@add_jc_meta
|
2022-01-25 17:07:47 -08:00
|
|
|
def parse(data, raw=False, quiet=False, ignore_exceptions=False)
|
2021-10-24 13:10:47 -07:00
|
|
|
```
|
|
|
|
|
|
|
|
Main text parsing generator function. Returns an iterator object.
|
|
|
|
|
2022-01-25 18:03:34 -08:00
|
|
|
Parameters:
|
|
|
|
|
|
|
|
data: (iterable) line-based text data to parse
|
|
|
|
(e.g. sys.stdin or str.splitlines())
|
2022-01-19 17:30:14 -08:00
|
|
|
|
2022-01-25 18:03:34 -08:00
|
|
|
raw: (boolean) unprocessed output if True
|
|
|
|
quiet: (boolean) suppress warning messages if True
|
2022-02-04 12:14:16 -08:00
|
|
|
ignore_exceptions: (boolean) ignore parsing exceptions if True
|
2021-10-24 13:10:47 -07:00
|
|
|
|
2022-01-25 18:03:34 -08:00
|
|
|
Yields:
|
2021-10-24 13:10:47 -07:00
|
|
|
|
2022-01-25 18:03:34 -08:00
|
|
|
Dictionary. Raw or processed structured data.
|
2021-10-24 13:10:47 -07:00
|
|
|
|
2022-01-25 18:03:34 -08:00
|
|
|
Returns:
|
2021-10-24 13:10:47 -07:00
|
|
|
|
2022-02-14 10:31:41 -08:00
|
|
|
Iterator object (generator)
|
2021-10-24 13:10:47 -07:00
|
|
|
|
2022-01-25 19:18:54 -08:00
|
|
|
### Parser Information
|
2021-10-24 13:10:47 -07:00
|
|
|
Compatibility: linux, darwin, cygwin, win32, aix, freebsd
|
|
|
|
|
2022-02-03 15:47:46 -08:00
|
|
|
Version 1.3 by Kelly Brazil (kellyjonbrazil@gmail.com)
|