1
0
mirror of https://github.com/pbnjay/grate.git synced 2025-01-20 21:18:23 +02:00

11 Commits

Author SHA1 Message Date
Jeremy Jay
35d03d67eb expose data types through interface 2021-02-22 00:01:17 -05:00
Jeremy Jay
792f566b59 add docs and simple example 2021-02-16 16:41:04 -05:00
Jeremy Jay
2896a36e3a implement concurrent search and sync aggregation 2021-02-14 14:17:25 -05:00
Jeremy Jay
a5be267bf7 more tweaks to memory usage in xls this time
did not reduce total allocations much (bytes.Reader is more efficient
than I thought), but reduced walltime from 99s to 55s for a large collection
2021-02-13 00:06:04 -05:00
Jeremy Jay
b953163a8d tweaks to reduce memory usage in xlsx
on a test dataset, usage goes from 6.0GB => 3.3GB
and walltime improves from 31.6s => 19.4s

largest remaining driver is the slow/hungry xml.Decoder
2021-02-12 13:44:46 -05:00
Jeremy Jay
749e19458a allow openers to have a priority 2021-02-12 00:59:01 -05:00
Jeremy Jay
88daa500dc accidently implemented xlsx. refactor to include it 2021-02-12 00:50:50 -05:00
Jeremy Jay
ee3b4224e0 refactor xls2tsv and add some features 2021-02-10 01:34:35 -05:00
Jeremy Jay
bf6d144fa3 improve error handling/drop panics 2021-02-08 15:36:08 -05:00
Jeremy Jay
80c3b4cc81 many bugfixes and edge cases, impl most cell types 2021-02-08 11:02:37 -05:00
Jeremy Jay
c01ac5f01b tool to create tsv from xls 2021-02-07 22:53:28 -05:00