9d31c704d5
Don't shrink window log when streaming with a dictionary
...
Fixes #2442 .
1. When creating a dictionary keep the same behavior as before.
Assume the source size is 513 bytes when adjusting parameters.
2. When calling ZSTD_getCParams() or ZSTD_adjustCParams() keep
the same behavior as before.
3. When attaching a dictionary keep the same behavior of ignoring
the dictionary size. When streaming this will select the
largest parameters and not adjust them down. But, the CDict
will use the correctly sized parameters, which seems like the
right tradeoff.
4. When not attaching a dictionary (either forced not to, or
using a prefix dictionary) we select parameters based on the
dictionary size + source size, and assume the source size is
small, which is the same behavior as before. But, now we don't
adjust the window log (and hash and chain log) down when the
source size is unknown.
When the source size is unknown all cdicts should attach, except
when the user disables attaching, or `forceWindow` is used. This
means that when streaming with a CDict we end up in the good case
where we get small CDict parameters, and large source parameters.
TODO: Add a streaming + dictionary regression test case.
2021-01-04 15:54:09 -08:00
66e811d782
[license] Update year to 2021
2021-01-04 17:53:52 -05:00
cfff4c1cd5
Merge pull request #2439 from senhuang42/skippable_frame_api
...
Generate skippable frame API
2020-12-28 11:22:07 -08:00
dd1a7e41ee
Add ifndef guards for _LARGEFILE_SOURCE and _LARGEFILE64_SOURCE
...
This ensures the symbols aren't redefined, which would result in a compiler
error.
I was getting redefined symbols for _LARGEFILE64_SOURCE when building for
32-bit x86 Linux on an older CentOS release in a CI environment. With this
change, I'm able to compile the single file library in this environment.
Closes #2443 .
2020-12-26 10:02:45 -07:00
5c41490bfe
Use pre-defined constants
2020-12-21 11:52:05 -05:00
7e11bd012b
Implement skippable frame function
2020-12-21 11:13:22 -05:00
9648bf027b
try to keep libzstd.a "as is" once created
...
to be compatible with scenarios such as
`make -j allmost`
2020-12-20 17:10:57 -08:00
7c495e8ea2
updated version number to v1.4.8
2020-12-18 15:52:11 -08:00
a7cb4af573
added emphasis on the alignment condition of workspace
...
and made it a programming mistake (`assert()`)
rather than a runtime error.
2020-12-18 15:04:09 -08:00
ae85676d44
Fix alignment of scratchBuffer in HUF_compressWeights()
...
The scratch buffer must be 4-byte aligned. This causes test failures in
32-bit systems, where the stack isn't aligned.
Fixes Issue #2428 .
2020-12-17 14:30:27 -08:00
0b39531d75
moving all references to release branch
...
was previously `master`
2020-12-16 23:00:35 -08:00
f647a759fe
updated version number to v1.4.7
...
and updated doc
2020-12-15 20:53:05 -08:00
b8c3a473ec
Merge pull request #2420 from terrelln/huf-comment
...
[huf_compress] Refactor and comment HUF_buildCTable()
2020-12-14 16:14:07 -08:00
c56723ab03
replace final links by direct copy
...
link can behave slightly differently from real binaries,
breaking a few scripts relying on "real binary" assumption.
2020-12-10 13:25:08 -08:00
f861e8c07b
Merge pull request #2421 from felixhandte/pc-no-sed
...
Don't Use Regexes to Build Pkg-Config File
2020-12-09 18:58:17 -05:00
9dab03db90
Create Enum to Represent Static/Dynamic Allocation Distinction in cwksp
2020-12-09 14:57:37 -05:00
db9e73cb07
Don't ASAN-Poison Statically-Allocated Workspaces
...
Addresses #2286 .
2020-12-09 13:00:47 -05:00
b521183c74
Avoid Use of Regexes in Building Package-Config File
2020-12-08 20:10:05 -05:00
1bbcf07bd5
[huf_compress] Refactor and comment HUF_buildCTable()
...
Comment and refactor `HUF_buildCTable()` and the helper functions
it calls as I read and understand the code. Hopefully this refactor
makes the code a bit more clear.
2020-12-08 13:57:01 -08:00
b86e3c9304
Merge pull request #2415 from facebook/fix_aliasing
...
fix gcc-10 strict aliasing warnings
2020-12-04 21:30:57 -08:00
fad175f9c1
Merge pull request #2412 from animalize/dict_compressionlevel
...
use ZSTD_CLEVEL_DEFAULT in zdict.c
2020-12-04 17:09:30 -08:00
6132df8dd3
fix gcc-10 strict aliasing warnings
...
by exposing HUF_CElt declaration.
2020-12-04 16:43:19 -08:00
68c14bdff2
minor speed improvement to HUF_readCTable()
...
faster by ~+1-2%
2020-12-04 16:33:39 -08:00
c238db046f
Merge pull request #2414 from terrelln/mt-progress
...
[lib] Ensure that multithreaded compression always makes some progress
2020-12-04 16:30:08 -08:00
4c58cb8383
[lib] Ensure that multithreaded compression always makes some progress
2020-12-03 20:25:14 -08:00
1aec77ea89
use ZSTD_CLEVEL_DEFAULT in zdict.c
2020-12-03 12:46:57 +08:00
6672689e7e
Merge pull request #2406 from terrelln/linux-wrapper-api
...
[linux] Add the linux wrapper API
2020-12-02 16:49:03 -08:00
91c1b57be9
Merge pull request #2409 from facebook/test_makefile
...
Minor refactor
2020-12-02 15:33:54 -08:00
894ae36675
Merge pull request #2390 from animalize/clamp_level
...
Clamp compression level
2020-12-02 14:35:58 -08:00
2cbd038528
Move max nb seq check to per-block
2020-12-02 12:11:32 -05:00
3cda5fae77
[minor][lib] Remove double semicolon
2020-12-02 01:08:08 -08:00
9f8b180d5d
fixed API documentation
2020-12-02 00:15:07 -08:00
6112b82526
Merge pull request #2348 from dscheg/dev
...
Fix dll path in case of cross-compilation
2020-12-01 17:59:56 -08:00
3efe9c902b
Add sequence nb validation to compressSequences(), adjust minMatch comparisons
2020-12-01 10:54:45 -05:00
4c5f337248
Use cctx's minMatch instead of global MINMATCH, make fuzzer use validation
2020-11-30 15:41:20 -05:00
61f71753d4
Pass dictBufferCapacity to COVER_selectDict()
...
closes #2371
2020-11-22 23:45:18 +05:00
c5fbd55dac
Merge pull request #2387 from senhuang42/compress_sequence_API
...
[RFC] New sequence compression API
2020-11-20 16:54:20 -05:00
7742f076b4
Add experimental param for sequence validation
2020-11-20 11:57:41 -05:00
0e32928b7d
Remove unnecessary repcode backup, apply style choices, use function pointer
2020-11-20 11:02:19 -05:00
e924a0fa51
Explicit cast for visual warnings
...
Github has automatic commits now! Cool
Co-authored-by: Nick Terrell <nickrterrell@gmail.com >
2020-11-19 17:32:40 -05:00
dcbbf7c09f
Unroll isRLE loop
2020-11-19 12:38:13 -05:00
05c0229668
Clean up visual conversion warnings
2020-11-18 15:36:29 -05:00
3c4454769b
Improve documentation on ZSTD_compressSequences()
2020-11-18 09:52:24 -05:00
d6d7ba2a1f
Modification to offset validation to include entire sequence
2020-11-17 10:13:22 -05:00
8f3136a9c7
Fix assert edge case, improve documentation in zstd.h
2020-11-16 18:05:35 -05:00
f6baad87d6
Fix warnings and make validation enabled by default
2020-11-16 12:00:06 -05:00
55b90ef010
Fix unit tests to agree with new changes
2020-11-16 11:36:37 -05:00
7f563b0519
Add new sequence format as an experimental CCtx param
2020-11-16 10:49:17 -05:00
347824ad73
Overhaul logic to simplify, add in proper validations, fix match splitting
2020-11-16 10:49:17 -05:00
46824cb018
Add new sequence compress api params to cctx
2020-11-16 10:49:17 -05:00
