adbb536d00
fixed single file library test
...
copy "zstd_errors.h" alongside "zstd.h" to pass compilation test.
2024-10-31 14:38:36 -07:00
d9553fd218
elevated ZSTD_getErrorCode() to stable status
...
answering #4183
2024-10-31 14:15:50 -07:00
15c29168b7
Merge pull request #4180 from facebook/split_param
...
Block splitter control parameter
2024-10-31 14:15:29 -07:00
bbaba45589
change experimental parameter name
...
from ZSTD_c_useBlockSplitter to ZSTD_c_splitAfterSequences.
2024-10-31 13:43:40 -07:00
4f93206d62
changed variable name to ZSTD_c_blockSplitterLevel
...
suggested by @terrelln
2024-10-29 11:12:09 -07:00
f593ccda04
removed trace left over
2024-10-28 16:57:01 -07:00
fcbf6b014a
fixed minor conversion warning
2024-10-28 16:47:38 -07:00
37706a677c
added a test
...
test both that the new parameter works as intended,
and that the over-split protection works as intended
2024-10-28 16:31:15 -07:00
226ae73311
expose new parameter ZSTD_c_blockSplitter_level
2024-10-28 16:31:15 -07:00
01474bf73b
add internal compression parameter preBlockSplitter_level
...
not yet exposed to the interface.
Also: renames `useBlockSplitter` to `postBlockSplitter`
to better qualify the difference between the 2 settings.
2024-10-28 16:31:15 -07:00
5bae43b411
Merge pull request #4178 from facebook/split_fromBorders
...
Add fastest block-splitter variant
2024-10-28 16:30:49 -07:00
4e1a87976f
update regression results
2024-10-25 16:33:00 -07:00
5b4ce643f0
update ZSTD_splitBlock() documentation
2024-10-25 16:25:02 -07:00
e557abc8a0
new block splitting variant _fromBorders
...
less precise but still suitable for `fast` strategy.
2024-10-25 16:13:55 -07:00
7fb5347e88
Merge pull request #4176 from facebook/sample11
...
Added faster block splitter variants for levels 3-7
2024-10-25 16:13:17 -07:00
c63b5d2294
update regression results
2024-10-24 14:47:55 -07:00
da2c0dffd8
add faster block splitting heuristic, suitable for dfast strategy
2024-10-24 14:37:00 -07:00
2366a87ddc
fix minor visual conversion warning
2024-10-24 13:38:12 -07:00
ea082692aa
added block splitter variant for greedy & lazy (levels 5 to 7)
2024-10-24 13:27:01 -07:00
326c45bb8e
complete sample11 with reduced fingerprint size
2024-10-24 13:17:56 -07:00
ca6e55cbf5
reduce splitBlock arguments
2024-10-24 13:17:56 -07:00
94d7b07425
organize specialization at recordFingerprint level
2024-10-24 13:17:56 -07:00
566763fdc9
new variant, sampling by 11
2024-10-24 13:17:56 -07:00
2dddf09056
Merge pull request #4136 from facebook/preSplit
...
Block splitter
2024-10-24 13:17:36 -07:00
70c77d20d6
update regression results
...
first block is no longer splitted since adding the @savings over-split protection
2024-10-24 11:43:16 -07:00
90095f056d
apply limit conditions for all splitting strategies
...
instead of just for blind split.
This is in anticipation of adversarial input,
that would intentionally target the sampling pattern of the split detector.
Note that, even without this protection, splitting can never expand beyond ZSTD_COMPRESSBOUND(),
because this upper limit uses a 1KB block size worst case scenario,
and splitting never creates blocks thath small.
The protection is more to ensure that data is not expanded by more than 3-bytes per 128 KB full block,
which is a much stricter limit.
2024-10-24 11:36:56 -07:00
bbda1acf85
update regression results
2024-10-23 15:56:56 -07:00
c80645a055
stricter limits to ensure expansion factor with blind-split strategy
...
issue reported by @terrelln
2024-10-23 14:55:10 -07:00
7d3e5e3ba1
split all full 128 KB blocks
...
this helps make the streaming behavior more consistent,
since it does no longer depend on having more data presented on the input.
suggested by @terrelln
2024-10-23 14:18:48 -07:00
b68ddce818
rewrite fingerprint storage to no longer need 64-bit members
...
so that it can be stored using standard alignment requirement (sizeof(void*)).
Distance function still requires 64-bit signed multiplication though,
so it won't change the issue regarding the bug in ubsan for clang 32-bit on github ci.
2024-10-23 11:50:57 -07:00
57239c4d3b
fixed minor strict pedantic C90 issue
2024-10-23 11:50:57 -07:00
18b1e67223
fixed extraneous return
...
strict C90 compliance test
2024-10-23 11:50:57 -07:00
d2eeed53dc
updated compression results
...
due to integration of `sample5` strategy, leading to better compression ratios on a range of levels
2024-10-23 11:50:57 -07:00
0be334d208
fixes static state allocation check
...
detected by @felixhandte
2024-10-23 11:50:57 -07:00
06b7cfabf8
rewrote ZSTD_cwksp_initialAllocStart() to be easier to read
...
following a discussion with @felixhandte
2024-10-23 11:50:57 -07:00
16450d0732
rewrite penalty update
...
suggested by @terrelln
2024-10-23 11:50:57 -07:00
1ec5f9f1f6
changed loop exit condition so that there is no need to assert() within the loop.
2024-10-23 11:50:57 -07:00
4662f6e646
renamed: FingerPrint => Fingerprint
...
suggested by @terrelln
2024-10-23 11:50:57 -07:00
ea85dc7af6
conservatively estimate over-splitting in presence of incompressible loss
...
ensure data can never be expanded by more than 3 bytes per full block.
2024-10-23 11:50:57 -07:00
5ae34e4c96
ensure lastBlock is correctly determined
...
reported by @terrelln
2024-10-23 11:50:57 -07:00
7bad787d8b
made ZSTD_isPower2() an inline function
2024-10-23 11:50:57 -07:00
a167571db5
added a faster block splitter variant
...
that samples 1 in 5 positions.
This variant is fast enough for lazy2 and btlazy2,
but it's less good in combination with post-splitter at higher levels (>= btopt).
2024-10-23 11:50:57 -07:00
1c62e714ab
minor split optimization
...
let's fill the initial stats directly into target fingerprint
2024-10-23 11:50:57 -07:00
dac26eaeac
updated regression test results
2024-10-23 11:50:57 -07:00
4ce91cbf2b
fixed workspace alignment on non 64-bit systems
2024-10-23 11:50:57 -07:00
cae8d13294
splitter workspace is now provided by ZSTD_CCtx*
2024-10-23 11:50:56 -07:00
4685eafa81
fix alignment test
...
for non 64-bit systems
2024-10-23 11:50:56 -07:00
433f4598ad
fixed minor conversion warnings on Visual
2024-10-23 11:50:56 -07:00
73a6653653
ZSTD_splitBlock_4k() uses externally provided workspace
...
ideally, this workspace would be provided from the ZSTD_CCtx* state
2024-10-23 11:50:56 -07:00
7f015c2fd7
replaced uasan32 test by asan32 test
2024-10-23 11:50:56 -07:00
Yann Collet
Yann Collet
Yann Collet