Yann Collet
b4a40a845f
move Sequences definition to zstd_compress_internal.h
...
they should not be in common/zstd_internal.h,
since these definitions are not shared beyond lib/compress/.
2024-12-20 10:36:55 -08:00
Yann Collet
a00f45a037
created ZSTD_storeSeqOnly()
...
makes it possible to register a sequence without copying its literals.
2024-12-20 10:36:04 -08:00
Yann Collet
125f05282b
publish new symbol ZSTD_compressSequencesAndLiterals()
2024-12-20 10:36:04 -08:00
Bent
5a7f5c745c
removed debug echos
2024-12-20 07:33:53 -08:00
Bent
d06e8778bc
added UNAME_TARGET_SYSTEM build flag to configure shared lib flags
2024-12-20 07:33:53 -08:00
Yann Collet
50ca9984ad
minor: more accurate parameter
...
just pass ldm_bucketLog, instead of the entire ldm* state
2024-12-14 11:57:14 -08:00
郑苏波 (Super Zheng)
5e0a83ec25
Disallow 32-bit mode in clang section
...
Fix register %rbx is only available in 64-bit mode
2024-12-04 06:47:32 -08:00
Yann Collet
2b36d4bc1c
Merge pull request #4202 from nhz2/fix-compressBound-typo
...
Fix typo in ZSTD_compressBound docs
2024-11-26 17:59:25 -08:00
Dimitri Papadopoulos
fcf88ae39b
Fix new typos found by codespell
2024-11-26 11:15:39 +01:00
nhz2
10beb7cb53
Fix typo in ZSTD_compressBound docs
2024-11-24 19:05:15 -05:00
Yann Collet
2e02cd330d
inform manual users that it's automatically generated
...
suggested by @Eugeny1
2024-10-31 15:06:48 -07:00
Yann Collet
d9553fd218
elevated ZSTD_getErrorCode() to stable status
...
answering #4183
2024-10-31 14:15:50 -07:00
Yann Collet
bbaba45589
change experimental parameter name
...
from ZSTD_c_useBlockSplitter to ZSTD_c_splitAfterSequences.
2024-10-31 13:43:40 -07:00
Yann Collet
4f93206d62
changed variable name to ZSTD_c_blockSplitterLevel
...
suggested by @terrelln
2024-10-29 11:12:09 -07:00
Yann Collet
fcbf6b014a
fixed minor conversion warning
2024-10-28 16:47:38 -07:00
Yann Collet
37706a677c
added a test
...
test both that the new parameter works as intended,
and that the over-split protection works as intended
2024-10-28 16:31:15 -07:00
Yann Collet
226ae73311
expose new parameter ZSTD_c_blockSplitter_level
2024-10-28 16:31:15 -07:00
Yann Collet
01474bf73b
add internal compression parameter preBlockSplitter_level
...
not yet exposed to the interface.
Also: renames `useBlockSplitter` to `postBlockSplitter`
to better qualify the difference between the 2 settings.
2024-10-28 16:31:15 -07:00
Yann Collet
5b4ce643f0
update ZSTD_splitBlock() documentation
2024-10-25 16:25:02 -07:00
Yann Collet
e557abc8a0
new block splitting variant _fromBorders
...
less precise but still suitable for `fast` strategy.
2024-10-25 16:13:55 -07:00
Yann Collet
da2c0dffd8
add faster block splitting heuristic, suitable for dfast strategy
2024-10-24 14:37:00 -07:00
Yann Collet
2366a87ddc
fix minor visual conversion warning
2024-10-24 13:38:12 -07:00
Yann Collet
326c45bb8e
complete sample11 with reduced fingerprint size
2024-10-24 13:17:56 -07:00
Yann Collet
ca6e55cbf5
reduce splitBlock arguments
2024-10-24 13:17:56 -07:00
Yann Collet
94d7b07425
organize specialization at recordFingerprint level
2024-10-24 13:17:56 -07:00
Yann Collet
566763fdc9
new variant, sampling by 11
2024-10-24 13:17:56 -07:00
Yann Collet
90095f056d
apply limit conditions for all splitting strategies
...
instead of just for blind split.
This is in anticipation of adversarial input,
that would intentionally target the sampling pattern of the split detector.
Note that, even without this protection, splitting can never expand beyond ZSTD_COMPRESSBOUND(),
because this upper limit uses a 1KB block size worst case scenario,
and splitting never creates blocks thath small.
The protection is more to ensure that data is not expanded by more than 3-bytes per 128 KB full block,
which is a much stricter limit.
2024-10-24 11:36:56 -07:00
Yann Collet
c80645a055
stricter limits to ensure expansion factor with blind-split strategy
...
issue reported by @terrelln
2024-10-23 14:55:10 -07:00
Yann Collet
7d3e5e3ba1
split all full 128 KB blocks
...
this helps make the streaming behavior more consistent,
since it does no longer depend on having more data presented on the input.
suggested by @terrelln
2024-10-23 14:18:48 -07:00
Yann Collet
b68ddce818
rewrite fingerprint storage to no longer need 64-bit members
...
so that it can be stored using standard alignment requirement (sizeof(void*)).
Distance function still requires 64-bit signed multiplication though,
so it won't change the issue regarding the bug in ubsan for clang 32-bit on github ci.
2024-10-23 11:50:57 -07:00
Yann Collet
57239c4d3b
fixed minor strict pedantic C90 issue
2024-10-23 11:50:57 -07:00
Yann Collet
18b1e67223
fixed extraneous return
...
strict C90 compliance test
2024-10-23 11:50:57 -07:00
Yann Collet
0be334d208
fixes static state allocation check
...
detected by @felixhandte
2024-10-23 11:50:57 -07:00
Yann Collet
06b7cfabf8
rewrote ZSTD_cwksp_initialAllocStart() to be easier to read
...
following a discussion with @felixhandte
2024-10-23 11:50:57 -07:00
Yann Collet
16450d0732
rewrite penalty update
...
suggested by @terrelln
2024-10-23 11:50:57 -07:00
Yann Collet
1ec5f9f1f6
changed loop exit condition so that there is no need to assert() within the loop.
2024-10-23 11:50:57 -07:00
Yann Collet
4662f6e646
renamed: FingerPrint => Fingerprint
...
suggested by @terrelln
2024-10-23 11:50:57 -07:00
Yann Collet
ea85dc7af6
conservatively estimate over-splitting in presence of incompressible loss
...
ensure data can never be expanded by more than 3 bytes per full block.
2024-10-23 11:50:57 -07:00
Yann Collet
5ae34e4c96
ensure lastBlock is correctly determined
...
reported by @terrelln
2024-10-23 11:50:57 -07:00
Yann Collet
7bad787d8b
made ZSTD_isPower2() an inline function
2024-10-23 11:50:57 -07:00
Yann Collet
a167571db5
added a faster block splitter variant
...
that samples 1 in 5 positions.
This variant is fast enough for lazy2 and btlazy2,
but it's less good in combination with post-splitter at higher levels (>= btopt).
2024-10-23 11:50:57 -07:00
Yann Collet
1c62e714ab
minor split optimization
...
let's fill the initial stats directly into target fingerprint
2024-10-23 11:50:57 -07:00
Yann Collet
4ce91cbf2b
fixed workspace alignment on non 64-bit systems
2024-10-23 11:50:57 -07:00
Yann Collet
cae8d13294
splitter workspace is now provided by ZSTD_CCtx*
2024-10-23 11:50:56 -07:00
Yann Collet
4685eafa81
fix alignment test
...
for non 64-bit systems
2024-10-23 11:50:56 -07:00
Yann Collet
73a6653653
ZSTD_splitBlock_4k() uses externally provided workspace
...
ideally, this workspace would be provided from the ZSTD_CCtx* state
2024-10-23 11:50:56 -07:00
Yann Collet
31d48e9ffa
fixing minor formatting issue in 32-bit mode with logs enabled
2024-10-23 11:50:56 -07:00
Yann Collet
6dc52122e6
fixed c90 comment style
2024-10-23 11:50:56 -07:00
Yann Collet
20c3d176cd
fix assert
2024-10-23 11:50:56 -07:00
Yann Collet
0d4b520657
only split full blocks
...
short term simplification
2024-10-23 11:50:56 -07:00
