1
0
mirror of https://github.com/facebook/zstd.git synced 2025-03-06 16:56:49 +02:00

4652 Commits

Author SHA1 Message Date
Yann Collet
dff5407cd0
Merge pull request #3966 from facebook/debug_lineNumber
add line number to debug traces
2024-03-15 10:26:06 -07:00
Yann Collet
686e7e4b4b updated version to v1.5.6 2024-03-14 15:38:14 -07:00
Elliot Gorokhovsky
559762da12
Remove duplicate and incorrect docs in zstd_decompress.c (#3967) 2024-03-14 15:55:01 -04:00
Yann Collet
9cc3304614 add line number to debug traces 2024-03-14 12:11:11 -07:00
Felix Handte
515c07a131
Merge pull request #3964 from felixhandte/promote-tgt-c-blk-size-to-stable
Promote `ZSTD_c_targetCBlockSize` Parameter to Stable API
2024-03-14 13:06:46 -04:00
Yann Collet
216099a73f
Merge pull request #3962 from facebook/cover_lessIncludes
reduce the amount of #include in cover.h
2024-03-13 16:52:54 -07:00
W. Felix Handte
3613448fb8 Promote ZSTD_c_targetCBlockSize Parameter to Stable API
This feature has demonstrated itself to be useful in web compression and we
want to encourage other folks to use it. But we currently make it difficult
to do so since it's locked away in the experimental API.

The API itself is really straightforward and I think it's fine to commit to
maintaining support / compatibility for this API even if in the future the
underlying implementation may continue to evolve.

Note that this commit changes its enum name and also its numeric value. Users
who respected the instructions of using the experimental API should be fine
with both of these changes since they should only have referred to it by the.

Conceivably someone could have done bad feature detection of this capability
by doing `#ifdef ZSTD_c_targetCBlockSize` which will now return false since
it's no longer a macro... but I think that's an acceptable hypothetical
breakage.
2024-03-13 17:07:10 -04:00
Nick Terrell
ff0afbad58 [asm][aarch64] Mark that BTI and PAC are supported
Mark that `huf_decompress_amd64.S` supports BTI and PAC, which it trivially does because it is empty for aarch64.

The issue only requested BTI markings, but it also makes sense to mark PAC, which is the only other feature.

Also run add a test for this mode to the ARM64 QEMU test. Before this PR it warns on `huf_decompress_amd64.S`, after it doesn't.

Fixes Issue #3841.
2024-03-13 16:15:51 -04:00
Yann Collet
c8ab027227 reduce the amount of includes in "cover.h" 2024-03-13 11:29:28 -07:00
Nick Terrell
94c102038b [cpu] Backport fix for rbx clobbering on Windows with Clang
Backport folly fix for rbx clobbering: f22f88b8b9

This supercedes PR #3646.
2024-03-13 09:45:40 -04:00
Yonatan Komornik
b20703f273
Updates ZSTD_RowFindBestMatch comment (#3947)
Updates the comment on the head of `ZSTD_RowFindBestMatch` to make sure it's aligned with recent changes to the hash table.
2024-03-12 15:10:07 -07:00
Elliot Gorokhovsky
f65b9e27ce
Exercise ZSTD_findDecompressedSize() in the simple decompression fuzzer (#3959)
* Improve decompression fuzzer

* Fix legacy frame header fuzzer crash, add unit test
2024-03-12 17:07:06 -04:00
Yann Collet
db996d253e
Merge pull request #3933 from facebook/fix3819
prevent XXH64 from being autovectorized by XXH512 by default
2024-03-12 09:46:48 -07:00
Yann Collet
3b573ba1f0
Merge pull request #3936 from facebook/debug5
minor: fix incorrect debug level
2024-03-12 09:45:20 -07:00
Yann Collet
fe7c645a7e
Merge pull request #3937 from facebook/offset0
new method to deal with offset==0 erroneous edge case
2024-03-12 09:25:06 -07:00
acceptacross
8ba5bc4729
chore: fix some typos (#3949)
Signed-off-by: acceptacross <csqcqs@gmail.com>
2024-03-12 11:44:42 -04:00
Elliot Gorokhovsky
e0872806df
Use ZSTD_LEGACY_SUPPORT=5 in make test (#3943) 2024-03-12 10:08:26 -04:00
Yann Collet
372fddf4e6
Merge pull request #3860 from likema/fix-xxhash-aix-51
Fix building xxhash on AIX 5.1
2024-03-09 15:34:02 -08:00
Yann Collet
a9fb8d4c41 new method to deal with offset==0
in this new method, when an `offset==0` is detected,
it's converted into (size_t)(-1), instead of 1.

The logic is that (size_t)(-1) is effectively an extremely large positive number,
which will not pass the offset distance test at next stage (`execSequence()`).
Checked the source code, and offset is always checked (as it should),
using a formula which is not vulnerable to arithmetic overflow:
```
RETURN_ERROR_IF(sequence.offset > (size_t)(oLitEnd - virtualStart),
```

The benefit is that such a case (offset==0) is always detected as corrupted data
as opposed to relying on the checksum to detect the error.
2024-03-08 15:26:06 -08:00
Yann Collet
aed172a8fe minor: fix incorrect debug level 2024-03-08 14:29:44 -08:00
Yann Collet
007cda88ca prevent XXH64 from being autovectorized by XXH512 by default
backport fix https://github.com/Cyan4973/xxHash/pull/924 from libxxhash
2024-03-07 16:43:13 -08:00
Yann Collet
8689633fdf
Merge pull request #3840 from aimuz/fix-reserved
lib/decompress: check for reserved bit corruption in zstd
2024-03-05 13:40:12 -08:00
Yann Collet
e385c3dd46
Merge pull request #3753 from facebook/make2
minor Makefile refactoring
2024-03-03 19:13:00 -08:00
Yann Collet
8d31e8ec42 sizeBlockSequences() also tracks uncompressed size
and only defines a sub-block boundary when
it believes that it is compressible.

It's effectively an optimization,
avoiding a compression cycle to reach the same conclusion.
2024-02-26 14:31:12 -08:00
Yann Collet
d23b95d21d minor refactor for clarity
since we can ensure that nbSubBlocks>0
2024-02-26 14:06:34 -08:00
Yann Collet
86db60752d optimization: bail out faster in presence of incompressible data 2024-02-26 13:27:59 -08:00
Yann Collet
ef82b214ad nit: comment indentation
as reported by @terrelln
2024-02-26 13:23:59 -08:00
Yann Collet
aa8592c532 minor: reformulate nbSubBlocks assignment 2024-02-26 13:21:14 -08:00
Yann Collet
e0412c2062 fix extraneous semicolon ';'
as reported by @terrelln
2024-02-26 12:26:54 -08:00
Yann Collet
1fafd0c4ae fix minor visual static analyzer warning
it's a false positive,
but change the code nonetheless to make it more obvious to the static analyzer.
2024-02-25 19:45:32 -08:00
Yann Collet
038a8a906b targetCBlockSize: modified splitting strategy to generate blocks of more regular size
notably avoiding to feature a larger first block
2024-02-25 17:39:29 -08:00
Yann Collet
f8372191f5 reduced minimum compressed block size
with the intention to match the transport layer size,
such as Ethernet and 4G mobile networks.
2024-02-24 01:59:16 -08:00
Yann Collet
f77f634d41 update API documentation 2024-02-24 01:28:17 -08:00
Yann Collet
4b51526412 fix partial block uncompressed 2024-02-24 01:24:58 -08:00
Yann Collet
6719794379 fixed some regressionTests
but not all
2024-02-23 18:48:29 -08:00
Yann Collet
0591e7eea1 minor: fix overly cautious conversion warning 2024-02-23 16:05:09 -08:00
Yann Collet
3b40100058 fix long sequences (> 64 KB) 2024-02-23 15:35:12 -08:00
Yann Collet
6b11fc436c fix issue with incompressible sections 2024-02-23 14:53:56 -08:00
Yann Collet
cc4530924b speed optimized version of targetCBlockSize
note that the size of individual compressed blocks will vary more wildly with this modification.
But it seems good enough for a first test, and fix the speed regression issue.
Further refinements can be attempted later.
2024-02-23 14:03:26 -08:00
Yann Collet
621a263fb2
Merge pull request #3903 from gruenich/feature/reduce-scope-of-variables
Reduce scope of variables
2024-02-22 09:42:02 -08:00
Yann Collet
e62e15df19 fix clangbuild
notably -Wconversion and -Wdocumentation
2024-02-20 22:43:22 -08:00
Christoph Grüninger
b921f1aad6 Reduce scope of variables
This improves readability, keeps variables local, and
prevents the unintended use (e.g. typo) later on.
Found by Cppcheck (variableScope)
2024-02-11 22:00:03 +01:00
Yann Collet
b0e8580dc7 fix fuzz issue 5131069967892480 2024-02-08 16:38:20 -08:00
Yann Collet
22574d848d fix issue 5921623844651008
ossfuzz managed to create a scenario which triggers an `assert`.
This fixes it, by giving +1 more space for the backward search pass.
2024-02-06 13:01:14 -08:00
Yann Collet
b88c593d8f added or updated code comments
as suggested by @terrelln,
to make the code of the optimal parser a bit more understandable.
2024-02-05 18:32:25 -08:00
Yann Collet
6c35fb2e8c fix msan warnings 2024-02-05 01:21:06 -08:00
Yann Collet
641749fc09 fix uasan dictionary_stream_round_trip fuzz test 2024-02-05 00:36:10 -08:00
Yann Collet
fe2e2ad36d use ZSTD_memcpy()
which can be redirected in Linux kernel mode
2024-02-03 19:57:38 -08:00
Yann Collet
0ae21d8c31 removed trace control 2024-02-03 19:32:59 -08:00
Yann Collet
5474edbe60 fixed wrong assert
by introducing ZSTD_OPT_SIZE
2024-02-03 19:31:53 -08:00