Yann Collet
02be2a830f
build macro ZSTD_DECODER_INTERNAL_BUFFER
...
just to make the topic more accessible for potential users.
2021-10-25 08:09:04 -07:00
binhdvo
6a7ede3dfc
Reduce size of dctx by reutilizing dst buffer ( #2751 )
...
* Reduce size of dctx by reutilizing dst buffer
Co-authored-by: Binh Vo <binhvo@fb.com>
2021-10-25 10:38:01 -04:00
Yann Collet
0a794f5afe
Merge pull request #2822 from marxin/fix-zstd-thread-pool-documentation
...
Support thread pool section in HTML documentation.
2021-10-22 16:46:08 -07:00
Nick Terrell
dad8a3cf34
Merge pull request #2825 from terrelln/huf-asm-comments
...
[asm] Switch to C style comments
2021-10-20 18:06:37 -07:00
Nick Terrell
abd717a5fa
[asm] Switch to C style comments
...
Switch to C style comments for increased portability, and consistency.
2021-10-20 11:37:05 -07:00
Yann Collet
9d62957b31
Merge pull request #2800 from animalize/fix_c89
...
Fix a C89 error in msvc
2021-10-18 14:32:04 -07:00
Martin Liska
1c2b02eee9
Support thread pool section in HTML documentation.
2021-10-15 18:35:39 +02:00
Felix Handte
23c1a2d260
Merge pull request #2774 from felixhandte/zstd-dfast-pipelined-single
...
Pipelined Implementation of ZSTD_dfast
2021-10-13 16:38:43 -04:00
W. Felix Handte
0bfc935add
Convert Outer Control Structure to Loop
2021-10-12 13:34:17 -04:00
sen
66e554fdad
Merge pull request #2813 from marxin/streaming-compress-enhance
...
Enhance streaming_compression examples.
2021-10-12 13:05:54 -04:00
Martin Liska
926d47004d
Enhance streaming_compression examples.
...
Add level argument to the first test and be more verbose about
used compression level and number of threads.
2021-10-12 08:59:34 +02:00
Nick Terrell
b77d95b053
Merge pull request #2820 from terrelln/nb-compares
...
[binary-tree] Fix underflow of nbCompares
2021-10-11 09:59:57 -07:00
Nick Terrell
26486db9ab
Merge pull request #2819 from terrelln/ldm-hash-rate-log
...
[ldm] Fix ZSTD_c_ldmHashRateLog bounds check
2021-10-08 14:58:29 -07:00
Nick Terrell
802745e88a
Merge pull request #2818 from terrelln/indentation-fix
...
[nit] Fix buggy indentation
2021-10-08 14:57:52 -07:00
Nick Terrell
9c9a5f3ff3
Merge pull request #2817 from terrelln/multiple-ddict-fix
...
[multiple-ddicts] Fix NULL checks
2021-10-08 14:57:29 -07:00
Nick Terrell
c6c482fe07
[binary-tree] Fix underflow of nbCompares
...
Fix underflow of `nbCompares` by switching to an `int` and comparing
`nbCompares > 0`. This is a minimal fix, because I don't want to change
the logic. These loops seem to be doing `nbCompares + 1` comparisons.
The bug was reported by Dan Carpenter and found by Smatch static
checker.
https://lore.kernel.org/all/20211008063704.GA5370@kili/
2021-10-08 13:22:55 -07:00
Nick Terrell
31316cf158
[multiple-ddicts] Fix NULL checks
...
The bug was reported by Dan Carpenter and found by Smatch static
checker.
https://lore.kernel.org/all/20211008063704.GA5370@kili/
2021-10-08 11:24:58 -07:00
Nick Terrell
1bbb372e3e
[ldm] Fix ZSTD_c_ldmHashRateLog bounds check
...
There is no minimum value check, so the parameter could be negative.
Switch to the standard pattern of using `BOUNDCHECK()`.
The bug was reported by Dan Carpenter and found by Smatch static
checker.
https://lore.kernel.org/all/20211008063704.GA5370@kili/
2021-10-08 11:17:40 -07:00
Nick Terrell
399644b1f1
[nit] Fix buggy indentation
...
The bug was reported by Dan Carpenter and found by Smatch static
checker.
https://lore.kernel.org/all/20211008063704.GA5370@kili/
2021-10-08 11:13:11 -07:00
W. Felix Handte
79ca830766
Style: Add Comments to Variables and Move a Couple into the Loop
2021-10-05 16:18:09 -04:00
W. Felix Handte
168d0a3c89
Fix Flaky Test
...
This test depended on `_extDict` and `_noDict` compressing identically, which
is not a guarantee we make, AFAIK.
2021-10-05 16:18:09 -04:00
W. Felix Handte
c2c32839dc
Update results.csv
2021-10-05 16:18:00 -04:00
W. Felix Handte
62536ef7da
Simplify DMS Implementation by Removing noDict Support
2021-10-05 14:54:37 -04:00
W. Felix Handte
051b473e7e
Fall Back in _extDict to New _noDict Rather than Old Merged Impl
2021-10-05 14:54:37 -04:00
W. Felix Handte
fcab4841aa
Nit: Rename Function
2021-10-05 14:54:37 -04:00
W. Felix Handte
47fd762ecc
Nit: Unnest Blocks that Don't Declare Anything
2021-10-05 14:54:37 -04:00
W. Felix Handte
2cdfad538c
Search One Last Position
2021-10-05 14:54:37 -04:00
W. Felix Handte
6ae44c0db8
Advance Long Index Lookup (+0.5% Speed)
...
This lookup can be advanced to before the short match check because either way
we will use it (in the next loop iter or in `_search_next_long`).
2021-10-05 14:54:37 -04:00
W. Felix Handte
2ddef7c872
Write Back Advanced Hash in Long Matches as Well (+Ratio)
...
Since we're now hashing the position ahead even if we find a long match and
don't search that next position, we can write it back into the hashtable even
in long matches. This seems to cost us no speed, and improves compression
ratio slightly!
2021-10-05 14:54:37 -04:00
W. Felix Handte
39f2491bfc
Use Look-Ahead Hash for Next Long Check after Short Match (+0.5% Speed)
...
This costs a little ratio, unfortunately.
2021-10-05 14:54:37 -04:00
W. Felix Handte
db4e1b5479
Hash Long One Position Ahead (+2.5% Speed)
...
Aside from maybe a latency win in the loop, this means that when we find a
short match, we've already done the hash we need to check the next long match.
2021-10-05 14:54:37 -04:00
W. Felix Handte
a1ac7205d0
Pull Match Found Stuff Out of the Loop
2021-10-05 14:54:37 -04:00
W. Felix Handte
072ffaad67
Extract Working Variables
2021-10-05 14:54:37 -04:00
W. Felix Handte
1bdf041071
Track Step Rather than Recalculating (+0.5% Speed)
2021-10-05 14:54:37 -04:00
W. Felix Handte
258c0623e1
Extract Single-Segment Variant of ZSTD_dfast
2021-10-05 14:54:37 -04:00
stanjo74
52598d54e9
Limit train samples ( #2809 )
...
* Limit training samples size to 2GB
* simplified DISPLAYLEVEL() macro to use global vqriable instead of local.
* refactored training samples loading
* fixed compiler warning
* addressed comments from the pull request
* addressed @terrelln comments
* missed some fixes
* fixed type mismatch
* Fixed bug passing estimated number of samples rather insted of the loaded number of samples.
Changed unit conversion not to use bit-shifts.
* fixed a declaration after code
* fixed type conversion compile errors
* fixed more type castting
* fixed more type mismatching
* changed sizes type to size_t
* move type casting
* more type cast fixes
2021-10-04 17:47:52 -07:00
Yann Collet
7868f38019
Merge pull request #2747 from Helflym/dev
...
Add AIX support in Makefile
2021-10-01 08:13:39 -07:00
Nick Terrell
a7826560aa
Merge pull request #2803 from terrelln/linux-kernel-ci
...
[contrib][linux-kernel] Add standard warnings and -Werror to CI
2021-09-29 09:48:47 -07:00
Nick Terrell
3a4d421c0f
Merge pull request #2802 from solbjorn/fix-kernel-wundef
...
[contrib][linux] Fix -Wundef inside Linux kernel tree
2021-09-29 09:48:17 -07:00
Nick Terrell
db9999a0b6
Merge pull request #2810 from 3nids/patch-1
...
add missing BUNDLE DESTINATION
2021-09-29 09:47:17 -07:00
sen
358f1778dc
Merge pull request #2755 from senhuang42/skip_long_matches_lazy
...
Skip most long matches in lazy hash table update
2021-09-29 12:44:07 -04:00
Denis Rouzaud
abc1a91fe2
add missing BUNDLE DESTINATION
...
fixes build on iOS
2021-09-29 08:27:12 +02:00
Ma Lin
894f05e88d
Fix ZSTD_countTrailingZeros() bug
...
`>> 3` is wrong.
2021-09-29 07:20:09 +08:00
Sen Huang
9360367371
Update regression test
2021-09-28 08:29:11 -07:00
Sen Huang
4b7f45cb04
Pull hot loop into its own function
2021-09-28 08:19:44 -07:00
Sen Huang
ccdcbf4621
Try beginning and end of match
2021-09-28 08:19:44 -07:00
Sen Huang
b8fd6bf30c
Skip most long matches in lazy hash table update
2021-09-28 08:19:39 -07:00
Nick Terrell
9ef055d706
Merge pull request #2808 from terrelln/huf-oss-fuzz-fix
...
[huf] Fix OSS-Fuzz assert
2021-09-27 15:00:52 -07:00
Felix Handte
8b7a19fcd4
Merge pull request #2805 from nolange/smaller_code_with_disabled_features
...
Smaller code with disabled features
2021-09-27 17:43:21 -04:00
Nick Terrell
a07ddb47f7
[huf] Fix OSS-Fuzz assert
...
PR #2784 introduced a bug in the decompressor that caused some valid
inputs to fail to decompress. The bitstream isn't reloaded after the 4X*
loop if the number of elements remaining is small enough, causing us to
read more bits than are available in the bitcontainer.
This was caught by the MSAN fuzzer in OSS-Fuzz because the assembly
implementation isn't used in the MSAN build.
Credit to OSS-Fuzz.
2021-09-27 13:56:07 -07:00
