1
0
mirror of https://github.com/facebook/zstd.git synced 2025-07-04 06:40:29 +02:00
Commit Graph

517 Commits

Author SHA1 Message Date
d1a6a9d285 Reference requested dict ID at decompression time 2021-01-07 12:29:11 -05:00
5a6d3eef2b Allocate memory for DDict hash set when parameter is set 2021-01-07 12:29:11 -05:00
fd5b608f1c Add parameter to control multiple DDicts 2021-01-07 12:29:11 -05:00
f933668d3f Implement hashset for dictIDs 2021-01-07 12:29:11 -05:00
66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
95e74616d5 fix multiple minor conversion warnings
unrelated to #2386, just cleaning up while I'm updating this file ...
2020-11-06 09:57:05 -08:00
2769e4d459 fix incorrect assert
fix #2386, reported by @Neumann-A
2020-11-06 09:44:04 -08:00
e3e0775cc8 [API] Add ZSTD_c_stable{In,Out}Buffer parameters
This commit adds the parameters and sets the value in the CCtxParams
but it does not do anything with the value.
2020-10-30 10:54:39 -07:00
2e7d174130 Reset all decompression parameters in ZSTD_DCtx_reset()
* Reset all decompression parameters in `ZSTD_DCtx_reset()` when
  resetting parameters.
* Add a test case.
2020-10-01 14:19:21 -07:00
9261476b7d [lib] Wrap customMem xor checks in parens for readability
This clarifies operator precedence, and quiets cppcheck in
the Kernel Test Robot. I think this is a slight bonus to
readability, so I am accepting the suggestion.
2020-09-23 23:26:07 -07:00
c465f24457 ZSTD_ prefix mem{cpy,move,set},malloc,calloc,free 2020-08-26 12:26:03 -07:00
a686d306d2 Rename ZSTD_{malloc,calloc,free} to ZSTD_custom{Malloc,Calloc,Free} 2020-08-26 12:25:08 -07:00
80f577baa2 Move standard includes to zstd_deps.h 2020-08-26 12:25:08 -07:00
f82d9865b9 Merge pull request #2278 from senhuang42/ignore_checksum_advanced_param
New advanced decompression param to ignore checksums
2020-08-25 12:08:53 -07:00
614e446000 Merge pull request #2271 from terrelln/small-blocks
Small block optimizations
2020-08-24 18:54:33 -07:00
a030560d62 Add new DCtx param: validateChecksum and update unit tests 2020-08-24 17:28:00 -04:00
1302f8d676 [fix] Always return dstSize_tooSmall when it is the case 2020-08-24 13:38:13 -07:00
44c54a3e31 Addressing comments: more comments, cleanup, remove extra function, checksum logic 2020-08-24 16:14:19 -04:00
ffaa0df76d Document change in CLI for --no-check during decompression in --help menu 2020-08-24 09:49:12 -04:00
e3f5f9658a Added CLI tests for --no-check, fixed ignore checksum logic 2020-08-22 16:05:40 -04:00
20eb095882 Added unit test to fuzzer.c, changed definition param name 2020-08-22 13:26:33 -04:00
47685ac856 Move enum into zstd.h, and fix pesky switch() logic 2020-08-21 18:18:53 -04:00
08d3567ba8 Add function prototype 2020-08-21 16:51:43 -04:00
6a8dbdcd1f Modify decompression loop to gnore checksums if flag is enabled 2020-08-21 16:46:46 -04:00
2f39124342 Rename to ZSTD_d_forceIgnoreChecksum, add to DCtx, add function to set the advanced param 2020-08-21 16:23:39 -04:00
612e947c5e wire up bmi2 support 2020-08-17 16:35:28 -07:00
6004c1117f speed up small blocks 2020-08-16 23:03:38 -07:00
f800e72a3c [lib] Fix assertion when dictionary is prefix 2020-05-12 14:33:59 -07:00
4b88bd3ee0 [lib][fuzz] Assert sequences are valid in round trip tests 2020-05-11 20:38:49 -07:00
6028827fee Rewrite Include Paths to be Relative
Addresses #1998.
2020-05-04 15:20:26 -04:00
5e5f262612 Add (Possibly Empty) Info Strings to All Variadic Error Handling Macro Invocations 2020-05-04 10:58:55 -04:00
77a2945c43 Add some comments 2020-04-27 20:04:04 -07:00
f33de06c3e [lib] Fix single-pass mode for empty frames 2020-04-27 20:04:01 -07:00
a4ff217baf [lib] Add ZSTD_d_stableOutBuffer 2020-04-27 18:09:44 -07:00
a93fadfcd9 Further replication removed
`CHECK_F` is now in `error_private.h`. Minor tidy.
2020-04-07 11:25:16 +02:00
7af7735fa3 Merge remote-tracking branch 'upstream/dev' into single-file-lib 2020-04-07 11:13:02 +02:00
edd9a07322 Code replicated in compression and decompression moved to shared headers
`CHECK_F` macro moved to `error_private.h` (shared between `fse_compress.c` and `fse_decompress.c`). `ZSTD_limitCopy()` moved to `zstd_internal.h` (shared between `zstd_compress.c` and `zstd_decompress.c`). Erroneous build artefact `zstd.h` removed from repo.
2020-04-07 11:02:06 +02:00
0154866749 moving consts to zstd_internal and reusing them 2020-04-03 14:26:15 -07:00
0a172c5e43 converting to if 2020-04-03 14:21:24 -07:00
3a4c8cc9b3 adding dctx to function name 2020-04-03 14:14:46 -07:00
ae47d50355 only computing sizes once 2020-04-03 14:12:23 -07:00
a4cbe79ccb Using in and out size together 2020-04-03 14:09:21 -07:00
936aa63ff1 adding oversized check on decompression 2020-04-03 13:25:32 -07:00
05574ec141 adding oversizeDuration to dctx and macros 2020-04-03 13:08:29 -07:00
7c420344d2 Single-file decoder script can now (optionally) create an encoder
To complement the single-file decoder a new script was added to create an amalgamated single-file of all of the Zstd source, along with examples and (simple) tests.
2020-04-03 19:07:46 +02:00
ac58c8d720 Fix copyright and license lines
* All copyright lines now have -2020 instead of -present
* All copyright lines include "Facebook, Inc"
* All licenses are now standardized

The copyright in `threading.{h,c}` is not changed because it comes from
zstdmt.

The copyright and license of `divsufsort.{h,c}` is not changed.
2020-03-26 17:02:06 -07:00
fa6a772f38 Initialize dctx->bType to silence valgrind false positive 2020-01-23 17:54:48 -08:00
6e3cd5b024 Move ZSTD_checkContinuity() to zstd_decompress_block.c 2020-01-23 12:27:39 -08:00
036b30b555 Fix super block compression and stream raw blocks in decompression (#1947)
Super blocks must never violate the zstd block bound of input_size + ZSTD_blockHeaderSize. The individual sub-blocks may, but not the super block. If the superblock violates the block bound we are liable to violate ZSTD_compressBound(), which we must not do. Whenever the super block violates the block bound we instead emit an uncompressed block.

This means we increase the latency because of the single uncompressed block. I fix this by enabling streaming an uncompressed block, so the latency of an uncompressed block is 1 byte. This doesn't reduce the latency of the buffer-less API, but I don't think we really care.

* I added a test case that verifies that the decompression has 1 byte latency.
* I rely on existing zstreamtest / fuzzer / libfuzzer regression tests for correctness. During development I had several correctness bugs, and they easily caught them.
* The added assert that the superblock doesn't violate the block bound will help us discover any missed conditions (though I think I got them all).

Credit to OSS-Fuzz.
2020-01-10 18:02:11 -08:00
03ffda7b88 fix UBSAN's invalid-null-argument error in zstd_decompress.c (#1939) 2020-01-08 16:17:42 -08:00