1
0
mirror of https://github.com/facebook/zstd.git synced 2025-07-06 07:37:29 +02:00
Commit Graph

220 Commits

Author SHA1 Message Date
49f8459d11 fixed minor cast warning 2017-06-21 18:43:39 -07:00
4d3bdcf130 reduced CLevelMax for --no-big-tests 2017-06-21 18:20:02 -07:00
bfc2f00080 --no-big-tests for zstreamtest
Apply --no-big-tests for tsan tests
2017-06-21 17:57:14 -07:00
dce789281b fixed : decompression of skippable frames in streaming mode 2017-06-21 15:53:42 -07:00
7bd1a2900e added ZSTD_dictMode_e to control dictionary loading mode 2017-06-21 11:50:33 -07:00
688952062b minor declaration statement warning fix 2017-06-19 11:53:42 -07:00
d7a3bffba9 new api : setting compression parameters is refused if a dictionary is already loaded 2017-06-19 11:53:01 -07:00
ed1d039127 newapi fuzzer tests : random flush orders in main loop 2017-06-19 11:07:33 -07:00
f5deae8a67 new api fuzzer tests : frame parameters are randonly set 2017-06-18 23:41:38 -07:00
ea3630a889 new api fuzzer set : dictionary is randomly set 2017-06-18 23:31:55 -07:00
8dee0ec99e new api fuzzer tests : compression parameters are randomly set 2017-06-18 23:25:15 -07:00
01743a36e7 fuzzer tests for new API 2017-06-16 17:56:41 -07:00
8c910d2097 updated ZSTDMT streaming API
ZSTDMT streaming API is now similar
and has same capabilites as single-thread streaming API.
It makes it easier to blend them together.
2017-06-03 01:15:02 -07:00
25989e361c updated ZSTD_estimate?DictSize() to pass parameter byReference
resulting ?Dict object is smaller when created byReference.
Seems better than a documentation note.
2017-05-25 15:07:37 -07:00
009d604e00 ZSTD_compress_generic() supports multiple successive frames
also : clarified streaming API implementation
2017-05-19 10:17:59 -07:00
a5ffe3d370 pushed enum values for strategy by one (ZSTD_fast==1)
this makes it possible to use `0` to mean:
"do not change strategy"
2017-05-12 16:29:19 -07:00
30ab64e21d added test for ZSTD_estimateCStreamSize() 2017-05-10 11:30:19 -07:00
f16f4497ca added ZSTD_estimateDStreamSize() 2017-05-09 16:18:17 -07:00
0be6fd3429 merged CCtx and CStream as a single same object
To be changed : ZSTD_sizeof_CCtx(), ZSTD_estimateCCtxSize()
2017-05-08 16:08:01 -07:00
a92cbb7004 Added a secondary test, checking dictID presence after setting noDictIdFLag=1 2017-04-27 15:08:56 -07:00
7d283cdfa2 added test for ZSTD_initCStream_usingCDict_advanced() 2017-04-27 14:48:34 -07:00
31533bacce Changed ZSTD_createCDict_advanced()
It now only uses compressionParameters as argument.
It produces many changes throughout user code,
though hopefully they tend to be simple :
just provide the cParams part from existing ZSTD_parameters.

Some programs might depend on ZSTD_createCDict_advanced() to pass frame parameters.
This change will force them to revisit this strategy and fix it,
since frame parameters are effectively silently ignored in current version.
2017-04-27 00:29:04 -07:00
2c5514c759 fixed ZSTDMT_initCStream_advanced()
Must use the new ZSTD_compressBegin_usingCDict_advanced()
to enforce correct frame parameters
2017-04-18 22:52:41 -07:00
0bb381dad8 added test for ZSTD_initCStream_advanced()
when params.fParams.contentSizeFlags = 1
and pledgedSrcSize = 0
then srcSize should be considered 0 (empty), and not "unknown"
2017-04-18 15:08:52 -07:00
4ee6b15dac force contentSizeFlag=0 when using ZSTD_initCStream_usingCDict()
because by definition srcSize is not known when using this prototype.
added relevant test

Note : this use was already working, because at a later stage
(both ZSTD_compressBegin_usingCDict() and ZSTD_copyCCtx())
pledgedSrcSize=0 is translated into "unknown", no matter the frame parameter.
This is not correct, but of little importance,
as the medium term plan is to no longer set fParams within CDict
2017-04-11 11:59:44 -07:00
4b987ad8ce Introduce ZSTD_initCStream_internal()
This is now the regroup point for ZSTD_initCStream*() functions

ZSTD_initCStream_advanced() now properly checks for parameters validity.

Also : added <assert.h> usage inside zstd_compress.c
Needs ZSTD_DEBUG=1 macro to be triggered.
Will be triggered by default from `tests` directory
2017-04-10 17:50:44 -07:00
ce80098f14 improved zstreamtest --mt to trap bug #644 2017-04-05 16:34:09 -07:00
62ecad3819 Fix ZSTD_initCStream_usingCDict() to use dictionary 2017-04-03 21:05:59 -07:00
30c7698970 optimize ZSTDMT_compress() memory usage
does no longer allocate temporary buffers
when there is enough room in dstBuffer to decompress directly there.
(previous method would skip that for 1st chunk only).

Also : fix ZSTD_compressBound() for small srcSize
2017-03-31 18:27:03 -07:00
042ba122ae Change g_displayLevel to int and fix DISPLAYUPDATE flush 2017-03-23 11:21:59 -07:00
7ebf2de02d Add ability to strongly limit fuzzer test size with flag 2017-03-20 11:25:00 -07:00
f5e50519e0 Prevent fuzz testers from combining large dicts with high clevel 2017-03-15 15:04:54 -07:00
a33ae64204 fixed decoding skippable frames 2017-02-28 01:15:28 -08:00
684858e7b7 fix memory leaks 2017-02-21 18:17:24 +01:00
0ed3901b05 Update overlength match test case 2017-02-16 13:36:57 -08:00
887eaa9e21 Fix wildcopy overwriting data still in window 2017-02-15 16:43:45 -08:00
2db7249265 Make pledgedSrcSize meaning clear for other functions
- Added tests
- Moved new size functions to static link only
2017-02-09 11:49:58 -08:00
4e709712e1 Decompressed size functions now handle multiframes and distinguish cases
- Add ZSTD_findDecompressedSize
    - Traverses multiple frames to find total output size
- Add ZSTD_getFrameContentSize
    - Gets the decompressed size of a single frame by reading header
- Deprecate ZSTD_getDecompressedSize
2017-02-08 14:50:10 -08:00
48bed91606 Merge pull request #527 from facebook/zstdmt
zstdmt refinements
2017-01-31 16:36:46 -08:00
92c98a5b21 zstreamtest uses random section sizes for fuzzing 2017-01-30 12:50:31 -08:00
cd23dd24af zstreamtest uses random overlapLog for fuzzing 2017-01-30 12:46:35 -08:00
64bf8ffce6 report @terrelln patch to ST fuzzer tests 2017-01-27 17:25:07 -08:00
d98bf49224 Fix segfault in zstreamtest MT
It was reading beyond the end of the input buffer because no errors were
detected. Once that was fixed, it wasn't making forward progress because
no errors were detected and it was waiting for input.
2017-01-27 15:42:36 -08:00
8dafb1acf5 CLI : automatically set overlap size to max (windowSize) for max compression level 2017-01-25 17:01:13 -08:00
bb0027405a fixed zstdmt corruption issue when enabling overlapped sections
see Asana board for detailed explanation on why and how to fix it
2017-01-25 16:25:38 -08:00
d7e3cb58c5 Resolved merge conflict dev+zstdmt 2017-01-20 16:44:50 -08:00
19d670ba9d Added ZSTDMT_initCStream_advanced() variant
Correctly compress with custom params and dictionary
Added relevant fuzzer test in zstreamtest

Also :
new macro ZSTDMT_SECTION_LOGSIZE_MIN, which sets a minimum size for a full job
(note : a flush() command can still generate a partial job anytime)
2017-01-19 15:32:07 -08:00
f22adae984 fixed minor warning (unused variable) in fuzzer 2017-01-19 13:46:30 -08:00
736788f8e8 added streaming fuzzer tests for MT API
Also : fixed corner case, where nb of jobs completed becomes > jobQueueSize
which is possible when many flushes are issued
while there is not enough dst buffer to flush completed ones.
2017-01-19 12:15:29 -08:00
33fce03045 added test checking dictID when using ZSTD_initCStream_usingCDict()
It shows that dictID is not properly added into frame header
2017-01-16 19:46:22 -08:00