e19b0822bc
Test large skippable frames
2017-11-01 13:10:03 -07:00
1ff8a8c109
Merge pull request #891 from facebook/contentSize
...
Content size
2017-10-17 17:24:51 -07:00
e600b5d0da
[test] Exercise all codes in dictionary tables
2017-10-16 18:05:36 -07:00
beb9b4b398
fixed ZSTDMT_initCStream() when contentSizeFlag==1 by default
...
and a wrong test in zstreamtest --mt
2017-10-13 19:09:30 -07:00
213ef3b510
fixed ZSTD_initCStream_advanced() behavior, which depends on contentSizeFlag,
...
and a stream fuzzer test, which was incorrect
(relied on 0 being unconditionnally transformed into `ZSTD_CONTENTSIZE_UNKNOWN`)
2017-10-13 19:01:58 -07:00
3c1e3f8ec9
contentSizeFlag enabled by default would also fail for streaming and MT operations
...
fixed
2017-10-13 18:32:06 -07:00
e0065cf660
make test : removed zstreamtest unit tests for variants
...
slightly reduced time to create dictionary
at beginning of unit tests
2017-09-28 18:34:38 -07:00
47c6a95d07
zstreamtest : run unit tests only during "normal" session
...
not during --mt, --newapi and --opaque
this avoids running them 4x during `make test`
2017-09-28 18:27:22 -07:00
e4ec427720
Merge branch 'dev' into shorterTests
...
fixed conflicts
2017-09-28 12:19:28 -07:00
bbef058ae6
zstreamtest --newapi : reduced maximum allocated memory
2017-09-28 11:48:45 -07:00
d6770f80af
minor : rewrite unit tests using CHECK_Z macro
2017-09-28 02:14:48 -07:00
9fe50ed623
fixed maximum windowLog for zstreamtest --newapi
...
for compatibility with low memory VM
2017-09-28 01:42:06 -07:00
9b5b47ac93
ensure adjustCParams adjust hLog and cLog even without srcSize
...
It would previously exit when srcSize is unknown.
But in the case of custom parameters,
hLog and cLog can still be too large in comparison with windowLog.
Reduces maximum memory allocated during zstreamtest --newapi
2017-09-28 01:25:40 -07:00
aa800c4793
reduced memory usage of zstreamtest --newapi
...
to run on memory-constrained VM
2017-09-27 18:00:15 -07:00
bc32b40b98
reduced zstreamtest --mt memory load
...
adjust compression level, hence memory usage, depending on nb threads
in order to run correctly on memory-starved VM.
2017-09-27 17:27:38 -07:00
54a827fff0
Merge branch 'dev' into newFormats
...
Fixed conflicts in zstdmt_compress.c
2017-09-27 16:39:40 -07:00
bfabd1d4dc
fixed zstreamtest decoding error
...
same error (wrong output buffer size) was present
on --mt and --new_api tests.
2017-09-27 01:01:11 -07:00
ca306c1c84
fixed a bug in zstreamtest
...
decoder output buffer would receive a wrong size.
In previous version, ZSTD_decompressStream() would blindly trust the caller that pos <= size.
In this version, this condition is actively checked,
and the function returns an error code if this condition is not respected.
This check could also be done with an assert(),
but since this is a user-facing interface, it seems better to keep this check at runtime.
2017-09-27 00:39:41 -07:00
c233bdbaee
Increase maximum window size
...
* Maximum window size in 32-bit mode is 1GB, since allocations for 2GB fail
on my Mac.
* Maximum window size in 64-bit mode is 2GB, since that is the largest
power of 2 that works with the overflow prevention.
* Allow `--long=windowLog` to set the window log, along with
`--zstd=wlog=#`. These options also set the window size during
decompression, but don't override `--memory=#` if it is set.
* Present a helpful error message when the window size is too large during
decompression.
* The long range matcher defaults to a hash log 7 less than the window log,
which keeps it at 20 for window log 27.
* Keep the default long range matcher window size and the default maximum
window size at 27 for the API and CLI.
* Add tests that use the maximum window size and hash size for compression
and decompression.
2017-09-26 14:00:01 -07:00
96f0cde31a
minor function rename
...
ZSTD_estimateCStreamSize_advanced_usingCParams -> ZSTD_estimateCStreamSize_usingCParams
_usingX is clear.
_advanced feels redundant
2017-09-24 16:47:02 -07:00
f1571dad8f
Merge pull request #838 from stellamplau/ldm-mergeDev
...
Add long distance matcher
2017-09-13 13:24:08 -07:00
3d8e313f64
Reduce ldm hash table size in test
2017-09-11 17:21:28 -07:00
eb3327c10a
Merge branch 'dev' of https://github.com/facebook/zstd into ldm-mergeDev
2017-09-11 15:00:01 -07:00
b3f33ccfb3
use ZSTD_decodingBufferSize_min() inside ZSTD_decompressStream()
...
Use same definition as public one
minor : reduce allocated buffer size in some cases
(when frameContentSize is known and == windowSize)
2017-09-09 14:37:28 -07:00
3128e03be6
updated license header
...
to clarify dual-license meaning as "or"
2017-09-08 00:09:23 -07:00
9e4060200b
Add tests and fix pointer alignment
2017-09-06 09:14:05 -07:00
67d4a6161c
Add ldmBucketSizeLog param
2017-09-02 21:55:29 -07:00
a1f04d518d
Move hashEveryLog to cctxParams and update cli
2017-09-01 15:05:47 -07:00
767a0b3be1
Move ldm hashLog, bucketLog, and mml to cctxParams
2017-09-01 12:24:59 -07:00
17d8e0bdcc
Merge remote-tracking branch 'upstream/longRangeMatcher' into ldm-integrate
2017-09-01 10:19:38 -07:00
8081becadc
Add long distance matching as a CCtxParam
2017-09-01 09:18:58 -07:00
d7ad99b2ab
Merge branch 'longRangeMatcher' into dev
2017-08-31 18:08:37 -07:00
6a546efb8c
Add long distance matcher
...
Move last literals section to ZSTD_block_internal
2017-08-31 12:53:19 -07:00
90a31bfa16
Pass dictMode to ZSTDMT_initCStream; fix nits
...
- Return error code in estimate{CCtx,CStream}Size functions
2017-08-30 16:19:07 -07:00
82d636b76a
Rename applyCCtxParams()
2017-08-29 18:03:06 -07:00
c88fb9267f
Replace 'byReference' with enum
2017-08-29 11:55:02 -07:00
b5b9275e67
Rename estimateCCtxSize_advanced() and estimateCStreamSize_advanced()
2017-08-29 10:49:29 -07:00
eb7bbab36a
Remove ZSTD_p_refDictContent and dictContentByRef
2017-08-25 11:11:45 -07:00
6f1a21c7e9
Remove formatting-only changes
2017-08-23 10:24:19 -07:00
73c73bf16a
Reduce code duplication in zstreamtest
2017-08-21 12:41:19 -07:00
91b30dbe84
Remove test parameter
2017-08-21 10:09:06 -07:00
f181f33bdf
Disable tests and refactor
2017-08-21 01:59:08 -07:00
023b24e6d4
Add cctx param tests
2017-08-20 22:55:07 -07:00
32fb407c9d
updated a bunch of headers
...
for the new license
2017-08-18 16:52:05 -07:00
052a95f77c
fix : ZSTDMT_compress_advanced() correctly generates checksum
...
when params.fParams.checksumFlag==1.
This use case used to be impossible when only ZSTD_compress() was available
2017-07-11 17:18:26 -07:00
4616fad18b
improved ZSTDMT_compress() memory usage
...
does not need the input buffer for streaming operations
also : reduced a few tests time length
2017-07-10 17:16:41 -07:00
afb0aca739
zstreamtest : big tests are only enabled in 64-bits mode
...
to avoid requesting too much memory in 32-bits mode during MT tests
2017-06-29 18:19:09 -07:00
62f7efc04a
added a test to ensure ZSTD_refPrefix() doesn't impact the following compression job
2017-06-28 16:25:13 -07:00
33a6639039
fixed ZSTD_refPrefix with Multithread-enabled CCtx
2017-06-28 11:09:43 -07:00
1fd2df9c2c
fixed zstreamtest
...
modifying compression parameters is not allowed after ZSTD_CCtx_loadDictionary()
but it is still allowed after ZSTD_refPrefix()
2017-06-28 09:57:59 -07:00
