1
0
mirror of https://github.com/facebook/zstd.git synced 2025-07-04 06:40:29 +02:00
Commit Graph

78 Commits

Author SHA1 Message Date
132e6efd76 switched ZSTDMT_compress_advanced() last argument to overlapLog
overlapRLog (== 9 - overlapLog) was a bit "strange"
as all other public entry points use overlapLog
2017-07-13 02:22:58 -07:00
052a95f77c fix : ZSTDMT_compress_advanced() correctly generates checksum
when params.fParams.checksumFlag==1.
This use case used to be impossible when only ZSTD_compress() was available
2017-07-11 17:18:26 -07:00
49af41820d clarified status of zstdmt_compress.h API 2017-07-05 17:21:13 -07:00
d5c046c609 implemented shortcut for zstd_compress_generic() in MT mode
added ZSTDMT_compress_advanced() API
2017-06-30 14:51:01 -07:00
b44ab82f7a ensure new ZSTD_strategy starts at value 1 2017-06-20 14:11:49 -07:00
f35e2de61c linked newAPI to ZSTDMT 2017-06-05 18:32:48 -07:00
8c910d2097 updated ZSTDMT streaming API
ZSTDMT streaming API is now similar
and has same capabilites as single-thread streaming API.
It makes it easier to blend them together.
2017-06-03 01:15:02 -07:00
58e8d793e1 made debug definitions common within zstd_internal.h 2017-06-02 18:20:48 -07:00
b877e834b1 minor indent 2017-06-02 13:47:11 -07:00
c35e535002 added support for multithreading parameters 2017-06-01 18:44:06 -07:00
c4a5a21c5c created ZSTDMT_sizeof_CCtx() and POOL_sizeof()
required by ZSTD_sizeofCCtx() while adding a ZSTDMT_CCtx*
2017-06-01 17:56:14 -07:00
ae728a43b8 removed defaultCustomMem
now ZSTD_customCMem is promoted as new default.

Advantages : ZSTD_customCMem = { NULL, NULL, NULL},
so it's natural default after a memset.

ZSTD_customCMem is public constant
(defaultCustomMem was private only).

Also : makes it possible to introduce ZSTD_calloc(),
which can now default to stdlib's calloc()
when it detects system default.

Fixed zlibwrapper which depended on defaultCustomMem.
2017-05-30 17:11:39 -07:00
44e45e8423 added ZSTDMT_createCCtx_advanced()
make it possible to use custom allocators
2017-05-30 16:12:06 -07:00
48bed91606 Merge pull request #527 from facebook/zstdmt
zstdmt refinements
2017-01-31 16:36:46 -08:00
88df1aed61 changed advanced parameter overlapLog
Follows a positive logic (increasing value => increasing overlap)
which is easier to use
2017-01-30 11:00:00 -08:00
b42dd27ef5 Add include guards and extern C 2017-01-27 16:00:19 -08:00
8dafb1acf5 CLI : automatically set overlap size to max (windowSize) for max compression level 2017-01-25 17:01:13 -08:00
06e7697f96 added test of new parameter ZSTD_p_forceWindow 2017-01-25 16:39:03 -08:00
dc8dae596a overlapped section, for improved compression
Sections 2+ read a bit of data from previous section
in order to improve compression ratio.
This also costs some CPU, to reference read data.

Read data is currently fixed to window>>3 size
2017-01-24 22:32:12 -08:00
512cbe8c10 zstdmt cli and API allow selection of section sizes
By default, section sizes are 4x window size.
This new setting allow manual selection of section sizes.
The larger they are, the (slightly) better the compression ratio,
but also the higher the memory allocation cost,
and eventually the lesser the nb of possible threads,
since each section is compressed by a single thread.

It also introduces a prototype to set generic parameters,
ZSTDMT_setMTCtxParameter()

The idea is that it's possible to add enums
to extend the list of parameters that can be set this way.
This is more long-term oriented than a fixed-size struct.
Consider it as a test.
2017-01-24 17:08:53 -08:00
3488a4a473 ZSTDMT now supports frame checksum 2017-01-24 11:48:40 -08:00
c593348722 ZSTDMT_initCStream_usingDict() can outlive dict
Like ZSTD_initCStream_usingDict(),
ZSTDMT_initCStream_usingDict() now keep a copy of dict internally.
This way, dict can be released :
it does not longer have to outlive all future compression sessions.
2017-01-22 16:44:15 -08:00
317604e0ad fixed : compilation of zstreamtest in dll mode 2017-01-20 17:18:41 -08:00
2e3b659ae1 fixed minor warnings (Visual, conversion, doxygen) 2017-01-20 14:43:09 -08:00
500014af49 zstd cli can now compress using multi-threading
added : command -T#
added : ZSTD_resetCStream() (zstdmt_compress)
added : FIO_setNbThreads()  (fileio)
2017-01-19 17:04:28 -08:00
19d670ba9d Added ZSTDMT_initCStream_advanced() variant
Correctly compress with custom params and dictionary
Added relevant fuzzer test in zstreamtest

Also :
new macro ZSTDMT_SECTION_LOGSIZE_MIN, which sets a minimum size for a full job
(note : a flush() command can still generate a partial job anytime)
2017-01-19 15:32:07 -08:00
107bcbbbc2 zstdmt : changed internal naming from frame to chunk
Since the result of mt compression is a single frame,
changed naming, which implied the concatenation of multiple frames.

minor : ensures that content size is written in header
2017-01-12 01:25:46 +01:00
3d93f2fce7 first zstdmt sketch 2016-12-27 07:19:36 +01:00