1
0
mirror of https://github.com/facebook/zstd.git synced 2025-03-07 09:26:03 +02:00

78 Commits

Author SHA1 Message Date
Bimba Shrestha
0fa6c90e89 Moving to same category as -v 2020-01-31 10:38:19 -08:00
Bimba Shrestha
f4ae6c0b94 Moving documentation and elaborating 2020-01-30 17:54:42 -08:00
Bimba Shrestha
6bf05a53ab Documenting in the man file 2020-01-30 17:11:38 -08:00
Bimba Shrestha
f25a6e9f8f Adding new cli endpoint --patch-from= (#1940)
* Adding new cli endpoint --diff-from=

* Appveyor conversion nit

* Using bool set trick instead of direct set

* Removing --diff-from and only leaving --diff-from=#

* Throwing error when both dictFileName vars are set

* Clean up syntax

* Renaming diff-from to patch-from

* Revering comma separated syntax clean up

* Updating playtests with patch-from

* Uncommenting accidentally commented

* Updating remaining docs and var names to be patch-from instead of diff-from

* Constifying

* Using existing log2 function and removing newly created one

* Argument order (moving prefs to end)

* Using comma separated syntax

* Moving to outside #ifndef
2020-01-10 14:25:24 -08:00
Bimba Shrestha
ef1684f29a [doc] Adding -M# decompression operation modifier documentation (#1938)
* Adding -M# operation modifier documentation

* Adding long name, typo and update desc
2020-01-03 16:36:47 -08:00
Yann Collet
c71bd45a3b Merge branch 'dev' into ahmed_file 2019-11-26 11:20:26 -08:00
Bimba Shrestha
b032a6e1d9 Updated some more outdated bounds 2019-11-19 09:55:13 -08:00
Bimba Shrestha
982830b081 Fixing hashLog error 2^26=64Mib 2019-11-14 10:48:55 -08:00
Yann Collet
be34969e5c minor man page fix 2019-11-06 15:07:35 -08:00
Yann Collet
d7f258d845 change command name to --filelist=
updated documentation and man page
2019-10-28 13:54:36 -07:00
Yann Collet
989e0f0cee Added --output-dir-flat=
which is a preferred construction when applicable.

Also, added --output-dir-flat into man page.
2019-10-21 14:39:18 -07:00
Yann Collet
ca73c218be added mode documentation on ZSTD_CLEVEL
providing range restriction [1-19] explicitly
partially answering #1829
2019-10-18 17:22:45 -07:00
Nick Magerko
493f95c7df Fix merge conflicts 2019-08-22 11:51:41 -07:00
Nick Magerko
fd486a846a Differentiate --stream-size from --size-hint 2019-08-22 09:37:47 -07:00
Nick Magerko
2cdda8b3c4 Minor documentation update 2019-08-22 09:13:28 -07:00
Nick Magerko
05d7479a50 Document --size-hint 2019-08-20 14:08:26 -07:00
Nick Magerko
bbd83c2ab3 Update man page 2019-08-19 09:11:22 -07:00
Tyler Tran
f8c1d7979c modifying minor nit 2019-07-22 16:36:44 -07:00
Tyler Tran
02da4497f0 Adding documentation for shrink flag PR #1656 2019-07-22 16:33:22 -07:00
Ori Peleg
bdeb4786b5 Fixed -r typo 2019-04-10 13:37:41 +03:00
Yann Collet
517d8c984c
Merge pull request #1449 from facebook/ovlog_def
overlapLog default values
2018-12-18 09:45:53 -08:00
Nick Terrell
bdfcaecc0a [zstdcli] Add --no-progress flag
The `--no-progress` flag disables zstd's progress bars, but leaves
the summary.

I've added simple tests to `playTests.sh` to make sure the parsing
works.
2018-12-14 11:50:25 -08:00
Yann Collet
1993f5d412 fixed ovlog tests
and updated man page
2018-12-12 21:09:14 -08:00
Yann Collet
34aa401afd updated documentation
introducing ZSTD_btultra2
2018-12-06 17:22:19 -08:00
Yann Collet
41c7d0b1e1 changed hashEveryLog into hashRateLog 2018-11-21 14:36:57 -08:00
Yann Collet
e874dacc08 changed searchLength into minMatch
refactored all relevant API and calls
for consistency.
2018-11-20 14:56:07 -08:00
Nick Terrell
f9a671ac37 [cli] Add rsyncable 2018-11-14 17:00:16 -08:00
Yann Collet
6c51bf420c bounds for --adapt mode
can supply min and max compression level through advanced command :
--adapt=min=#,max=#
2018-09-25 16:03:28 -07:00
Yann Collet
45010da074 updated man page
and added `--adapt` test in `playTests.sh`
2018-09-19 17:37:22 -07:00
Yann Collet
2f78228f65 Merge branch 'dev' into adapt 2018-09-19 12:43:42 -07:00
Jennifer Liu
21721b75a3 Change default f to 20 2018-09-04 17:15:14 -07:00
Jennifer Liu
f87383507d Update comment about default dictionary builder 2018-08-30 15:46:39 -07:00
Jennifer Liu
9d6ed9def3 Merge fastCover into DictBuilder (#1274)
* Minor fix

* Run non-optimize FASTCOVER 5 times in benchmark

* Merge fastCover into dictBuilder

* Fix mixed declaration issue

* Add fastcover to symbol.c

* Add fastCover.c and cover.h to build

* Change fastCover.c to fastcover.c

* Update benchmark to run FASTCOVER in dictBuilder

* Undo spliting fastcover_param into cover_param and f

* Remove convert param functions

* Assign f to parameter

* Add zdict.h to Makefile in lib

* Add cover.h to BUCK

* Cast 1 to U64 before shifting

* Remove trimming of zero freq head and tail in selectSegment and rebenchmark

* Remove f as a separate parameter of tryParam

* Read 8 bytes when d is 6

* Add trimming off zero frequency head and tail

* Use best functions from COVER and remove trimming part(which leads to worse compression ratio after previous bugs were fixed)

* Add finalize= argument to FASTCOVER to specify percentage of training samples passed to ZDICT_finalizeDictionary

* Change nbDmer to always read 8 bytes even when d=6

* Add skip=# argument to allow skipping dmers in computeFrequency in FASTCOVER

* Update comments and benchmarking result

* Change default method of ZDICT_trainFromBuffer to ZDICT_optimizeTrainFromBuffer_fastCover

* Add dictType enum and fix bug about passing zParam when converting to coverParam

* Combine finalize and skip into a single parameter

* Update acceleration parameters and benchmark on 3 sample sets

* Change default splitPoint of FASTCOVER to 0.75 and benchmark first 3 sample sets

* Initialize variables outside of for loop in benchmark.c

* Update benchmark result for hg-manifest

* Remove cover.h from install-includes

* Add explanation of f

* Set default compression level for trainFromBuffer to 3

* Add assertion of fastCoverParams in DiB_trainFromFiles

* Add checkTotalCompressedSize function + some minor fixes

* Add test for multithreading fastCovr

* Initialize segmentFreqs in every FASTCOVER_selectSegment and move mutex_unnlock to end of COVER_best_finish

* Free segmentFreqs

* Initialize segmentFreqs before calling FASTCOVER_buildDictionary instead of in FASTCOVER_selectSegment

* Add FASTCOVER_MEMMULT

* Minor fix

* Update benchmarking result
2018-08-23 12:06:20 -07:00
Yann Collet
0853f86044 adaptive mode uses default window size of 8 MB 2018-08-13 13:13:22 -07:00
Yann Collet
f3aa510738 rateLimiter does not "catch up" when input speed is slow 2018-08-13 11:38:55 -07:00
Yann Collet
e7a49c6683 introduced command --adapt 2018-08-11 20:48:06 -07:00
Nick Terrell
45821fac0c
Merge pull request #1225 from jennifermliu/dev
Split samples when building dictionary for COVER
2018-07-13 13:26:15 -07:00
Jennifer Liu
612b346ed5 Add explanation for split=100 2018-07-11 15:50:28 -07:00
Jennifer Liu
5021441d86 Change default splitPoint to 100 2018-07-10 11:19:33 -07:00
Jennifer Liu
bfad1af031 Update doc for split==100 2018-07-05 11:05:31 -07:00
Jennifer Liu
348e5f77a9 Add split=# to cli 2018-06-29 17:54:41 -07:00
Yann Collet
1fd621ff6d minor man page update
regarding advanced parameter `tlen`
which was recently changed.
`0` in association with `ZSTD_fast` now means "normal fast mode".
2018-06-27 18:49:02 -07:00
George Lu
547096d672 update man 2018-05-31 18:03:52 -07:00
Yann Collet
b824d213cb fix #1115 2018-05-12 10:21:30 -07:00
Björn Ketelaars
e5ea8d272a fix typo in programs/zstd.{1,1.md}
s/nodictID/no-dictID/g
2018-04-05 06:44:46 +02:00
Tor E Hagemann
c7a5e60bc6
Update zstd.1.md 2018-03-30 15:25:32 -07:00
Yann Collet
a57d43d4d4 updated documentation of targetLength 2018-03-12 11:35:01 -07:00
Yann Collet
6a9b41b731 create command --fast[=#]
access negative compression levels from command line
for both compression and benchmark modes.

also : ensure proper propagation of parameters
through ZSTD_compress_generic() interface.

added relevant cli tests.
2018-03-11 20:01:23 -07:00
Yann Collet
75689838e4 specify new command --single-thread 2018-02-09 15:55:41 -08:00
Yann Collet
9f8ed23b5b bumped version number to v1.3.4
also added a paragraph on using compression level with training mode
as this is a recurrent question (see for example #1004)
2018-01-27 22:23:26 -08:00