1
0
mirror of https://github.com/facebook/zstd.git synced 2025-03-07 01:10:04 +02:00

2240 Commits

Author SHA1 Message Date
Victor Zhang
6b046f5841 PR feedback 2025-01-02 15:05:58 -08:00
Victor Zhang
54c3d998a0 Support for libc variants without fseeko/ftello
Some older Android libc implementations don't support `fseeko` or `ftello`.
This commit adds a new compile-time macro `LIBC_NO_FSEEKO` as well as a usage in CMake for old Android APIs.
2025-01-02 14:02:10 -08:00
Yann Collet
04a2a0219c update type names
naming convention: Type names should start with a Capital letter (after the prefix)
2024-12-29 14:25:33 -08:00
Yann Collet
56cfb7816a codemod: ZSTD_paramSwitch_e -> ZSTD_ParamSwitch_e 2024-12-20 10:36:58 -08:00
Yann Collet
125f05282b publish new symbol ZSTD_compressSequencesAndLiterals() 2024-12-20 10:36:04 -08:00
daniellerozenblit
17beeb5d1a
Change CLI to employ multithreading by default (#4211)
* Change CLI to employ multithreading by default

* Document changes to benchmarking, print number of threads for display level >= 4, and add lower bound of 1 for the default number of threads
2024-12-12 13:09:29 -05:00
Dimitrios Apostolou
194062a4e7 Fix printing of filesize if >4GB 2024-11-20 16:11:17 +01:00
Yann Collet
b3035b36c6 blind fix for QNX
following notification from @rainbowball.
fix #4186.

Note: there is currently no QNX compilation test in CI
so this is a "blind" fix,
and this target can be silently broken again in the future.
2024-11-05 00:09:13 -08:00
Yann Collet
f34bc9cee6 improve man page on benchmark mode
update the man page in troff format,
and the README with latest `--help` content and complementary details about benchmark mode.

also: display level 0 when doing decompression benchmark
2024-10-23 00:16:13 -07:00
Yann Collet
0079d515b1 Modify benchmark to only load sources once
After a regrettable update,
the benchmark module ended up reloading sources for every compression level.

While the delay itself is likely torelable,
the main issue is that the `--quiet` mode now also displays a loading summary between each compression line.
This wasn't the original intention, which is to produce a compact view of all compressions.

This is fixed in this version,
where sources are loaded only once, for all compression levels,
and loading summary is only displayed once.
2024-10-22 02:18:48 -07:00
Yann Collet
8edd147686 fix missing fclose()
fix #4151
2024-10-01 09:52:45 -07:00
Yann Collet
039f404faa update documentation to specify that Dictionary can be used for benchmark
fix #4139
2024-09-25 16:56:01 -07:00
Yann Collet
bc96d4b077
Merge pull request #4119 from xionghul/dev
Fix zstd-pgo run error
2024-09-24 17:55:43 -07:00
Xionghu Luo
bf4a43fcd4 Fix zstd-pgo run error
The -Werror=missing-profile caused thread/zlib/lzma/lz4 detection failure
when build with profile-use, thus caused ZSTD_MULTITHREAD etc. is not
defined for profile-use, then there will be many profile mismatch information
in output and the final binary reports run error sometimes as below:

Error : ZSTD_CCtx_setParameter(ctx, ZSTD_c_nbWorkers, adv->nbWorkers) failed : Unsupported parameter

Signed-off-by: Xionghu Luo <xionghuluo@tencent.com>
2024-08-08 15:13:27 +08:00
Yann Collet
14b8d398fd fix c90 comment style 2024-08-06 12:47:30 -07:00
Yann Collet
89451cafbd formatString_u() can display numbers > 100
fixes #4110
2024-08-06 11:44:37 -07:00
@RubenKelevra
b320d096a4
clarify when to use '--single-thread' with '--patch-from'
Using '--single-thread' with '--patch-from' on compression levels above 15 will lead to significantly worse compression ratios.

Corrected the man page not suggest anymore to do this.
2024-07-07 19:43:49 +02:00
Quentin Boswank
f19c98228f Fix $filter and Msys/Cygwin
- switched the patter and input of $filter into the right places
- added pattern wildcard to MSYS_NT & CYGWIN_NT as they change with windows versions
- correctly identify MSYS2, even in an env like MINGW64
2024-06-05 18:37:27 +02:00
Jure Bajic
4c6a519fdd [fix] Add newline when file exceeds 128KB 2024-05-26 11:34:24 +02:00
Yann Collet
a2f145f059 decompression errors always display the full origin filename
instead of the truncated size-limited version.
2024-04-01 11:49:21 -07:00
Yann Collet
0fcdc62c2d
Merge pull request #3969 from facebook/v156_prep
bump version number
2024-03-15 10:26:49 -07:00
Yann Collet
686e7e4b4b updated version to v1.5.6 2024-03-14 15:38:14 -07:00
Felix Handte
490163ac73
Merge pull request #3963 from felixhandte/also-handle-hidden-files-output-dir-mirror
Remove Erroneous Exclusion of Hidden Files and Folders in `--output-dir-mirror`
2024-03-14 10:21:00 -04:00
Felix Handte
04d91d5219
Merge pull request #3960 from felixhandte/use-utimensat-on-posix-2001
Use `utimensat()` on FreeBSD
2024-03-13 17:10:35 -04:00
W. Felix Handte
86b8e39a84 Remove Erroneous Exclusion of Hidden Files and Folders in --output-dir-mirror 2024-03-13 16:33:30 -04:00
W. Felix Handte
d6ee2d5d24 Use utimensat() on FreeBSD
FreeBSD only claims to support POSIX 2001 [0]. But they do in fact support
`utimensat()`. This adds a specific check to opt them in to using it. This
value was selected by consulting [1].

See discussion on #3952.

Further addresses #3748.

[0] 937a005585/sys/sys/unistd.h (L96)

[1] https://docs.freebsd.org/en/books/porters-handbook/versions/
2024-03-13 13:26:21 -07:00
Yann Collet
7c2aa73a8e
Merge pull request #3958 from facebook/doc3698
updated documentation
2024-03-13 07:15:45 -07:00
Yann Collet
ff6713fd72 fix duplicated paragraph
reported by @zougloub
2024-03-12 13:09:10 -07:00
Yann Collet
5473b72a05 updated documentation
following recommendations by @zougloub at #3698
2024-03-12 12:27:33 -07:00
Yann Collet
83ec3d0164 no longer truncate file name in verbose mode
fix #3702
2024-03-12 11:27:42 -07:00
daniellerozenblit
5a66afa051
Add common file extensions to --exclude-compressed (#3951) 2024-03-12 13:49:06 -04:00
Felix Handte
a6ff1194d8
Merge pull request #3952 from felixhandte/stop-hardcoding-posix-version-on-bsd
Stop Hardcoding the POSIX Version on BSDs
2024-03-12 12:49:19 -04:00
Yann Collet
8688c6ed92
Merge pull request #3942 from facebook/fix3719
Fix #3719 : mixing -c, -o and --rm
2024-03-12 09:44:52 -07:00
acceptacross
8ba5bc4729
chore: fix some typos (#3949)
Signed-off-by: acceptacross <csqcqs@gmail.com>
2024-03-12 11:44:42 -04:00
Yonatan Komornik
edab9eed66
Fix AsyncIO reading seed queueing (#3940)
Fixes a bug in AsyncIO where we queue reads after opening a file so our queue will always be saturated (or as saturated as possible).
Previous code was looping up to `availableJobsCount` not realizing `availableJobsCount` was also decreasing in each iteration, so instead of queueing 10 jobs we'd queue 5 (and instead of 2 we'd queue 1).
This PR fixes the loop to queue as long as `availableJobsCount` is not 0.
2024-03-11 16:28:32 -07:00
W. Felix Handte
f99a450ca4 Stop Hardcoding the POSIX Version on BSDs
BSDs should all have a `unistd.h` header.
2024-03-11 17:25:51 -04:00
Yann Collet
1362699e87 minor man page clarification 2024-03-11 12:23:37 -07:00
Yann Collet
fbd9e628ae added tests 2024-03-11 12:17:34 -07:00
Yann Collet
c610a01d7d fix #3719
only disable `--rm` at end of command line parsing,
so that `-c` only disables `--rm` if it's effectively selected,
and not if it's overriden by a later `-o FILE` command.
2024-03-11 11:38:55 -07:00
Yann Collet
4fb0a77314 update -V documentation
to answer #3727 comment
2024-03-03 23:24:40 -08:00
Yann Collet
e385c3dd46
Merge pull request #3753 from facebook/make2
minor Makefile refactoring
2024-03-03 19:13:00 -08:00
Yann Collet
4d2bf7f0f2 removed sprintf usage from zstdcli.c
some static analyzers flag this standard C90 function as unsafe.
2024-02-24 23:03:40 -08:00
Yann Collet
cc4530924b speed optimized version of targetCBlockSize
note that the size of individual compressed blocks will vary more wildly with this modification.
But it seems good enough for a first test, and fix the speed regression issue.
Further refinements can be attempted later.
2024-02-23 14:03:26 -08:00
Yann Collet
68a232c591 benchmark more can test targetCBlockSize 2024-02-23 13:13:03 -08:00
Yann Collet
621a263fb2
Merge pull request #3903 from gruenich/feature/reduce-scope-of-variables
Reduce scope of variables
2024-02-22 09:42:02 -08:00
Yann Collet
7170f51dd2 fix include order 2024-02-20 23:36:04 -08:00
Yann Collet
e62e15df19 fix clangbuild
notably -Wconversion and -Wdocumentation
2024-02-20 22:43:22 -08:00
Yann Collet
588dfbcc97 fix c89 compatibility by removing snprintf()
note that this function has been in the code for a long while,
so why does it only start failing CI tests now ?
2024-02-20 19:24:50 -08:00
Yann Collet
7a225c0c46 internal benchmark: can select size of generated synthetic sample 2024-02-20 15:47:09 -08:00
Yann Collet
83598aa106 datagen generates lorem ipsum by default 2024-02-20 15:24:25 -08:00