1
0
mirror of https://github.com/facebook/zstd.git synced 2025-03-07 09:26:03 +02:00

10843 Commits

Author SHA1 Message Date
Yann Collet
e0ee0fccf8 updated benchmarks for v1.5.6 2024-03-27 18:19:31 -07:00
Yann Collet
aaad8e2e8c
Merge pull request #3988 from facebook/dependabot/github_actions/github/codeql-action-3.24.9
Bump github/codeql-action from 3.24.7 to 3.24.9
2024-03-27 18:05:42 -07:00
Yann Collet
ebf24b7b77 fix nodejs deprecation warning
by updating msys2 action
2024-03-27 16:10:15 -07:00
Yann Collet
80a9a2dfcd
Merge pull request #3987 from ManuelBlanc/msbuild_vswhere
Use vswhere to find MSBuild; add VS2022 support
2024-03-27 15:48:40 -07:00
Po-Chuan Hsieh
103a85e6f6
Use md5sum rather than gmd5sum for FreeBSD
Reference:	https://man.freebsd.org/cgi/man.cgi?query=md5sum
2024-03-27 20:53:52 +08:00
Nick Terrell
81a5e5d438 [fuzz] Turn off -Werror by default
This was causing OSS-Fuzz errors, due to compiler differences.
* Fix the issue
* Also turn off -Werror so we don't fail fuzzer builds for warnings
* Turn on -Werror in our CI
2024-03-26 16:34:36 -07:00
dependabot[bot]
101e601c79
Bump github/codeql-action from 3.24.7 to 3.24.9
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.24.7 to 3.24.9.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](3ab4101902...1b1aada464)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-25 05:31:42 +00:00
ManuelBlanc
9c442d6fc2 Improve MSBuild search; add latest option 2024-03-23 18:34:07 -04:00
ManuelBlanc
65ab6c267e Use vswhere to find MSBuild; add VS2022 support
See:
https://github.com/microsoft/vswhere/wiki/Find-MSBuild
2024-03-23 09:13:49 -04:00
Yann Collet
78f732cfdd
Merge pull request #3985 from facebook/scorecard_permission
try to silence some scorecard warnings
2024-03-21 13:56:20 -07:00
Yann Collet
273d1279ca try to silence some scorecard warnings 2024-03-21 13:25:48 -07:00
Yann Collet
c6e5257240
Merge pull request #3977 from facebook/doc_advanced
Doc update
2024-03-21 12:33:15 -07:00
Elliot Gorokhovsky
dc1f7b560b
fix -Werror=pointer-arith in fuzzers (#3983) 2024-03-21 15:16:38 -04:00
Yann Collet
6679d0ca7b
Merge pull request #3982 from embg/fuzzer_readme
Document the process for adding a new fuzzer
2024-03-21 11:10:03 -07:00
Yann Collet
76fbb5f3d4
Merge pull request #3975 from terrelln/2024-03-18-cmake-docs
[cmake] Emit warnings for contradictory build settings
2024-03-21 10:41:54 -07:00
Yann Collet
1d3f664fce
Merge pull request #3979 from yoniko/Werror-fuzz
Fail on errors when building fuzzers
2024-03-21 10:41:34 -07:00
Yann Collet
86caab507f
Merge pull request #3981 from terrelln/2024-03-19-generate-sequences
Fix & fuzz ZSTD_generateSequences
2024-03-21 10:41:21 -07:00
Nick Terrell
731f4b70fc Fix & fuzz ZSTD_generateSequences
This function was seriously flawed:
* It didn't do output bounds checks
* It produced invalid sequences when an uncompressed or RLE block was emitted
* It produced invalid sequences when the block splitter was enabled
* It produced invalid sequences when ZSTD_c_targetCBlockSize was enabled

I've attempted to fix these issues, but this function is just a bad idea,
so I've marked it as deprecated and unsafe. We should replace it with
`ZSTD_extractSequences()` which operates on a compressed frame.
2024-03-21 07:18:05 -07:00
Elliot Gorokhovsky
741b87bbe1
Fuzzing and bugfixes for magicless-format decoding (#3976)
* fuzzing and bugfixes for magicless format

* reset dctx before each decompression

* do not memcmp empty buffers

* nit: decompressor errata
2024-03-20 19:22:34 -04:00
Elliot Gorokhovsky
f62b2663b9 Add docs on how to add a new fuzzer 2024-03-19 14:05:23 -07:00
Yonatan Komornik
3487a60950 Fail on errors when building fuzzers
Fails on errors when building fuzzers with `fuzz.py` (adds `Werror`).
Currently allows `declaration-after-statement`, `c++-compat` and
`deprecated` as they are abundant in code (some fixes to
`declaration-after-statement` are presented in this commit).
2024-03-18 15:51:28 -07:00
Yonatan Komornik
6a0052a409
Fix bugs in simple decompression fuzzer (#3978)
Fixes 2 issue in `simple_decompress.c`:
1. Wrong type used for storing the results of `ZSTD_findDecompressedSize` resulting in never matching to `ZSTD_CONTENTSIZE_ERROR` or `ZSTD_CONTENTSIZE_UNKNOWN`.

2. Experimental API is used (`ZSTD_findDecompressedSize`) without defining `ZSTD_STATIC_LINKING_ONLY`.
2024-03-18 15:36:40 -07:00
Yann Collet
6f1215b874 fix ZSTD_TARGETCBLOCKSIZE_MIN test
when requested CBlockSize is too low,
bound it to the minimum
instead of returning an error.
2024-03-18 14:10:08 -07:00
Yann Collet
c5da438dc0 fix typo 2024-03-18 12:33:22 -07:00
Yann Collet
3d18d9a9ce updated API manual 2024-03-18 12:30:54 -07:00
Yann Collet
902c7ec1fe add doc on CCtx UB state 2024-03-18 12:30:35 -07:00
Yann Collet
5d82c2b57c add a paragraph on UB DCtx state after error 2024-03-18 12:17:41 -07:00
Yann Collet
f5728da365 update targetCBlockSize documentation 2024-03-18 12:04:02 -07:00
Nick Terrell
42b02f5185 [cmake] Emit warnings for contradictory build settings
Document that the `ZSTD_BUILD_{SHARED,STATIC}` take precedence over `BUILD_SHARED_LIBS` when exactly one is ON.

Thanks to @teo-tsirpanis for pointing out the potentially confusing behavior.
2024-03-18 09:31:22 -07:00
Alexander Kanavin
cd4dba74de pzstd: use c++14 without conditions
Doing this check with a direct c++ snippet is prone to portability problems:

- \043 is not portable between shells: dash expands it to #,
bash does not;

- using # directly works with make 4.3 but does not with make 4.2.

Let's just use the c++ version that covers both the code and the gtest.
2024-03-18 12:15:26 -04:00
Elliot Gorokhovsky
7d970bd83c
Implement one-shot fallback for magicless format (#3971) 2024-03-18 10:55:53 -04:00
Nick Terrell
a595e5812a [cmake] Fix up PR #3716
* Make a variable `PublicHeaders` for Zstd's public headers
* Add `PublicHeaders` to `Headers`, which was missing
* Only export `${LIBRARY_DIR}` publicly, not `common/`
* Switch the `target_include_directories()` to `INTERFACE` because zstd uses relative includes internally, so doesn't need any include directories to build
* Switch installation to use the `PublicHeaders` variable, and test that the right headers are installed
2024-03-18 09:18:51 -04:00
dsvi
79cd0ff712 Makes it possible to use the lib through FetchContent or ExternalProject_Add 2024-03-18 09:18:51 -04:00
Yann Collet
3008f74ee2
Merge pull request #3973 from facebook/dependabot/github_actions/github/codeql-action-3.24.7
Bump github/codeql-action from 3.24.6 to 3.24.7
2024-03-18 00:44:55 -07:00
Yann Collet
f79c1c476e
Merge pull request #3972 from facebook/dependabot/github_actions/actions/cache-4
Bump actions/cache from 3 to 4
2024-03-18 00:44:12 -07:00
dependabot[bot]
9dca0602f4
Bump github/codeql-action from 3.24.6 to 3.24.7
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 3.24.6 to 3.24.7.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](8a470fddaf...3ab4101902)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-18 05:37:56 +00:00
dependabot[bot]
88301b58c1
Bump actions/cache from 3 to 4
Bumps [actions/cache](https://github.com/actions/cache) from 3 to 4.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-03-18 05:37:52 +00:00
Yann Collet
0fcdc62c2d
Merge pull request #3969 from facebook/v156_prep
bump version number
2024-03-15 10:26:49 -07:00
Yann Collet
dff5407cd0
Merge pull request #3966 from facebook/debug_lineNumber
add line number to debug traces
2024-03-15 10:26:06 -07:00
Yann Collet
351498b932 update CHANGELOG for v1.5.6 2024-03-14 17:01:34 -07:00
Yann Collet
686e7e4b4b updated version to v1.5.6 2024-03-14 15:38:14 -07:00
Elliot Gorokhovsky
559762da12
Remove duplicate and incorrect docs in zstd_decompress.c (#3967) 2024-03-14 15:55:01 -04:00
Yann Collet
9cc3304614 add line number to debug traces 2024-03-14 12:11:11 -07:00
Nick Terrell
a0a9bc6c95 [cmake] Always create libzstd target
If both `ZSTD_BUILD_SHARED` and `ZSTD_BUILD_STATIC` are set, then cmake exports the libraries `libzstd_shared` and `libzstd_static` only.
It does not export `libzstd`, which is only exported when exactly one of `ZSTD_BUILD_SHARED` and `ZSTD_BUILD_STATIC` is set.
This PR exports `libzstd` in that case, based on the value of the standard CMake variable [`BUILD_SHARED_LIBS`](https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html).
This ensures that `libzstd` can always be used to refer to the exported zstd library, since the build errors if neither `ZSTD_BUILD_SHARED` nor `ZSTD_BUILD_STATIC` are set.

I tested all the possible combinations of `ZSTD_BUILD_SHARED`, `ZSTD_BUILD_STATIC`, and `BUILD_SHARED_LIBS` and they always worked as expected:
* If only exactly one of `ZSTD_BUILD_SHARED` and `ZSTD_BUILD_STATIC` is set, that is used as `libzstd`.
* Otherwise, libzstd is set based on `BUILD_SHARED_LIBS`.

Fixes #3859.
2024-03-14 15:04:46 -04:00
Felix Handte
515c07a131
Merge pull request #3964 from felixhandte/promote-tgt-c-blk-size-to-stable
Promote `ZSTD_c_targetCBlockSize` Parameter to Stable API
2024-03-14 13:06:46 -04:00
Felix Handte
490163ac73
Merge pull request #3963 from felixhandte/also-handle-hidden-files-output-dir-mirror
Remove Erroneous Exclusion of Hidden Files and Folders in `--output-dir-mirror`
2024-03-14 10:21:00 -04:00
Yann Collet
216099a73f
Merge pull request #3962 from facebook/cover_lessIncludes
reduce the amount of #include in cover.h
2024-03-13 16:52:54 -07:00
Felix Handte
04d91d5219
Merge pull request #3960 from felixhandte/use-utimensat-on-posix-2001
Use `utimensat()` on FreeBSD
2024-03-13 17:10:35 -04:00
W. Felix Handte
3613448fb8 Promote ZSTD_c_targetCBlockSize Parameter to Stable API
This feature has demonstrated itself to be useful in web compression and we
want to encourage other folks to use it. But we currently make it difficult
to do so since it's locked away in the experimental API.

The API itself is really straightforward and I think it's fine to commit to
maintaining support / compatibility for this API even if in the future the
underlying implementation may continue to evolve.

Note that this commit changes its enum name and also its numeric value. Users
who respected the instructions of using the experimental API should be fine
with both of these changes since they should only have referred to it by the.

Conceivably someone could have done bad feature detection of this capability
by doing `#ifdef ZSTD_c_targetCBlockSize` which will now return false since
it's no longer a macro... but I think that's an acceptable hypothetical
breakage.
2024-03-13 17:07:10 -04:00
W. Felix Handte
86b8e39a84 Remove Erroneous Exclusion of Hidden Files and Folders in --output-dir-mirror 2024-03-13 16:33:30 -04:00