1
0
mirror of https://github.com/facebook/zstd.git synced 2025-03-07 01:10:04 +02:00

10180 Commits

Author SHA1 Message Date
Dimitri Papadopoulos
547794ef40
Fix typos found by codespell 2023-02-18 10:31:48 +01:00
Yann Collet
4ebaf36582
Merge pull request #3490 from eli-schwartz/meson-tests-noprograms
meson: always build the zstd binary when tests are enabled
2023-02-16 11:27:27 -08:00
Sutou Kouhei
8420502ef9 Don't require CMake 3.18 or later
fix #3500

CMake 3.18 or later was required by #3392. Because it uses
`CheckLinkerFlag`. But requiring CMake 3.18 or later is a bit
aggressive. Because Ubuntu 20.04 LTS still uses CMake 3.16.3:
https://packages.ubuntu.com/search?keywords=cmake

This change disables `-z noexecstack` check with old CMake. This will
not break any existing users. Because users who need `-z noexecstack`
must already use CMake 3.18 or later.
2023-02-16 10:08:45 -08:00
Felix Handte
1c42844668
Merge pull request #3479 from felixhandte/faster-file-ops
Use `f`-variants of `chmod()` and `chown()`
2023-02-16 13:07:34 -05:00
Felix Handte
3c50854c05
Merge pull request #3511 from felixhandte/fix-release-artifact-upload-permission
Fix Permissions on Publish Release Artifacts Job
2023-02-15 13:35:04 -05:00
daniellerozenblit
345ed63976
Merge pull request #3509 from daniellerozenblit/fix-window-resize-test
Fix cli-tests issues
2023-02-15 13:32:07 -05:00
W. Felix Handte
d54ad3c234 Fix Permissions on Publish Release Artifacts Job
Publishing release artifacts requires the `contents` permission, as documented
by: https://docs.github.com/en/rest/overview/permissions-required-for-github-apps.
2023-02-15 11:05:54 -05:00
Danielle Rozenblit
d3d0b92e5e add make test for 32bit 2023-02-15 06:03:02 -08:00
Danielle Rozenblit
7da1c6ddbf fix cli-tests issues 2023-02-14 11:33:26 -08:00
Danielle Rozenblit
2d8afd9ce1 add manual flag to mmap dictionary 2023-02-14 09:42:23 -08:00
Yonatan Komornik
6a86db11a4
CI workflow to test external compressors dependencies
Implemented CI workflow for testing compilation with external compressors and without them. This serves as a sanity check to avoid any code dependencies on libraries that may not always be present. (Reference: #3497 for a bug fix related to this issue.)
2023-02-13 18:00:13 -08:00
Yonatan Komornik
727d03161f
Make Github workflows permissions read-only by default (#3488)
* Make Github workflows permissions read-only by default

* Pins `skx/github-action-publish-binaries` action to specific hash
2023-02-13 16:57:05 -08:00
Alex Xu
886de7bc04
Use correct types in LZMA comp/decomp (#3497)
Bytef and uInt are zlib types, not available when zlib is disabled

Fixes: 1598e6c634ac ("Async write for decompression")
Fixes: cc0657f27d81 ("AsyncIO compression part 2 - added async read and asyncio to compression code (#3022)")
2023-02-13 16:30:56 -08:00
Danielle Rozenblit
8a189b1b29 refactor dictionary file stat 2023-02-13 15:23:06 -08:00
Yann Collet
30cb0a4f25
Merge pull request #3491 from facebook/win_artifacts
Github Action to generate Win64 artifacts
2023-02-13 11:30:47 -08:00
Danielle Rozenblit
4373c5ab88 mmap dicitonaries for large dictionaries in patch-from 2023-02-13 07:26:22 -08:00
dependabot[bot]
6894746eb1
Bump github/codeql-action from 2.2.1 to 2.2.4 (#3503)
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2.2.1 to 2.2.4.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](3ebbd71c74...17573ee1cc)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-13 06:35:31 -05:00
Yonatan Komornik
c78f434aa4
Fix zstd-dll build missing dependencies (#3496)
* Fixes zstd-dll build (https://github.com/facebook/zstd/issues/3492):
- Adds pool.o and threading.o dependency to the zstd-dll target
- Moves custom allocation functions into header to avoid needing to add dependency on common.o
- Adds test target for zstd-dll
- Adds github workflow that buildis zstd-dll
2023-02-12 12:32:31 -08:00
Elliot Gorokhovsky
a7de1d9f49
Fix all MSVC warnings (#3495)
* fix and test MSVC AVX2 build

* treat msbuild warnings as errors

* fix incorrect MSVC 2019 compiler warning

* fix MSVC error D9035: option 'Gm' has been deprecated and will be removed in a future release
2023-02-11 10:56:59 -05:00
Yann Collet
f8ae21680f try a different compiler 2023-02-10 20:49:21 -08:00
Yann Collet
5be3f19e1d simplify win64 artifact generation 2023-02-10 16:58:54 -08:00
Yann Collet
43bc470fe0 move shell to mingw 2023-02-10 14:27:47 -08:00
Yann Collet
f37b291bf5 Github Action to generate Win64 artifacts 2023-02-09 22:48:30 -08:00
Eli Schwartz
97ab0e2ab6
meson: always build the zstd binary when tests are enabled
We need to run it for the tests, even if programs are disabled. So if
they are disabled, create a build rule for the program, but don't
install it. Just make it available for the test itself.
2023-02-10 00:33:07 -05:00
Eli Schwartz
183a18a45c
meson: correctly specify the dependency relationship for playtests
It depends on the zstd program being built, and passes it as an env
variable. Just like datagen. But for datagen, we explicitly depend on
it, while for zstd, we assume it's built as part of "all".

This can be wrong in two cases:
- when running individual tests, meson can (re)build just what is needed
  for that one test
- a later patch will handle building zstd but not by default
2023-02-10 00:33:07 -05:00
Yann Collet
515266e31b minor nit updates to README.md 2023-02-09 15:51:18 -08:00
Yann Collet
95ffc767f6 updated man pages 2023-02-09 14:40:39 -08:00
Elliot Gorokhovsky
ff42ed1582
Rename "External Matchfinder" to "Block-Level Sequence Producer" (#3484)
* change "external matchfinder" to "external sequence producer"

* migrate contrib/ to new naming convention

* fix contrib build

* fix error message

* update debug strings

* fix def of invalid sequences in zstd.h

* nit

* update CHANGELOG

* fix .gitignore
2023-02-09 17:01:17 -05:00
Danielle Rozenblit
db3e1de729 initial commit 2023-02-09 07:47:21 -08:00
Danielle Rozenblit
cc4e941745 initial commit 2023-02-09 07:40:25 -08:00
Danielle Rozenblit
610c8b9e33 initial commit 2023-02-09 07:37:37 -08:00
Yann Collet
4aa3bc49da updated CHANGELOG in preparation for v1.5.4 release 2023-02-08 16:35:55 -08:00
Yann Collet
67fb9d9c77 Merge branch 'dev' of github.com:facebook/zstd into dev 2023-02-08 15:57:12 -08:00
Yann Collet
4de9d637e8 minor: fix missing newline character in help page 2023-02-08 15:56:49 -08:00
Nick Terrell
83f8a05f87 Fix empty-block.zst golden decompression file
This frame is invalid because the `Window_Size = 0`, and the
`Block_Maximum_Size = min(128 KB, Window_Size) = 0`. But the empty
compressed block has a `Block_Content` size of 2, which is invalid.

The fix is to switch to using a `Window_Descriptor` instead of the
`Single_Segment_Flag`. This sets the `Window_Size = 1024`.

Hexdump before this PR: `28b5 2ffd 2000 1500 0000 00`

Hexdump after this PR: `28b5 2ffd 0000 1500 0000 00`

For issue #3482.
2023-02-08 14:11:22 -08:00
Yann Collet
488f7c0ff1
Merge pull request #3476 from facebook/fix55714
fix oss-fuzz case 55714
2023-02-07 18:06:22 -08:00
Yann Collet
c689310b25 rewrite legacy v0.7 bound checks to be independent of address space overflow 2023-02-07 17:11:07 -08:00
Yann Collet
c5bf6b8b88 add requested check for legacy decoder v0.1
which uses a different technique to store literals,
and therefore must check for potential overwrites.
2023-02-07 14:47:16 -08:00
Yann Collet
9419747171 fix legacy decoders v0.4, v0.5 and v0.6 2023-02-07 14:02:12 -08:00
Yann Collet
67d7a659f8 port fix for v0.3 to v0.6
in case it would applicable for this version
2023-02-07 13:55:30 -08:00
Yann Collet
7a1a171658 port fix for v0.3 to v0.5
in case it would be applicable for this version too
2023-02-07 13:55:30 -08:00
Yann Collet
b20e4e95f2 copy fix for v0.3 to v0.4
in case it would be applicable for this legacy version too.
2023-02-07 13:55:30 -08:00
Yann Collet
7eb4471fec adapt v0.3 fix to v0.1
slightly different constraints on end of buffer conditions
2023-02-07 13:55:30 -08:00
Yann Collet
cfec005efd fix for v0.3 blindly ported to v0.2
in case it would be applicable here too.
2023-02-07 13:55:30 -08:00
Yann Collet
e04706c58c fix oss-fuzz case 55714
impacts legacy decoder v0.3 in 32-bit mode
2023-02-07 13:55:30 -08:00
Yann Collet
df21aceaa7
Merge pull request #3480 from facebook/benchFail
return error code when benchmark fails
2023-02-07 12:42:47 -08:00
Yann Collet
6740f8f0b8 add error message for the (rare) compression error scenario 2023-02-07 10:02:09 -08:00
Yann Collet
58e7067c7d added more accurate error messages
for the decompression-only benchmark mode.
2023-02-07 08:47:39 -08:00
Yann Collet
9cabd155fd return error code when benchmark fails
such scenario can happen, for example,
when trying a decompression-only benchmark on invalid data.
Other possibilities include an allocation error in an intermediate step.

So far, the benchmark would return immediately, but still return 0.
On command line, this would be confusing, as the program appears successful (though it does not display any successful message).

Now it returns !0, which can be interpreted as an error by command line.
2023-02-07 00:35:51 -08:00
W. Felix Handte
f746c37d00 Use File Descriptor in Setting Stat on Output File
Note that the `fd` is only valid while the file is still open. So we need to
move the setting calls to before we close the file. However! We cannot do so
with the `utime()` call (even though `futimens()` exists) because the follow-
ing `close()` call to the `fd` will reset the atime of the file. So it seems
the `utime()` call has to happen after the file is closed.
2023-02-06 14:02:31 -08:00