1
0
mirror of https://github.com/facebook/zstd.git synced 2025-07-04 06:40:29 +02:00
Commit Graph

53 Commits

Author SHA1 Message Date
c5bf6b8b88 add requested check for legacy decoder v0.1
which uses a different technique to store literals,
and therefore must check for potential overwrites.
2023-02-07 14:47:16 -08:00
7eb4471fec adapt v0.3 fix to v0.1
slightly different constraints on end of buffer conditions
2023-02-07 13:55:30 -08:00
8927f985ff Update Copyright Headers 'Facebook' -> 'Meta Platforms'
```
for f in $(find . \( -path ./.git -o -path ./tests/fuzz/corpora \) -prune -o -type f);
do
  sed -i 's/Facebook, Inc\./Meta Platforms, Inc. and affiliates./' $f;
done
```
2022-12-20 12:37:57 -05:00
e2fc93340f Merge branch 'dev' into http-to-https 2022-12-15 10:46:13 -05:00
728e73ebb4 [legacy] Remove FORCE_MEMORY_ACCESS and only use memcpy
Delete unaligned memory access code from the legacy codebase by removing all the
non-memcpy functions. We don't care about speed at all for this codebase, only
simplicity.
2022-12-14 17:54:35 -08:00
a78c91ae59 Use proper unaligned access attributes
Instead of using packed attribute hack, just use aligned attribute. It
improves code generation on armv6 and armv7, and slightly improves code
generation on aarch64. GCC generates identical code to regular aligned
access on ARMv6 for all versions between 4.5 and trunk, except GCC 5
which is buggy and generates the same (bad) code as packed access:
https://gcc.godbolt.org/z/hq37rz7sb
2022-12-14 16:00:37 -08:00
4dffc35f2e Convert references to https from http 2022-12-14 06:58:35 -08:00
b6b4c9a3da Merge pull request #2907 from Hello71/armv6-fix-legacy
Apply FORCE_MEMORY_ACCESS=1 to legacy
2021-12-06 15:41:22 -05:00
3d773d7013 Apply FORCE_MEMORY_ACCESS=1 to legacy
See #2633, #2881.
2021-12-05 22:51:44 -05:00
95f492ea17 Don't initialize the first parameter of _BitScanReverse* functions
Like the document example, no need to initialize `r` to 0.
https://docs.microsoft.com/en-us/cpp/intrinsics/bitscanreverse-bitscanreverse64
2021-09-25 16:36:53 +08:00
a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
cc907770bd Fix building on AIX 5.1 2020-10-09 18:34:00 +08:00
5717bd39ee [lib] Fix NULL pointer dereference
When the output buffer is `NULL` with size 0, but the frame content size
is non-zero, we will write to the NULL pointer because our bounds check
underflowed.

This was exposed by a recent PR that allowed an empty frame into the
single-pass shortcut in streaming mode.

* Fix the bug.
* Fix another NULL dereference in zstd-v1.
* Overflow checks in 32-bit mode.
* Add a dedicated test.
* Expose the bug in the dedicated simple_decompress fuzzer.
* Switch all mallocs in fuzzers to return NULL for size=0.
* Fix a new timeout in a fuzzer.

Neither clang nor gcc show a decompression speed regression on x86-64.
On x86-32 clang is slightly positive and gcc loses 2.5% of speed.

Credit to OSS-Fuzz.
2020-05-06 12:09:02 -07:00
6028827fee Rewrite Include Paths to be Relative
Addresses #1998.
2020-05-04 15:20:26 -04:00
c7da66c9cf Purge C++-Style Comments (// ...), Make Compilation Succeed Under C90 2020-05-04 10:59:15 -04:00
ac58c8d720 Fix copyright and license lines
* All copyright lines now have -2020 instead of -present
* All copyright lines include "Facebook, Inc"
* All licenses are now standardized

The copyright in `threading.{h,c}` is not changed because it comes from
zstdmt.

The copyright and license of `divsufsort.{h,c}` is not changed.
2020-03-26 17:02:06 -07:00
1f7228c040 Use clz ^ 31 instead of 31 - clz; better codegen for GCC 2019-09-23 21:23:09 +02:00
0fd322f812 [legacy] Fix ZSTDv0*_decodeSequence()
* Version <= 0.5 could read beyond the end of `dumps`, which points into
  the input buffer.
* Check the validity of `dumps` before using it, if it is out of bounds
  return garbage values. There is no return code for this function.
* Introduce `MEM_readLE24()` for simplicity, since I don't want to trust
  that there is an extra byte after `dumps`.
2019-04-19 11:34:52 -07:00
2536771134 [legacy] Fix Huffman jump table reads in v01 and v05 2019-04-18 16:20:42 -07:00
a880ca239b Spelling (#1582)
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
2019-04-12 11:18:11 -07:00
0a3fa6f909 Add legacy mode in documentation 2019-03-18 20:33:15 -07:00
20aa1b455c Stylistic changes 2019-03-17 19:35:43 -07:00
60796e76b0 Add legacy support to decompressBound 2019-03-15 16:10:37 -07:00
73773c6b6a fixed legacy compilation tests
for some reason, these tests started failing recently on CircleCI
2018-09-27 18:15:14 -07:00
f2d6db45cd [zstd] Add -Wmissing-prototypes 2018-09-27 15:24:48 -07:00
c173dbd6e7 no longer supported starting C++17 2017-12-04 18:00:53 -08:00
3128e03be6 updated license header
to clarify dual-license meaning as "or"
2017-09-08 00:09:23 -07:00
32fb407c9d updated a bunch of headers
for the new license
2017-08-18 16:52:05 -07:00
5152fb2cb2 Convert all tabs to spaces 2017-03-29 18:51:58 -07:00
9050e1925e Change name to to findFrameCompressedSize and add skippable support 2017-02-22 12:12:34 -08:00
d7bfcac18a Expose frameSrcSize to experimental API 2017-02-10 11:55:44 -08:00
4e709712e1 Decompressed size functions now handle multiframes and distinguish cases
- Add ZSTD_findDecompressedSize
    - Traverses multiple frames to find total output size
- Add ZSTD_getFrameContentSize
    - Gets the decompressed size of a single frame by reading header
- Deprecate ZSTD_getDecompressedSize
2017-02-08 14:50:10 -08:00
35168679bd Merge pull request #478 from terrelln/wildcopy-ub
Fix execSequence wildcopy undefined behavior
2016-12-13 11:33:00 +01:00
064a143520 Fix execSequence wildcopy undefined behavior
execSequence relied on pointer overflow to handle cases where
`sequence.matchLength < 8`.  Instead of passing an `size_t` to
wildcopy, pass a `ptrdiff_t`.
2016-12-12 19:01:23 -08:00
e474aa55b4 Fix decompression buffer overrun
Allows an adversary to write up to 3 bytes beyond the end of the buffer.
Occurs if the match overlaps the `extDict` and `currentPrefix`, and the
match length in the `currentPrefix` is less than `MINMATCH`, and
`op-(16-MINMATCH) >= oMatchEnd > op-16`.
2016-12-12 18:05:30 -08:00
d760529a05 Fix stack buffer overrun when weightTotal == 0
If `weightTotal == 0`, then `BIT_highbit32(weightTotal)` is
undefined behavior in the case that it calls `__builtin_clz()`.
If `tableLog == HUF_TABLELOG_ABSOLUTEMAX` then we will access one
byte beyond the end of the buffer.
2016-10-19 11:39:11 -07:00
bb68062c59 Unitialized memory read in ZSTD_decodeSeqHeaders()
Caused by two things:
1. Not checking that `ip` is in range except for the first byte.
2. `ZSTDv0{5,6}_decodeLiteralsBlock()` could return a value larger than `srcSize`.
2016-10-18 16:41:33 -07:00
ccfcc643da Check if dict is empty before reading first byte 2016-10-17 11:46:03 -07:00
45db83f98d ZSTD_decodeLiteralsBlock renamed to ZSTDv01_decodeLiteralsBlock 2016-09-05 14:46:24 +02:00
476964f6a1 ZSTD_decodeSeqHeaders renamed to ZSTDv01_decodeSeqHeaders 2016-09-05 13:34:57 +02:00
c13faa1b0f legacy decoders: restored #include <intrin.h> for VC++ 2016-09-05 13:25:07 +02:00
8161e7321a unified error codes for legacy decoders 2016-09-05 12:29:51 +02:00
1563bfeabc fixing FORCE_INLINE for older compilers (#330) 2016-09-02 11:44:21 -07:00
4ded9e591c added boilerplate 2016-08-30 11:06:28 -07:00
87c18b2ebd fixed multiple minor warnings for XCode 2016-08-26 01:43:47 +02:00
48849f86f0 fixed compilation with Intel Compiler with Windows 2016-08-10 14:26:35 +02:00
ddb8ebd5b3 Stricter gnu90 tests 2016-05-05 04:59:53 +02:00
ffec740d37 fixed visual and clang errors 2016-01-21 15:50:11 +01:00
494c786a3b fixed outstanding strict aliasing warnings 2016-01-06 12:54:02 +01:00