zstd/tests/cli-tests/compression/golden.sh

#!/bin/sh

set -e

GOLDEN_DIR="$ZSTD_REPO_DIR/tests/golden-compression/"
cp -r "$GOLDEN_DIR" golden/

zstd -rf golden/ --output-dir-mirror golden-compressed/
zstd -r -t golden-compressed/

zstd --target-compressed-block-size=1024 -rf golden/ --output-dir-mirror golden-compressed/
zstd -r -t golden-compressed/

# PR #3517 block splitter corruption test
zstd -rf -19 --zstd=mml=7 golden/ --output-dir-mirror golden-compressed/
zstd -r -t golden-compressed/
Fix off-by-one error in superblock mode (#3221) Fixes #3212. Long literal and match lengths had an off-by-one error in ZSTD_getSequenceLength. Fix the off-by-one error, and add a golden compression test that catches the bug. Also run all the golden tests in the cli-tests framework. 2022-08-03 11:28:39 -07:00			`#!/bin/sh`

			`set -e`

			`GOLDEN_DIR="$ZSTD_REPO_DIR/tests/golden-compression/"`
			`cp -r "$GOLDEN_DIR" golden/`

			`zstd -rf golden/ --output-dir-mirror golden-compressed/`
			`zstd -r -t golden-compressed/`

			`zstd --target-compressed-block-size=1024 -rf golden/ --output-dir-mirror golden-compressed/`
			`zstd -r -t golden-compressed/`
[bug-fix] Fix rare corruption bug affecting the block splitter The block splitter confuses sequences with literal length == 65536 that use a repeat offset code. It interprets this as literal length == 0 when deciding the meaning of the repeat offset, and corrupts the repeat offset history. This is benign, merely causing suboptimal compression performance, if the confused history is flushed before the end of the block, e.g. if there are 3 consecutive non-repeat code sequences after the mistake. It also is only triggered if the block splitter decided to split the block. All that to say: This is a rare bug, and requires quite a few conditions to trigger. However, the good news is that if you have a way to validate that the decompressed data is correct, e.g. you've enabled zstd's checksum or have a checksum elsewhere, the original data is very likely recoverable. So if you were affected by this bug please reach out. The fix is to remind the block splitter that the literal length is actually 64K. The test case is a bit tricky to set up, but I've managed to reproduce the issue. Thanks to @danlark1 for alerting us to the issue and providing us a reproducer! 2023-02-22 14:49:08 -08:00
			`# PR #3517 block splitter corruption test`
			`zstd -rf -19 --zstd=mml=7 golden/ --output-dir-mirror golden-compressed/`
			`zstd -r -t golden-compressed/`