1
0
mirror of https://github.com/facebook/zstd.git synced 2025-07-04 06:40:29 +02:00
Commit Graph

49 Commits

Author SHA1 Message Date
4c25ea329b Disable unused variable warning in msan configurations 2023-04-20 11:14:08 +01:00
9420bce8a4 Add init once memory (#3528) (#3529)
- Adds memory type that is guaranteed to have been initialized at least once in the workspace's lifetime.
- Changes tag space in row hash to be based on init once memory.
2023-03-13 13:20:49 -07:00
c78f434aa4 Fix zstd-dll build missing dependencies (#3496)
* Fixes zstd-dll build (https://github.com/facebook/zstd/issues/3492):
- Adds pool.o and threading.o dependency to the zstd-dll target
- Moves custom allocation functions into header to avoid needing to add dependency on common.o
- Adds test target for zstd-dll
- Adds github workflow that buildis zstd-dll
2023-02-12 12:32:31 -08:00
1d636b4ba0 Bug fix redzones by unpoisoning only the intended buffer and not the followup redzone. 2023-01-24 12:54:43 -08:00
5d693cc38c Coalesce Almost All Copyright Notices to Standard Phrasing
```
for f in $(find . \( -path ./.git -o -path ./tests/fuzz/corpora -o -path ./tests/regression/data-cache -o -path ./tests/regression/cache \) -prune -o -type f); do sed -i '/Copyright .* \(Yann Collet\)\|\(Meta Platforms\)/ s/Copyright .*/Copyright (c) Meta Platforms, Inc. and affiliates./' $f; done

git checkout HEAD -- build/VS2010/libzstd-dll/libzstd-dll.rc build/VS2010/zstd/zstd.rc tests/test-license.py contrib/linux-kernel/test/include/linux/xxhash.h examples/streaming_compression_thread_pool.c lib/legacy/zstd_v0*.c lib/legacy/zstd_v0*.h
nano ./programs/windres/zstd.rc
nano ./build/VS2010/zstd/zstd.rc
nano ./build/VS2010/libzstd-dll/libzstd-dll.rc
```
2022-12-20 12:52:34 -05:00
8927f985ff Update Copyright Headers 'Facebook' -> 'Meta Platforms'
```
for f in $(find . \( -path ./.git -o -path ./tests/fuzz/corpora \) -prune -o -type f);
do
  sed -i 's/Facebook, Inc\./Meta Platforms, Inc. and affiliates./' $f;
done
```
2022-12-20 12:37:57 -05:00
e9dd923fa4 only declare debug functions in debug mode 2022-01-26 14:47:24 -08:00
41ad7332dd Updated expression for better readability 2022-01-04 09:07:11 -08:00
8c53e526db fix performance issue in scenario #2966 (part 1)
When re-using a compression state, across multiple successive compressions,
the state should minimize the amount of allocation and initialization required.

This mostly matters in situations where initialization is an overwhelming task
compared to compression itself.
This can happen when the amount to compress is small,
while the compression state was given the impression that it would be much larger,
aka, streaming mode without providing a srcSize hint.

This lean-initialization optimization was broken in 980f3bbf83 .

This commit fixes it, making this scenario once again on par with v1.4.9.

Note that this does not completely fix #2966,
since another heavy initialization, specific to row mode,
is also happening (and was not present in v1.4.9).
This will be fixed in a separate commit.
2021-12-31 15:16:19 -08:00
30b9db8ae4 changed macro name to ZSTD_ALIGNOF
for better consistency
2021-12-02 12:57:42 -08:00
e89e847820 added alignment test
and fix an incorrect alignment check in cwksp which was failing on m68k
2021-12-01 17:16:36 -08:00
ebbd675998 Fix typos 2021-11-13 10:04:04 +02:00
sen
980f3bbf83 [cwksp] Align all allocated "tables" and "aligneds" to 64 bytes (#2546)
* Perform 64-byte alignment of wksp tables and aligneds internally

* Clean up cwskp_finalize() function to only do two allocs

* Refactor aligned/buffer reservation code, remove ASAN req for alignment reservations

* Change from allocating 128 bytes always to allocating only buffer space as needed for tables/aligned

* Back out aligned/table reservation order restriction

* Add stricter bounds for new/resized wksps, fix comment in zstd_cwksp.h
2021-04-01 20:07:19 -04:00
a494308ae9 [copyright][license] Switch to yearless copyright and some cleanup in the linux-kernel files
* Switch to yearless copyright per FB policy
* Fix up SPDX-License-Identifier lines in `contrib/linux-kernel` sources
* Add zstd copyright/license header to the `contrib/linux-kernel` sources
* Update the `tests/test-license.py` to check for yearless copyright
* Improvements to `tests/test-license.py`
* Check `contrib/linux-kernel` in `tests/test-license.py`
2021-03-30 10:30:43 -07:00
f9802d80a0 fix typos (work done by Andrea Gelmini) 2021-01-07 18:47:23 +01:00
66e811d782 [license] Update year to 2021 2021-01-04 17:53:52 -05:00
9dab03db90 Create Enum to Represent Static/Dynamic Allocation Distinction in cwksp 2020-12-09 14:57:37 -05:00
db9e73cb07 Don't ASAN-Poison Statically-Allocated Workspaces
Addresses #2286.
2020-12-09 13:00:47 -05:00
6bd6b6f7d3 [cwksp] Return NULL when 0 bytes are requested
This ensures that the buffer is never used.
2020-10-30 10:55:34 -07:00
caecd8c211 Allow user to override ASAN/MSAN detection
Rename ADDRESS_SANITIZER -> ZSTD_ADDRESS_SANITIZER and same for
MEMORY_SANITIZER. Also set it to 0/1 instead of checking for defined.
This allows the user to override ASAN/MSAN detection for platforms that
don't support it.
2020-09-24 19:42:04 -07:00
c465f24457 ZSTD_ prefix mem{cpy,move,set},malloc,calloc,free 2020-08-26 12:26:03 -07:00
a686d306d2 Rename ZSTD_{malloc,calloc,free} to ZSTD_custom{Malloc,Calloc,Free} 2020-08-26 12:25:08 -07:00
1aa6c7ccce Assert We Allocated Approximately What We Expected To 2020-05-12 16:55:03 -04:00
6028827fee Rewrite Include Paths to be Relative
Addresses #1998.
2020-05-04 15:20:26 -04:00
5e5f262612 Add (Possibly Empty) Info Strings to All Variadic Error Handling Macro Invocations 2020-05-04 10:58:55 -04:00
0154866749 moving consts to zstd_internal and reusing them 2020-04-03 14:26:15 -07:00
ac58c8d720 Fix copyright and license lines
* All copyright lines now have -2020 instead of -present
* All copyright lines include "Facebook, Inc"
* All licenses are now standardized

The copyright in `threading.{h,c}` is not changed because it comes from
zstdmt.

The copyright and license of `divsufsort.{h,c}` is not changed.
2020-03-26 17:02:06 -07:00
6cf04c0344 Merge pull request #1834 from facebook/winFix
Windows fixes
2019-10-21 13:45:17 -07:00
1795133c45 refactored FIO_compressMultipleFilenames() prototype
for consistency
2019-10-17 15:32:03 -07:00
bd6a20b8a0 Expand Default Redzone Size 2019-10-10 13:45:55 -04:00
0ffae7e440 Stop Allocating Extra Space for Table Redzones 2019-10-10 13:40:16 -04:00
a07037b784 Don't Try to Redzone the Tables 2019-10-10 13:40:16 -04:00
8cffd6ed08 Avoid ASAN Failure in ZSTD_cwksp_free() 2019-10-10 13:40:16 -04:00
ef0b5707c5 Refactor Freeing CCtxes / CDicts Inside Workspaces 2019-10-10 13:40:16 -04:00
143b296cf6 Surround Workspace Allocs with Dead Zone 2019-10-10 13:40:16 -04:00
19a0955ec9 Add ZSTD_cwksp_alloc_size() to Help Calculate Needed Workspace Size 2019-10-10 13:40:16 -04:00
35c30d6ca7 Poison Unused Workspace Memory 2019-10-10 13:40:16 -04:00
20c69077d1 Shrink Table Valid End During Alloc Alignment / Phase Change 2019-09-11 17:14:59 -04:00
51d90668ba Add Assertions to Confirm that Workspace Pointers are Correctly Ordered 2019-09-11 17:14:59 -04:00
a10c191613 __msan_poison() Workspace When Preparing for Re-Use 2019-09-11 17:14:45 -04:00
1999b2ed9b Update DEBUGLOG Statements 2019-09-11 11:21:00 -04:00
17b6da2e0f Track Usable Table Space in Compression Workspace 2019-09-10 18:25:37 -04:00
81208fd7c2 Forward Declare ZSTD_cwksp_available_space to Fix Build 2019-09-09 19:10:09 -04:00
91bf1babd1 Inline Workspace Functions 2019-09-09 18:53:53 -04:00
1120e4d962 Clean Up TODOs and Comments pt. II 2019-09-09 14:04:39 -04:00
8549ae9f1d Hide Workspace Movement Behind Helper Function 2019-09-09 13:34:08 -04:00
901bba4ca6 Re-Implement Workspace Shrinking when Oversized 2019-09-09 13:34:08 -04:00
881bcd80ca Cleanup from Move 2019-09-09 13:34:08 -04:00
b511a84adc Move Workspace Functions to Their Own File 2019-09-09 13:34:08 -04:00