* fix and test MSVC AVX2 build
* treat msbuild warnings as errors
* fix incorrect MSVC 2019 compiler warning
* fix MSVC error D9035: option 'Gm' has been deprecated and will be removed in a future release
Tell CMake to explicitly compile our assembly as C code, because we
require it is compiled by a C compiler, and it is only enabled for
clang/gcc.
Fixes#3193.
```
for f in $(find . \( -path ./.git -o -path ./tests/fuzz/corpora \) -prune -o -type f);
do
sed -i 's/Facebook, Inc\./Meta Platforms, Inc. and affiliates./' $f;
done
```
playTests.sh has an option to run really slow tests. This is enabled by
default in Meson, but what we really want is to do like the Makefile,
and run the fast ones by default, but with an option to run the slow
ones instead.
It's entirely possible some people don't have valgrind installed, but
still want to run the tests. If they don't have it installed, then they
probably don't intend to run those precise test targets anyway.
Also, this solves an error when running the tests in an automated
environment. The valgrind tests have a hard dependency on behavior such
as `./zstd` erroring out with the message "stdin is a console, aborting"
which does not work if the automated environment doesn't have a console.
As a rough heuristic, automated environments lacking a console will
*probably* also not have valgrind, so avoiding that test definition
neatly sidesteps the issue.
Also, valgrind is not easily installable on macOS, at least homebrew
says it isn't available there. This makes it needlessly hard to
enable the testsuite on macOS.
In commit 031de3c69ccbf3282ed02fb49369b476730aeca8 a feature of Meson
0.50.0 was added, but the minimum specified version of Meson is 0.48.0.
Meson therefore emitted a warning:
WARNING: Project targets '>=0.48.0' but uses feature introduced in '0.50.0': required arg in compiler.has_header.
And if anyone actually used Meson 0.48.0 to build with, it would error
out with mysterious claims that the build file itself is invalid, rather
than telling the user to install a newer version of Meson.
Solve this by bumping the minimum version to align with reality. This
e.g. drops support for Debian oldstable (buster)'s packaged version of
Meson, but still works if backports are enabled, or if the user can
`pip install` a newer version.
In commit 031de3c69ccbf3282ed02fb49369b476730aeca8 some code was added
that returned a boolean, but was treated as if it returned a dependency
object. This wasn't tested and could not work. Moreover, zstd no longer
built at all unless the entire programs directory was disabled and not
even evaluated.
Fix the return type checking.
It uses non-portable compiler options unconditionally. Elsewhere, we
check the compiler ID and only add the right ones, globally. Do the same
here.
NDEBUG can actually be handled by a core option, so while we are moving
things around, do so.
Unfortunately, this doesn't fix things entirely. The remaining issue is
not Meson's issue though -- MSVC simply does not like this source code
and somehow chokes on innocent code with the inscrutable "syntax error"
and "illegal token".
Run the scraper command to establish the project version immediately,
rather than wait for the build to be configured. This simplifies the
code and ensures that project introspection works correctly.
With this patch the pkg-config generation when using the CMake build
system is improved in the following ways:
- Libs.private is now filled when needed
- The JoinPaths module is now used to join paths, leading to simpler
code
- The .pc file is always generated, regardless of the platform, as it
can also be consumed on Windows
Here's how the .pc file is affected by these changes, in comparison to
the one generated with the official Makefiles:
$ diff -s lib/libzstd.pc build/cmake/build-old/lib/libzstd.pc
15c15
< Libs.private: -pthread
---
> Libs.private:
$ diff -s lib/libzstd.pc build/cmake/build-new/lib/libzstd.pc
Files lib/libzstd.pc and build/cmake/build-new/lib/libzstd.pc are
identical
Partial, Meson-only implementation of #2976 for non-MSVC builds.
Due to the prevalence of private symbol reuse, linking to a shared
library is simply utterly unreliable, but we still want to defer to the
shared library for installable applications. By linking to both, we can
share symbols where possible, and statically link where needed.
This means we no longer need to manually track every file that needs to
be extracted and reused.
The flip side is that MSVC completely does not support this, so for MSVC
builds we just link to a full static copy even where
-Ddefault_library=shared.
As a side benefit, by using library inclusion rather than including
extra explicit object files, the zstd program shrinks in size slightly
(~4kb).
These need to be explicitly included as we use their private symbols,
but we don't need to recompile them when we can reuse the existing
objects.
Minus 7 compile steps.
The poolTests program already linked to libzstd, and later to
libtestcommon with included libzstd objects. So this was redundant.
Minus 4 compile steps.
While trying to raise an exception on failures, it instead raised an
exception for misusing the exception. CalledProcessError is only
supposed to be used when given a return code and a command, and it
prints:
Command '{cmd}' returned non-zero exit status {ret}
Passing an error message string instead, just errored out with:
TypeError: __init__() missing 1 required positional argument
Instead use the subprocess module's base error which does accept string
messages. Everything that used to error out, still errors out, but now
they do so with a slightly prettier console message.
libm is not guaranteed to exist. POSIX requires the math functions to
exist, but doesn't require to have it be a standalone library.
On platforms where libm exists as a standalone library, it will always
be found by meson -- it is shipped with libc.
If it is not found, then we can safely assume the linker will make the
math functions available by default.
See https://mesonbuild.com/howtox.html#add-math-library-lm-portably
Fixes building with bin_tests=true on Windows.
* Refactored fileio.c:
- Extracted asyncio code to fileio_asyncio.c/.h
- Moved type definitions to fileio_types.h
- Moved common macro definitions needed by both fileio.c and fileio_asyncio.c to fileio_common.h
* Bugfix - rename fileio_asycio to fileio_asyncio
* Added copyrights & license to new files
* CR fixes
* Async IO decompression:
- Added --[no-]asyncio flag for CLI decompression.
- Replaced dstBuffer in decompression with a pool of write jobs.
- Added an ability to execute write jobs in a separate thread.
- Added an ability to wait (join) on all jobs in a thread pool (queued and running).
When testing amalgamating other projects it was found: invalid Unicode errors were tripping Python's text IO, and the header search order appears differs from the shell version.
