From 2500dcfa5f6f99dc015231d5561c35d9a4016965 Mon Sep 17 00:00:00 2001 From: Sean Purcell Date: Thu, 9 Mar 2017 13:59:26 -0800 Subject: [PATCH] Add testing description --- TESTING.md | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100644 TESTING.md diff --git a/TESTING.md b/TESTING.md new file mode 100644 index 000000000..1fa5fe8c2 --- /dev/null +++ b/TESTING.md @@ -0,0 +1,44 @@ +Testing +======= + +Zstandard CI testing is split up into three sections: +short, medium, and long tests. + +Short Tests +----------- +Short tests run on CircleCI for new commits on every branch and pull request. +They consist of the following tests: +- Compilation on all supported targets (x86, x86_64, ARM, AArch64, PowerPC, and PowerPC64) +- Compilation on various versions of gcc, clang, and g++ +- `tests/playTests.sh` on x86_64, without the tests on long data (CLI tests) +- Small tests (`tests/legacy.c`, `tests/longmatch.c`, `tests/symbols.c`) on x64_64 + +Medium Tests +------------ +Medium tests run on every commit and pull request to `dev` branch, on TravisCI. +They consist of the following tests: +- The following tests run with UBsan and Asan on x86_64 and x86, as well as with + Msan on x86_64 + - `tests/playTests.sh --test-long-data` + - Fuzzer tests: `tests/fuzzer.c`, `tests/zstreamtest.c`, and `tests/decodecorpus.c` +- `tests/zstreamtest.c` under Tsan (streaming mode, including multithreaded mode) +- Valgrind Test (`make -C tests valgrindTest`) (testing CLI and fuzzer under valgrind) +- Fuzzer tests (see above) on ARM, AArch64, PowerPC, and PowerPC64 + +Long Tests +---------- +Long tests run on all commits to `master` branch, +and once a day on the current version of `dev` branch, +on TravisCI. +They consist of the following tests: +- Entire test suite (including fuzzers and some other specialized tests) on: + - x86_64 and x86 with UBsan and Asan + - x86_64 with Msan + - ARM, AArch64, PowerPC, and PowerPC64 +- Streaming mode fuzzer with Tsan (for the `zstdmt` testing) +- ZlibWrapper tests, including under valgrind +- Versions test (ensuring `zstd` can decode files from all previous versions) +- `pzstd` with asan and tsan, as well as in 32-bits mode +- Testing `zstd` with legacy mode off +- Testing `zbuff` (old streaming API) +- Entire test suite and make install on OS X