1
0
mirror of https://github.com/facebook/zstd.git synced 2025-03-07 09:26:03 +02:00
Elliot Gorokhovsky 2a402626dd
External matchfinder API (#3333)
* First building commit with sample matchfinder

* Set up ZSTD_externalMatchCtx struct

* move seqBuffer to ZSTD_Sequence*

* support non-contiguous dictionary

* clean up parens

* add clearExternalMatchfinder, handle allocation errors

* Add useExternalMatchfinder cParam

* validate useExternalMatchfinder cParam

* Disable LDM + external matchfinder

* Check for static CCtx

* Validate mState and mStateDestructor

* Improve LDM check to cover both branches

* Error API with optional fallback

* handle RLE properly for external matchfinder

* nit

* Move to a CDict-like model for resource ownership

* Add hidden useExternalMatchfinder bool to CCtx_params_s

* Eliminate malloc, move to cwksp allocation

* Handle CCtx reset properly

* Ensure seqStore has enough space for external sequences

* fix capitalization

* Add DEBUGLOG statements

* Add compressionLevel param to matchfinder API

* fix c99 issues and add a param combination error code

* nits

* Test external matchfinder API

* C90 compat for simpleExternalMatchFinder

* Fix some @nocommits and an ASAN bug

* nit

* nit

* nits

* forward declare copySequencesToSeqStore functions in zstd_compress_internal.h

* nit

* nit

* nits

* Update copyright headers

* Fix CMake zstreamtest build

* Fix copyright headers (again)

* typo

* Add externalMatchfinder demo program to make contrib

* Reduce memory consumption for small blockSize

* ZSTD_postProcessExternalMatchFinderResult nits

* test sum(matchlen) + sum(litlen) == srcSize in debug builds

* refExternalMatchFinder -> registerExternalMatchFinder

* C90 nit

* zstreamtest nits

* contrib nits

* contrib nits

* allow block splitter + external matchfinder, refactor

* add windowSize param

* add contrib/externalMatchfinder/README.md

* docs

* go back to old RLE heuristic because of the first block issue

* fix initializer element is not a constant expression

* ref contrib from zstd.h

* extremely pedantic compiler warning fix, meson fix, typo fix

* Additional docs on API limitations

* minor nits

* Refactor maxNbSeq calculation into a helper function

* Fix copyright
2022-12-28 16:45:14 -05:00
..
2022-12-23 14:18:11 -08:00
2022-12-28 16:45:14 -05:00
2018-10-03 18:42:44 -07:00
2022-03-12 08:58:04 +01:00

Cmake contributions

Contributions to the cmake build configurations are welcome. Please use case sensitivity that matches modern (i.e. cmake version 2.6 and above) conventions of using lower-case for commands, and upper-case for variables.

How to build

As cmake doesn't support command like cmake clean, it's recommended to perform an "out of source build". To do this, you can create a new directory and build in it:

cd build/cmake
mkdir builddir
cd builddir
cmake ..
make

Then you can clean all cmake caches by simply delete the new directory:

rm -rf build/cmake/builddir

And of course, you can directly build in build/cmake:

cd build/cmake
cmake
make

To show cmake build options, you can:

cd build/cmake/builddir
cmake -LH ..

Bool options can be set to ON/OFF with -D[option]=[ON/OFF]. You can configure cmake options like this:

cd build/cmake/builddir
cmake -DZSTD_BUILD_TESTS=ON -DZSTD_LEGACY_SUPPORT=OFF ..
make

referring

Looking for a 'cmake clean' command to clear up CMake output

CMake Style Recommendations

Indent all code correctly, i.e. the body of

  • if/else/endif
  • foreach/endforeach
  • while/endwhile
  • macro/endmacro
  • function/endfunction

Use spaces for indenting, 2, 3 or 4 spaces preferably. Use the same amount of spaces for indenting as is used in the rest of the file. Do not use tabs.

Upper/lower casing

Most important: use consistent upper- or lowercasing within one file !

In general, the all-lowercase style is preferred.

So, this is recommended:

add_executable(foo foo.c)

These forms are discouraged

ADD_EXECUTABLE(bar bar.c)
Add_Executable(hello hello.c)
aDd_ExEcUtAbLe(blub blub.c)

End commands

To make the code easier to read, use empty commands for endforeach(), endif(), endfunction(), endmacro() and endwhile(). Also, use empty else() commands.

For example, do this:

if(FOOVAR)
   some_command(...)
else()
   another_command(...)
endif()

and not this:

if(BARVAR)
   some_other_command(...)
endif(BARVAR)

Other resources for best practices

https://cmake.org/cmake/help/latest/manual/cmake-developer.7.html#modules