1
0
mirror of https://github.com/facebook/zstd.git synced 2025-07-09 00:45:44 +02:00
Commit Graph

106 Commits

Author SHA1 Message Date
c5fab8848a Document searchFuncs Table 2020-09-10 22:10:02 -04:00
85a95840e4 Further Consolidate Dict Mode Checks 2020-09-10 22:10:02 -04:00
efa33861f2 Attempt to Fix MSVC Warnings 2020-09-10 22:10:02 -04:00
ed43832770 Simplify Match Limit Checks
Seems like a ~1.25% speedup.
2020-09-10 22:10:02 -04:00
06d240b8a7 Use All Available Space in the Hash Table to Extent Chain Table Reach
Rather than restrict our temp chain table to 2 ** chainLog entries, this
commit uses all available space to reach further back to gather longer
chains to pack into the DDSS chain table.
2020-09-10 22:10:02 -04:00
b2b0641ea0 Rewrite Table Fill to Retain Cache Entries Beyond Chain Window 2020-09-10 22:10:02 -04:00
916238d9dc Avoid Malloc in Table Fill; Pack Tmp Structure into Hash Table 2020-09-10 22:10:02 -04:00
f42c5bddd9 Truncate Chain at Last Possible Attempt
Make the chain table denser?
2020-09-10 22:10:02 -04:00
20a020edbc Prefetch Chain Table Matches 2020-09-10 22:10:02 -04:00
9b9feb84f2 Lay Out Chain Table Chains Contiguously
Rather than interleave all of the chain table entries, tying each entry's
position to the corresponding position in the input, this commit changes the
layout so that all the entries in a single chain are laid out next to each
other. The last entry in the hash table's bucket for this hash is now a packed
pointer of position + length of this chain.

This cannot be merged as written, since it allocates temporary memory inside
ZSTD_dedicatedDictSearch_lazy_loadDictionary().
2020-09-10 22:10:02 -04:00
66509c7bf4 Only Insert Positions Inside the Chain Window 2020-09-10 22:10:02 -04:00
d214d8c859 Shorten Dict Mode Conditionals in Order to Improve Readability 2020-09-10 18:51:52 -04:00
f49c1563ff Force-Inline ZSTD_insertAndFindFirstIndex_internal()
Without this, gcc was declining to inline the function in `ZSTD_noDict` mode,
resulting in a ~10% slowdown.
2020-09-10 18:51:52 -04:00
cab86b074f Clean Up Search Function Selection 2020-09-10 18:51:52 -04:00
2ffbde0d95 Fix -Wshorten-64-to-32 Error 2020-09-10 18:51:52 -04:00
d332f57897 Permit Matching Against Lowest Valid Position
This comparison was previously faulty: the lowest valid position is itself
valid, and we should therefore be allowed to match against it.
2020-09-10 18:51:52 -04:00
7b9a755ac9 Remove Chain Limit on Hash Cache Entries; Slightly Improve Compression
Entries in the hashTable chain cache aren't subject to the same aliasing that
the circular chain table is subject to. As such, we don't need to stop when we
cross the chain limit. We can delve deeper. :)
2020-09-10 18:51:52 -04:00
e8b4011b52 Split Lookups in Hash Cache and Chain Table into Two Loops
Sliiiight speedup.
2020-09-10 18:51:52 -04:00
9e83c782f8 Simplify DDS Hash Table Construction
No need to walk the chainTable; we can just keep shifting the entries in the
hashTable.
2020-09-10 18:51:52 -04:00
5390fee4f7 Rename and Move DD_BLOG Constant to ZSTD_LAZY_DDSS_BUCKET_LOG 2020-09-10 18:51:52 -04:00
5e91ae27eb Prefetch First Batch of Match Positions; +11% Speed in Level 5 w/ 1 Dict 2020-09-10 18:51:52 -04:00
df386b3d8d Fix Off-By-One Error in Counting DDS Search Attempts
This caused us to double-search the first position and fail to search the
last position in the chain, slowing down search and making it less effective.
2020-09-10 18:51:52 -04:00
a494111385 Move Prefetch Before Insertion; Speed Up ~6% 2020-09-10 18:51:52 -04:00
eede46a47e Misc Refactor of DDS Search Code 2020-09-10 18:51:52 -04:00
34b545acb0 Add a ZSTD_dedicatedDictSearch ZSTD_dictMode_e to Allow Const Propagation
Speed +1.5%.
2020-09-10 18:51:52 -04:00
e29bc3a009 using dict mls instead of src mls 2020-09-10 18:51:52 -04:00
145c2d12f9 add hashtable head prefetching 2020-09-10 18:51:52 -04:00
5d5507788d change method name for consistency 2020-09-10 18:51:52 -04:00
628559d0e4 loading dict using new algorithm 2020-09-10 18:51:52 -04:00
22705f0c93 adding dedicatedDictSearch algorithm 2020-09-10 18:51:52 -04:00
50550a14ad adding dedicated dict load method to lazy 2020-09-10 18:51:52 -04:00
f91ed5c766 [lib] s/current/curr because it collides with Linux Kernel macro 2020-09-09 14:35:39 -07:00
70c80e19e6 [greedy] Fix performance instability 2020-05-12 17:51:16 -07:00
3c1eba4d99 [lib] Fix lazy repcode validity checks 2020-05-12 12:25:06 -07:00
4e0515916d [lib] Fix repcode validation in no dict mode 2020-05-12 11:57:15 -07:00
4b88bd3ee0 [lib][fuzz] Assert sequences are valid in round trip tests 2020-05-11 20:38:49 -07:00
80d3585e31 [lib] Fix lazy parser with dictionary + repcodes 2020-05-11 19:04:30 -07:00
ac58c8d720 Fix copyright and license lines
* All copyright lines now have -2020 instead of -present
* All copyright lines include "Facebook, Inc"
* All licenses are now standardized

The copyright in `threading.{h,c}` is not changed because it comes from
zstdmt.

The copyright and license of `divsufsort.{h,c}` is not changed.
2020-03-26 17:02:06 -07:00
659e9f05cf Fix null pointer addition 2019-11-20 18:36:04 -08:00
ddab2a94e8 Pass iend into ZSTD_storeSeq() to allow ZSTD_wildcopy() 2019-09-20 00:56:20 -07:00
facbe8b2c2 factored the logic selecting lowest match index
as suggested by @terrelln
2019-08-05 15:18:43 +02:00
98e7c344cd fixed strategies btopt+ 2019-08-02 14:42:53 +02:00
b4257b04e7 fixed strategy btlazy2 2019-08-02 14:26:26 +02:00
5cf1b24aca fixed strategies greedy, lazy & lazy2
restore dictionary compression ratio
2019-08-02 14:21:39 +02:00
98692c2838 fixed compression ratio regression when dictionary-compressing medium-size inputs at levels 1-3 2019-08-01 15:58:17 +02:00
58adb1059f extended exact window size to greedy/lazy modes 2019-05-31 16:08:48 -07:00
327cf6fac1 nextToUpdate3 does not need to be maintained outside of zstd_opt.c
It's re-synchronized with nextToUpdate at beginning of each block.
It only needs to be tracked from within zstd_opt block parser.

Made the logic clear, so that no code tried to maintain this variable.

An even better solution would be to make nextToUpdate3
an internal variable of ZSTD_compressBlock_opt_generic().
That would make it possible to remove it from ZSTD_matchState_t,
thus restricting its visibility to only where it's actually useful.

This would require deeper changes though,
since the matchState is the natural structure to transport parameters into and inside the parser.
2019-05-28 15:26:52 -07:00
e874dacc08 changed searchLength into minMatch
refactored all relevant API and calls
for consistency.
2018-11-20 14:56:07 -08:00
626040ab53 changed PREFETCH() macro into PREFETCH_L2()
which is more accurate
2018-11-12 17:05:32 -08:00
b8235be865 Avoid Searching Dictionary in ZSTD_btlazy2 When an Optimal Match is Found
Bailing here is important to avoid reading past the end of the input buffer.
2018-10-08 15:59:32 -07:00