1
0
mirror of https://github.com/facebook/zstd.git synced 2025-07-04 22:58:46 +02:00
Commit Graph

42 Commits

Author SHA1 Message Date
a880ca239b Spelling (#1582)
* spelling: accidentally

* spelling: across

* spelling: additionally

* spelling: addresses

* spelling: appropriate

* spelling: assumed

* spelling: available

* spelling: builder

* spelling: capacity

* spelling: compiler

* spelling: compressibility

* spelling: compressor

* spelling: compression

* spelling: contract

* spelling: convenience

* spelling: decompress

* spelling: description

* spelling: deflate

* spelling: deterministically

* spelling: dictionary

* spelling: display

* spelling: eliminate

* spelling: preemptively

* spelling: exclude

* spelling: failure

* spelling: independence

* spelling: independent

* spelling: intentionally

* spelling: matching

* spelling: maximum

* spelling: meaning

* spelling: mishandled

* spelling: memory

* spelling: occasionally

* spelling: occurrence

* spelling: official

* spelling: offsets

* spelling: original

* spelling: output

* spelling: overflow

* spelling: overridden

* spelling: parameter

* spelling: performance

* spelling: probability

* spelling: receives

* spelling: redundant

* spelling: recompression

* spelling: resources

* spelling: sanity

* spelling: segment

* spelling: series

* spelling: specified

* spelling: specify

* spelling: subtracted

* spelling: successful

* spelling: return

* spelling: translation

* spelling: update

* spelling: unrelated

* spelling: useless

* spelling: variables

* spelling: variety

* spelling: verbatim

* spelling: verification

* spelling: visited

* spelling: warming

* spelling: workers

* spelling: with
2019-04-12 11:18:11 -07:00
aae5bc538a Merge pull request #1470 from facebook/U32
fix confusion between unsigned <-> U32
2018-12-23 12:35:39 -08:00
ededcfca57 fix confusion between unsigned <-> U32
as suggested in #1441.

generally U32 and unsigned are the same thing,
except when they are not ...

case : 32-bit compilation for MIPS (uint32_t == unsigned long)

A vast majority of transformation consists in transforming U32 into unsigned.
In rare cases, it's the other way around (typically for internal code, such as seeds).

Among a few issues this patches solves :
- some parameters were declared with type `unsigned` in *.h,
  but with type `U32` in their implementation *.c .
- some parameters have type unsigned*,
  but the caller user a pointer to U32 instead.

These fixes are useful.

However, the bulk of changes is about %u formating,
which requires unsigned type,
but generally receives U32 values instead,
often just for brevity (U32 is shorter than unsigned).
These changes are generally minor, or even annoying.

As a consequence, the amount of code changed is larger than I would expect for such a patch.

Testing is also a pain :
it requires manually modifying `mem.h`,
in order to lie about `U32`
and force it to be an `unsigned long` typically.
On a 64-bit system, this will break the equivalence unsigned == U32.
Unfortunately, it will also break a few static_assert(), controlling structure sizes.
So it also requires modifying `debug.h` to make `static_assert()` a noop.
And then reverting these changes.

So it's inconvenient, and as a consequence,
this property is currently not checked during CI tests.
Therefore, these problems can emerge again in the future.

I wonder if it is worth ensuring proper distinction of U32 != unsigned in CI tests.
It's another restriction for coding, adding more frustration during merge tests,
since most platforms don't need this distinction (hence contributor will not see it),
and while this can matter in theory, the number of platforms impacted seems minimal.

Thoughts ?
2018-12-21 18:09:41 -08:00
c8d1fda982 update aarch64 test to xenial
in an attempt to circumvent the `ld` bug
2018-12-21 15:08:48 -08:00
7b74405150 refactor HUF_compress_internal for clarity
changed workspace parameter convention
to always provide workspaceSize,
so that size can be explicitly checked.

Also, use more enum to make the meaning of some parameters more explicit.
2018-10-26 13:21:37 -07:00
f2d6db45cd [zstd] Add -Wmissing-prototypes 2018-09-27 15:24:48 -07:00
1515f0bb0d fixed more issues detected by recent version of scan-build
test run on Linux
2018-08-16 15:20:25 -07:00
6e66bbf5dd fixed several minor issues detected by scan-build
only notable one :
writeNCount() resists better vs invalid distributions
(though it should never happen within zstd anyway)
2018-08-14 16:55:35 -07:00
dc5a67cb7b Disallow tableLog == srcLog 2018-08-02 11:12:17 -07:00
9889bca530 [FSE] Fix division by zero
When the primary normalization method fails, and
`(1 << tableLog) == (maxSymbolValue + 1)`, and every symbol gets assigned
normalized weight 1 or -1 in the first loop, then the next division can
raise `SIGFPE`.
2018-07-27 17:30:03 -07:00
2d76defbfe grouped all histogram functions into hist.c
renamed functions with HIST_* prefix
2018-06-13 19:49:31 -04:00
fa41bcc2c2 grouped debug functions into debug.h
There were 2 competing set of debug functions
within zstd_internal.h and bitstream.h.
They were mostly duplicate, and required care to avoid messing with each other.

There is now a single implementation, shared by both.

Significant change :
The macro variable ZSTD_DEBUG does no longer exist,
it has been replaced by DEBUGLEVEL,
which required modifying several source files.
2018-06-13 15:43:09 -04:00
776128d16f fix corner case when requiring cost of an FSE symbol
ensure that, when frequency[symbol]==0,
result is (tableLog + 1) bits
with both upper-bit and fractional-bit estimates.

Also : enable BIT_DEBUG in /tests
2018-05-24 13:59:11 -07:00
1a26ec6e8d opt: init statistics from dictionary
instead of starting from fake "default" statistics.
2018-05-10 17:59:12 -07:00
ac6105463a opt: minor improvements to log traces
slight improvement when using fractional-bit evaluation (opt:dictionay)
2018-05-09 15:46:11 -07:00
4d5bd32a00 added traces to look at symbol costs
evaluation looks correct.
2018-05-09 12:00:12 -07:00
a95a88af57 removed huf_compress_impl.h
re-imported all functions inside huf_compress.c
for easier source editing.

Also updated a bunch of code comments
for clarification.
2018-03-13 14:14:05 -07:00
653383f74a minor nit from Mac XCode 2018-02-22 15:44:26 -08:00
76cb38d085 [zstd] Backport kernel patch from @ColinIanKing
* Make the U32 table in `FSE_normalizeCount()` static.
* Patch from https://lkml.kernel.org/r/20170922145946.14316-1-colin.king@canonical.com.
* Clang makes non-static tables static anyways. gcc however, does [weird things](https://godbolt.org/g/fvTcED).
* Benchmarks showed no difference in speed.
2017-09-25 16:18:23 -07:00
cae3e3c652 [fse] Fix FSE_optimalTableLog() for srcSize==1 2017-09-18 14:11:18 -07:00
f9e6590715 Merge pull request #796 from terrelln/is-error
[FSE][HUF] Inline error checks
2017-08-15 12:37:28 -07:00
07c6ff588e [FSE][HUF] Inline error checks
Caught by Clang's optimization remarks.
2017-08-15 11:23:28 -07:00
565e925eb7 [libzstd] Fix FORCE_INLINE macro 2017-08-14 21:12:05 -07:00
ae20d413da [libzstd] Fix CHECK_V_F macros 2017-07-25 12:52:01 -07:00
405d2a1027 Explicitly convert scratchBuffer to unsigned* 2017-04-04 16:35:31 -07:00
16a739cab0 Switch call of FSE_count() to FSE_count_wksp() 2017-04-04 16:17:21 -07:00
5152fb2cb2 Convert all tabs to spaces 2017-03-29 18:51:58 -07:00
23776ce290 fixed ERROR_GENERIC on dstSize_tooSmall
required by users which depends on this error code to size dest buffer
2017-03-23 17:59:50 -07:00
83038d236a Fix bug in FSE distribution normalization 2017-02-22 13:52:48 -08:00
4596037042 updated fse version
feature minor refactoring (removing FSE_abs())
also : fix a few minor issues recently introduced in examples
2017-02-15 12:00:03 -08:00
643d9a234b replaced usage of FSE_buildCTable by FSE_buildCTable_wksp, using less stack space in the process 2016-12-01 16:24:04 -08:00
e928f7e16d introduced ext_wksp variants of count to reduce stack memory usage 2016-12-01 16:13:35 -08:00
979cab412b fixed some minor visual silent cast warnings.
introduced FSE_count_parallel_wksp().
2016-11-30 18:10:38 -08:00
5e00b848a8 FSE_compress_wksp() uses less stack space 2016-11-30 16:46:13 -08:00
d79a9a00d9 Introduced FSE_compress_wksp() and FSE_buildCTable_wksp() to reduce stack memory usage 2016-11-30 15:52:20 -08:00
1563bfeabc fixing FORCE_INLINE for older compilers (#330) 2016-09-02 11:44:21 -07:00
d5c5a77990 minor comments clarifications 2016-07-20 13:35:14 +02:00
52a0622beb RepsCodes are saved into Dict
(uncomplete : need decompression to regenerate them)
2016-06-16 01:05:04 +02:00
d0e2cd15cb Merged fse_static into fse.h . Now requires FSE_STATIC_LINKING_ONLY macro. 2016-06-05 00:58:01 +02:00
f22a0d653d huff0 dynamic reduction 2016-05-20 14:36:36 +02:00
45f84ab7cc all programs use ZSTD_VERSION_STRING from zstd.h to display their version 2016-05-20 12:34:40 +02:00
23a0889301 separation of lib/ into common/, compress/, decompress/, dictBuilder/, legacy/ 2016-04-22 12:43:18 +02:00