When using ff_init_vlc_sparse() to create a VLC, three input tables are
used: A table for lengths, one for codes and one for symbols; the latter
one can be omitted, then a default one will be used. These input tables
will be traversed twice, once to get the long codes (which will be put
into subtables) and once for the small codes. The long codes are then
sorted so that entries that should be in the same subtable are
contiguous.
This commit adds an alternative to ff_init_vlc_sparse():
ff_init_vlc_from_lengths(). It is based upon the observation that if
lengths, codes and symbols tables are permuted (in the same way) so that
the codes are ordered from left to right in the corresponding tree and
if said tree is complete (i.e. every non-leaf node has two children),
the codes can be easily computed from the lengths and are therefore
redundant. This means that if one initializes such a VLC with explicitly
coded lengths, codes and symbols, the codes can be avoided; and even if
one has no explicitly coded symbols, it might still be beneficial to
remove the codes even when one has to add a new symbol table, because
codes are typically longer than symbols so that the latter often fit
into a smaller type, saving space.
Furthermore, given that the codes here are by definition ordered from
left to right, it is unnecessary to sort them again; for the same
reason, one does not have to traverse the input twice. This function
proved to be faster than ff_init_vlc_sparse() whenever it has been
benchmarked.
This function is usable for static tables (they can simply be permuted
once) as well as in scenarios where the tables are naturally ordered
from left to right in the tree; the latter e.g. happens with Smacker,
Theora and several other formats.
In order to make it also usable for (static) tables with incomplete trees,
negative lengths are used to indicate that there is an open end of a
certain length.
Finally, ff_init_vlc_from_lengths() has one downside compared to
ff_init_vlc_sparse(): The latter uses tables that can be reused by
encoders. Of course, one could calculate the needed table at runtime
if one so wishes, but it is nevertheless an obstacle.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This function is so extremely simple that it is preferable to make it
inline rather than deal with all the complications arising from it being
an exported symbol.
Keep avpriv_align_put_bits() around until the next major bump to
preserve ABI compatibility.
Allocating one temporary entry more than needed was made necessary by
the COPY loop below writing an element before having checked that it
should be written at all. But given that this behaviour changed, the
need for overallocating is gone.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
If a static VLC table gets initialized a second time (or concurrently by
two threads) and if said VLC table uses symbols that have the sign bit
of VLC_TYPE (a typedef for int16_t) set, initializing the VLC fails. The
reason is that the type of the symbol in the temporary array is an
uint16_t and so comparing it to the symbol read from the VLC table will
fail, because only the lower 16bits coincide. Said failure triggers an
assert.
Reviewed-by: Lynne <dev@lynne.ee>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This is easily possible because ff_init_vlc_sparse() already transforms
both LE as well as BE codes to a normal form internally before
processing them further. This will be used in subsequent commits.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The comment referred to the INIT_VLC_USE_STATIC flag which has been
removed in 2009 in 595324e143b57a52e2329eb47b84395c70f93087; the
function it referred to was removed even earlier in commit
83422c1940 in 2008.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Fixes: out of array read
Fixes: 14563/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AGM_fuzzer-5646451545210880
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read
Fixes: 14563/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AGM_fuzzer-5646451545210880
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* commit 'ffa190d0479d2370dd89c95692f822cbff2cc24c':
Move VLC and RL_VLC_ELEM structure definitions to a separate header
Merged-by: Clément Bœsch <u@pkh.me>
Use the newly created vlc.h directly instead of including get_bits when needed.
The VLC and RL_VLC_ELEM structures are independent from the bitreader.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Commit 3a0a2f33a6 claims large performance
advantages for AV_QSORT over libc's qsort. The reason is that I suspect
that libc's qsort (at least on non LTO builds, like the typical FFmpeg config)
can't inline the comparison callback:
https://stackoverflow.com/questions/5290695/is-there-any-way-a-c-c-compiler-can-inline-a-c-callback-function.
AV_QSORT has two things going for it:
1. The guaranteed inlining of qsort itself. This yields a negligible
boost that may be ignored.
2. The more serious possibility of potentially allowing the comparison
function to be inlined - this is likely responsible for the large boosts
reported.
There is a comment explaining that this is a place that could use some
performance improvement. Thus AV_QSORT is used to achieve that.
Benchmarks deemed unnecessary due to existing claims about AV_QSORT.
Tested with FATE.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
Also fix typo found by Lou Logan:
Sacrifying -> Sacrificing
Reviewed-by: Lou Logan <lou@lrcd.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Neccessary -> Necessary
formated -> formatted
thee -> the
eventhough -> even though
seperately -> separately
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Such buffers are not malloced thus freeing would be bad.
Note though this condition never could have happened so this
is more for correctness sake and not a true bugfix
Fixes CID1061047
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This reverts commit 4b6869d6e0.
Conflicts:
libavcodec/bitstream.c
This code can cause assertion failures on artificial OOM situations
It will be replaced by a solution that doesnt have this issue in the
following commits.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
