VLC_MULTI_ELEM contains an uint8_t array that is supposed
to be treated as an array of uint16_t when the used symbols
have a size of two; otherwise it should be treated as just
an array of uint8_t, but it was not always treated that way:
vlc_multi_gen() initialized the first entry of the array
by writing the symbol via AV_WN16; on big endian systems,
the intended value was instead written into the second entry
of the array (where it would likely be overwritten lateron
during initialization).
read_vlc_multi() also treated this case incorrectly: In case
the code is so long that it needs a classical multi-stage lookup,
the symbol has been written to the destination as if via AV_WN16.
On little endian systems, this sets the correct first symbol and
clobbers (zeroes) the next one, but the next one will be overwritten
lateron anyway, so it won't be recognized. But on big-endian systems,
the first symbol will be set to zero and the actually read symbol
will be put into the slot for the next one (where it will be overwritten
lateron).
This commit fixes this; this fixes the magicyuv and utvideo FATE-tests
on big endian arches.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This makes the code more testable as uninitialized fields are 0
and not random values from the last call
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
These entries do not correspond to VLC symbols that can be used
they do corrupt various variables like min/max bits
This also no longer assumes that there is a single non subtable
entry
Probably fixes some infinite loops too
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
For lots of static VLCs, the number of bits is not read from
VLC.bits, but rather a compile-constant that is hardcoded
at the callsite of get_vlc2(). Only VLC.table is ever used
and not using it directly is just an unnecessary indirection.
This commit adds helper functions and macros to avoid the VLC
structure when initializing VLC tables; there are 2x2 functions:
Two choices for init_sparse or from_lengths and two choices
for "overlong" initialization (as used when multiple VLCs are
initialized that share the same underlying table).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Fixes: use after free
Fixes: 62153/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MAGICYUV_fuzzer-4702814909366272
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Therefore use a proper prefix for this API, e.g.
ff_init_vlc_sparse -> ff_vlc_init_sparse
ff_free_vlc -> ff_vlc_free
INIT_VLC_LE -> VLC_INIT_LE
INIT_VLC_USE_NEW_STATIC -> VLC_INIT_USE_STATIC
(The ancient INIT_VLC_USE_STATIC has been removed
in 595324e143b57a52e2329eb47b84395c70f93087, so that
the NEW has been dropped.)
Finally, reorder the flags and change their values
accordingly.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Mostly taken from the documentation for ff_init_vlc_from_lengths();
also remove the documentation in vlc.c.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
In C, qualifiers for arrays are broken:
const VLC_TYPE (*foo)[2] is a pointer to an array of two const VLC_TYPE
elements and unfortunately this is not compatible with a pointer
to a const array of two VLC_TYPE, because the latter does not exist
as array types are never qualified (the qualifier applies to the base
type instead). This is the reason why get_vlc2() doesn't accept
a const VLC table despite not modifying the table at all, as
there is no automatic conversion from VLC_TYPE (*)[2] to
const VLC_TYPE (*)[2].
Fix this by using a structure VLCElem for the VLC table.
This also has the advantage of making it clear which
element is which.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
bitstream.c is currently the disjoint union of three parts:
The first part is ff_log2_run, the second part are some auxiliary
functions for the PutBits-API; and the third part is the code
for creating VLCs. This commit moves the latter into a file of its own.
This has the advantage of making one of the hacks in tableprint_vlc.h
redundant as vlc.c does not include config.h (whereas the PutBits-API
part does).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
