1
0
mirror of https://github.com/FFmpeg/FFmpeg.git synced 2025-02-04 06:08:26 +02:00

43248 Commits

Author SHA1 Message Date
Andreas Rheinhardt
943ce3a756 avcodec/bitstream: Consistently treat symbol as VLC_TYPE
If a static VLC table gets initialized a second time (or concurrently by
two threads) and if said VLC table uses symbols that have the sign bit
of VLC_TYPE (a typedef for int16_t) set, initializing the VLC fails. The
reason is that the type of the symbol in the temporary array is an
uint16_t and so comparing it to the symbol read from the VLC table will
fail, because only the lower 16bits coincide. Said failure triggers an
assert.

Reviewed-by: Lynne <dev@lynne.ee>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit df6ec7f83b4fb65d760259e01182dc28b6cf3f2f)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
da5ac1563e avcodec/vp3: Check allocations of VLCs
It would lead to crashs lateron if they failed.

Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 786b1b0c44d3e5d71c3e69a3fe260baa95172e02)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
4e5a97e393 avcodec/vp3: Fix memleak upon init failure
Up until now, there was no cleanup in case initializing the Theora VLC
tables failed, leading to memleaks. This commit gets rid of them by
setting the FF_CODEC_CAP_INIT_CLEANUP flag for all decoders in vp3.c;
this also allows to remove some (now redundant) cleanup code.

Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit a01ca21bbbd41ad86ca58f2c7575c92a36a4b722)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
78ac14f280 avcodec/movtextenc: Fix undefined left shifts outside the range of int
Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 2f9fc35028364b0140fd6e0d2e4dbaffebed1acd)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
915e68c051 avcodec/movtextenc: Fix memleak on (re)allocation error
Up until now, the mov_text encoder used the dynamic array API for its
list of style attributes; it used the (horrible) av_dynarray_add() which
works with an array of pointers; on error it frees its array but not
the buffers referenced by the pointers said array contains. It also
returns no error code, encouraging not to check for errors.

These properties imply that this function may only be used if the buffers
referenced by the list either need not be freed at all or if they are
freed by other means (i.e. if the list contains non-ownership pointers).

In this case, the style attributes are owned by the pointers of the
dynamic list. Ergo the old style attributes leak on a subsequent
reallocation failure. But given that the (re)allocation isn't checked
for success, the style attribute intended to be added to the list also
leaks because the only pointer to it gets overwritten in the belief that
it is now owned by the list.

This commit fixes this by switching to av_fast_realloc() and an array
containing the styles directly instead of pointers to individually
allocated style attributes. The current style attributes are now no longer
individually allocated, instead they are part of the context.

Furthermore, av_fast_realloc() allows to easily distinguish between
valid and allocated elements, thereby allowing to reuse the array
(which up until now has always been freed after processing an
AVSubtitleRect).

Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 9a731e9fec53f121e0fd5981f22c9c5093db0793)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
0f70a8ae19 avcodec/movtextenc: Don't presume every style to have a font
Fixes segfaults in the absence of fonts; this can happen because the
file didn't contain any or because the allocation of the font-string
failed.

Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 0dd7b8232d38317abc195edc48434ac1fd3e80fd)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
f4d7dba3ce avcodec/movtextenc: Reset array counter after freeing array
Otherwise the mov_text encoder can segfault when given subtitles with more
than one AVSubtitleRect if one of the first nb_rects - 1 rects contained
a style attribute.

Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 8d4431955ccca80933a837e1bc7f44679039335b)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
bb8d18a39b avcodec/movtextenc: Fix potential use of uninitialized value
Background colour was never initialized if no style was available.
Use a sane default of zero (i.e. completely transparent).

Fixes Coverity issue #1461471.

Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 56b3726ed2ac89df31a939a8f5f00fa66a0ad2ed)
2021-02-27 07:20:58 +01:00
Andriy Gelman
1eb8f6885e avcodec/movtextenc: fix writing to bytestream on BE arches
Fixes fate-binsub-movtextenc on PPC64

Currently tags are written in reverse order on BE arches. This is fixed
by using MKBETAG() and AV_RB32() to be arch agnostics.

Also s->font_count is of type int. On BE arches with 32bit int,
count = AV_RB16(&s->font_count) will read two most significant bytes
instead of the least significant bytes. This is fixed by assigning
s->font_count to count first.

The final change is modifying the type of len. On BE arches
the most significant byte of the int was written instead of the least
significant byte.

Signed-off-by: Andriy Gelman <andriy.gelman@gmail.com>
(cherry picked from commit d4c46dc32856bd9c7c7ab29ee727676c7855fa1c)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
51faa4bd60 avcodec/movtextdec: Fix leaks on (re)allocation failure
Up until now, the 3GPP Timed Text decoder used av_dynarray_add()
for a list of style entries. Said entries are individually allocated
and owned by the pointers in the dynamic array and are therefore
unsuitable for av_dynarray_add() which simply frees the array,
but not the entries on error. In this case the intended new entry
also leaks because it has been forgotten to free it.

This commit fixes this. It is now allocated in one go and not
reallocated multiple times (and it won't be overallocated any more).
After all, the final number of elements (pending errors) is already
known in advance.

Furthermore, the style entries are now the entries of the new array,
i.e. they are no longer allocated separately. This also removes one
level of indirection.

Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 94ad68ee17420996c9b003f142717d82b52c0915)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
b4a96efdda avcodec/movtextdec: Simplify checking for invalid extradata
Every font entry occupies at least three bytes, so checking early
whether there is that much data available is a low-effort way to exclude
invalid extradata. Doing so leads to an overall simplification.

Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit a42695c07244991ceabf9996d086dda3fcc28fc1)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
59b8634411 avcodec/movtextdec: Fix leaks of strings upon reallocation failure
Up until now, the 3GPP Timed Text decoder used av_dynarray_add()
for a list of font entries, a structure which contains an allocated
string. The font entries are owned by the pointers in the dynamic array
and are therefore unsuitable for av_dynarray_add() which simply frees
the array, but not the font entries and of course not the strings. The
latter all leak if reallocating the dynamic array fails.

This commit fixes this. It stops reallocating the array altogether:
After all, the final number of elements (pending errors) is already
known in advance.

Furthermore, the font entries are now the entries of the new array,
i.e. the font entries are no longer allocated separately. This also
removes one level of indirection.

Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 38cdd27e9b36183a74c19859a8092aca6e82d712)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
2e49daad79 avcodec/movtextdec: Reset counter of fonts when freeing them
If allocating fonts fails when reading the header, all fonts are freed,
yet the counter of fonts is not reset and no error is returned; when
subtitles are decoded lateron, the inexistent list of fonts is searched
for the matching font for this particular entry which of course leads to
a segfault.

Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 5758620560f1aa329a26ca1585dc0dbd903522c4)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
9c6feb104d avcodec/sheervideo: Don't leave context in inconsistent state upon error
This has happened if the format changed midstream and if the new packet
is so small that it is instantaneously rejected: In this case the VLC
tables were for the new format, although the context says that they are
still the ones for the old format. It can also happen if the format
changed midstream and the allocation of the new tables fails. If the
next packet is a packet for the old format, the decoder thinks it
already has the correct VLC tables, leading to a segfault.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 8969b9aa061790a5e87694aab17741cc7647d099)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
3f9c5f242f avcodec/dvenc: Fix undefined left shift of negative numbers
The earlier code was based on the assumption that AVFrame.linesize can
not be negative.

Fixes ticket #8280.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 6770057ac97e78c799f06eb3769fecdb0833d44f)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
211f23d789 avcodec/utils: Also free encoder extradata on avcodec_open2() error
It is owned by libavcodec for encoders.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit b8e0ceda11f7a12d763c6a744943347fc808b8b7)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
39d6d7f7e8 avcodec/utils: Don't forget cleaning up when allocating priv_data fails
Allocating an AVCodecContext's priv_data used to be the first object
allocated in avcodec_open2(), so it was unnecessary to goto free_and_end
(which does the cleanup) upon error here. But this is no longer so since
f3a29b750a5979ae6847879fba758faf1fae88d0.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit d1dcc20126f3df52762bf5972a282d1699ef1ca8)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
3fe22f2a0d avcodec/utils: Improve check for freeing codec private options
Don't check for AVCodec.priv_data_size (which is always true if
AVCodec.priv_class is set). Instead check for AVCodecContext.priv_data
to actually exist.

(Note: av_opt_free(NULL) is a no-op.)

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit c6e54d14c55221b7380a0e86d7a42b60ff9e5eeb)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
62dc4c2df5 avcodec/utils: Only call codec->close if init has been called
avcodec_open2() also called the AVCodec's close function if an error
happened before init had ever been called if the AVCodec has the
FF_CODEC_CAP_INIT_CLEANUP flag set. This is against the documentation of
said flag: "The codec allows calling the close function for deallocation
even if the init function returned a failure."

E.g. the SVQ3 decoder is not ready to be closed if init has never been
called.

Fixes: NULL dereference
Fixes: 25762/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_SVQ3_fuzzer-5716279070294016

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 5bc74d06dad35d00b5925b1c76208aeaf40a2dbb)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
cb08668ff9 avcodec/mpegaudiodec_template: Check return value of subdecoder
After all, allocating an AVFloatDSPContext might have failed.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit dac9e88a99ea34c2f812b8f7b6781a84ac86360a)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
c937f21e26 avcodec/j2kenc: Fix leaks on init failure
The JPEG2000 encoder did not clean up after itself on error.
This commit fixes this by modifying the cleanup function to be able to
handle only partially allocated structures and by setting the
FF_CODEC_CAP_INIT_CLEANUP flag.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 3d83de4187e9bb07af2ea8a0ec071094fca7a500)
2021-02-27 07:20:58 +01:00
Andreas Rheinhardt
f8747758bb avcodec/flashsv2enc: Check allocations for success before usage
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 319dbcf4402978f7a493b81d0f61be7a46ddbeba)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
140d871e08 avcodec/flashsvenc: Avoid allocation of buffer, fix memleak
Up until now, the flashsv encoder tried to allocate two buffers in its
init function; if only one of these allocations succeeds, the other
buffer leaks. Fix this by making one of these buffers part of the
context (its size is a compile-time constant).

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit ec6f4c51586cc213d16749f930970ba8afd0e38a)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
5000bae830 avcodec/ac3enc_template: Don't free uninitialized pointers on error
The ac3 encoders (fixed- and floating-point AC-3 as well as the EAC-3
encoder) all allocate an array whose elements are pointers to other
buffers. The array is not zeroed initially so that if an allocation of
one of the subbuffers fails, the other pointers are uninitialized.
This causes problems when cleaning, so zero the array initially.

(Only the fixed-point AC-3 encoder was affected by this, because
the other two don't clean up at all in case of errors during init.)

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit ae36fad624307dcd1bbe0c954a017293a1ce34d3)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
7cfa8e1910 avcodec/svq3: Fix segfault on allocation error, avoid allocations
The very first thing the SVQ3 decoder currently does is allocating several
SVQ3Frames, a structure which contains members that need to be freed on
their own. If one of these allocations fails, the decoder calls its own
close function to not leak the already allocated SVQ3Frames. Yet said
function presumes that the SVQ3Frames have been successfully allocated
as there is no check before freeing the members that need to be freed.

This commit fixes this by making these frames part of the SVQ3Context,
thereby avoiding the allocations altogether. Notice that the pointers
to the frames have been retained in order to allow to just swap them as
the code already does.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 96061c5a4f690c3ab49e4458701bb013fd3dd57f)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
2e05113b92 avcodec/sonic: Fix leaks upon allocation errors
The Sonic decoder and encoders allocate several buffers in their init
function and return immediately if one of these allocations fails; this
will lead to leaks if there was an earlier successfull allocation. Fix
this by setting the FF_CODEC_CAP_INIT_CLEANUP flag.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 7d91f9271ee336da93b6871bf3306348ac1595a7)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
0dd79193a7 avcodec/qtrleenc: Fix memleak upon allocation failure
The qtrle encoder allocates several buffers and an AVFrame in its init
function. If one of these allocations fails, but others succeed, the
successfully allocated objects leak. This is fixed by setting the
FF_CODEC_CAP_INIT_CLEANUP flag.

Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 2a71cbeb019fabd70f04ca9d2ec5d0bff3b3e3d2)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
ce629ee216 avcodec/ffv1enc: Fix memleaks on init failure
The FFV1 encoder has so far not cleaned up after itself in this case;
but it can be done easily by setting the FF_CODEC_CAP_INIT_CLEANUP flag.

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit f9215d0bb20242299f3654fd4646511665b20c4c)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
5e95dc426e avcodec/ffv1: Fix segfaults on allocation error
When allocating FFV1 slice contexts fails, ff_ffv1_init_slice_contexts()
frees everything that it has allocated, yet it does not reset the
counter for the number of allocated slice contexts. This inconsistent
state leads to segfaults lateron in ff_ffv1_close(), because said
function presumes that the slice contexts have been allocated.
Fix this by making sure that the number of slice contexts on error is
consistent (namely zero).

(This issue only affected the FFV1 decoder, because the encoder does not
clean up after itself on init failure.)

Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit a0750f412ade5a969b1f90e038d707d531c97342)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
e7780f1615 avcodec/mlpenc: Fix memleak upon init failure
If an error happens during init after an allocation has succeeded,
the already allocated data leaked up until now. Fix this by setting the
FF_CODEC_CAP_INIT_CLEANUP flag.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit b78031cf1670c176e97f62ad6a4865d3a44c1507)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
7b10d643a0 avcodec/indeo5: Fix memleaks upon allocation error
ff_ivi_init_planes() might error out after having allocated some arrays.
Set the FF_CODEC_CAP_INIT_CLEANUP flag in order to free these arrays in
this case.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit d8fc69bd6a20394adf42406bf80bd06b4c8d7384)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
83cd7a29dc avcodec/ivi: Fix segfault on allocation error
If allocating the tiles array for indeo 4/5 fails, the context is in an
inconsistent state, because the counter for the number of tiles is > 0.
This will lead to a segfault when freeing the tiles' substructures.
Fix this by setting the number of tiles to zero if the allocation was
unsuccessful.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit e411a3af11e5d00a311f594a45c17f74898cb46e)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
d0045b41a3 avcodec/magicyuvenc: Fix memleak upon init failure
If an error happens during init after an allocation has succeeded,
the already allocated data leaked up until now. Fix this by setting the
FF_CODEC_CAP_INIT_CLEANUP flag.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit a8ebb5632018adad733f89fa4efe74d7e92155ac)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
9ce88b5b53 avcodec/gif: Fix leaks upon allocation error
If one of several allocations the gif encoder performs in its init
function fails, the successful allocations leak. Fix this by adding the
FF_CODEC_CAP_INIT_CLEANUP flag.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 8da8774d8eb1d5dec830f36d2e6b0ace1b16e2f5)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
9c6d8ae33c avcodec/avrndec: Fix memleak on error
If ff_codec_open2_recursive() fails, the already allocated
AVCodecContext leaks. Fix this by setting the FF_CODEC_CAP_INIT_CLEANUP
flag.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 3c2128df7ffdb32a3ef34044f9a4c2f15dc1776f)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
682574e1da avcodec/avrndec: Check allocation for success
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit d8e4d26de6a96ffd7813aad5bb0680b494192031)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
634c4ce8b0 avcodec/atrac1: Check allocation of AVFloatDSPContext
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit e75ccc81234a22eee90e2a31808639e754b97cd1)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
88a563b27d avcodec/binkaudio: Don't use static storage for context-dependent data
Move it to the context instead.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 2777bae7f2fbb96843bc2d12c9190a12ffdfef55)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
7f1f850f11 avcodec/bink: Fix memleak upon init failure
The init function first allocates an AVFrame and then some buffers; if
one of the buffers couldn't be allocated, the AVFrame leaks. Solve this
by setting the FF_CODEC_CAP_INIT_CLEANUP flag.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 4f672889481e7b3dc03c04b02a86836e94104e63)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
c71de42131 avcodec/wnv1: Use LE bitstream reader, avoid copying packet, fix memleak
The Winnov WNV1 format is designed for a little-endian bitstream reader;
yet our decoder reversed every byte bitwise (in a buffer only
allocated for this purpose) to use a big-endian bitstream reader. This
commit stops this.

Two things needed to be done to achieve this: The codes in the table used
to initialize a VLC reader needed to be reversed bitwise (when
initializing a VLC in LE mode, it is expected that the first bit to be
read is in the least significant bit; with BE codes the first bit to be
read is the most significant bit of the code) and the following
expression needed to be adapted:

ff_reverse[get_bits(&w->gb, 8 - w->shift)]

But this is easy: When only the bits read are reversed, they coincide
with what a little-endian bitstream reader reads that reads the
original, not-reversed data. But ff_reverse always reverses the full
eight bits and this also performs a shift by (8 - (8 - w->shift)) on top
of reversing the bits read. So the above line needs to be changed to

get_bits(&w->gb, 8 - w->shift) << w->shift

and this also shows why the variable shift is named the way it is.

Finally, this also fixes a hypothetical memleak: For gigantic packets,
initializing a GetBitContext can fail and in this case, the buffer
containing the reversed data would leak.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 0166b1d1a6d052ef49aba3523d64f3c6d4f26372)
2021-02-27 07:20:57 +01:00
Andreas Rheinhardt
d9da39572b avcodec/truemotion2: Avoid duplicating array, fix memleak
TrueMotion 2.0 uses Huffmann trees. To parse them, the decoder allocates
arrays for the codes, their lengths and their value; afterwards a VLC
table is initialized using these values. If everything up to this point
succeeds, a new buffer of the same size as the already allocated arrays
for the values is allocated and upon success the values are copied into
the new array; all the old arrays are then freed. Yet if allocating the
new array fails, the old arrays get freed, but the VLC table doesn't.

This leak is fixed by not allocating a new array at all; instead the old
array is simply reused, ensuring that nothing can fail after the
creation of the VLC table.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 5ff2ff6bd9cd9e08729060d330e381a09972c498)
2021-02-27 07:20:57 +01:00
Michael Niedermayer
a3d147899c avcodec/hapdec: Change compressed_offset to unsigned 32bit
Fixes: out of array access
Fixes: 29345/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HAP_fuzzer-5401813482340352
Fixes: 30745/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HAP_fuzzer-5762798221131776

Suggested-by: Anton
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 89fe1935b18621af06587c76bcde6adcdc8f2249)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-02-20 14:21:24 +01:00
Michael Niedermayer
c4ae8618f4 avcodec/vp8: Move end check into MB loop in vp78_decode_mv_mb_modes()
Fixes: Timeout (long -> 5sec)
Fixes: 30269/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VP7_fuzzer-5430325004075008

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Peter Ross <pross@xvid.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6a797ceafe2a96aa1682a1eca421eddd4c498275)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-02-20 14:21:24 +01:00
Michael Niedermayer
2d155dcb7e avcodec/fits: Check gcount and pcount being non negative
Fixes: signed integer overflow: 9223372036854775807 - -30069403896 cannot be represented in type 'long'
Fixes: 30046/clusterfuzz-testcase-minimized-ffmpeg_dem_FITS_fuzzer-5807144773484544

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c000a9128815e7cee4316dc45605259bbaa138ff)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-02-20 14:21:24 +01:00
Michael Niedermayer
12b329a51d avcodec/pnm_parser: Check av_image_get_buffer_size() for failure
Fixes: out of array access
Fixes: 30135/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PBM_fuzzer-4997145650397184
Fixes: 30208/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_PGMYUV_fuzzer-5605891665690624.fuzz

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5314a4996cc76e2a8534c74a66f5181e95ac64fc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-02-20 14:21:24 +01:00
Michael Niedermayer
186df3419c avcodec/jpeg2000dec: Check atom_size in jp2_find_codestream()
Fixes: Infinite loop
Fixes: 29722/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_JPEG2000_fuzzer-6412228041506816

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2a2082a41bca9dbb22c45288972f2da309443cf8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-02-20 14:21:24 +01:00
Michael Niedermayer
4a4f4cc814 avcodec/jpeglsdec: Fix k=16 in ls_get_code_regular()
Fixes: Timeout
Fixes: left shift of 33046 by 16 places cannot be represented in type 'int'
Fixes: 29258/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MJPEG_fuzzer-4889231489105920
Fixes: 29515/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MJPEG_fuzzer-6161940391002112

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 980900d991606cbc3747b37d6e83c7aae98cbecc)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-02-20 14:21:24 +01:00
Michael Niedermayer
fc0453d3e4 avcodec/hevc_sei: Check payload size in decode_nal_sei_message()
Fixes: out of array access
Fixes: 29392/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HEVC_fuzzer-4821602850177024.fuzz

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0791a515d38fd35c1e2a309ec8f4015153687b8c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-02-20 14:21:24 +01:00
Michael Niedermayer
ad7c1ed262 avcodec/vp9dsp_template: Fix integer overflows in itxfm_wrapper
Fixes: signed integer overflow: 2147483641 + 32 cannot be represented in type 'int'
Fixes: 27452/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_VP9_fuzzer-5078752576667648

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4dfb7ff528c02afbafba14676c139ecb82164c44)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-02-20 14:21:24 +01:00
Michael Niedermayer
506406b803 avcodec/mxpegdec: fix SOF counting
Fixes: Timeout (>10sec -> 15ms)
Fixes: 27652/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MXPEG_fuzzer-5125920868007936

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 401495def62638a205569cac0f7861c7faba4d18)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2021-02-20 14:21:24 +01:00