In case of resolution changes rv20_decode_picture_header() closes and
reopens its MpegEncContext; it checks the latter for errors, yet when
an error happens, it might happen that no new attempt at
reinitialization is performed when decoding the next frame; this leads
to crashes lateron.
This commit fixes this by making sure that initialization will always
be attempted if the context is currently not initialized.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(cherry picked from commit 8ffd3ef9d94f33b411348c594a49d994b55c9550)
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 9bab7de175d7c942a6ebddae6ba0cacdf360827e)
When slice-threading is used, ff_mpv_common_init() duplicates
the first MpegEncContext and allocates some buffers for each
MpegEncContext (the first as well as the copies). But the count of
allocated MpegEncContexts is not updated until after everything has
been allocated and if an error happens after the first one has been
allocated, only the first one is freed; the others leak.
This commit fixes this: The count is now set before the copies are
allocated. Furthermore, the copies are now created and initialized
before the first MpegEncContext, so that the buffers exclusively owned
by each MpegEncContext are still NULL in the src MpegEncContext so
that no double-free happens upon allocation failure.
Given that this effectively touches every line of the init code,
it has also been factored out in a function of its own in order to
remove code duplication with the same code in
ff_mpv_common_frame_size_change() (which was never called when using
more than one slice (and if it were, there would be potential
double-frees)).
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit ff0706cde8b1a1f483e26c0ccac117c23b23d604)
This mostly reverts commit 4b2863ff01b1fe93d9a518523c9098d17a9d8c6f.
Said commit removed the freeing code from ff_mpv_common_init(),
ff_mpv_common_frame_size_change() and ff_mpeg_framesize_alloc() and
instead added the FF_CODEC_CAP_INIT_CLEANUP to several codecs that use
ff_mpv_common_init(). This introduced several bugs:
a) Several decoders using ff_mpv_common_init() in their init function were
forgotten: This affected FLV, Intel H.263, RealVideo 3.0 and V4.0 as well as
VC-1/WMV3.
b) ff_mpv_common_init() is not only called from the init function of
codecs, it is also called from AVCodec.decode functions. If an error
happens after an allocation has succeeded, it can lead to memleaks;
furthermore, it is now possible for the MpegEncContext to be marked as
initialized even when ff_mpv_common_init() returns an error and this can
lead to segfaults because decoders that call ff_mpv_common_init() when
decoding a frame can mistakenly think that the MpegEncContext has been
properly initialized. This can e.g. happen with H.261 or MPEG-4.
c) Removing code for freeing from ff_mpeg_framesize_alloc() (which can't
be called from any init function) can lead to segfaults because the
check for whether it needs to allocate consists of checking whether the
first of the buffers allocated there has been allocated. This part has
already been fixed in 76cea1d2ce3f23e8131c8664086a1daf873ed694.
d) ff_mpv_common_frame_size_change() can also not be reached from any
AVCodec.init function; yet the changes can e.g. lead to segfaults with
decoders using ff_h263_decode_frame() upon allocation failure, because
the MpegEncContext will upon return be flagged as both initialized and
not in need of reinitialization (granted, the fact that
ff_h263_decode_frame() clears context_reinit before the context has been
reinited is a bug in itself). With the earlier version, the context
would be cleaned upon failure and it would be attempted to initialize
the context again in the next call to ff_h263_decode_frame().
While a) could be fixed by adding the missing FF_CODEC_CAP_INIT_CLEANUP,
keeping the current approach would entail adding cleanup code to several
other places because of b). Therefore ff_mpv_common_init() is again made
to clean up after itself; the changes to the wmv2 decoder and the SVQ1
encoder have not been reverted: The former fixed a memleak, the latter
allowed to remove cleanup code.
Fixes: double free
Fixes: ff_free_picture_tables.mp4
Fixes: ff_mpeg_update_thread_context.mp4
Fixes: decode_colskip.mp4
Fixes: memset.mp4
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit d4b9e117ceb6356cbcdc9ca81ec9c6c4b90efdae)
There might be segfaults on failure.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(cherry picked from commit e93875b756b575438d7b825332739719d4fbc600)
If only one of the two arrays used for the ICC profile could be
successfully allocated, it might be overwritten and leak when
the next ICC entry is encountered. Fix this by using a common struct,
so that one has only one array to allocate.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(cherry picked from commit a5b2f06b0c69221e375edd918a335c68b33d5667)
In this case it also fixes a potential for compilation failures:
Not all compilers can handle the case in which a function with
a forward declaration declared with an attribute to always inline it
is called before the function body appears. E.g. GCC 4.2.1 on OS X 10.6
doesn't like it.
Reviewed-by: Pavel Koshevoy <pkoshevoy@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(cherry picked from commit e5d6af7b35d97f5c1252ecf8bd61f3295909fb6c)
Up until now, initializing the mutexes/condition variables wasn't
checked by ff_frame_thread_init(). This commit changes this.
Given that it is not documented to be save to destroy a zeroed but
otherwise uninitialized mutex/condition variable, one has to choose
between two approaches: Either one duplicates the code to free them
in ff_frame_thread_init() in case of errors or one records which have
been successfully initialized. This commit takes the latter approach:
For each of the two structures with mutexes/condition variables
an array containing the offsets of the members to initialize is added.
Said array is used both for initializing and freeing and the only thing
that needs to be recorded is how many of these have been successfully
initialized.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit c85fcc96b79a502b1d2cd022c6f0c638e4527732)
In case an error happened when setting up the child threads,
ff_frame_thread_init() would up until now call ff_frame_thread_free()
to clean up all threads set up so far, including the current, not
properly initialized one.
But a half-allocated context needs special handling which
ff_frame_thread_frame_free() doesn't provide.
Notably, if allocating the AVCodecInternal, the codec's private data
or setting the options fails, the codec's close function will be
called (if there is one); it will also be called if the codec's init
function fails, regardless of whether the FF_CODEC_CAP_INIT_CLEANUP
is set. This is not supported by all codecs; in ticket #9099 it led
to a crash.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit e9b66175793e5c2af19beefe8e143f6e4901b5df)
Some old DV AVI files have the DSF-Flag of frames set to 0, although it
is PAL (maybe rendered with an old Ulead Media Studio Pro) ... this causes
ffmpeg/VLC-player to produce/play corrupted video (other players/editors
like VirtualDub work fine).
Fixes ticket #8333 and replaces/extends hack for ticket #2177
Signed-off-by: Marton Balint <cus@passwd.hu>
(cherry picked from commit 6ef5d8ca86c1d17e486300daa476dc6e0799f2f6)
This avoids use of uninitialized data
also several checks are inside the band reading code
so it is important that it is run at least once
Fixes: out of array accesses
Fixes: 28209/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-5684714694377472
Fixes: 32124/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-5425980681355264
Fixes: 30519/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-4558757155700736
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit da8c86dd8be244100da226a11b416c20d0c914d6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Previously the code skipped all security checks when these where encountered but prior data was incorrect.
Also replace an always true condition by an assert
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3b88c88fa1888c47b0767d84bfebf1fd656c7846)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array accesses
Fixes: 29754/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-6333598414274560
Fixes: 30519/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-6298424511168512
Fixes: 30739/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_CFHD_fuzzer-5011292836462592
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 20473a93d2a385a7137ef03e8b2604fa5e79eaa7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3ce8d092448827842c451807f03010ad5129fd8f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dc57762cb43619f91fd2a5d95510fa3b14cfeaaf)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e1484bc455dff500f8b35b58d434924bca0e03d6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(cherry picked from commit d5ddfec6c3ce0ff743e3d67c42a1a59d3afe8e8c)
When using external Huffman tables fails during init, the decoder
reverts back to using the default Huffman tables; and when doing so,
the current VLC tables leak because init_default_huffman_tables()
doesn't free them before overwriting them.
Sample:
samples.ffmpeg.org/archive/all/avi+mjpeg+pcm_s16le++mjpeg-interlace.avi
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(cherry picked from commit 3cc685b7bcc2bcc781b5632bf2e0af6ed941ae27)
render_charset() used static buffers that are always completely
initialized before every use, so that it is unnecessary for the
values in these arrays to be kept after leaving the function.
Given that this is not only unnecessary, but harmful due to the
possibility of data races if several instances of a64multi/a64multi5
run simultaneously these buffers have been replaced by ordinary buffers
on the stack (they are small enough for this).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(cherry picked from commit 0ca09335aa47fee181c36187143403811b5452f6)
The current code tries the access the codecpar of a nonexistent
audio stream when seeking. Stop that. Fixes ticket #9121.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(cherry picked from commit af867e59d9ac3db5aaf7df4691f032e14ea51588)
An AVMD5 struct would leak if an error happened after its allocation.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 56bd071e5487fcf516d942bbc84d7a77b0bdf9b6)
A buffer may leak in case of YUVA444P10 with dimensions that are not
both divisible by 16.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit d789d72d30d8cb6c0a9b5b1140589da1eae44e1e)
When allocating a BSF fails, it could happen that the BSF's close
function has been called despite a failure to allocate the private data.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(cherry picked from commit 9bf2b32da048732ad79e075dfefd40777e9f935a)
Fixes a crash with ISOBMFF extradata containing no OBUs.
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(cherry picked from commit 8081a0b10f5325b60e885d8823593485019b8a18)
This is a regression since switching to the generic CBS BSF code.
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(cherry picked from commit b917218c35d240d9266c163103353e51facb0cae)
The DSS demuxer currently decrements a counter that should be positive
at the beginning of read_packet; should it become negative, it means
that the data to be read can't be read contiguosly, but has to be read
in two parts. In this case the counter is incremented again after the
first read if said read succeeded; if not, the counter stays negative.
This can lead to problems in further read_packet calls; in tickets #9020
and #9023 it led to segfaults if one tries to seek lateron if the seek
failed and generic seek tried to read from the beginning. But it could
also happen when av_new_packet() failed and the user attempted to read
again afterwards.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(cherry picked from commit afa511ad34452b1806a6cfa2dd785168140843e6)
There would be leaks in case of failure.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(cherry picked from commit 543e4a194252050cf1abcded7c75e4b889e3db4f)
Since the very beginning (since de6d9b6404bfd1c589799142da5a95428f146edd)
the AC-3 encoder used AC3_MAX_CODED_FRAME_SIZE (namely 3840) for the
size of the output buffer (without any check at all).
This causes problems when encoding EAC-3 for which the maximum is too small,
smaller than the actual size of the buffer: One can run into asserts used
by the PutBits API. Ticket #8513 is about such a case and this commit
fixes it by using the real size of the buffer.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
(cherry picked from commit 968c158abde36ebb7520706a69eebe3e8eacbd81)
Affected the vsynth*-flashsv2 FATE-tests.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit b7b73e83e3d5c78a5fea96a6bcae02e1f0a5c45f)
Affected many FATE-tests.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 3ef65fd4d1d0cdfa0b60351c719bef93d3664ea2)
Affected the filter-codecview-mvs FATE-test.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 3c151e79991181c8b7b9f9b888ac46eba91c5012)
Affected the g2m4 FATE-test.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit a86f3e983ec113689af48dd49ae043d85c8dc8bd)
Affected the base64 FATE test.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit bbf8431b1bb947bdb3c9cb34718879ad1fc4e1be)
Affected the FATE tests filter-gradfun-sample and sierra-vmd-video.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 566bf56791ea96df2bbc334feec0f1cdfc93b2da)
Affected the FATE tests mss2-wmv and mss1-pal.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 8429661db86b1e76bcae9d8381389098e57417de)
Affected the FATE tests vsynth*-zlib, mszh and zlib.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit dd9cbd1cc31e72ad9f11d10d086a46d339a924ca)
Before commit f1e17eb446577180ee9976730aacb46563766518, the qtrle
encoder had undefined pointer arithmetic: Outside of a loop, two
pointers were set to point to the ith element (with index i-1) of
a line of a frame. At the end of each loop iteration, these pointers
were decremented, so that they pointed to the -1th element of the line
after the loop. Furthermore, one of these pointers can be NULL (in which
case all pointer arithmetic is automatically undefined behaviour).
Commit f1e17eb44 added a check in order to ensure that the elements
never point to the -1th element of the array: The pointers are only
decremented if they are bigger than the frame's base pointer
(i.e. AVFrame.data[0]). Yet this check does not work at all in case of
negative linesizes; furthermore in case the pointer that can be NULL is
NULL initializing it still involves undefined pointer arithmetic.
This commit fixes both of these issues: First, non-NULL pointers are
initialized to point to the element after the ith element and
decrementing is moved to the beginning of the loop. Second, if a pointer
is NULL, it is just made to point to the other pointer, as this allows
to avoid checks before decrementing it.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 911fe69c5f5946461d9e7c785b19e3841dabd873)
If keeping a reference to an earlier frame failed, the next frame must
be an I frame for lack of reference frame. This commit implements this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit d5fc16a6a87a24360312b812a560e7a3664e3791)
Affected 26 FATE tests like swr-resample_async-s16p-44100-8000.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 64977ed7ae1e437910ca837ccb282e07d9200249)
Affected ProRes without alpha; affected 32 FATE tests, e.g. prores-422,
prores-422_proxy, prores-422_lt or matroska-prores-header-insertion-bz2.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit f83976344ecdc776ec827b06b19c2fcaa2b8ccdb)
Affected the filter-hdcd-* FATE tests.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
(cherry picked from commit 9eadd616b7ec31f9a6b691ff3faa2c6c3716335f)
