Up until now, both the msmpeg4 decoders and encoders initialized several
RLTables common to them (the decoders also initialized the VLCs of these
RLTables). This is an obstacle to making these codecs init-threadsafe.
So move this initialization to ff_msmpeg4_common_init() that already
contains this initialization code. This allows to reuse the AVOnce used
for initializing ff_v2_dc_lum/chroma_table which automatically makes
initializing these RLTables thread-safe.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This mostly reverts commit 4b2863ff01.
Said commit removed the freeing code from ff_mpv_common_init(),
ff_mpv_common_frame_size_change() and ff_mpeg_framesize_alloc() and
instead added the FF_CODEC_CAP_INIT_CLEANUP to several codecs that use
ff_mpv_common_init(). This introduced several bugs:
a) Several decoders using ff_mpv_common_init() in their init function were
forgotten: This affected FLV, Intel H.263, RealVideo 3.0 and V4.0 as well as
VC-1/WMV3.
b) ff_mpv_common_init() is not only called from the init function of
codecs, it is also called from AVCodec.decode functions. If an error
happens after an allocation has succeeded, it can lead to memleaks;
furthermore, it is now possible for the MpegEncContext to be marked as
initialized even when ff_mpv_common_init() returns an error and this can
lead to segfaults because decoders that call ff_mpv_common_init() when
decoding a frame can mistakenly think that the MpegEncContext has been
properly initialized. This can e.g. happen with H.261 or MPEG-4.
c) Removing code for freeing from ff_mpeg_framesize_alloc() (which can't
be called from any init function) can lead to segfaults because the
check for whether it needs to allocate consists of checking whether the
first of the buffers allocated there has been allocated. This part has
already been fixed in 76cea1d2ce.
d) ff_mpv_common_frame_size_change() can also not be reached from any
AVCodec.init function; yet the changes can e.g. lead to segfaults with
decoders using ff_h263_decode_frame() upon allocation failure, because
the MpegEncContext will upon return be flagged as both initialized and
not in need of reinitialization (granted, the fact that
ff_h263_decode_frame() clears context_reinit before the context has been
reinited is a bug in itself). With the earlier version, the context
would be cleaned upon failure and it would be attempted to initialize
the context again in the next call to ff_h263_decode_frame().
While a) could be fixed by adding the missing FF_CODEC_CAP_INIT_CLEANUP,
keeping the current approach would entail adding cleanup code to several
other places because of b). Therefore ff_mpv_common_init() is again made
to clean up after itself; the changes to the wmv2 decoder and the SVQ1
encoder have not been reverted: The former fixed a memleak, the latter
allowed to remove cleanup code.
Fixes: double free
Fixes: ff_free_picture_tables.mp4
Fixes: ff_mpeg_update_thread_context.mp4
Fixes: decode_colskip.mp4
Fixes: memset.mp4
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Both motion vector tables have the same number of elements, hence one
can inline said number and remove the field containing the number of
elements from the structure.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
msmpeg4dec and ituh263dec both create VLCs with identical parameters out
of ff_mvtab. Given that ff_msmpeg4_decode_init() always (indirectly) calls
ff_h263_decode_init_vlc(), the VLC initialized by the latter can be
directly used by msmpeg4dec. Doing so saves a bit more than 2KB from the
.bss segment as well as the code to initialize a VLC.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Some of the RLTables used by msmpeg4dec actually coincide with other
RLTables: ff_rl_table[5] coincides with ff_h263_rl_inter (and
ff_rl_table[2] with ff_mpeg4_rl_intra). Given that ff_h263_rl_inter is
always initialized before msmpeg4dec's RLTables are initialized, one can
just reuse the VLC tables by copying the pointers; after all, there are
no ownership issues for static data. This saves 70912B from the .bss
segment, translating into actual memory savings when this decoder is
actually used.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
For the RLTables ff_rl_table[0..2] only the very first VLC is only ever
used, so it makes no sense to create 32 of them. This saves 285200B from
the .bss segment; this amount of memory is actually saved when this
decoder is used.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Frames that small are not valid and of limited use for error concealment, while
being very computationally intensive to process.
Fixes: Timeout
Fixes: 11318/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_MSMPEG4V1_fuzzer-5710884555456512
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: left shift of negative value -1
Fixes: 1480/clusterfuzz-testcase-minimized-5188321007370240
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/targets/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
32bit is not sufficient for all cases
Fixes: signal_sigabrt_7ffff6ac8cc9_686_cov_1897408623_microsoft_new_way_to_shove_mpeg2_in_asf.dvr_ms
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* commit '3dc6272bed7890a49080e18eacf3c7a4a6594b0d':
Remove a number of unnecessary dsputil.h #includes
Conflicts:
libavcodec/h264pred.c
libavcodec/vc1dsp.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Fixes use of uinitialized memory
Fixes: msan_uninit-mem_7f01b8799186_7278_msnaudio.asf
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'b4d24b471bc52f1f78a43ee330199e70483e51c3':
build: Remove configure-generated .config file on distclean
msmpeg4: Split decoding related functions to a separate file
Conflicts:
Makefile
libavcodec/Makefile
libavcodec/msmpeg4.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This fixes standalone compilation of the msmpeg4v2, msmpeg4v3
and wmv2 encoders, that previously failed to link due to the
decoder codepaths requiring error_resilience.
Signed-off-by: Martin Storsjö <martin@martin.st>
