* commit '1f097d168d9cad473dd44010a337c1413a9cd198':
h264: reset data partitioning at the beginning of each decode call
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Prevents using GetBitContexts with data from previous calls.
Fixes access to freed memory.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
This avoids them being cleared before the full initialization finished
Fixes out of array read
Fixes: asan_heap-oob_f0c5e6_7071_cov_1605985132_mov_h264_aac__Demo_FlagOfOurFathers.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Such changes are forbidden in H.264 and lead to race conditions
Fixes out of array read
Fixes: signal_sigsegv_f9796a_1613_cov_3114610371_FM1_BT_B.h264
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This handles macroblock edges for the chroma components in the same way
as for the luma compoment for 4:4:4 streams. The Spec explicitly states
that the deblocking filter is not applied to edges at the boundary of
the picture.
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
* commit '00dbff4c3e048b4abd01bf805725aabff0fa5ee1':
h264: do not call field_end if we do not have a current picture
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Otherwise the ER code might try to use some already freed references.
Fixes possible access to freed memory.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
Higher modes are not allowed for 16x16/chroma, which is what this
function is used for. Otherwise this function would return 0 (vertical
prediction) for invalid higher modes, which could result in invalid
reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
In this case we may not have a current frame, while first_field being
set implies we do.
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
* commit '9eef9eb3014b2ed9c3ff4aac510a9f04edb555cf':
h264: check that execute_decode_slices() is not called too many times
Conflicts:
libavcodec/h264.c
The check is replaced by an assert() as the mb index should not ever go out
of bounds.
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '9a026c72982faf20e1c8dfbe48f0b312cdea69c8':
h264: rebuild the default ref list if the reference count changes
Conflicts:
libavcodec/h264.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
There is no point in delaying the check and it avoids bugs with a
half-initialized context.
Fixes invalid reads.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
* commit '58312b2472d3a44d7458865c459d59ef2e02bf1a':
h264: reset data_partitioning if decoding the slice header for NAL_DPA fails
Merged-by: Michael Niedermayer <michaelni@gmx.at>
If it was set before then we can end up trying to decode a slice without
a valid slice header, which can lead to invalid memory access.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC:libav-stable@libav.org
This is a temporary workaround to allow deprecating
avcodec_get_frame_defaults(). The proper solution will be using a
properly allocated AVFrame in Picture.
* commit '5b10ef729f610fcbc9c485e7b643ce53268144cb':
h264: parse frame packing arrangement SEI messages and save relevant stereo3d information
Conflicts:
libavcodec/h264.c
libavcodec/h264_sei.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'a553c6a347d3d28d7ee44c3df3d5c4ee780dba23':
lavc: use buf[0] instead of data[0] in checks whether a frame is allocated
Conflicts:
libavcodec/h264_refs.c
libavcodec/mpegvideo.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* commit '458446acfa1441d283dacf9e6e545beb083b8bb0':
lavc: Edge emulation with dst/src linesize
Conflicts:
libavcodec/cavs.c
libavcodec/h264.c
libavcodec/hevc.c
libavcodec/mpegvideo_enc.c
libavcodec/mpegvideo_motion.c
libavcodec/rv34.c
libavcodec/svq3.c
libavcodec/vc1dec.c
libavcodec/videodsp.h
libavcodec/videodsp_template.c
libavcodec/vp3.c
libavcodec/vp8.c
libavcodec/wmv2.c
libavcodec/x86/videodsp.asm
libavcodec/x86/videodsp_init.c
Changes to the asm are not merged, they are left for volunteers or
in their absence for later.
The changes this merge introduces are reordering of the function
arguments
See: face578d56
Merged-by: Michael Niedermayer <michaelni@gmx.at>
Allow supporting files for which the image stride is smaller than
the maximum block size + number of subpel mc taps, e.g. a 64x64 VP9
file or a 16x16 VP8 file with -fflags +emu_edge.
The not handling of frame gaps has lead to the lack of a dummy reference
frame, which has lead to the failure of decode_slice_header() which has
lead to one SEI recovery message being skiped which had introduced a
slightly suboptimal recovery point for at least 1 h264 file compared to
JM.
Found-by: Carl & BugMaster
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
