Fixes: out of array read
Fixes: 15409/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_H264_fuzzer-5758846959616000
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7d3581e6bb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 697984b9db)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The way videotoolbox hooks in as a hwaccel is pretty hacky. The VT decode
API is not invoked until end_frame(), so alloc_frame() returns a dummy
frame with a 1-byte buffer. When end_frame() is eventually called, the
dummy buffer is replaced with the actual decoded data from
VTDecompressionSessionDecodeFrame().
When the VT decoder fails, the frame returned to the h264 decoder from
alloc_frame() remains invalid and should not be used. Before
9747219958, it was accidentally being
returned all the way up to the API user. After that commit, the dummy
frame was unref'd so the user received an error.
However, since that commit, VT hwaccel failures started causing random
segfaults in the h264 decoder. This happened more often on iOS where the
VT implementation is more likely to throw errors on bitstream anomolies.
A recent report of this issue can be see in
http://ffmpeg.org/pipermail/libav-user/2016-November/009831.html
The issue here is that the dummy frame is still referenced internally by the
h264 decoder, as part of the reflist and cur_pic_ptr. Deallocating the
frame causes assertions like this one to trip later on during decoding:
Assertion h->cur_pic_ptr->f->buf[0] failed at src/libavcodec/h264_slice.c:1340
With this commit, we leave the dummy 1-byte frame intact, but avoid returning it
to the user.
This reverts commit 9747219958.
Signed-off-by: wm4 <nfxjfg@googlemail.com>
Should fix "libavcodec/h264_refs.c:372:13: warning: variable 'i' is used uninitialized whenever switch default is taken"
Found-by: durandal_17
Suggested-by: jkqxz
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* commit 'b13fc1e344011949929975a3451f78f226aa1de3':
h264: do not pass H264Context to h264_slice_header_parse()
Conflicts:
libavcodec/h264dec.h
Did not merge the h264_slice_header_parse() part. We use a few other
members of H264Context for error checking in that function.
Merged-by: Timothy Gu <timothygu99@gmail.com>
The code conflicts with moving the h264_init_ps() call point
Without this, ff_h264_parse_ref_count() fills ref and list count and
h264_init_ps() subsequently wipes them out on a "success" path.
Subsequently things crash as the wiped fields are used.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* commit 'ed9a20ebe4a89de119ea97bdccf688ece8c6648c':
h264: split reading the ref list modifications and actually building the ref list
ref_modifications.val are read as u32 instead of u8 in FFmpeg.
Merged-by: Clément Bœsch <clement@stupeflix.com>
* commit '7b50d60442af8d9527e9da46818011fe15a5265a':
h264: call ff_h264_fill_mbaff_ref_list() when constructing the normal ref list
Merged-by: Clément Bœsch <clement@stupeflix.com>
Move the NAL unit types into it. This will allow to stop including the
whole decoder-specific h264dec.h in some code that is unrelated to the
decoder and only needs some enum values.
While the value of those variables will be constant for the whole frame,
they are only used in two functions called from slice header decoding.
Moving them to the per-slice context allows us to make the H264Context
passed to slice_header_parse() constant.
* commit '755f79f84cbeb5d749fb120e55e0098a2d7663a0':
h264_refs: make the H264Context const where possible
ff_h264_decode_ref_pic_list_reordering() and h264_initialise_ref_list()
do not have a const H264Context * as they modify the default_ref inside
that context.
Merged-by: Clément Bœsch <u@pkh.me>
Do it right before the MMCOs are applied to the DPB. This will allow
moving the frame_start() call out of the slice header parsing, since
generating the implicit MMCOs needs to be done after frame_start().
They are stored in the slice header, so technically they are per-slice
(though they must be the same in every slice). This will simplify the
following commits.
This will allow postponing the reference list construction (and by
consequence some other functions, like frame_start) until the whole
slice header has been parsed.
Currently it's done in the code that initialises the ref list for
MBAFF, which is not a logical place for it. Move it to the function that
parses the pred table from the bitstream, which is analogous to what is
done for the implicit weight table as well.
* commit 'c8dcff0cdb17d0aa03ac729eba12d1a20f1f59c8':
h264: factor out calculating the POC count into a separate file
Merged-by: Clément Bœsch <u@pkh.me>
* commit '3176217c60ca7828712985092d9102d331ea4f3d':
h264: decouple h264_ps from the h264 decoder
Main changes:
- a local GetBitContext is created for the various
ff_h264_decode_seq_parameter_set() attempts
- just like the old code, remove_sps() is adjusted so it doesn't remove
the pps.
Fixes decode with Ticket #631http://ffmpeg.org/pipermail/ffmpeg-user/attachments/20111108/dae58f17/attachment.mp4
but see next point as well.
- ff_h264_update_thread_context() is updated to work even when SPS
isn't set as it breaks current skip_frame code. This makes sure we
can still decode the sample from ticket #631 without the need for
-flags2 +chunks. (Thanks to Michael)
- keep {sps,pps}_ref pointers that stay alive even when the active
pps/sps get removed from the available lists (patch by michaelni with
additionnal frees in ff_h264_free_context() from mateo)
- added a check on sps in avpriv_h264_has_num_reorder_frames() to fix
crashes with mpegts_with_dvbsubs.ts from Ticket #4074http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket4074/mpegts_with_dvbsubs.ts
- in h264_parser.c:h264_parse(), after the ff_h264_decode_extradata() is
called, the pps and sps from the local parser context are updated with
the pps and sps from the used h264context. This fixes fate-flv-demux.
- in h264_slice.c, "PPS changed between slices" error is not triggered
anymore in one condition as it makes fate-h264-xavc-4389 fails with
THREADS=N (Thanks to Michael)
Merged-by: Clément Bœsch <clement@stupeflix.com>
Merged-by: Michael Niedermayer <michael@niedermayer.cc>
Merged-by: Matthieu Bouron <matthieu.bouron@stupeflix.com>
* commit 'e481458bc308ee838deaeacac51929514762e7a7':
h264: factor out pred weight table parsing into a separate file
Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Make the SPS/PPS parsing independent of the H264Context, to allow
decoupling the parser from the decoder. The change is modelled after the
one done earlier for HEVC.
Move the dequant buffers to the PPS to avoid complex checks whether they
changed and an expensive copy for frame threads.
