alloc_frame_buffer in ff_alloc_picture asserts that the linesize
of planes 1 and 2 are the same. If the pixfmt has a single uv
plane, like NV12, this won't be true.
So, let's only do this check if there are more than 2 planes.
We never hit this with previous hw formats because they don't set
linesize to meaningful values, but the cuda hw format sets the
values based on the underlying data layout.
Use mask buffer.
Signed-off-by: Kaustubh Raste <kaustubh.raste@imgtec.com>
Reviewed-by: Manojkumar Bhosale <Manojkumar.Bhosale@imgtec.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This is a partial fix - to fix videotoolbox fully (with multithreaded
hwaccel decoding), much more work would be required.
Before this change, an assertion failure would trigger:
$ ffmpeg -hwaccel videotoolbox -hwaccel_output_format videotoolbox_vld -i h264.ts -f null -y /dev/null
Assertion (frame->private_ref && frame->private_ref->size == sizeof(FrameDecodeData)) || !(avctx->codec->capabilities & (1 << 1)) failed at libavcodec/decode.c:620
Signed-off-by: Aman Gupta <aman@tmm1.net>
This includes the SEI_PREFIX/SEI_SUFFIX NALUs, which can contain
updates like HEVC_SEI_TYPE_ACTIVE_PARAMETER_SETS. Previously, hevc
samples with this SEI present would not playback correctly.
See for example https://github.com/lhc70000/iina/issues/1123
Signed-off-by: Aman Gupta <aman@tmm1.net>
Removes the avctx->extradata_size requirement when creating avcC/hvcC, since
avctx->extradata is only used in the esds code path.
This fixes an issue where the VideoToolbox decoder would not work unless
avformat_find_stream_info() was called.
Signed-off-by: Aman Gupta <aman@tmm1.net>
This fixes decoding of H264 video samples with SPS and PPS changes.
See for example https://s3.amazonaws.com/tmm1/videotoolbox/spschange.ts,
which previously stalled the decoder and failed to produce any new frames
after the SPS change.
Also see https://s3.amazonaws.com/tmm1/videotoolbox/ppschange.ts, which
uses multiple PPS and would previously cause VT decode failures.
If the VideoToolbox session needs to be restarted, and
videotoolbox_start() fails for some reason (for instance, if the video
is interlaced and the decoder is running on iOS), avcodec will return
AVERROR_EXTERNAL. This can be used by the API user to switch to another
decoder.
Signed-off-by: Aman Gupta <aman@tmm1.net>
This callback will be used by the VideoToolbox H264 hwaccel so that it
can receive SPS and PPS NALUs. VideoToolbox requires PPS changes to be
fed into the decoder session, and for the session to be recreated when
the SPS changes.
Signed-off-by: Aman Gupta <aman@tmm1.net>
The only reason videotoolbox wouldn't produce frames is if the data fed
to it was invalid, so returning AVERROR_INVALIDDATA makes sense here.
Further, it means AVERROR_EXTERNAL can be used in further commits to signal
fatal VideoToolbox errors, letting the user know that they need to fallback to
another decoder.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
This allows decode_slice to be invoked multiple times before end_frame,
causing slices to accumulate before being fed into the VT decoder.
An upcoming commit will re-use decode_slice for parameter NALUs, so
they can be propagated into the VT decoder session along with slice
data.
Signed-off-by: Aman Gupta <aman@tmm1.net>
These helpers will be used in later commits to automatically restart
the decoder session when SPS changes are encountered.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Fixes: runtime error: signed integer overflow: 1939661764 - -454942263 cannot be represented in type 'int'
Fixes: 3191/clusterfuzz-testcase-minimized-5688798451073024
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: left shift of negative value -801112064
Fixes: 3492/clusterfuzz-testcase-minimized-5784775283441664
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: signed integer overflow: 1219998458 - -1469874012 cannot be represented in type 'int'
Fixes: 3443/clusterfuzz-testcase-minimized-5369987105554432
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 3984/clusterfuzz-testcase-minimized-5265759929368576
Fixes: runtime error: signed integer overflow: -1085585801 + -1094995529 cannot be represented in type 'int'
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Variables used in inline assembly need to be marked with attribute((used)).
Static constants already were, via the define of DECLARE_ASM_CONST.
But DECLARE_ALIGNED does not add this attribute, and some of the variables
defined with it are const only used in inline assembly, and therefore
appeared dead. This change adds a macro DECLARE_ASM_ALIGNED that marks
variables as used.
This change makes FFMPEG work with Clang's ThinLTO.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
* commit '5c22c90c1d5050f1206e46494b193320ac2397cb':
vp9_superframe_bsf: cache packets by creating new references instead of moving pointers
See 37f4a093f7f95241e4fcd582758504491e85e488
7a02b364b68c0bf7f065f5c217fae458f0efdb8d
Merged-by: James Almer <jamrial@gmail.com>
* commit '0ccddbad200c1d9439c5a836501917d515cddf76':
smacker: limit recursion depth of smacker_decode_bigtree
See 946ecd19ea752399bccc751c9339ff74b815587e
Merged-by: James Almer <jamrial@gmail.com>
* commit 'cd4663dc80323ba64989d0c103d51ad3ee0e9c2f':
smacker: add sanity check for length in smacker_decode_tree()
See b829da363985cb2f80130bba304cc29a632f6446
Merged-by: James Almer <jamrial@gmail.com>
* commit '1746c7c8f2f9a6c5eacb486426dd0a579b4b7498':
libspeexenc: Use speex_lib_get_mode instead of the speex_foo_mode data symbols
Merged-by: James Almer <jamrial@gmail.com>
* commit 'b46a77f19ddc4b2b5fa3187835ceb602a5244e24':
lavc: external hardware frame pool initialization
Includes the fix from e724bdfffbd3c27aac53d1f32f20f105f37caef0
Merged-by: James Almer <jamrial@gmail.com>
This reverts commit 1a4315f24d2c5857ebeb379fe39f0124746ee30a, reversing
changes made to 869401cefc22e221edbf135bacaa1f21014fd001.
At least one distro (Arch) ships gsm.h inside the gsm/ folder
