Move some functions from dsputil. The idea is that videodsp contains
functions that are useful for a large and varied set of video decoders.
Currently, it contains emulated_edge_mc() and prefetch().
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
The VC1 decoder uses edge_emu_buffer simultaneously for luma and chroma
and needs more space. That was not a problem before f1d8763a02
since the size for edge_emu_buffer was always calculated with 2 byte per
pixel since the linesize was not known.
Fixes occasionally fate errors in vc1_sa10143.
Since we can't know which stride a custom get_buffer() implementation is
going to use we have to allocate this scratch buffers after the linesize
is known. It was pretty safe for 8 bit per pixel pixel formats since we
always allocated memory for up to 16 bits per pixel. It broke hoever
with cmdutis.c's alloc_buffer() and high pixel bit depth since it
allocated larger edges than mpegvideo expected.
Fixes fuzzed sample nasa-8s2.ts_s244342.
It is not posible to call get_buffer during frame-mt codec
initialization. Libavformat might pass huge amounts of data as
extradata after parsing broken files. The 'extradata' for the fuzzed
sample sample_varPAR_s5374_r001-02.avi is 2.8M large and contains
multiple slices.
This requires to move the avcodec_default_free_buffers() call to
ff_MPV_common_end() since otherwise delayed pictures would get freed
during a size change.
Direct rendering capable decoders call get_buffer() which will set the
frame parameters.
Prevents frames with wrong parameters when a decoder outputs delayed
frames after a resolution or pixel format change.
Use this in VP8/H264-8bit loopfilter functions so they can be used if
there is no aligned stack (e.g. MSVC 32bit or ICC 10.x).
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
This changes the LOCAL_ALIGNED definition on systems where
DECLARE_ALIGNED is used so it matches the manual alignment
case, ensuring invalid use will not compile on x86 only to
fail on everything else.
Signed-off-by: Mans Rullgard <mans@mansr.com>
The initial testing of the VFW binary codec was flawed,
likely due to an AviSynth bug.
Re-testing using VirtualDub and various professional editing
applications has revealed it should have been flipped.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Prevent an invalid write into coeffs[scantable[-1]] if zeros_left
itself was an invalid VLC code (and thus -1).
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
This prevents undefined behaviour of signed left shift if the coded
value is larger than 2^31. Large values are most likely invalid and
caused errors or by feeding random.
Validate every use of svq3_get_ue_golomb() and changed the place there
the return value was compared with negative numbers. dirac.c was clean,
fixed rv30 and svq3.
