In these cases, there is no extradata but only the properly set
width/height values by the demuxer.
This makes sure VP6 in F4V files is cropped properly.
This is similar to what is done for H264 for letting the container
width/height override what's in the bitstream, since 30f515091.
Signed-off-by: Martin Storsjö <martin@martin.st>
Store XMM6 and XMM7 in the shadow space in functions that
clobbers them. This way we don't have to adjust the stack
pointer as often, reducing the number of instructions as
well as code size.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
This fixes reads out of bounds.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
We read 2 bytes for 15 out of 16 pixels, therefore we need to
have at least 30 bytes, not 16.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
Decoded data is always written in pairs within this decoder.
This fixes writes out of bounds.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
Previously, we returned 0, meaning successful decoding but 0
bytes consumed, leading to an infinite loop.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
If the input buffer contains more data after the deflate stream,
the loop previously left running infinitely, with inflate returning
Z_STREAM_END.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
In lpc_prediction(), we write up to array element 'lpc_order' in
an array allocated to hold 'max_samples_per_frame' elements.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
This fixes breakage in a few fate tests on certain setups
(that for some reason didn't break on OS X) after the previous
commit (8812a8057). Currently, some video streams are initialized
in ff_MPV_common_init with width/height set at 0 and only changed
to a proper video size with ff_MPV_common_frame_size_change later.
The breakage was diagnosed by Anton Khirnov.
Signed-off-by: Martin Storsjö <martin@martin.st>
The actual usefulness of the hack is not known, and it does cause
infinite loops with some broken input files.
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
Previously, s->context_initialized was left set to 1
if ff_vc1_decode_init_alloc_tables failed, skipping the
initialization completely on the next decode call.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
If the height is zero, the decompression will probably end up
failing due to not fitting into the allocated buffer later
anyway, so this doesn't need any more elaborate check.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
This is similar to the fix in 35cbc98b.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
The init function reads one byte of extradata.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
This is required, since invalid parameters actually could
pass the switch check below.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
A change in framesize caused a perpetual loss of synchronization.
So read (and use) the frame size from the frame header instead of
setting it only once.
Signed-off-by: Diego Biurrun <diego@biurrun.de>
This avoids crashes when initializing the range coder for
the first slice context.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
This is similar to an existing check for the second-last frame
from 062421e3.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
This avoids a potential division by zero.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
Also return a proper error code.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
Also pass on any returned error code.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
Previously the wrong buffer pointer was checked, when buf
instead of *buf was checked. But checking the return value
instead is even better.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
If idx equaled num_coeffs - 1 on entry to the loop, the previous
check failed to break out of the loop.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
The volatile is not required here, and prevents a miscompilation with GCC
4.8.1 when building on x86 with --cpu=i686
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
The sample buffering logic does not take into account that the blocksize
could change. Reset the buffer if the channel configuration changes,
since if there are leftover samples, it is most likely a broken or
misconcatenated stream. This could lead to negative numbers for
missing_samples during decoding.
Thanks to Michael Niedermeyer for pointing these out.
Also don't pointlessly set the buffer size to 1 after copying
one packet.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
This makes sure that linesize * start_y doesn't overflow, so that
emulated_edge_mc can get back the original value if needed.
Reported-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Signed-off-by: Martin Storsjö <martin@martin.st>
