This prevents having to sign-extend on 64-bit systems with 32-bit ints,
such as x86-64. Also fixes crashes on systems where we don't do it and
arguments are not in registers, such as Win64 for all weight functions.
Revert "swscale: update context offsets after removal of AlpMmxFilter."
(commit a95e3fa90b)
and
Revert "swscale: Remove some write-only variables related to alpha handling."
(commit 9d03cb9fc5).
They broke alpha handling - it's the evil inline asm that still uses that
variable, so it's not truely write-only.
The parser uses VLC tables initialized in vc1_common_init(), therefore
we should call this function on parser init also.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Return 0 means "please return the same data again", i.e. it causes an
infinite loop. Instead, return an error.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
This prevents failed assertions further down in the packet processing
where we require non-negative values for packet_size_left.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Return 0 indicates "please return the same data again", i.e. it causes
an infinite loop. Instead, return that we consumed the buffer if we
finished decoding succesfully, or return an error if an error occurred.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
If bufsize < headersize, init_get_bits() will be called with a negative
number, causing it to fail and any subsequent call to get_bits() will
crash because it reads from a NULL pointer.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
On EOF, get_bits() will continuously return 0, causing an infinite
loop.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Disadvantage is that it no longer allows modifying brightness through
adjustment of the RGB lookup table. Advantage is that now monowhite/black
no longer need to be identified as a RGB format.
WMApro actually support 13-bits block sizes (potentially even up to 14),
and thus we should support that also. If we get block sizes beyond what
the decoder can handle (14 is possible depending on s->decode_flags),
error out instead of crashing.
They were introduced in an earlier commit that introduced use of named
arguments. One cause was a typo, a second cause appears to be a bug in
x264asm that I work around by not using named arguments.
At very small dimensions, this calculation could lead to zero-sized
filters, which leads to uninitialized output, zero-sized allocations,
loop overflows in SIMD that uses do{..}while(i++<filtersize); instead
of for(i=0;i<filtersize;i++){..} and several other similar failures.
Therefore, require a minimum filtersize of 1.
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
Conversion of the luma intra prediction mode to one of the constrained
("alzheimer") ones can happen by crafting special bitstreams, causing
a crash because we'll call a NULL function pointer for 16x16 block intra
prediction, since constrained intra prediction functions are only
implemented for chroma (8x8 blocks).
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
CC: libav-stable@libav.org
This fixes crashes in e.g. PNG decoding with SSE2 enabled. In fact, many
x86 optimizations for codecs assume that our buffer strides are 16-byte
aligned.
Also slightly move around code not allocate a new frame if we won't
decode it. This prevents us from putting undecoded frames in frame
pointers, which (in mt decoding) other threads will use and wait on
as references, causing a deadlock (if we skipped decoding) or a crash
(if we didn't initialized next_framep[] at all).
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
This will be useful to test more aggressively for failures to mark XMM
registers as clobbered in Win64 builds, and prevent regressions thereof.
Based on a patch by Ramiro Polla <ramiro.polla@gmail.com>
On x86-64, it indeed uses all 16 registers (and on x86-32, this gets
clipped to 8). Not marking it properly causes callers of this function
to fail randomly because of XMM register clobbering.
Fixes problems where rgbToRgbWrapper() is called even though it doesn't
support this particular conversion (e.g. converting from RGB444 to
anything). Thirdly, fixes issues where rgbToRgbWrapper() is called for
non-native endiannness conversions (e.g. RGB555BE on a LE system).
Fourthly, fixes crashes when converting from e.g. monowhite to
monowhite, which calls planarCopyWrapper() and overwrites/reads because
n_bytes != n_pixels.
The safe bitstream reader does not allow using skip_bits_long() to seek to a
point before the start of the buffer, which was needed by the mp3 decoder.
This change instead calculates the start point of the first valid granule and
skips to that position.
Signed-off-by: Justin Ruggles <justin.ruggles@gmail.com>
This fixes integer multiplication overflows in RGB48 output
(vertical) scaling as detected by IOC. What happens is that for
certain types of filters (lanczos, spline, bicubic), the
intermediate sum of coefficients in the middle of a filter can
be larger than the fixed-point equivalent of 1.0, even if the
final sum is 1.0. This is fine and we support that.
However, at frame edges, initFilter() will merge the coefficients
for the off-screen pixels into the top or bottom pixel, such as
to emulate edge extension. This means that suddenly, a single
coefficient can be larger than the fixed-point equivalent of
1.0, which the vertical scaling routines do not support.
Therefore, remove the merging of coefficients for edges for
the vertical scaling filter, and instead add edge detection
to the scaler itself so that it copies the pointers (not data)
for the edges (i.e. it uses line[0] for line[-1] as well), so
that a single coefficient is never larger than the fixed-point
equivalent of 1.0.
This fixes the same overflow as in the RGB48/16-bit YUV scaling;
some filters can overflow both negatively and positively (e.g.
spline/lanczos), so we bias a signed integer so it's "half signed"
and "half unsigned", and can cover overflows in both directions
while maintaining full 31-bit depth.
Signed-off-by: Mans Rullgard <mans@mansr.com>
For certain types of filters where the intermediate sum of coefficients
can go above the fixed-point equivalent of 1.0 in the middle of a filter,
the sum of a 31-bit calculation can overflow in both directions and can
thus not be represented in a 32-bit signed or unsigned integer. To work
around this, we subtract 0x40000000 from a signed integer base, so that
we're halfway signed/unsigned, which makes it fit even if it overflows.
After the filter finishes, we add the scaled bias back after a shift.
We use the same trick for 16-bit bpc YUV output routines.
Signed-off-by: Mans Rullgard <mans@mansr.com>
When turned on, H264/CAVLC gets ~15% (CVPCMNL1_SVA_C.264) slower for
ultra-high-bitrate files, or ~2.5% (CVFI1_SVA_C.264) for lower-bitrate
files. Other codecs are affected to a lesser extent because they are
less optimized; e.g., VC-1 slows down by less than 1% (all on x86).
The patch generated 3 extra instructions (cmp, cmovae and mov) per
call to get_bits().
The performance penalty on ARM is within the error margin for most
files, up to 4% in extreme cases such as CVPCMNL1_SVA_C.264.
Based on work (for GCI) by Aneesh Dogra <lionaneesh@gmail.com>, and
inspired by patch in Chromium by Chris Evans <cevans@chromium.org>.
The keyframe after a POC reset may not be the first to be returned to
the user. Therefore, don't reset the expected next POC once we return
a keyframe to the user, but once we know that the next frame in the
return-queue is a keyframe.
Fixes the following conformance suite samples:
HCBP1_HHI_A.264, HCBP2_HHI_A.264, HCMP1_HHI_A.264 (main)
HCHP1_HHI_B.264, HCHP2_HHI_A.264, HCHP3_HHI_A.264 (frext)
This fixes false positives of has_codec_delay_been_guessed() for
streams where not every input picture generates an output picture,
such as interlaced H264.
Ronald S. Bultje
