It was only validating that normal data wasn't filling the buffer.
However, extra data may be written afterwards.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Currently, the encoder will try to reduce it down to 150000, but the
decoder will complain starting at 131072 (WV_MAX_SAMPLES). Therefore,
change the loop limit.
Fixes ticket #3881.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Some streams were found to have what appeared to be truncated SPS.
Their syntax seem to be valid at least until the end of the VUI, so
try that syntax if the parsing would overflow the SPS in the
conforming syntax.
Fixes ticket #3872.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
bytestream2_* will not cause buffer overflow, but in that case, this means
the allocation would be incorrect and the encoded result invalid. Therefore,
assert no overflow occurred.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
With huge sampling rates, the table derivation method does not converge fast
enough. While fixing it using e.g. Newton-Rhapson-like methods (the curve is
nicely convex) is possible, it is much simpler to reject these cases.
The value of 96000 was arbitrarily chosen as a realistic value, though
1000000 would still work and converge.
Fixes ticket #3868.
Suggested-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The allocation didn't account for headers, that can be easily 79 bytes.
As a result, buffers allocated for a few samples (e.g. 5 in the original
bug) could be undersized.
Fixed ticket #2881.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Not actually used in huffyuvenc, but rather in setting the frame
threading.
Example for some files:
context=0: 851974 27226 1137281
context=1,ND=0: 471819 22604 972351
context=1,ND=1: 472875 22673 972582
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Prior to 56.1.100, incorrect ALAC files for 24bps content were produced, in
particular not decoding losslessly.
Add an option to allow correctly decoding those streams.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The packet buffer allocation considers the alpha channel as DCT-coded,
while it is actually run-coded and thus requires a larger buffer.
CC: libav-stable@libav.org
Signed-off-by: Diego Biurrun <diego@biurrun.de>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
The buffer allocation may be incorrect (e.g. with an alpha plane),
and currently causes the buffer to be set to NULL by init_put_bits,
causing a crash later on.
So, detect that situation, and if detected, reallocate the buffer
and ask for a sample that shows the problem.
CC: libav-stable@libav.org
Signed-off-by: Diego Biurrun <diego@biurrun.de>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Some files seem to have an off-by-one error. In most cases, it appears to
be on the image width. Therefore, if the decoded image doesn't fit in the
screen:
- If it is wider than the screen (and the lzw decoding buffer), reject it;
- Otherwise, decode the indicated amount, but only write a truncated amount
to the screen.
Fixes ticket #3538.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The raw coded bits are extracted prior to decorrelation, as is correctly
performed by the decoder, and not after.
Fixes ticket #2768.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The issue is that, when the main packet data buffer is changed, streamcopy
uses a temporary new packet to store that buffer, frees the old packet, and
replace it with the new packet.
However, in doing so, it forgets about the side data, which gets freed, but
is still needed and referenced. Then, when the packet gets freed again in
the normal code path, it attempts to free its side data which has already
been freed.
Therefore, simply avoid the first free on side data by removing that side
data from the packet.
Fixes ticket #3773.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The encoder produces files that are no longer compatible with previous
versions of the decoder, and may actually cause decoding issues for other
software, so indicate that change to allow decoder quirks.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The packet buffer allocation considered as dct-coded, while it is
actually run-coded and thus requires a larger buffer.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
SMPTE 268M-2003 specifies that each line starts at a 4-bytes boundary.
Therefore, modify correspondingly the input buffer strides and size.
Partially fixes ticket #3692: DLAD_8b_3c_big.dpx still has inverted
colors, which might be related to endianness.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
S268M-2003 specifies that each line start is aligned on a 4-byte boundary.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Reduces the number of calls to tmvp derivation from 933685 to 586271 on
a sequence.
Reviewed-by: Mickaël Raulet <mraulet@insa-rennes.fr>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
The position is either rounded or not checked, so delay the wait to
check the proper value.
Reviewed-by: Mickaël Raulet <mraulet@insa-rennes.fr>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
After finishing parsing VPS/SPS/PPS/slice header, check remaining bits,
and if an overconsumption occurred, report invalid data.
Liked-by: BBB
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This makes the SPS parsing a little, but barely, safer.
Reviewed-by: Mickaël Raulet <mraulet@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
They might be left uninitialized otherwise since 3ad04608.
Fixes ticket #3840.
Found-by: Carl Eugen Hoyos <ce@hoyos.ws>
Reported-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
It previously used the output, cropped size, causing overreads/writes.
Fixes ticket #3839.
This issue was introduced by d249e682, which is not part of any release
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The x86 asm expects int32_t so use that type.
Reviewed-by: Mickaël Raulet <mraulet@insa-rennes.fr>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The unpacks/shuffles later on makes it unnecessary.
Before:
1508 decicycles in h, 2096759 runs, 393 skips
2512 decicycles in v, 2095422 runs, 1730 skips
After:
1477 decicycles in h, 2096745 runs, 407 skips
2484 decicycles in v, 2095297 runs, 1855 skips
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This should help cache locality. On win64:
Before: 1397x cycles, 16216 bytes
After: 1369x cycles, 16040 bytes
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Reduces on a sequence number of calls from 933685 to 586271.
Reviewed-by: Mickaël Raulet <mraulet@insa-rennes.fr>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The position is either rounded or not checked, so delay the wait to
check the proper value.
Reviewed-by: Mickaël Raulet <mraulet@insa-rennes.fr>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
When dealing with MVs, both components may be processed at a time.
On Win64, 560 to 539 cycles for derive_spatial_merge_candidates.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
There's a lag of one CTB line for SAO behind deblocking filter, except for
last line. However, once SAO has been completed on a line, all its pixels,
i.e. up to y+ctb_size are filtered and ready to be used as reference.
Without SAO, when deblocking filter finishes a CTB line, only the bottom
bottom 4 pixels may be filtered when next CTB is process by the deblocing.
The await_progess for hevc then checks whether the bottom pixels of a PU
requires access beyond that point, so the reporting should effectively
report up to the the above limits.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Normally, a Laplace distribution is more typical of the residuals
encoded, but for noisy input, it's both better and simpler to be
safe and use a 1/d^2 distribution. Second hunk could use some
renormalization but it has effectively little impact.
Output size of ffvhuff on various 4:2:0 sequences:
context=0,1/d: 851974 27226 1137281
context=0,1/d²: 619081 25069 1051500
context=0,1/d³: 501983 30454 1290561
context=0,lapl: 500650 31754 1304082
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The reader reads in chunks of 11 bits at most, and at most 3 times. The unsafe
reader therefore may read 6 chunks instead of 1 in worst case, ie 8 bytes,
which is within the padding tolerance.
The reader ends up being ~10% faster. Cumulative effect of unsafe reading and
code block swapping on 3 sequences is for 1 thread, decoding time goes from
23.3s to 19.0s.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The old code was reserving the 0xFFFF entry to represent an inexisting
entry/codeword. These entries are now detected through their length
being <= 0. As this entry is often used for the residuals (-1,-1), which
should be among the most frequent, it is particularly important to not
reserve it.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The effect is not really deterministic, as it seems to be a combination
on x86_64 of fewer registers used, different jump offsets and, for all
archs, of likely branches.
Speedup is around 15%.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Those macros take a byte number as shift argument, as this argument
differs between MMX and SSE2 instructions.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This is done by padding the coefficient buffer with 0s, because the order
may be only a multiple of 4, and the DSP function requires batches of 8.
However, no sample with such a case was found, so request one if it uses
that kind of order.
Approximate relative speedup depending on instruction set:
plain C: -6%
mmxext: 51%
sse2: 54%
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
An invalid entry already has the property of having a negative number
of bits, so remove the check on the reserved value, and rearrange the
code as a consequence.
346800 decicycles in 422, 262079 runs, 65 skips
168197 decicycles in gray, 262077 runs, 67 skips
Overall time: 7.878s
319076 decicycles in 422, 262096 runs, 48 skips
159875 decicycles in gray, 262057 runs, 87 skips
Overall time: 7.394s
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
It's no longer used inside another specific macro, so rename it.
Also remove duplicated definition and realign code.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
When the joint table does not contain a valid entry, the decoding restarts
from scratch. By implementing the trick of jumping to the 2nd level of the
individual table (and inlining the whole), a speed improvement of 5-10%
is possible.
On a 1000-frames YUV4:2:0 video, before:
362851 decicycles in 422, 262094 runs, 50 skips
182488 decicycles in gray, 262087 runs, 57 skips
Object size: 23584
Overall time: 8.377
After:
346800 decicycles in 422, 262079 runs, 65 skips
168197 decicycles in gray, 262077 runs, 67 skips
Object size: 23188
Overall time: 7.878
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The xmm reg count was incorrect, and manual loading of the gprs
furthermore allows to noticeable reduce the number needed.
The modified functions are used in weighted prediction, so only a
few samples like WP_* exhibit a change. For this one and Win64
(some widths removed because of too few occurrences):
WP_A_Toshiba_3.bit, ff_hevc_put_hevc_uni_w
16 32
before: 2194 3872
after: 2119 3767
WP_B_Toshiba_3.bit, ff_hevc_put_hevc_bi_w
16 32 64
before: 2819 4960 9396
after: 2617 4788 9150
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
