The change was not correct and broke H264
This reverts commit cd83f899c94f691b045697d12efa21f83eb2329f.
(cherry picked from commit 95b59bfb9d9e47de8438183a035e02667946f27c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 997de2e8107cc4256e50611463d609b18fe9619f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
If that is the case, the loop setting predictor_state in
sonic_decode_frame causes out of bounds reads of int_samples, which has
only frame_size number of elements.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9637c2531f7eb040ad1c3cb46cb40a63dfc77b80)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Due to this typo max_center can be too large, causing nlsf to be set to
too large values, which in turn can cause nlsf[i - 1] + min_delta[i] to
overflow to a negative value, which is not allowed for nlsf and can
cause an out of bounds read in silk_lsf2lpc.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f61d44b74aaae1d306d8a0d38b7b3d4292c89ced)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
A negative codec_id cannot be handled by the found_decoder API of
AVStream->info: if the codec_id is not recognized, found_decoder is set
to -codec_id, which has to be '<0' according to the API documentation.
This can cause NULL pointer dereferencing in try_decode_frame.
Also make sure the codec_type matches the expected one for codec_id.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ecf63b7cc24b9fd3e6d604313325dd1ada4db662)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Also correct the check to reject log < 7, because UPDATE_CACHE only
guarantees 25 meaningful bits.
This fixes undefined behavior:
runtime error: shift exponent is negative
Testing with START/STOP timers in get_ue_golomb, one for the first
branch (A) and one for the second (B), shows that there is practically no
slowdown, e.g. for the cavs decoder:
With the check in the B branch:
629 decicycles in get_ue_golomb B, 4194260 runs, 44 skips
433 decicycles in get_ue_golomb A,268434102 runs, 1354 skips
Without the check:
624 decicycles in get_ue_golomb B, 4194273 runs, 31 skips
433 decicycles in get_ue_golomb A,268434203 runs, 1253 skips
Since the B branch is executed far less often than the A branch, this
change is negligible, even more so for the h264 decoder, where the ratio
B/A is a lot smaller.
Fixes: mozilla bug 1230239
Fixes: fbeb8b2c7c996e9b91c6b1af319d7ebc/asan_heap-oob_195450f_2743_e8856ece4579ea486670be2b236099a0.bit
Found-by: Tyson Smith
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 22e960ad478e568f4094971a58c6ad8f549c0180)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If the input contains too many too large values, the imdct can overflow.
Even if it didn't, the output would be larger than the valid range of 29
bits.
Note that this is a very delicate limit: Allowing values up to 1<<25
does not prevent input larger than 1<<29 from arriving at
sbr_sum_square, while limiting values to 1<<23 breaks the
fate-aac-fixed-al_sbr_hq_cm_48_5.1 test.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit fdc94db37e89165964fdf34f1cd7632e44108bd0)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a9c20e922cee435c9ad2dc78f6c50651f353329c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes a division by zero in the aac_fixed decoder.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ff8816f7172b94028131ee2426ba35e875d973ae)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes out-of-bounds reads in avoid_clipping.
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5b0da6999fdd0135b6f269d3691d74720f773c85)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If minq is negative, the range of sf_idx can be larger than
SCALE_MAX_DIFF allows, causing assertion failures later in
encode_scale_factors.
Reviewed-by: Claudio Freire <klaussfreire@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7a4652dd5da0502ff21c183b5ca7d76b1cfd6c51)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If size is zero, avio_get_str fails, leaving the buffer uninitialized.
This causes invalid reads in av_set_options_string.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a611375db532c3d5363d97b10fadd0211811a4fd)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This also as a sideeffect fixes the non aligned case
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a066ff89bcbae6033c2ffda9271cad84f6c1b807)
Fixes: Ticket4914
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b51e7554e74cbf007a1cab83c7bed3ad9fa2793a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Similar to 33fefdb44.
Fix trac ticket #4921.
Signed-off-by: Nicolas George <george@nsup.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1acc90eaa54ad82a21474ed759b8ed3a0f3d482d)
Conflicts:
libavformat/tee.c
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes Ticket5047
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 861f47ddf463926da2cba9e12665e7f004419f4b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: CID1341577
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 537e901fe66c326f78e916ee9393830ee366131d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The transpose_4x4H is wrong which cost me much time to find this bug. The orders of r2 and r3 are wrong,
this bug waste me much time while I make aarch64 arm instruction which used the function.
(cherry picked from commit c18176bd551b4616757080376707637e30547fd0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes a mpegts file with hevc that fails estimating duration. Increasing number of
retries fixes the issue.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d8c2f1a28073d451c7db31291c333cb15ca3d0b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes CID1238790
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e6971db12b8ae49712b77378fa8141de4904082b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: CID1341571
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc8b1e694cc395fdf5e2917377ef11263c937d85)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
If the chroma components are subsampled, smaller buffers are allocated
for them. In that case the maximal block_offset for the chroma
components is not as large as for the luma component.
This fixes out of bounds writes causing segmentation faults or memory
corruption.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5adb5d9d894aa495e7bf9557b4c78350cbfc9d32)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
VP8E_UPD_ENTROPY, VP8E_UPD_REFERENCE, VP8E_USE_REFERENCE were removed
from libvpx and the remaining values were never used here
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Zern <jzern@google.com>
(cherry picked from commit 6540fe04a3f9a11ba7084a49b3ee5fa2fc5b32ab)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: d41d8cd98f00b204e9800998ecf8427e/signal_sigsegv_321165b_7641_077dfcd8cbc80b1c0b470c8554cd6ffb.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 214085852491448631dcecb008b5d172c11b8892)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 99d142c47e6ba3510a74b872a1a2ae72/asan_heap-oob_11b36f4_3811_0f5c69e7609a88a580135678de1df844.dxa
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a08681f1e614152184615e2bcd71c3d63835f810)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c7d6ec947c053699950af90f695413a5640b3872)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 79798f7c57b098c78e0bbc6becd64b9888b013d1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes CID1338334
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 32bf6550cb9cc9f487a6722fe2bfc272a93c1065)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
ts->mux_rate is int (signed 32-bit) type. The period calculations
will start to overflow when mux_rate > 5mbps. This fixes overflows
by converting first to 64-bit type.
Fixes#5044.
Signed-off-by: Timo Teräs <timo.teras@iki.fi>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 64f7db554ee83846f207e82a08946a6a5a6acfe2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The fps variable is explicitly set to -1 in case of some errors, the check must
thus be signed or the code setting it needs to use 0 as error code
the type of the field could be changed as well but its in an installed header
Fixes: integer overflow
Fixes: 9982cc157b1ea90429435640a989122f/asan_generic_3ad004a_3799_22cf198d9cd09928e2d9ad250474fa58.mov
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b46dcd5209a77254345ae098b83a872634c5591b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes integer overflow
Fixes: mozilla bug 1229167
Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit f03c2ceec174877e03bb302f5971fbe9ffbe4856)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 0a7ff0c1d93da9cef28a315ec91b692a/asan_heap-oob_4a52e5_3604_9c56dbb20e308f4faeef7b35f688521a.ape
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cd7524fdd13dc8d0cf22e2cfd8300a245542b13a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read
Fixes: 76c515fc3779d1b838667c61ea13ce92/asan_heap-oob_1fc0d07_8913_794a4629a264ebdb25b58d3a94ed1785.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 891dc8f87536ac2ec695c70d081345224524ad99)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The code expects actual positive numbers and gives completely wrong
results if INT64_MIN is treated as positive
Instead clip it into the valid range that is add 1 and treat it as
negative
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 25e37f5ea92d4201976a59ae306ce848d257a7e6)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3a9cb18855d29c96a5d9d2f5ad30448cae3a2ddf)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes part of mozilla bug 1229167
Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8e7f4520226d2d9ad6a58ad6c32d1455a8b244b2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 59bb925e90201fa0f87f0a31945d43b5/asan_heap-oob_4a52e5_3388_66027f11e3d072f1e02401ecc6193361.jvt
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit aa427537b529cd584cd73222980286d36a00fe28)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>