Fixes index out of bounds error
Fixes: aac_index_out_of_bounds.wmv
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 590863876d1478547640304a31c15809c3618090)
Conflicts:
libavcodec/aacdec_template.c
Fixes: dvdec_left_shift.avi
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 93ac72a98dff592ffc174cfb36a8975dfbf145ae)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: mjpeg_left_shift.avi
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d86d7b2486cd5c31db8e820d8a89554abf19567e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: mss2_left_shift.wmv
Found-by: Piotr Bandurski <ami_stuff@o2.pl>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e273dade78943e22b71d0ddb67cd0d737fc26edf)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: test_case-mdc.264 (b47be15a120979f5a1a945c938cbef33)
Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 13f266b50cc7554028d22480b7e4383968e64a63)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Prevents that following scalers in the filter chain will do unintentional color range conversions.
Fixes Ticket #5096
Signed-off-by: Thomas Mundt <loudmax@yahoo.de>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 73ce8162f3499cf0e86d1d80dea53324bd62bcb3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Found-by: Tyson Smith <twsmith@mozilla.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5b4da8a38a5ed211df9504c85ce401c30af86b97)
Conflicts:
libavcodec/motion_est.h
Fixes mozilla bug 1230423
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7cc01c25727a96eaaa0c177234b626e47c8ea491)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes Ticket4960
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1edf129cbc897447a289ca8b045853df5df1bab3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This causes a overall slowdown of 0.1 % (tested with mpeg4 single thread encoding of matrixbench at QP=3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit cccb0ffccc3723acc7aab3a859b24743596dd9c0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This can happen if s->ls changes from 0 to 1, but picture allocation is
skipped due to s->interlaced.
In that case ff_jpegls_decode_picture could be called even though the
s->picture_ptr frame has the wrong pixel format and thus a wrong
linesize, which results in a too small zero buffer being allocated.
This fixes an out-of-bounds read in ls_decode_line.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7ea2db6eafa0a8a9497aab20be2cfc8742a59072)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes Ticket4926
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b83d8be6bff7d645469a623aee0b380541da15cf)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f6830cf5ba03fdcfcd81a0358eb32d4081a2fcce)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Otherwise the too small buffer is directly used in the frame, causing
segmentation faults, when trying to use the frame.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 699e68371ec7e381e5cc48e3d96e29c669261af7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes NULL pointer dereferencing.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9fcfe4a3cdf9a5af0c37758b178965b7b99582d4)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If it is negative, it can cause the byte position to move backwards in
avio_skip, which in turn makes sm_size negative and thus size larger
than the size of the packet buffer, causing invalid writes in avio_read.
Also fix potential overflow of avio_tell(bc) + value_len.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ce10f572c12b0d172c72d31d8c979afce602bf0c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This is used to check if the input buffer is large enough, so if this
overflows it can cause a false negative leading to a segmentation fault
in bytestream2_get_bufferu.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9d38f06d05efbb9d6196c27668eb943e934943ae)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes ubsan runtime error: null pointer passed as argument 2, which is
declared to never be null
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9f82506c79874edd7b09707ab63d9e72078de8f9)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This macro unconditionally used out[-1], which causes an out of bounds
read, if out is the very beginning of the buffer.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 90b99a81071d10e6b5efe86a4602d54d4f45bbcb)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
More don't fit into the integer output.
Also use get_bits_long, since get_bits only supports reading up to 25
bits, while get_bits_long supports the full integer range.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 4d5c3b02e9d2c9a630ca433fabca43285879e0b8)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes out of array read
Fixes mozilla bug 1233606
Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b92b4775a0d07cacfdd2b4be6511f3cb362c977b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes ubsan warning
Fixes Mozilla bug 1230276
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c8ea57664fe3ad611c9ecd234670544ddff7ca55)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This also suppresses a ubsan warning
Fixes Mozilla bug 1230247
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ef8f6464a55db730cab8c48a1a51fa4e6ca12107)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The change was not correct and broke H264
This reverts commit cd83f899c94f691b045697d12efa21f83eb2329f.
(cherry picked from commit 95b59bfb9d9e47de8438183a035e02667946f27c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 997de2e8107cc4256e50611463d609b18fe9619f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
If that is the case, the loop setting predictor_state in
sonic_decode_frame causes out of bounds reads of int_samples, which has
only frame_size number of elements.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 9637c2531f7eb040ad1c3cb46cb40a63dfc77b80)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Due to this typo max_center can be too large, causing nlsf to be set to
too large values, which in turn can cause nlsf[i - 1] + min_delta[i] to
overflow to a negative value, which is not allowed for nlsf and can
cause an out of bounds read in silk_lsf2lpc.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit f61d44b74aaae1d306d8a0d38b7b3d4292c89ced)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
A negative codec_id cannot be handled by the found_decoder API of
AVStream->info: if the codec_id is not recognized, found_decoder is set
to -codec_id, which has to be '<0' according to the API documentation.
This can cause NULL pointer dereferencing in try_decode_frame.
Also make sure the codec_type matches the expected one for codec_id.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit ecf63b7cc24b9fd3e6d604313325dd1ada4db662)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Also correct the check to reject log < 7, because UPDATE_CACHE only
guarantees 25 meaningful bits.
This fixes undefined behavior:
runtime error: shift exponent is negative
Testing with START/STOP timers in get_ue_golomb, one for the first
branch (A) and one for the second (B), shows that there is practically no
slowdown, e.g. for the cavs decoder:
With the check in the B branch:
629 decicycles in get_ue_golomb B, 4194260 runs, 44 skips
433 decicycles in get_ue_golomb A,268434102 runs, 1354 skips
Without the check:
624 decicycles in get_ue_golomb B, 4194273 runs, 31 skips
433 decicycles in get_ue_golomb A,268434203 runs, 1253 skips
Since the B branch is executed far less often than the A branch, this
change is negligible, even more so for the h264 decoder, where the ratio
B/A is a lot smaller.
Fixes: mozilla bug 1230239
Fixes: fbeb8b2c7c996e9b91c6b1af319d7ebc/asan_heap-oob_195450f_2743_e8856ece4579ea486670be2b236099a0.bit
Found-by: Tyson Smith
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 22e960ad478e568f4094971a58c6ad8f549c0180)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If minq is negative, the range of sf_idx can be larger than
SCALE_MAX_DIFF allows, causing assertion failures later in
encode_scale_factors.
Reviewed-by: Claudio Freire <klaussfreire@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 7a4652dd5da0502ff21c183b5ca7d76b1cfd6c51)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If size is zero, avio_get_str fails, leaving the buffer uninitialized.
This causes invalid reads in av_set_options_string.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit a611375db532c3d5363d97b10fadd0211811a4fd)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This also as a sideeffect fixes the non aligned case
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a066ff89bcbae6033c2ffda9271cad84f6c1b807)
Fixes: Ticket4914
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b51e7554e74cbf007a1cab83c7bed3ad9fa2793a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The transpose_4x4H is wrong which cost me much time to find this bug. The orders of r2 and r3 are wrong,
this bug waste me much time while I make aarch64 arm instruction which used the function.
(cherry picked from commit c18176bd551b4616757080376707637e30547fd0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes a mpegts file with hevc that fails estimating duration. Increasing number of
retries fixes the issue.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 2d8c2f1a28073d451c7db31291c333cb15ca3d0b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes CID1238790
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e6971db12b8ae49712b77378fa8141de4904082b)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: CID1341571
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit bc8b1e694cc395fdf5e2917377ef11263c937d85)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
If the chroma components are subsampled, smaller buffers are allocated
for them. In that case the maximal block_offset for the chroma
components is not as large as for the luma component.
This fixes out of bounds writes causing segmentation faults or memory
corruption.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
(cherry picked from commit 5adb5d9d894aa495e7bf9557b4c78350cbfc9d32)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes out of array read
Fixes: 2f95ddd996db8a6281d2e18c184595a7/asan_heap-oob_192fe91_3330_58e4441181e30a66c19f743dcb392347.bit
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit dad354f38ddc9bfc834bc21358a1d0ad41532ca0)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
