level can be 5, but there are only four codebooks.
Fixes ubsan runtime error: index 5 out of bounds for type 'int8_t
[4][96]'
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If links don't work, fall back to using the full source path as was
previously done.
This should fix build failures with MSVC.
Reviewed-by: Hendrik Leppkes <h.leppkes@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Previously the full source path was embedded inconsistently in the debug
information between in-tree/out-of-tree builds.
The 'vpath %.inc' becomes necessary for finding
libavfilter/all_channel_layouts.inc in out-of-tree builds.
The full source path is still embedded in the debug information, but
it's now independent of whether building in-tree or out-of-tree.
The biggest improvement of this patch is that gdb now always searches
for the path relative to the source directory. It still also searches
for the full path.
Previously it searched only for the full path in out-of-tree builds,
making the debug information generated by Debian's buildds rather hard
to use.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
s->ac_val[0][0] is of type 'int16_t [16]', but points into a larger buffer.
Here it is used as base pointer to find the correct position in the
larger buffer by adding 's->block_index[n] * 16' and thus as 'int16_t *'.
This fixes clang's ubsan runtime error: index out of bounds for type
'int16_t [16]'
Fixes: test_case.mp4
Found-by: Tyson Smith <twsmith@mozilla.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The function is used on unaligned buffers (such as those provided
by AVPacket), accessing them as uint16_t causes SIGBUS crashes on
architectures like SPARC.
This fixes ubsan runtime error: load of misaligned address for type
'const uint16_t', which requires 2 byte alignment
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
asf_read_payload can unset eof_reached, so check it also before calling
that function.
This fixes infinite loops.
Reviewed-by: Alexandra Hájková <alexandra.khirnova@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Both avio_skip and detect_unknown_subobject use int64_t for the size
parameter.
This fixes a segmentation fault due to infinite recursion.
Reviewed-by: Alexandra Hájková <alexandra.khirnova@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes ubsan runtime error: signed integer overflow: 8388608 +
2140274688 cannot be represented in type 'int'
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
src and dst are only 8-bit-aligned, so accessing them as uint16_t causes
SIGBUS crashes on architectures like sparc.
This fixes ubsan runtime error: load of misaligned address for type
'const uint16_t', which requires 2 byte alignment
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes runtime error: null pointer passed as argument 2, which is
declared to never be null
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
This is used to check if the input buffer is larger enough, so if this
overflows it can cause a false negative leading to a segmentation fault
in bytestream2_get_bufferu.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
More don't fit into the integer output.
Also use get_bits_long, since get_bits only supports reading up to 25
bits, while get_bits_long supports the full integer range.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
The transformation to bytes must happen after alignment to get the same
resulting pointers as before.
This fixes segmentation faults in the assembler code.
The regression was introduced in commit 9553689.
Reviewed-by: Kieran Kunhya <kierank@obe.tv>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
It is used to store the difference between pointers, so ptrdiff_t is the
correct type.
This prevents potential overflows.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes segmentation faults due to out of bounds writes, when
color_start is interpreted as negative number.
This regression was introduced in commit 57631f.
Reviewed-by: Mats Peterson <matsp888@yahoo.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The previous check only caught sizes from -AV_INPUT_BUFFER_PADDING_SIZE
to -1.
This fixes ubsan runtime error: signed integer overflow: 2147483647 + 32
cannot be represented in type 'int'
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Both avio_skip and detect_unknown_subobject use int64_t for the size
parameter.
This fixes a segmentation fault due to infinite recursion.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Alexandra Hájková <alexandra.khirnova@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
The previous check only caught sizes from -AV_INPUT_BUFFER_PADDING_SIZE
to -1.
This fixes ubsan runtime error: signed integer overflow: 2147483647 + 32
cannot be represented in type 'int'
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes NULL pointer dereferencing if the codec is forced to
adpcm_thp even though a different one was detected.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This causes non-unique elements in floor_setup->data.t1.list, which
makes the stream undecodable according to the specification.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
It causes the angle channel number to equal the magnitude channel
number, which makes the stream undecodable according to the
specification.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes ubsan runtime error: left shift by 8 places cannot be
represented in type 'int'
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Also use a unsigned constant for the shift calculation, as 1 << 31 is
undefined for int32_t. This is also fixed oggparsetheora.
This fixes ubsan runtime error: shift exponent is too large for
32-bit type 'int'
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This can happen if s->ls changes from 0 to 1, but picture allocation is
skipped due to s->interlaced.
In that case ff_jpegls_decode_picture could be called even though the
s->picture_ptr frame has the wrong pixel format and thus a wrong
linesize, which results in a too small zero buffer being allocated.
This fixes an out-of-bounds read in ls_decode_line.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
A negative bits_per_coded_sample doesn't make sense.
If it is too large, the size calculation for av_get_packet overflows,
resulting in allocation of a too small buffer.
Also make sure width and height are sane.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Otherwise the too small buffer is directly used in the frame, causing
segmentation faults, when trying to use the frame.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If it is negative, it can cause the byte position to move backwards in
avio_skip, which in turn makes sm_size negative and thus size larger
than the size of the packet buffer, causing invalid writes in avio_read.
Also fix potential overflow of avio_tell(bc) + value_len.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This is used to check if the input buffer is large enough, so if this
overflows it can cause a false negative leading to a segmentation fault
in bytestream2_get_bufferu.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Fixes ubsan runtime error: null pointer passed as argument 2, which is
declared to never be null
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This macro unconditionally used out[-1], which causes an out of bounds
read, if out is the very beginning of the buffer.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
More don't fit into the integer output.
Also use get_bits_long, since get_bits only supports reading up to 25
bits, while get_bits_long supports the full integer range.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If that is the case, the loop setting predictor_state in
sonic_decode_frame causes out of bounds reads of int_samples, which has
only frame_size number of elements.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This macro unconditionally used out[-1], which causes an out of bounds
read, if out is the very beginning of the buffer.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Due to this typo max_center can be too large, causing nlsf to be set to
too large values, which in turn can cause nlsf[i - 1] + min_delta[i] to
overflow to a negative value, which is not allowed for nlsf and can
cause an out of bounds read in silk_lsf2lpc.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Due to this typo max_center can be too large, causing nlsf to be set to
too large values, which in turn can cause nlsf[i - 1] + min_delta[i] to
overflow to a negative value, which is not allowed for nlsf and can
cause an out of bounds read in silk_lsf2lpc.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
A negative codec_id cannot be handled by the found_decoder API of
AVStream->info: if the codec_id is not recognized, found_decoder is set
to -codec_id, which has to be '<0' according to the API documentation.
This can cause NULL pointer dereferencing in try_decode_frame.
Also make sure the codec_type matches the expected one for codec_id.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Also correct the check to reject log < 7, because UPDATE_CACHE only
guarantees 25 meaningful bits.
This fixes undefined behavior:
runtime error: shift exponent is negative
Testing with START/STOP timers in get_ue_golomb, one for the first
branch (A) and one for the second (B), shows that there is practically no
slowdown, e.g. for the cavs decoder:
With the check in the B branch:
629 decicycles in get_ue_golomb B, 4194260 runs, 44 skips
433 decicycles in get_ue_golomb A,268434102 runs, 1354 skips
Without the check:
624 decicycles in get_ue_golomb B, 4194273 runs, 31 skips
433 decicycles in get_ue_golomb A,268434203 runs, 1253 skips
Since the B branch is executed far less often than the A branch, this
change is negligible, even more so for the h264 decoder, where the ratio
B/A is a lot smaller.
Fixes: mozilla bug 1230239
Fixes: fbeb8b2c7c996e9b91c6b1af319d7ebc/asan_heap-oob_195450f_2743_e8856ece4579ea486670be2b236099a0.bit
Found-by: Tyson Smith
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This should fix this test failing on kfreebsd, a regression since
6e5dbe7, which decreased the CMP_TARGET by 1.
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If the input contains too many too large values, the imdct can overflow.
Even if it didn't, the output would be larger than the valid range of 29
bits.
Note that this is a very delicate limit: Allowing values up to 1<<25
does not prevent input larger than 1<<29 from arriving at
sbr_sum_square, while limiting values to 1<<23 breaks the
fate-aac-fixed-al_sbr_hq_cm_48_5.1 test.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If the chroma components are subsampled, smaller buffers are allocated
for them. In that case the maximal block_offset for the chroma
components is not as large as for the luma component.
This fixes out of bounds writes causing segmentation faults or memory
corruption.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If minq is negative, the range of sf_idx can be larger than
SCALE_MAX_DIFF allows, causing assertion failures later in
encode_scale_factors.
Reviewed-by: Claudio Freire <klaussfreire@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If size is zero, avio_get_str fails, leaving the buffer uninitialized.
This causes invalid reads in av_set_options_string.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Otherwise the ffm demuxer can return the same packets endlessly, if
there is no server attached.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Doing that doesn't make sense, because the only purpose of sbr_dequant
is to process the data from read_sbr_data.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
It is used as size argument of ff_canopus_parse_info_tag, which uses it
as size argument to bytestream2_init, which only supports sizes up to
INT_MAX.
Changing it's type to unsigned simplifies the check.
Reviewed-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Previously only the edit_rate of material_track was checked.
If it's negative, it causes assertion failures in av_rescale_rnd.
Reviewed-by: Tim Nicholson <nichot20-at-yahoo.com@ffmpeg.org>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This prevents various values from getting an insanely huge exponent.
If someone knows a cleaner solution, thats welcome!
This is similar to commit 8978c74 for aacsbr.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Otherwise v=INT_MIN doesn't get normalized and thus triggers av_assert2
in other functions.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The correct result can't be expressed in SoftFloat.
Currently it returns a random value from an out of bounds read.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The buffer needs s->bpp bytes, at maximum currently 10.
Assert that s->bpp is not larger.
This fixes a stack buffer overflow.
Reviewed-by: wm4 <nfxjfg@googlemail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The return type of strlen is size_t, i.e. unsigned, so if pd->buf_size
is 3, the right side overflows leading to a wrong result of the
comparison and subsequently a heap buffer overflow.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Also replace the last two usages of avpriv_float_dsp_init with
avpriv_float_dsp_alloc.
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Andreas Cadhalpun
