flcoeffs2[pos] should be the log2 of flcoeffs1[pos].
flcoeffs1[0] can be 0 here, thus flcoeffs2[pos] gets set to -inf,
causing problems further down.
This seems to have been copied from imc_decode_level_coefficients in
commit 4eb4bb3 without updating the position.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
flcoeffs2[pos] should be the log2 of flcoeffs1[pos].
flcoeffs1[0] can be 0 here, thus flcoeffs2[pos] gets set to -inf,
causing problems further down.
This seems to have been copied from imc_decode_level_coefficients in
commit 4eb4bb3 without updating the position.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
It asserts that the frame linesize is larger than 37, but it can be
smaller and decoding such frames works.
Before commit cc884a35 src_stride > 7*MB_SIZE was necessary, because the
blocks were interleaved in the tmp buffer and the last block was added
with an offset of 6*MB_SIZE.
It was changed for src_stride <= 7*MB_SIZE to write the blocks
sequentially, hence the larger tmp_step.
After that the assert was only necessary to make sure that the buffer
remained large enough.
Since commit bd2b6b33 s->scratchbuf is used as tmp buffer.
As part of commit 86e107a7 the minimal scratchbuf size was increased to
256*7*MB_SIZE, which is enough for any src_stride <= 7*MB_SIZE.
Also add a comment explaining the tmp_step calculation.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Otherwise a frame with non-zero nb_samples but without any data can be
returned.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The loop always needs at least 8 bytes for chunk_type and chunk_size.
If fewer are left, bytestream2_get_le32 just returns 0 without
reading any bytes, leading to an infinite loop.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Every chunk needs at least 8 bytes for chunk_type and chunk_size.
Prevent a possible infinite loop.
CC: libav-stable@libav.org
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
More than 32 bits can't be stored in an integer and get_bits should not
be used with more than 25 bits.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If threading is used, the first (thread_count - 1) packets are read
before any frame/error is returned. Counting this as successful decoding
is wrong, because it also happens when no single frame could be decoded.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Otherwise sm_size can be larger than size, which results in a negative
packet size.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If nb_samples is zero, the bit_rate calculation results in a division by
zero.
Since ff_get_buffer fails if frame->nb_samples is zero, this can be
fixed by moving the bit_rate calculation after that function call.
That also makes it possible to reuse the already calculated
frame->nb_samples value.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes a segfault when decoding multi-channel MP3onMP4 files.
This is similar to commit cb72230d for MPADSPContext.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
QP_store is only 8-bit-aligned, so accessing it as uint32_t causes
SIGBUS crashes on sparc.
The AV_RN32/AV_WN32 macros only do unaligned access in the
HAVE_FAST_UNALIGNED case.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The current check is too strict for newer makeinfo versions.
Existing version strings are:
makeinfo (GNU texinfo) 4.13
makeinfo (GNU texinfo) 5.2
texi2any (GNU texinfo) 5.9.93
Probably version 6 will come in the not too far future.
Reviewed-by: Timothy Gu <timothygu99@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
In the TTA extradata re-construction the values are written with
avio_wl16 and if they don't fit into uint16_t, this triggers an
av_assert2 in avio_w8.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
And default to 8000 if it is invalid.
An invalid sample rate can trigger av_assert2 in av_rescale_rnd.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This is more robust.
And only check if there is actually a frame returned.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Also use the frame pixel format instead of the one from the codec
context, which is more robust.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Also use the frame pixel format instead of the one from the codec
context, which is more robust.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Neccessary -> Necessary
formated -> formatted
thee -> the
eventhough -> even though
seperately -> separately
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This reverts commit 04f0002, which made it impossible to enable VSX with
a generic cpu.
This changes the behavior back to what it was before commit b0af404.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Inconsistencies between the dimensions/pixel format of avctx and the
frame can confuse API users.
For example this can crash the demuxing_decoding example.
Back up the previous values and restore them, when decoding the next
frame. This is necessary, because these can be different between the
returned frame and the last decoded frame.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If chan2 is not smaller than the number of channels, it can cause
segmentation faults due to dereferencing a NULL pointer.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If one of the dimensions is larger than 8176, s->mb_width or
s->mb_height is larger than 511, leading to an int16_t overflow of
s->mv_max.{x,y}. This then causes av_clip to be called with amin > amax.
Changing the type to int avoids the overflow and has no negative
effect, because s->mv_max is only used in clamp_mv for clipping.
Since mv_max.{x,y} is positive and mv_min.{x,y} negative, av_clip can't
increase the absolute value. The input to av_clip is an int16_t, and
thus the output fits into int16_t as well.
For additional safety, s->mv_{min,max}.{x,y} are clipped to int16_t range
before use.
Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Otherwise the check 'tile_size < size' treats a negative size as
unsigned, causing the check to pass. This subsequently leads to
segmentation faults.
This was originally fixed as part of Libav commit 72ca83, so the
original author is one of the following developers:
Anton Khirnov <anton@khirnov.net>
Diego Biurrun <diego@biurrun.de>
Luca Barbato <lu_zero@gentoo.org>
Martin Storsjö <martin@martin.st>
Reviewed-by: Ronald S. Bultje <rsbultje@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If opj_image_create fails to allocate an image it returns NULL, which
causes a segmentation fault at 'img->x0 = 0'.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If avio_read fails, the buffer can contain uninitialized data.
This fixes 'Conditional jump or move depends on uninitialised value(s)'
valgrind warnings, and addresses a few memleaks.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
libopenjpeg can return images with components without data.
This fixes segmentation faults.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
A negative sample duration is invalid according to the spec, but there
are samples that use it for the DTS calculation, e.g.:
http://files.1f0.de/samples/mp4-negative-stts-problem.mp4
These currently get out of A/V sync.
Also change the logging type to AV_LOG_WARNING, because decoding the
sample can continue.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
In this case the mov demuxer can return a large number of empty packets.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
shine_encode_buffer expects written to be an int pointer, while the
previous shine_encode_frame expected it to be a long pointer.
Thus encoding with libshine currently always fails with
"internal buffer too small", because a negative return value of
shine_encode_buffer is interpreted as a very large long value.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If avio_read fails, the buffer can contain uninitialized data.
This fixes 'Conditional jump or move depends on uninitialised value(s)'
valgrind warnings.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If it fails, the buffers can be (partially) uninitialized.
This fixes 'Conditional jump or move depends on uninitialised value(s)'
valgrind warnings.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
nut->last_syncpoint_pos doesn't necessarily change between resync
attempts, so find_any_startcode can return the same startcode again.
Thus remember where the last resync happened and don't try to resync
before that.
This can't be done locally in nut_read_packet, because this wouldn't
prevent infinite resync loops, where after the resync a packet is
returned and while reading a following packet the resync happens again.
Reviewed-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
av_image_fill_pointers always aligns the palette, but the padding
bytes don't (and can't) get initialized in av_image_copy.
Thus initialize them in av_image_alloc.
This fixes 'Syscall param write(buf) points to uninitialised byte(s)'
valgrind warnings.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Since commit 676a395a aac->frame->data is not necessarily allocated at
the end of aac_decode_frame_int if avctx->channels is 0.
In this case a bogus frame without any data, but non-zero nb_samples is
returned.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Since len is an unsigned int, the comparison is currently treated as
unsigned and thus ignores all errors from avio_read.
Thus cast len to int, which is unproblematic, because at that point len
is between 0 and 4.
This fixes 'Conditional jump or move depends on uninitialised value'
valgrind warnings in is_tag.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If in compression mode rice_limit = 0 leads to call
`show_bits(gb, k)` in `decode_scalar` with k = 0.
Request a sample in case it is valid and it should be accepted.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
CC: libav-stable@libav.org
In init_planes p->xblen and p->yblen are set to:
p->xblen = s->plane[0].xblen >> s->chroma_x_shift;
p->yblen = s->plane[0].yblen >> s->chroma_y_shift;
These are later used as block_w and block_h arguments of
s->vdsp.emulated_edge_mc. If one of them is 0 it triggers an av_assert2
in emulated_edge_mc:
av_assert2(start_x < end_x && block_w > 0);
av_assert2(start_y < end_y && block_h > 0);
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
s->ref_pics[i] is later used as ref argument of interpolate_refplane,
where it is dereferenced.
If it is NULL, it causes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If bytes is large enough, bytes*8 can overflow and become negative.
In that case 'bufsize -= bytes*8' causes bufsize to increase instead of
decrease.
This leads to a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
buf_idx + data_unit_size can overflow, causing the '> buf_size' check to
wrongly fail.
This causes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The minimum of the ath(x, ATH_ADD) function depends on ATH_ADD.
This patch uses the first order approximation to determine it.
For ATH_ADD = 4 this results in the value at 3407.06812 (-5.24241638)
not the one at 3410 (-5.24237967).
CC: libav-stabl@libav.org
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
If max in clean_index is set to a negative ast->sample_size, the
following loop never ends:
while (max < 1024)
max += max;
Thus set ast->sample_size to 0 if it would otherwise be negative.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
If bit_rate is negative, it can trigger an av_assert2 in av_rescale_rnd.
Since av_rescale returns int64_t, but st->codec_bit_rate is int, it can
also overflow into a negative value.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
index_scale is set to matroska->time_scale of type uint64_t.
When index_scale is int, the assignment can overflow and e.g. result
in index_scale = 0. This causes a floating point exception due to the
division by index_scale.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
AV_PIX_FMT_MONOBLACK has the AV_PIX_FMT_FLAG_BITSTREAM flag, i.e.
linesize can be smaller than width.
Since x_offset is only check against the width, this can lead to
x_offset * bpp >= image_linesize.
In this case ptr could be set to a position outside the image_buf in
png_handle_row, leading to memory corruption and thus crashes.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
During the loop ret can get changed. Since it is not set on all failure
paths, decode_frame_common can return 0 even though an error occurred.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This check was removed in commit 08aec6f6, but
s->last_picture.f->data[0] is still used in handle_p_frame_apng
unconditionally.
This fixes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The existing check has two problems:
1) i + count can overflow, so that the check '< 256' returns true.
2) In the (i == 'N') case occurs a j-- so that the loop runs once more.
This can trigger the assertion 'nut->header_len[0] == 0' or cause
segmentation faults or infinite hangs.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
s->decoded_buffer is allocated with a min_size of:
2 * FFALIGN(blockstodecode, 8) * sizeof(*s->decoded_buffer)
Then it is assigned to s->decoded[0] (and s->decoded_buffer + FFALIGN(blockstodecode, 8)
to s->decoded[1]) and passed as out buffer to decode_array_0000.
In this function 64 elements of the out buffer are written
unconditionally and outside the array if blockstodecode is too small.
This causes memory corruption, leading to segmentation faults or other
crashes.
Thus change decode_array_0000 to write at most blockstodecode elements
of the out buffer.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Otherwise range_start_decoding is not necessarily run and thus
ctx->rc.range still 0 in range_dec_normalize leading to an infinite
loop.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
