Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 584cc1ade10a3297ef9c107ef3a2081c04024156)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
In init_planes p->xblen and p->yblen are set to:
p->xblen = s->plane[0].xblen >> s->chroma_x_shift;
p->yblen = s->plane[0].yblen >> s->chroma_y_shift;
These are later used as block_w and block_h arguments of
s->vdsp.emulated_edge_mc. If one of them is 0 it triggers an av_assert2
in emulated_edge_mc:
av_assert2(start_x < end_x && block_w > 0);
av_assert2(start_y < end_y && block_h > 0);
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 75fc81c8318505aa7946e05a9bee08d47241fc66)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
s->ref_pics[i] is later used as ref argument of interpolate_refplane,
where it is dereferenced.
If it is NULL, it causes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit d93181ef3eacdb862d93448f31c97765a523d1db)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If bytes is large enough, bytes*8 can overflow and become negative.
In that case 'bufsize -= bytes*8' causes bufsize to increase instead of
decrease.
This leads to a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 9e66b39aa87eb653a6e5d15f70b792ccbf719de7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
buf_idx + data_unit_size can overflow, causing the '> buf_size' check to
wrongly fail.
This causes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 984f50deb2d48f6844d65e10991b996a6d29e87c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If bit_rate is negative, it can trigger an av_assert2 in av_rescale_rnd.
Since av_rescale returns int64_t, but st->codec_bit_rate is int, it can
also overflow into a negative value.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0eec40b713eee84e2aec8af35ccce059817cad2a)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The later is not correct
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5d309d309108684f742bbf5fc2393f1c519cda72)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e54540655f229d06667dc7fa7005f2a20e101e80)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
index_scale is set to matroska->time_scale of type uint64_t.
When index_scale is int, the assignment can overflow and e.g. result
in index_scale = 0. This causes a floating point exception due to the
division by index_scale.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit eb9fb508b0e09d85d234fe694333b2005e1d7a7e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
AV_PIX_FMT_MONOBLACK has the AV_PIX_FMT_FLAG_BITSTREAM flag, i.e.
linesize can be smaller than width.
Since x_offset is only check against the width, this can lead to
x_offset * bpp >= image_linesize.
In this case ptr could be set to a position outside the image_buf in
png_handle_row, leading to memory corruption and thus crashes.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 372aa0777aaacf726de7cd7dd0e6797026a124ee)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
During the loop ret can get changed. Since it is not set on all failure
paths, decode_frame_common can return 0 even though an error occurred.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 8f760be4d312bb6e78f80d39b9d0062253332e08)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This check was removed in commit 08aec6f6, but
s->last_picture.f->data[0] is still used in handle_p_frame_apng
unconditionally.
This fixes a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 287dbb0771d558b336e377d0594e26c0a6291755)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The existing check has two problems:
1) i + count can overflow, so that the check '< 256' returns true.
2) In the (i == 'N') case occurs a j-- so that the loop runs once more.
This can trigger the assertion 'nut->header_len[0] == 0' or cause
segmentation faults or infinite hangs.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 7c24ca1bda2d4df1dc9b2b982941be532d60da21)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 361702660d2c37a63b7d6381d39e1e1de8405260)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes a segmentation fault when accessing the metadata.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 3ff1af2b0db7132d5717be6395227a94c8abab07)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
A negative frame rate triggers an av_assert2 in av_rescale_rnd.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 6621105877ce0d65724a8ab60b3a50160adbe65d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
s->decoded_buffer is allocated with a min_size of:
2 * FFALIGN(blockstodecode, 8) * sizeof(*s->decoded_buffer)
Then it is assigned to s->decoded[0] (and s->decoded_buffer + FFALIGN(blockstodecode, 8)
to s->decoded[1]) and passed as out buffer to decode_array_0000.
In this function 64 elements of the out buffer are written
unconditionally and outside the array if blockstodecode is too small.
This causes memory corruption, leading to segmentation faults or other
crashes.
Thus change decode_array_0000 to write at most blockstodecode elements
of the out buffer.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 699341d647f7af785fb8ceed67604467b0b9ab12)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Otherwise range_start_decoding is not necessarily run and thus
ctx->rc.range still 0 in range_dec_normalize leading to an infinite
loop.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 464c49155ce7ffc88ed39eb2511e7a75565c24be)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
GCC tool had a bug of PPC intrinsic interpret, which has been fixed in GCC 4.9.1. This bug lead to
errors in two of our previous patches. We found this when we update our GCC tools to 4.9.1 and by
reading the related info on GCC website. We fix our previous error in two separate commits
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 603c839398f89d3ef15c47530470fbb0051632c0)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If rice_limit is 0, k can be 0 in decode_scalar, which calls show_bits(gb, k).
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4b657a1b1eedcf38bcf36e89a2f4be6f76b5ce09)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
For max_order = 0 the clipping range is invalid. (amin = 2, amax = 1)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 58d605ee9b3277289278dc40e022311f8e083833)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes a NULL pointer dereference if vst->duration is 0.
The problem was introduced in commit 0588acaf.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 86d00ede4f9acb02690a0615490173648e1d933c)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b3408ae4c64cb674b1d5f0f30171759113ce722a)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Also change the type of begin, end and smp to ptrdiff_t to make the
comparison well-defined.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Thilo Borgmann <thilo.borgmann@mail.de>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit afc7748d1f6abc4b3b1cc957b0fa6941837db3d0)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The minimum of the ath(x, ATH_ADD) function depends on ATH_ADD.
This patch uses the first order approximation to determine it.
For ATH_ADD = 4 this results in the value at 3407.06812 (-5.24241638)
not the one at 3410 (-5.24237967).
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Approved-by: Claudio Freire <klaussfreire@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ca9849eecdf7db91d652c698018a5b096d8b78c7)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If begin is smaller than t, the subtraction 'begin -= t' wraps around,
because begin is unsigned. The same applies for end < t.
This causes segmentation faults.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit faf9fe2c224ea81a98afd53e2f0be0a2e13aeca9)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
avctx->bits_per_raw_sample is used in get_sbits_long, which only
supports up to 32 bits.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 4c2b88678b436f59132386d9be2fc143e3ee480d)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If the same idx is used for more than one i, at least one entry in
sconf->chan_pos remains uninitialized.
This can cause segmentation faults.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ef16501aebed43e34a3721336e8bee732eca2877)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes an invalid read if end is 0:
band_end = ff_ac3_bin_to_band_tab[end-1] + 1;
Depending on what is before the array, this can cause stack smashing,
when band_end becomes too large.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit bc4fee7f2a51635fa3c0f61d1e5164da1efeded3)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
If band->thr is 0.0f, the division is undefined, making norm_fac not a
number or infinity, which causes psy_band->threshold to become NaN.
This is passed on to other variables until it finally reaches
sce->sf_idx and is converted to an integer (-2147483648).
This causes a segmentation fault when it is used as array index.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: Claudio Freire <klaussfreire@gmail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e224aa41917454e7b5c23d9f2541425743ce595a)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 0be54ad280cf114c02306b7063147e8379f8ed1e)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
ac may be NULL and then accessing ac->avctx results in a segmentation fault.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 5b75689b987e4c4dd4f34d5c8be389547e9cc701)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes segmentation faults, when pic->linesize[0] is negative.
In that case 'line * pic->linesize[0] + pixel_ptr' is treated as
unsigned and wraps around.
This reverts commit 7d78a964.
The problem was introduced in commit f7e1367f, which should obsolete
that commit.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit ae6fd7300b4e9f81d3b5ba201096ffe7cccf26fb)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Generally, libavformat exports cover art pictures as video streams with
1 packet and AV_DISPOSITION_ATTACHED_PIC set. Only matroskadec exported
it as attachment with codec_id set to AV_CODEC_ID_MJPEG.
Obviously, this should be consistent, so change the Matroska demuxer to
export a AV_DISPOSITION_ATTACHED_PIC pseudo video stream.
Matroska muxing is probably incorrect too. I know that it can create
broken files with an audio track and just 1 video frame when e.g.
remuxing mp3 with APIC to mkv. But for now this commit does not change
anything about muxing, and also continues to write attachments with
AV_CODEC_ID_MJPEG should the muxer application have special knowledge
that the Matroska is broken in this way.
Fixes trac #4423.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 511585ce7f7272e5069ef011d6be5f073d268901)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes a crash, when trying to mux h264 into mxf_opatom.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Previous version reviewed-by: tomas.hardin@codemill.se
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit b61cb61ab8f9abca98cc8c4d67cbefdb30f1e82a)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
Reviewed-by: tomas.hardin@codemill.se
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 39ddda12f17c666bb08abb2493d4adf8835b22c9)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
By making sure we at each time only have one pointer set, either a
local variable or one in the context, we avoid potential double frees
in the cleanup routines. If chain->rtp_ctx is set, it is closed by
calling avformat_write_trailer, but that shouldn't be called unless
avformat_write_header succeeded.
This issue was pointed out by Andreas Cadhalpun.
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit cf402d6fa88acd647cdff993429583bec8a34fdc)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This fixes a typo from 8e32b1f096.
Signed-off-by: Martin Storsjö <martin@martin.st>
(cherry picked from commit c83dd2d2a458075a58895c384372f57c1ec26276)
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
This more completely checks for truncation
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 32e06c485be94a4967bff87022cbb065d3cbfd0d)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes Ticket4460
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit e88b3852aefaa39b2170ef185ad03dda18732821)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Fixes non deterministic crash in ticket4408/fuzz2.264
Likely fixes other samples as well
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
(cherry picked from commit 43b434210e597d484aef57c4139c3126d22b7e2b)
Conflicts:
libavcodec/h264.h
libavcodec/h264_slice.c
