Commit Graph
100 Commits
Author SHA1 Message Date
Michael Niedermayer c62d1780ff avformat/hlsenc: Check the return code of avformat_write_header()
Fixes: segfault
Fixes: Ticket5067

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-13 02:28:13 +01:00
Michael Niedermayer 861f47ddf4 avformat/mov: Enable parser for mp3s by old HandBrake
Fixes Ticket5047

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-13 02:11:56 +01:00
Michael Niedermayer 537e901fe6 avformat/mxfenc: Fix integer overflow in length computation
Fixes: CID1341577

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-13 02:11:56 +01:00
Michael Niedermayer 625b582d5a avcodec/aacsbr_template: Add Check to read_sbr_envelope()
The limit is a conservative guess, the spec does not seem to specify a limit

Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-12 19:05:07 +01:00
Michael Niedermayer b78885a3c5 avcodec/aacsbr: Split the env_facs table
This also removes a #ifdef and special case for the fixed point case

Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-12 12:19:07 +01:00
Michael Niedermayer edfc835a56 avformat/mpegtsenc: Add basic multi program support
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-11 22:27:25 +01:00
Michael Niedermayer 8f948b6244 ffmpeg: Add basic support to mux multiple programs
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-11 22:27:25 +01:00
Michael Niedermayer 676a93f2d8 avformat: Add av_program_add_stream_index()
This will be used by the subsequent commit(s)

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-11 22:27:25 +01:00
Michael Niedermayer 5e8b9e746f avutil/softfloat: Assert that the exponent did not overflow the legal range in av_normalize1_sf()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-11 13:39:21 +01:00
Michael Niedermayer 80ceb4696a avformat/rmdec: Fix use of uninitialized variable
Fixes: CID1341580

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-09 18:17:47 +01:00
Michael Niedermayer bc8b1e694c avutil/mathematics: Fix division by 0
Fixes: CID1341571

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-09 17:39:38 +01:00
Michael Niedermayer 8cfa912e25 avformat/flacdec: remove unused return code assignment
Fixes CID1271811

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-09 16:47:55 +01:00
Michael Niedermayer e6971db12b avformat/matroskaenc: Check codecdelay before use
Fixes CID1238790

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-09 16:18:14 +01:00
Michael Niedermayer 2140858524 avcodec/hevc: Fix integer overflow of entry_point_offset
Fixes out of array read
Fixes: d41d8cd98f00b204e9800998ecf8427e/signal_sigsegv_321165b_7641_077dfcd8cbc80b1c0b470c8554cd6ffb.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 22:34:30 +01:00
Michael Niedermayer a08681f1e6 avcodec/dirac_parser: Check that there is a previous PU before accessing it
Fixes out of array read
Fixes: 99d142c47e6ba3510a74b872a1a2ae72/asan_heap-oob_11b36f4_3811_0f5c69e7609a88a580135678de1df844.dxa

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 17:42:45 +01:00
Michael Niedermayer c7d6ec947c avcodec/dirac_parser: Add basic validity checks for next_pu_offset and prev_pu_offset
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 17:42:45 +01:00
Michael Niedermayer 79798f7c57 avcodec/dirac_parser: Fix potential overflows in pointer checks
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 17:42:45 +01:00
Michael Niedermayer 66e05f6ff5 avcodec/wmaprodec: Check bits per sample to be within the range not causing integer overflows
Fixes: 549d5aab1480d10f2a775ed90b0342f1/signal_sigabrt_7ffff6ae7cc9_5643_96bbb0cfe3e28be1dadfce1075016345.wma

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 14:35:37 +01:00
Michael Niedermayer 15d14ce47c avcodec/utils: Fix overflow in get_bit_rates computations
Fixes: 129ca3e28d73af7b1e24a9d4118e7a2d/signal_sigabrt_7ffff6ae7cc9_836_762b310fc3ef6087bd7771e5d8e90b9b.asf

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 13:20:03 +01:00
Michael Niedermayer 2de8bfd2ef avcodec/pcm: Fix overflow in bitrate computation
Fixes: 129ca3e28d73af7b1e24a9d4118e7a2d/signal_sigabrt_7ffff6ae7cc9_836_762b310fc3ef6087bd7771e5d8e90b9b.asf

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 13:20:03 +01:00
Michael Niedermayer 0c56f8303e avcodec/wmaprodec: Fix overflow of cutoff
Fixes: 129ca3e28d73af7b1e24a9d4118e7a2d/signal_sigabrt_7ffff6ae7cc9_836_762b310fc3ef6087bd7771e5d8e90b9b.asf

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 13:20:03 +01:00
Michael Niedermayer 7ed47e9729 avformat/smacker: fix integer overflow with pts_inc
Fixes: ce19e41f0ef1e52a23edc488faecdb58/asan_heap-oob_2504e97_4202_ffa0df1baed14022b9bfd4f8ac23d0cb.smk

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-05 13:20:03 +01:00
Michael Niedermayer 325b59368d swscale/utils: Fix for runtime error: left shift of negative value -1
Fixes: c106b36fa36db8ff8f3ed0c82be7bea2/asan_heap-oob_32699f0_6321_467b9a1d7e03d7cfd310b7e65dc53bcc.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-04 21:44:53 +01:00
Michael Niedermayer 4f2419888b avcodec/pgssubdec: Fix left shift of 255 by 24 places cannot be represented in type int
Fixes: b293a6479bb4b5286cff24d356bfd955/asan_generic_225c3c9_7819_cc526b657450c6cdef1371b526499626.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-04 21:39:14 +01:00
Michael Niedermayer ab7ff38052 avformat/flvdec: Fix left shift of 137 by 24 places cannot be represented in type int
Fixes: 3c857d4d90365731524716e6d051e43a/signal_sigsegv_7f4f59bcc29e_1386_20abd2c8e655cb9c75b24368e65fe3b1.flv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-04 18:54:39 +01:00
Michael Niedermayer 689a867413 avformat/msf: Check channels when reading the header
Fixes integer overflow
Fixes: 0c2625f236ced104d402b4a03c0d65c7/asan_generic_274e1ce_5990_9314e7a67c26aecf011b178ade9f217c.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-04 18:54:39 +01:00
Michael Niedermayer a85b02dcf7 avcodec/jpeg2000dec: Check bpno in decode_cblk()
Fixes: undefined shift
Fixes: c409ef86f892335a0a164b5871174d5a/asan_heap-oob_1dff564_2159_162b7234616deab02b544410455eb07b.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-04 18:54:39 +01:00
Michael Niedermayer 0c1d62ab9d avformat/oggparsetheora: Check duration to be not AV_NOPTS_VALUE (and positive) before use
Fixes: 1536b9b096a8f95b742bae9d3d761cc6/signal_sigsegv_294aaed_7322_4fad88a38dc8952dc20dcb60c1895758.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-04 13:47:58 +01:00
Michael Niedermayer 2ff61c3c1a avcodec/dnxhddec: Fix runtime error: left shift of negative value
Fixes: 2abd25478c62a675f335fac00b467023/asan_static-oob_10aff98_1227_8811480c6ef1e970a7977ceb7e5a9958.mxf

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-04 13:44:22 +01:00
Michael Niedermayer 9d44e23618 ffmpeg: Check pkt_dts for AV_NOPTS_VALUE
Fixes integer overflow
Fixes: 1536b9b096a8f95b742bae9d3d761cc6/signal_sigsegv_294aaed_2039_8d1797aeb823ea43858d0fa45c9eb899.ogv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-04 13:37:04 +01:00
Michael Niedermayer 18268f761b avcodec/vp3: Fix "runtime error: left shift of negative value"
Fixes: 5c6129154b356b80bcab86f9e3ee5d29/signal_sigabrt_7ffff6ae7cc9_7322_d26ac6d7cb6567db1b8be0159b387d0b.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-04 13:37:00 +01:00
Michael Niedermayer 9696a01f12 avformat/mpjpegdec: Fix "libavformat/mpjpegdec.c:269:9: warning: passing argument 3 of av_stristart from incompatible pointer type"
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-04 03:09:00 +01:00
Michael Niedermayer 77a644e6fa avcodec/h264_direct: Check CO-POCs before use
Fixes harmless integer overflow
Fixes mozilla bug 1230286

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-04 01:55:11 +01:00
Michael Niedermayer cafb195604 avformat/utils: Fix "libavformat/utils.c:927:35: runtime error: signed integer overflow: -2450238577049583619 - 9223090561878065151 cannot be represented in type long long"
Fixes Mozilla bug 1229205

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-04 00:44:06 +01:00
Michael Niedermayer 4e31176e14 avformat/riffdec: remove special case for bitrate > 32bit
AVCodecContext.bitrate is 64bit

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-03 20:20:09 +01:00
Michael Niedermayer 32bf6550cb avformat/riffdec: Initialize bitrate
Fixes CID1338334

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-03 20:20:09 +01:00
Michael Niedermayer b46dcd5209 avutil/timecode: Fix fps check
The fps variable is explicitly set to -1 in case of some errors, the check must
thus be signed or the code setting it needs to use 0 as error code
the type of the field could be changed as well but its in an installed header

Fixes: integer overflow
Fixes: 9982cc157b1ea90429435640a989122f/asan_generic_3ad004a_3799_22cf198d9cd09928e2d9ad250474fa58.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-03 04:09:48 +01:00
Michael Niedermayer 0afdfbe116 avcodec/jpeg2000: fix type of arguments of tag_tree_size
Fixes: out of array read
Fixes: 36b8096fefab16c4c9326a508053e95c/signal_sigsegv_1d9ce18_3233_1a55196b018106dfabeace071a432d9e.r3d

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-03 02:28:00 +01:00
Michael Niedermayer 26379d4fdd avcodec/vp3: ensure header is parsed successfully before tables
Fixes assertion failure
Fixes: 266ee543812e934f7b4a72923a2701d4/signal_sigabrt_7ffff6ae7cc9_7322_85218d61759d461bdf7387180e8000c9.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-03 00:13:27 +01:00
Michael Niedermayer ec7a3be11e avformat/utils: Move end_time1 AV_NOPTS_VALUE Check after rescale
Fixes integer overflow
Fixes: 266ee543812e934f7b4a72923a2701d4/signal_sigabrt_7ffff6ae7cc9_7322_85218d61759d461bdf7387180e8000c9.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-03 00:13:27 +01:00
Michael Niedermayer d872643cfe avformat/utils: Check AVFormatContext->duration in compute_chapters_end() before use
Fixes integer overflow
Fixes: 266ee543812e934f7b4a72923a2701d4/signal_sigabrt_7ffff6ae7cc9_7322_85218d61759d461bdf7387180e8000c9.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-03 00:13:27 +01:00
Michael NiedermayerandLuca Barbato 0fc61c6ab6 avi: Validate the stream-id for DV as well
Avoid false positives while syncing.

Bug-Id: ffmpeg/4086
Bug-Id: 879

Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
2015-12-02 23:49:10 +01:00
Michael Niedermayer 97b8db334a avutil/rational: Test av_rescale_rnd() with combinations of "special" values
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-02 21:38:11 +01:00
Michael Niedermayer f03c2ceec1 avutil/mathematics: return INT64_MIN (=AV_NOPTS_VALUE) from av_rescale_rnd() for overflows
Fixes integer overflow
Fixes: mozilla bug 1229167

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-02 21:38:11 +01:00
Michael Niedermayer cd7524fdd1 avcodec/apedec: Check length in long_filter_high_3800()
Fixes out of array read
Fixes: 0a7ff0c1d93da9cef28a315ec91b692a/asan_heap-oob_4a52e5_3604_9c56dbb20e308f4faeef7b35f688521a.ape

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-02 21:38:11 +01:00
Michael Niedermayer e1057babdf avcodec/mpegaudio_tablegen: Add (), fix fate
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-02 03:42:44 +01:00
Michael Niedermayer 736e2e2c30 avfilter/vf_shuffleframes: Assert that the case of an uninitialized ret does not occur
Fixes CID1258479

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-01 22:00:25 +01:00
Michael Niedermayer 25e37f5ea9 avutil/mathematics: Do not treat INT64_MIN as positive in av_rescale_rnd
The code expects actual positive numbers and gives completely wrong
results if INT64_MIN is treated as positive
Instead clip it into the valid range that is add 1 and treat it as
negative

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-01 13:26:12 +01:00
Michael Niedermayer 3a9cb18855 avutil/integer: Fix av_mod_i() with negative dividend
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-01 13:26:12 +01:00
Michael Niedermayer 8e7f452022 avformat/dump: Fix integer overflow in av_dump_format()
Fixes part of mozilla bug 1229167

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-12-01 13:26:03 +01:00
Michael Niedermayer a00cc2e40d ffserver: Clear avio context after closing it in rtp_new_av_stream()
Suggested-by: "Reynaldo H. Verdejo Pinochet" <reynaldo@osg.samsung.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-30 23:31:12 +01:00
Michael Niedermayer 40063a900b ffserver: Clear avio context after closing it
Fixes: ==13287== Invalid read of size 4
==13287==    at 0x45161A: flush_buffer (aviobuf.c:143)
==13287==    by 0x451971: avio_flush (aviobuf.c:200)
==13287==    by 0x512CCF: av_write_trailer (mux.c:1016)
==13287==    by 0x41A5E0: close_connection (ffserver.c:853)
==13287==    by 0x421EDC: rtsp_cmd_interrupt (ffserver.c:3245)
==13287==    by 0x420B9C: rtsp_parse_request (ffserver.c:2854)
==13287==    by 0x41A9C2: handle_connection (ffserver.c:930)
==13287==    by 0x41A04B: http_server (ffserver.c:700)
==13287==    by 0x423A60: main (ffserver.c:3897)
==13287==  Address 0xb6cd258 is 88 bytes inside a block of size 192 free'd
==13287==    at 0x4C2B5D9: free (vg_replace_malloc.c:446)
==13287==    by 0x1004DAC: av_free (mem.c:239)
==13287==    by 0x454835: avio_close_dyn_buf (aviobuf.c:1170)
==13287==    by 0x41F385: http_prepare_data (ffserver.c:2368)
==13287==    by 0x41F59B: http_send_data (ffserver.c:2416)
==13287==    by 0x41ABE2: handle_connection (ffserver.c:986)
==13287==    by 0x41A04B: http_server (ffserver.c:700)
==13287==    by 0x423A60: main (ffserver.c:3897)

Reviewed-by: "Reynaldo H. Verdejo Pinochet" <reynaldo@osg.samsung.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-30 23:31:12 +01:00
Michael NiedermayerandVittorio Giovara b74b88f30d g723_1: Handle values at the ends of the table in lsp2lpc()
Fixes out of array reads.

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
2015-11-30 10:58:45 -05:00
Michael Niedermayer a814f1d364 avcodec/vp3: always set pix_fmt in theora_decode_header()
Fixes assertion failure
Fixes: d0bb0662da342ec65f8f2a081222e6b9/signal_sigabrt_7ffff6ae7cc9_5471_82964f0a9ac2f4d3d59390c15473f6f7.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-30 04:25:00 +01:00
Michael Niedermayer 891dc8f875 avcodec/mpeg4videodec: Check available data before reading custom matrix
Fixes: out of array read
Fixes: 76c515fc3779d1b838667c61ea13ce92/asan_heap-oob_1fc0d07_8913_794a4629a264ebdb25b58d3a94ed1785.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-29 23:45:31 +01:00
Michael Niedermayer 55711e6fd0 avcodec/dnxhddec: Disable 12bit by default
The DC VLC table used is too small, fixing this requires a sample,
thus request a sample.
Some samples are said to work even though the table has the wrong size, thus
this is left enabled if the user enables experimental features.

Fixes: 2abd25478c62a675f335fac00b467023/asan_static-oob_10aff98_1227_8811480c6ef1e970a7977ceb7e5a9958.mxf

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Approved-by: kurosu
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-29 21:30:37 +01:00
Michael Niedermayer 98626a1a4c avfilter/avf_showvolume: Fix "warning: comparison of unsigned expression < 0 is always false"
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-29 15:47:42 +01:00
Michael Niedermayer aa427537b5 avcodec/h264_refs: Check that long references match before use
Fixes out of array read
Fixes: 59bb925e90201fa0f87f0a31945d43b5/asan_heap-oob_4a52e5_3388_66027f11e3d072f1e02401ecc6193361.jvt

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-29 03:27:28 +01:00
Michael Niedermayer abee0a1c60 avcodec/utils: Clear dimensions in ff_get_buffer() on failure
Fixes out of array access
Fixes: 482d8f2fd17c9f532b586458a33f267c/asan_heap-oob_4a52b6_7417_1d08d477736d66cdadd833d146bb8bae.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-28 20:10:51 +01:00
Michael Niedermayer 4f03bebc79 avcodec/utils: Use 64bit for aspect ratio calculation in avcodec_string()
Fixes integer overflow
Fixes: 3a45b2ae02f2cf12b7bd99543cdcdae5/asan_heap-oob_1dff502_8022_899f75e1e81046ebd7b6c2394a1419f4.mov

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-28 20:10:51 +01:00
Michael Niedermayer dad354f38d avcodec/hevc: Check max ctb addresses for WPP
Fixes out of array read
Fixes: 2f95ddd996db8a6281d2e18c184595a7/asan_heap-oob_192fe91_3330_58e4441181e30a66c19f743dcb392347.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-28 15:48:55 +01:00
Michael Niedermayer 861f2b2a53 avcodec/vp3: Fix several memleaks
Fixes: 1536b9b096a8f95b742bae9d3d761cc6/signal_sigsegv_294aaed_4460_b209bd1e7cebe458b53072a44191316d.ogg

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-28 01:01:41 +01:00
Michael Niedermayer 6105b7219a avcodec/vp3: Clear context on reinitialization failure
Fixes null pointer dereference
Fixes: 1536b9b096a8f95b742bae9d3d761cc6/signal_sigsegv_294aaed_2039_8d1797aeb823ea43858d0fa45c9eb899.ogv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-28 00:25:08 +01:00
Michael Niedermayer d85aa76115 avcodec/hevc: allocate entries unconditionally
Fixes out of array access
Fixes: 08664a2a7921ef48172f26495c7455be/asan_heap-oob_23036c6_3301_523388ef84285a0270caf67a43247b59.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 23:37:42 +01:00
Michael Niedermayer 36205501ba avcodec/pthread_slice: Allow calling ff_alloc_entries() multiple times to readjust the entry count
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 23:37:42 +01:00
Michael Niedermayer d5028f61e4 avcodec/hevc_cabac: Fix multiple integer overflows
Fixes: 04ec80eefa77aecd7a49a442cc02baea/asan_heap-oob_19544fa_3303_1905796cd9d8e15f86d664332caabc00.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 23:37:42 +01:00
Michael Niedermayer feb3f39614 avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_encode*()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 21:02:13 +01:00
Michael Niedermayer 75422280fb avcodec/jpeg2000dwt: Check ndeclevels before calling dwt_decode*()
Fixes out of array access
Fixes: 01859c9a9ac6cd60a008274123275574/asan_heap-oob_1dff571_8250_50d3d1611e294c3519fd1fa82198b69b.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 20:59:26 +01:00
Michael Niedermayer ef9f7bbfa4 avcodec/hevc: Check entry_point_offsets
Fixes out of array read
Fixes: 007c4a36608ebdf27ee260ad60a81184/asan_heap-oob_32076b4_2243_116b1cb29d91cc4974d6680e3d10bd91.bit

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 18:53:21 +01:00
Michael Niedermayer 8000d484b8 avcodec/cabac: Check initial cabac decoder state
Fixes integer overflows
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 13:52:47 +01:00
Michael Niedermayer a1f6b05f52 avcodec/cabac_functions: Fix "left shift of negative value -31767"
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2340_591e9810c7b09efe501ad84638c9e9f8.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Found-by: xiedingbao (Ticket4727)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-27 13:52:47 +01:00
Michael Niedermayer b3494e3c3e avcodec/pthread_slice: Remove rets_count
It appears rets_count is redundant

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-26 00:56:12 +01:00
Michael Niedermayer 4ea4d2f438 avcodec/h264_slice: Limit max_contexts when slice_context_count is initialized
Fixes out of array access
Fixes: 1430e9c43fae47a24c179c7c54f94918/signal_sigsegv_421427_2049_f2192b6829ab6e0eefcb035329c03c60.264

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-24 22:17:36 +01:00
Michael NiedermayerandMartin Storsjö 5b70fb8fee movenc-test: Fix integer overflows
Signed-off-by: Martin Storsjö <martin@martin.st>
2015-11-24 20:57:11 +02:00
Michael Niedermayer e4bfc726cd avfilter/vsrc_mandelbrot: Fix speed regression
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-24 03:17:32 +01:00
Michael Niedermayer 13834c1016 avcodec/mpegvideo_enc: Remove slice structured mode from H.263 as well as the code automatically enabing it
There is no such thing as a slice structured mode in the original version 1  H.263,
that mode was added in H.263+ in 1998. Also the headers for slice structured mode
are not part of the older version 1 and this would result in unplayable files

An alternative to this patch would be to merge the H263 and H263P AVCodecs and use
other means to distinguish the older and newer versions.

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-23 22:47:41 +01:00
Michael Niedermayer 188a1a17a6 avformat/movenc-test: Fix integer overflows
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-23 15:45:44 +01:00
Michael Niedermayer 764900d645 avcodec/dvdsubdec: Fix types for offset1/2 in ff_dlog()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-22 19:26:05 +01:00
Michael Niedermayer 9b7d39de81 avformat/ffmdec: Only return EAGAIN if a server is attached
This should fix a infinite loop

Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-20 20:08:26 +01:00
Michael Niedermayer e33d372023 ffserver: Use AVOption API to access ffm demuxer instead of direct access depending on ABI
server_attached is newly added so the demuxer knows if there is an attached server
that can update the write index. This is needed to fix a infinite loop

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-20 20:03:12 +01:00
Michael Niedermayer 3444c00aab avformat/ffmdec: Add cleaner API for ffserver to interface without depending on internal ABI
Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-20 20:01:05 +01:00
Michael Niedermayer 8eadabf941 avcodec/aacsbr_fixed: Replace a noise_facs_q check by an av_assert0
The replaced check should have become redundant

Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-20 03:46:43 +01:00
Michael Niedermayer 03a34ef7a9 avcodec/sbr: fix copy and paste error
Found-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-20 03:46:43 +01:00
Michael Niedermayer 58fba4b02f avcodec/aacsbr: Replace a noise_facs_q check by an av_assert0
The replaced check should have become redundant

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-19 13:52:05 +01:00
Michael Niedermayer d877b88f51 avcodec/aacsbr_template: Check values read in read_sbr_noise()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-19 13:12:43 +01:00
Michael Niedermayer bfd0e02dd6 avcodec/aacsbr: Split pre dequantization noise factors table
This allows removing a special case for the fixed point decoder and will
make error checks simpler

Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-19 13:12:20 +01:00
Michael Niedermayer 50ce510ac4 avcodec/pthread_slice: remove dummy_ret hack
This should avoid some tsan warnings

Found-by: Chris Cunningham <chcunningham@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-19 00:41:25 +01:00
Michael Niedermayer 6f37226b68 avcodec/h264_slice: Clear top_borders on allocation
In case of bitstream errors the deblock filter and slices can access uninitialized
top_borders from previous slices which did not fill them as they stoped halfway due
to error or where entirely missing.

This also makes code using these tables deterministic in case of missing or damaged
slices

Found-by: Tyson Smith
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-18 22:18:01 +01:00
Michael Niedermayer fc91eeab0b avutil/mem: Add av_fast_mallocz()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-18 22:05:16 +01:00
Michael Niedermayer 3a4d8281c6 avformat/genh: Fix tools/probetest failure
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-18 14:45:51 +01:00
Michael Niedermayer a5034b324c avformat/matroskadec: Check subtitle stream before dereferencing
Unrecognized streams are not allocated
Fixes: flicker-1.color1.vp91447030769.08.webm

Found-by: Chris Cunningham <chcunningham@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-17 19:02:28 +01:00
Michael Niedermayer bf6d41d8a2 avcodec/internal: Fix skiped typo
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-17 01:23:10 +01:00
Michael Niedermayer a62178be80 avcodec/pngdec: Replace assert by request for sample for unsupported TRNS cases
Fixes assertion failure
Fixes: 7f646252a30ee28b583aac1f82e7985e/signal_sigabrt_7ffff6ae7cc9_7353_62fc077bf2f454d39e188c69807193a6.png

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-16 18:48:02 +01:00
Michael Niedermayer 1ef336e912 avformat/utils: Do not init parser if probing is unfinished
Fixes assertion failure
Fixes: 136f8b8d47af7892306625e597dee655/signal_sigabrt_7ffff6ae7cc9_8941_ab11bea57c84796418f481f873dc31ba.dvr_ms

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-15 23:42:24 +01:00
Michael Niedermayer 0eb7de1973 avcodec/jpeg2000: Change coord to 32bit to support larger than 32k width or height
Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-15 22:11:05 +01:00
Michael Niedermayer 65d3359fb3 avcodec/jpeg2000dec: Fix potential integer overflow with tile dimensions
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-15 21:36:19 +01:00
Michael Niedermayer 6ef819c40b avcodec/jpeg2000dec: Check SIZ dimensions to be within the supported range
Fixes potential integer overflows
Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi

This fix is choosen to be simple to backport, better solution
for master is planed

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-15 21:36:19 +01:00
Michael Niedermayer a1a8cbcb35 avcodec/jpeg2000: Check comp coords to be within the supported size
Fixes assertion failure
Fixes: 03e0abe721b1174856d41a1eb5d6a896/signal_sigabrt_7ffff6ae7cc9_3813_e71bf3541abed3ccba031cd5ba0269a4.avi

This fix is choosen to be simple to backport, better solution
for master is planed

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-15 20:53:22 +01:00
Michael Niedermayer 016fd413f9 avcodec/jpeg2000: Use av_image_check_size() in ff_jpeg2000_init_component()
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-15 20:53:22 +01:00
Michael Niedermayer 7ad698e24e avcodec/wmaprodec: Check for overread in decode_packet()
Fixes assertion failure
Fixes: 0256e92df2df7e933b43a2c70e4c8040/signal_sigabrt_7ffff6ae7cc9_1358_999ac18684788221490757582ce9af84.wma

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2015-11-15 18:32:22 +01:00