e95fcfe8fb
avcodec/lpc: signed integer overflow in compute_lpc_coefs() (aacdec_fixed)
...
Fixes: runtime error: signed integer overflow: -1575818955 + -915383657 cannot be represented in type 'int'
Fixes: 2224/clusterfuzz-testcase-minimized-6208559949807616
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-21 02:25:23 +02:00
966a0a814d
avcodec/decode: Update decode_simple_internal() to get the side data correctly.
...
Use avci->last_pkt_props to get the side data. Using |pkt| doesn't work
when FF_API_MERGE_SD is set, as the compressed side data is expanded into
|tmp|, leaving the original |pkt| unchanged.
Signed-off-by: James Almer <jamrial@gmail.com >
2017-06-20 17:14:17 -03:00
8221c71703
avcodec/x86: allow future 8-bit simple idct to use slightly different coefficients
2017-06-20 16:12:25 +02:00
9d11fedd11
avcodec/mdec: override IDCT choice before initing DSP structs
2017-06-20 13:59:51 +02:00
d2597fb0c1
avcodec/x86: modify simple_idct10 macros to add an action paramter
2017-06-20 13:35:01 +02:00
8781330d80
avcodec/x86: cleanup simple_idct10
...
Use named arguments for the functions so we can remove a define. The
stride/linesize argument is now ptrdiff_t type so we no longer need to
sign extend the register.
2017-06-20 13:34:38 +02:00
e3db94302c
avcodec/x86/mpegenc: support transpose permuation type
2017-06-20 12:12:13 +02:00
fa30a0a548
avcodec/x86/mpegenc: check IDCT permutation type is a valid value
2017-06-20 12:12:13 +02:00
ae6f6d4e34
avcodec/x86/mpegvideo: Use intra scantable in dct_unquantize_h263_intra_mmx()
...
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-20 00:07:51 +02:00
cf231b68da
avcodec/h264: Fix mix of lossless and lossy MBs decoding
...
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com >
2017-06-19 14:33:40 -04:00
06dda70f1e
avcodec/h264_mb: Fix 8x8dct in lossless for new versions of x264
...
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com >
2017-06-19 14:33:39 -04:00
840b41b2a6
avcodec/h264_cabac: Fix CABAC+8x8dct in 4:4:4
...
Use the correct ctxIdxInc calculation for coded_block_flag.
Keep old behavior for old versions of x264 for backward compatibility.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com >
2017-06-19 14:33:39 -04:00
5f89747086
avcodec/wavpack: Fix undefined integer negation
...
Fixes: runtime error: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
Fixes: 2291/clusterfuzz-testcase-minimized-5538453481586688
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-19 17:54:40 +02:00
cf7edbd6c5
avcodec/aacdec_fixed: Check s for being too small
...
Fixes: runtime error: shift exponent -8 is negative
Fixes: 2286/clusterfuzz-testcase-minimized-5711764169687040
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-19 17:54:40 +02:00
f670c13f13
avcodec: Rename ff_mpv_decode_mb() to ff_mpv_reconstruct_mb
...
The new name more accuratly describes what the function does
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-19 17:54:40 +02:00
8bb59e6742
x86/aacpsdsp: add ff_ps_hybrid_analysis_ileave_sse
...
About 2x faster than the c version.
2017-06-18 22:34:22 -03:00
e229df9478
x86/aacpsdsp: add ff_ps_hybrid_synthesis_deint_{sse,sse4}
...
About 2x faster than the c version.
2017-06-18 22:33:27 -03:00
14b834c45a
avcodec/htmlsubtitles: Factor open brace handling into its own function
...
Suggested-by: wm4
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-18 14:50:35 +02:00
4132218b87
avcodec/htmlsubtitles: Replace very slow redundant sscanf() calls by cleaner and faster code
...
This reduces the worst case from O(n²) to O(n) time
Fixes Timeout
Fixes: 2127/clusterfuzz-testcase-minimized-6595787859427328
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-18 14:50:30 +02:00
27c2006805
avcodec/takdec: Fixes: integer overflow in AV_SAMPLE_FMT_U8P output
...
Fixes: runtime error: signed integer overflow: 2147483543 + 128 cannot be represented in type 'int'
Fixes: 2234/clusterfuzz-testcase-minimized-6266896041115648
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-18 14:38:41 +02:00
9a6503f496
avcodec/iff: Cleanup on init failure
...
Fixes: memleak
Fixes: 2272/clusterfuzz-testcase-minimized-5059103858622464
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-18 14:05:57 +02:00
c746f92a8e
avcodec/jpeg2000dsp: Reorder operations in ict_int() to avoid 2 integer overflows
...
Fixes: runtime error: signed integer overflow: 58065 * 51981 cannot be represented in type 'int'
Fixes: 2271/clusterfuzz-testcase-minimized-5778297776504832
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-17 22:36:54 +02:00
c94326c1fc
avcodec/hevcpred_template: Fix left shift of negative value
...
Fixes: runtime error: left shift of negative value -1
Fixes: 2250/clusterfuzz-testcase-minimized-5693382112313344
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-17 22:36:54 +02:00
1edbf5e20c
avcodec/hevcdec: Fix signed integer overflow in decode_lt_rps()
...
Fixes: runtime error: signed integer overflow: 2147483647 + 6 cannot be represented in type 'int'
Fixes: 2263/clusterfuzz-testcase-minimized-4800359627227136
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-17 00:34:48 +02:00
9b65dbf734
avcodec/gdv: Fix undefined shift
...
Fixes: runtime error: left shift of 255 by 24 places cannot be represented in type 'int'
Fixes: 2249/clusterfuzz-testcase-minimized-5388542379294720
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-16 23:32:28 +02:00
dfb61ea263
avcodec/jpeg2000dec: Check nonzerobits more completely
...
Fixes: runtime error: shift exponent 36 is too large for 32-bit type 'int'
Fixes: 2239/clusterfuzz-testcase-minimized-5639766592716800
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-16 20:32:39 +02:00
16d6cc2168
avcodec/wavpack: Change wp_log2() to unsigned
...
Fixes: runtime error: signed integer overflow: 2143315325 + 4186162 cannot be represented in type 'int'
Fixes: 2134/clusterfuzz-testcase-minimized-4619258405322752
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-16 20:32:39 +02:00
e77ddd31a8
avcodec/shorten: Sanity check maxnlpc
...
Fixes OOM
Fixes: 2131/clusterfuzz-testcase-minimized-4718045157130240
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-16 20:32:39 +02:00
623d217ed1
avcodec/aacps: move checks for valid length outside the stereo_interpolate dsp function
...
Signed-off-by: James Almer <jamrial@gmail.com >
2017-06-15 23:49:40 -03:00
b3446862bf
x86/vorbisdsp: optimize ff_vorbis_inverse_coupling_sse
...
About 7% faster.
2017-06-15 23:20:05 -03:00
c0607d88ee
avcodec/parser: assert that there is a past buffer if theres a reference into the past
...
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-16 01:35:35 +02:00
3c716682a8
avcodec/truemotion2: Move skip computation after checks
...
Fixes: runtime error: signed integer overflow: 630067357 * 4 cannot be represented in type 'int'
Fixes: 2233/clusterfuzz-testcase-minimized-5943031318446080
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-16 00:21:30 +02:00
e3fadc57c5
avcodec/jpeg2000: Fixes integer overflow in ff_jpeg2000_ceildivpow2()
...
Fixes: runtime error: negation of -2147483648 cannot be represented in type 'int'; cast to an unsigned type to negate this value to itself
Fixes: 2231/clusterfuzz-testcase-minimized-4565181982048256
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-16 00:21:30 +02:00
611b356274
avcodec/dnxhd_parser: Do not return invalid value from dnxhd_find_frame_end() on error
...
Fixes: Null pointer dereference
Fixes: CVE-2017-9608
Found-by: Yihan Lian
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-16 00:21:30 +02:00
b52b398c30
vc2enc: decrease default strictness level
...
Given how incredibly limited the official specifications are (limiting all use
to only the most common broadcasting formats), permit all supported inputs
by default. This makes the encoder more useful.
2017-06-15 18:30:08 +01:00
752dd1952a
vorbisenc: Stop tracking number of samples per frame
...
Each frame is now padded with 0 values if not enough samples are
present, and all frames are guaranteed to have exactly
1 << (venc->log2_blocksize[1] - 1) samples.
Signed-off-by: Tyler Jones <tdjones879@gmail.com >
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com >
2017-06-15 16:42:49 +01:00
f57f665183
vorbisenc: Apply and output correct length window and mdct
...
Usage of blocksize, window, mode, and mdct indexes are switched from
default 0 to a default of 1 to better align with specs. A flag of 0
should correspond with short windows, a flag of 1 with long.
Signed-off-by: Tyler Jones <tdjones879@gmail.com >
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com >
2017-06-15 16:42:49 +01:00
5a2ad7ede3
vorbisenc: Separate copying audio samples from windowing
...
Audio samples are shifted around when copying from the frame queue so that
analysis can be done without negatively impacting calculation of the MDCT.
Window coefficients are applied to the current two overlapped windows
simultaneously instead of applying overlap for the next frame ahead of time.
This improves readability when applying windows of varying lengths.
Signed-off-by: Tyler Jones <tdjones879@gmail.com >
Reviewed-by: Rostislav Pehlivanov <atomnuker@gmail.com >
2017-06-15 16:42:49 +01:00
bc40674462
avcodec/hevcdec: Check nb_sps
...
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-15 03:09:40 +02:00
1cb4ef526d
avcodec/hevc_refs: Check nb_refs in add_candidate_ref()
...
Fixes: runtime error: index 16 out of bounds for type 'int [16]'
Fixes: 2209/clusterfuzz-testcase-minimized-5012343912136704
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-15 03:09:40 +02:00
12245ab1f6
avcodec/mpeg4videodec: Check sprite delta upshift against overflowing.
...
Fixes: runtime error: signed integer overflow: -268386304 * 16 cannot be represented in type 'int'
Fixes: 2204/clusterfuzz-testcase-minimized-5616756909408256
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-15 00:16:20 +02:00
0a87be404a
avcodec/mpeg4videodec: Fix integer overflow in num_sprite_warping_points=2 case
...
Fixes: runtime error: signed integer overflow: 131072 + 2147352576 cannot be represented in type 'int'
Fixes: 2192/clusterfuzz-testcase-minimized-5370387988742144
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-15 00:16:20 +02:00
900fe8ee5d
avcodec/dnxhdenc: Assert that frame size is not assigned an error code
...
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc >
2017-06-15 00:16:20 +02:00
88a2e4504d
hevc: Fix scaling list prediction delta for the 32x32 inter matrix
...
Fixes ticket #6356 .
2017-06-14 23:08:26 +01:00
3882063174
vaapi: Add external control of allow-profile-mismatch
...
Uses the just-added ALLOW_PROFILE_MISMATCH flag.
(cherry picked from commit 7acb90333a
2017-06-14 22:27:06 +01:00
49ae8a5e87
lavc: Add flag to allow profile mismatch with hardware decoding
...
(cherry picked from commit 64a5260c69
2017-06-14 22:27:04 +01:00
b658b5399e
vaapi_encode: Use gop_size consistently in RC parameters
...
The non-H.26[45] codecs already use this form. Since we don't
currently generate I frames for codecs which support them separately
to IDR, the p_per_i variable is set to infinity by default so that it
doesn't interfere with any other calculation. (All the code for I
frames still exists, and it works for H.264 if set manually.)
(cherry picked from commit 6af014f402
2017-06-14 22:26:32 +01:00
28aedeed19
qsvenc: Allow use of hw_device_ctx to make the internal session
...
(cherry picked from commit 3d197514e6
2017-06-14 22:26:32 +01:00
8aa3c2df1a
qsvdec: Allow use of hw_device_ctx to make the internal session
...
(cherry picked from commit 8848ba0bd6
2017-06-14 22:26:32 +01:00
91c3b50d74
qsv: Add ability to create a session from a device
...
(cherry picked from commit 4936a48b1e
2017-06-14 22:26:32 +01:00
Michael Niedermayer
John Rummell
James Darnley
Anton Mitrofanov
James Almer
Rostislav Pehlivanov
Tyler Jones
Mark Thompson