This fixes some cases where the clipping was entirely missing.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Thanks (for the comments and review) -to: Reimar, beastd, Ronald
This reverts commit 77d88b872d.
The revert fixes actual overflows and a segfault as the variables
are signed and can be negative.
Conflicts:
libswscale/swscale.c
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
fate: Add tests for more AAC features.
aacps: Add missing newline in error message.
fate: Add tests for vc1/wmapro in ism.
aacdec: Add a fate test for 5.1 channel SBR.
aacdec: Turn off PS for multichannel files that use PCE based configs.
cabac: remove put_cabac_u/ueg from cabac-test.
swscale: RGB4444 and BGR444 input
FATE: add test for xWMA demuxer.
FATE: add test for SMJPEG demuxer and associated IMA ADPCM audio decoder.
mpegaudiodec: optimized iMDCT transform
mpegaudiodec: change imdct window arrangment for better pointer alignment
mpegaudiodec: move imdct and windowing function to mpegaudiodsp
mpegaudiodec: interleave iMDCT buffer to simplify future SIMD implementations
swscale: convert yuy2/uyvy/nv12/nv21ToY/UV from inline asm to yasm.
FATE: test to exercise WTV demuxer.
mjpegdec: K&R formatting cosmetics
swscale: K&R formatting cosmetics for code examples
swscale: K&R reformatting cosmetics for header files
FATE test: cvid-grayscale; ensures that the grayscale Cinepak variant is exercised.
Conflicts:
libavcodec/cabac.c
libavcodec/mjpegdec.c
libavcodec/mpegaudiodec.c
libavcodec/mpegaudiodsp.c
libavcodec/mpegaudiodsp.h
libavcodec/mpegaudiodsp_template.c
libavcodec/x86/Makefile
libavcodec/x86/imdct36_sse.asm
libavcodec/x86/mpegaudiodec_mmx.c
libswscale/swscale-test.c
libswscale/swscale.c
libswscale/swscale_internal.h
libswscale/x86/swscale_template.c
tests/fate/demux.mak
tests/fate/microsoft.mak
tests/fate/video.mak
tests/fate/wma.mak
tests/ref/lavfi/pixfmts_scale
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master: (21 commits)
ipmovie: do not read audio packets before the codec is known
truemotion2: check size before GetBitContext initialisation
avio: Only do implicit network initialization for network protocols
avio: Add an URLProtocol flag for indicating that a protocol uses network
adpcm: ADPCM Electronic Arts has always two channels
matroskadec: Fix a bug where a pointer was cached to an array that might later move due to a realloc()
fate: Add missing reference file from 9b4767e4.
mov: Support MOV_CH_LAYOUT_USE_DESCRIPTIONS for labeled descriptions.
4xm: Prevent buffer overreads.
mjpegdec: parse RSTn to prevent skipping other data in mjpeg_decode_scan
vp3: add fate test for non-zero last coefficient
vp3: fix streams with non-zero last coefficient
swscale: remove unused U/V arguments from yuv2rgb_write().
timer: K&R formatting cosmetics
lavf: cosmetics, reformat av_read_frame().
lavf: refactor av_read_frame() to make it easier to understand.
Report an error if pitch_lag is zero in AMR-NB decoder.
Revert "4xm: Prevent buffer overreads."
4xm: Prevent buffer overreads.
4xm: pass the correct remaining buffer size to decode_i2_frame().
...
Conflicts:
libavcodec/4xm.c
libavcodec/mjpegdec.c
libavcodec/truemotion2.c
libavformat/ipmovie.c
libavformat/mov_chan.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master: (46 commits)
mtv: Make sure audio_subsegments is not 0
v4l2: use V4L2_FMT_FLAG_EMULATED only if it is defined
avconv: add symbolic names for -vsync parameters
flvdec: Fix compiler warning for uninitialized variables
rtsp: Fix compiler warning for uninitialized variable
ulti: convert to new bytestream API.
swscale: Use standard multiple inclusion guards in ppc/ header files.
Place some START_TIMER invocations in separate blocks.
v4l2: list available formats
v4l2: set the proper codec_tag
v4l2: refactor device_open
v4l2: simplify away io_method
v4l2: cosmetics
v4l2: uniform and format options
v4l2: do not force interlaced mode
avio: exit early in fill_buffer without read_packet
vc1dec: fix invalid memory access for small video dimensions
rv34: fix invalid memory access for small video dimensions
rv34: joint coefficient decoding and dequantization
avplay: Don't call avio_set_interrupt_cb(NULL)
...
Conflicts:
Changelog
avconv.c
doc/APIchanges
doc/indevs.texi
libavcodec/adxenc.c
libavcodec/dnxhdenc.c
libavcodec/h264.c
libavdevice/v4l2.c
libavformat/flvdec.c
libavformat/mtv.c
libswscale/utils.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This fixes integer multiplication overflows in RGB48 output
(vertical) scaling as detected by IOC. What happens is that for
certain types of filters (lanczos, spline, bicubic), the
intermediate sum of coefficients in the middle of a filter can
be larger than the fixed-point equivalent of 1.0, even if the
final sum is 1.0. This is fine and we support that.
However, at frame edges, initFilter() will merge the coefficients
for the off-screen pixels into the top or bottom pixel, such as
to emulate edge extension. This means that suddenly, a single
coefficient can be larger than the fixed-point equivalent of
1.0, which the vertical scaling routines do not support.
Therefore, remove the merging of coefficients for edges for
the vertical scaling filter, and instead add edge detection
to the scaler itself so that it copies the pointers (not data)
for the edges (i.e. it uses line[0] for line[-1] as well), so
that a single coefficient is never larger than the fixed-point
equivalent of 1.0.
This fixes the same overflow as in the RGB48/16-bit YUV scaling;
some filters can overflow both negatively and positively (e.g.
spline/lanczos), so we bias a signed integer so it's "half signed"
and "half unsigned", and can cover overflows in both directions
while maintaining full 31-bit depth.
Signed-off-by: Mans Rullgard <mans@mansr.com>
We're shifting individual components (8-bit, unsigned) left by 24,
so making them unsigned should give the same results without the
overflow.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
For certain types of filters where the intermediate sum of coefficients
can go above the fixed-point equivalent of 1.0 in the middle of a filter,
the sum of a 31-bit calculation can overflow in both directions and can
thus not be represented in a 32-bit signed or unsigned integer. To work
around this, we subtract 0x40000000 from a signed integer base, so that
we're halfway signed/unsigned, which makes it fit even if it overflows.
After the filter finishes, we add the scaled bias back after a shift.
We use the same trick for 16-bit bpc YUV output routines.
Signed-off-by: Mans Rullgard <mans@mansr.com>
* qatar/master:
x86: cabac: replace explicit memory references with "m" operands
avplay: don't request a stereo downmix
wmapro: use av_float2int()
lavc: avoid invalid memcpy() in avcodec_default_release_buffer()
lavu: replace int/float punning functions
lavfi: install libavfilter/vsrc_buffer.h
Remove extraneous semicolons
sdp: Restore the original mp4 format h264 extradata if converted
rtpenc: Add support for mp4 format h264
rtpenc: Simplify code by introducing a separate end pointer
movenc: Use the actual converted sample for RTP hinting
Fix a bunch of common typos.
Conflicts:
doc/developer.texi
doc/eval.texi
doc/filters.texi
doc/protocols.texi
ffmpeg.c
ffplay.c
libavcodec/mpegvideo.h
libavcodec/x86/cabac.h
libavfilter/Makefile
libavformat/avformat.h
libavformat/cafdec.c
libavformat/flvdec.c
libavformat/flvenc.c
libavformat/gxfenc.c
libavformat/img2.c
libavformat/movenc.c
libavformat/mpegts.c
libavformat/rtpenc_h264.c
libavformat/utils.c
libavformat/wtv.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
libswscale/swscale.c:2744:40: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
libswscale/swscale.c:2745:41: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
libswscale/swscale.c:2746:41: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
libswscale/swscale.c:2747:78: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master: (42 commits)
swscale: fix signed overflow in yuv2mono_X_c_template
snow: fix integer overflows
svq1enc: remove stale altivec-related hack
snow: fix signed overflow in byte to 32-bit replication
adx: rename ff_adx_decode_header() to avpriv_adx_decode_header()
avformat: add CRI ADX format demuxer
adx: add an ADX parser.
adx: move header decoding to ADX common code
adx: calculate the number of blocks in a packet
adx: define and use 2 new macro constants BLOCK_SIZE and BLOCK_SAMPLES
adx: check for unsupported ADX formats
adx: simplify encoding by using put_sbits()
adx: calculate correct LPC coeffs
adx: use 12-bit coefficients instead of 14-bit to avoid integer overflow
adx: simplify adx_decode() by using get_sbits() to read residual samples
adx: fix the data offset parsing in adx_decode_header()
adx: remove unneeded post-decode channel interleaving
adx: validate header values
adx: cosmetics: general pretty-printing and comment clean-up
adx: remove useless comments
...
Conflicts:
Changelog
libavcodec/cook.c
libavcodec/fraps.c
libavcodec/nuv.c
libavcodec/pthread.c
libavcodec/version.h
libavformat/Makefile
libavformat/version.h
Merged-by: Michael Niedermayer <michaelni@gmx.at>
As old bits are shifted out of the accumulator, they cause signed
overflows when they reach the end. Making the variable unsigned fixes
this.
Signed-off-by: Mans Rullgard <mans@mansr.com>
* qatar/master:
id3v2: fix doxy comment - 'machine byte order' makes no sense on char arrays
VC1: restore mistakenly removed code
twinvq: check output buffer size before decoding
twinvq: return an error when the packet size is too small
lavf: export some forgotten symbols with non-av prefixes.
swscale: update altivec yuv2planeX asm to new per-plane API.
swscale: make yuv2yuvX_10_sse2/avx 8/9/16-bits aware.
yuv2planeX10 SIMD
swscale: decide whether to use yuv2plane1/X on a per-plane basis.
swscale: reintroduce full precision in 16-bit output.
Split up yuv2yuvX functions
Split out yuv2yuv1 luma and chroma in order to make them generic DSP functions
lavc: replace references to deprecated AVCodecContext.error_recognition to use AVCodecContext.err_recognition
lavc: translate non-flag-based er options into flag-based ef options at codec open
add -err_filter AVOptions to access flag-based error recognition
h264_weight: initialize "height" function argument properly.
presets: spelling error in libvpx 1080p50_60
avplay: fix fullscreen behaviour with SDL 1.2.14 on Mac OS X
Conflicts:
ffplay.c
libavformat/libavformat.v
libswscale/swscale.c
libswscale/x86/swscale_template.c
tests/ref/lavfi/pixfmts_scale
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master: (35 commits)
flvdec: Do not call parse_keyframes_index with a NULL stream
libspeexdec: include system headers before local headers
libspeexdec: return meaningful error codes
libspeexdec: cosmetics: reindent
libspeexdec: decode one frame at a time.
swscale: fix signed shift overflows in ff_yuv2rgb_c_init_tables()
Move timefilter code from lavf to lavd.
mov: add support for hdvd and pgapmetadata atoms
mov: rename function _stik, some indentation cosmetics
mov: rename function _int8 to remove ambiguity, some indentation cosmetics
mov: parse the gnre atom
mp3on4: check for allocation failures in decode_init_mp3on4()
mp3on4: create a separate flush function for MP3onMP4.
mp3on4: ensure that the frame channel count does not exceed the codec channel count.
mp3on4: set channel layout
mp3on4: fix the output channel order
mp3on4: allocate temp buffer with av_malloc() instead of on the stack.
mp3on4: copy MPADSPContext from first context to all contexts.
fmtconvert: port float_to_int16_interleave() 2-channel x86 inline asm to yasm
fmtconvert: port int32_to_float_fmul_scalar() x86 inline asm to yasm
...
Conflicts:
libavcodec/arm/h264dsp_init_arm.c
libavcodec/h264.c
libavcodec/h264.h
libavcodec/h264_cabac.c
libavcodec/h264_cavlc.c
libavcodec/h264_ps.c
libavcodec/h264dsp_template.c
libavcodec/h264idct_template.c
libavcodec/h264pred.c
libavcodec/h264pred_template.c
libavcodec/x86/h264dsp_mmx.c
libavdevice/Makefile
libavdevice/jack_audio.c
libavformat/Makefile
libavformat/flvdec.c
libavformat/flvenc.c
libavutil/pixfmt.h
libswscale/utils.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
rtp: factorize dynamic payload type fallback
flvdec: Ignore the index if it's from a creator known to be different
cmdutils: move grow_array out of #if CONFIG_AVFILTER
avconv: actually set InputFile.rate_emu
ratecontrol: update last_qscale_for sooner
Fix unnecessary shift with 9/10bit vertical scaling
prores: mark prores as intra-only in libavformat/utils.c:is_intra_only()
prores: return more meaningful error values
prores: improve error message wording
prores: cosmetics: prettyprinting, drop useless parentheses
prores: lowercase AVCodec name entry
Conflicts:
cmdutils.c
libavcodec/proresdec_lgpl.c
tests/ref/lavfi/pixfmts_scale
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master: (23 commits)
h264: hide reference frame errors unless requested
swscale: split hScale() function pointer into h[cy]Scale().
Move clipd macros to x86util.asm.
avconv: reindent.
avconv: rescue poor abused start_time global.
avconv: rescue poor abused recording_time global.
avconv: merge two loops in output_packet().
avconv: fix broken indentation.
avconv: get rid of the arbitrary MAX_FILES limit.
avconv: get rid of the output_streams_for_file vs. ost_table schizophrenia
avconv: add a wrapper for output AVFormatContexts and merge output_opts into it
avconv: make itsscale syntax consistent with other options.
avconv: factor out adding input streams.
avconv: Factorize combining auto vsync with format.
avconv: Factorize video resampling.
avconv: Don't unnecessarily convert ipts to a double.
ffmpeg: remove unsed variable nopts
RV3/4 parser: remove unused variable 'off'
add XMV demuxer
rmdec: parse FPS in RealMedia properly
...
Conflicts:
avconv.c
libavformat/version.h
libswscale/swscale.c
tests/ref/fate/lmlm4-demux
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This allows using more specific implementations for chroma/luma, e.g.
we can make assumptions on filterSize being constant, thus avoiding
that test at runtime.
