This fixes the same overflow as in the RGB48/16-bit YUV scaling;
some filters can overflow both negatively and positively (e.g.
spline/lanczos), so we bias a signed integer so it's "half signed"
and "half unsigned", and can cover overflows in both directions
while maintaining full 31-bit depth.
Signed-off-by: Mans Rullgard <mans@mansr.com>
We're shifting individual components (8-bit, unsigned) left by 24,
so making them unsigned should give the same results without the
overflow.
Signed-off-by: Ronald S. Bultje <rsbultje@gmail.com>
For certain types of filters where the intermediate sum of coefficients
can go above the fixed-point equivalent of 1.0 in the middle of a filter,
the sum of a 31-bit calculation can overflow in both directions and can
thus not be represented in a 32-bit signed or unsigned integer. To work
around this, we subtract 0x40000000 from a signed integer base, so that
we're halfway signed/unsigned, which makes it fit even if it overflows.
After the filter finishes, we add the scaled bias back after a shift.
We use the same trick for 16-bit bpc YUV output routines.
Signed-off-by: Mans Rullgard <mans@mansr.com>
* qatar/master:
x86: cabac: replace explicit memory references with "m" operands
avplay: don't request a stereo downmix
wmapro: use av_float2int()
lavc: avoid invalid memcpy() in avcodec_default_release_buffer()
lavu: replace int/float punning functions
lavfi: install libavfilter/vsrc_buffer.h
Remove extraneous semicolons
sdp: Restore the original mp4 format h264 extradata if converted
rtpenc: Add support for mp4 format h264
rtpenc: Simplify code by introducing a separate end pointer
movenc: Use the actual converted sample for RTP hinting
Fix a bunch of common typos.
Conflicts:
doc/developer.texi
doc/eval.texi
doc/filters.texi
doc/protocols.texi
ffmpeg.c
ffplay.c
libavcodec/mpegvideo.h
libavcodec/x86/cabac.h
libavfilter/Makefile
libavformat/avformat.h
libavformat/cafdec.c
libavformat/flvdec.c
libavformat/flvenc.c
libavformat/gxfenc.c
libavformat/img2.c
libavformat/movenc.c
libavformat/mpegts.c
libavformat/rtpenc_h264.c
libavformat/utils.c
libavformat/wtv.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
libswscale/swscale.c:2744:40: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
libswscale/swscale.c:2745:41: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
libswscale/swscale.c:2746:41: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
libswscale/swscale.c:2747:78: warning: to be safe all intermediate pointers in cast from ‘int16_t **’ to ‘const int16_t **’ must be ‘const’ qualified [-Wcast-qual]
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master: (42 commits)
swscale: fix signed overflow in yuv2mono_X_c_template
snow: fix integer overflows
svq1enc: remove stale altivec-related hack
snow: fix signed overflow in byte to 32-bit replication
adx: rename ff_adx_decode_header() to avpriv_adx_decode_header()
avformat: add CRI ADX format demuxer
adx: add an ADX parser.
adx: move header decoding to ADX common code
adx: calculate the number of blocks in a packet
adx: define and use 2 new macro constants BLOCK_SIZE and BLOCK_SAMPLES
adx: check for unsupported ADX formats
adx: simplify encoding by using put_sbits()
adx: calculate correct LPC coeffs
adx: use 12-bit coefficients instead of 14-bit to avoid integer overflow
adx: simplify adx_decode() by using get_sbits() to read residual samples
adx: fix the data offset parsing in adx_decode_header()
adx: remove unneeded post-decode channel interleaving
adx: validate header values
adx: cosmetics: general pretty-printing and comment clean-up
adx: remove useless comments
...
Conflicts:
Changelog
libavcodec/cook.c
libavcodec/fraps.c
libavcodec/nuv.c
libavcodec/pthread.c
libavcodec/version.h
libavformat/Makefile
libavformat/version.h
Merged-by: Michael Niedermayer <michaelni@gmx.at>
As old bits are shifted out of the accumulator, they cause signed
overflows when they reach the end. Making the variable unsigned fixes
this.
Signed-off-by: Mans Rullgard <mans@mansr.com>
* qatar/master:
id3v2: fix doxy comment - 'machine byte order' makes no sense on char arrays
VC1: restore mistakenly removed code
twinvq: check output buffer size before decoding
twinvq: return an error when the packet size is too small
lavf: export some forgotten symbols with non-av prefixes.
swscale: update altivec yuv2planeX asm to new per-plane API.
swscale: make yuv2yuvX_10_sse2/avx 8/9/16-bits aware.
yuv2planeX10 SIMD
swscale: decide whether to use yuv2plane1/X on a per-plane basis.
swscale: reintroduce full precision in 16-bit output.
Split up yuv2yuvX functions
Split out yuv2yuv1 luma and chroma in order to make them generic DSP functions
lavc: replace references to deprecated AVCodecContext.error_recognition to use AVCodecContext.err_recognition
lavc: translate non-flag-based er options into flag-based ef options at codec open
add -err_filter AVOptions to access flag-based error recognition
h264_weight: initialize "height" function argument properly.
presets: spelling error in libvpx 1080p50_60
avplay: fix fullscreen behaviour with SDL 1.2.14 on Mac OS X
Conflicts:
ffplay.c
libavformat/libavformat.v
libswscale/swscale.c
libswscale/x86/swscale_template.c
tests/ref/lavfi/pixfmts_scale
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master: (35 commits)
flvdec: Do not call parse_keyframes_index with a NULL stream
libspeexdec: include system headers before local headers
libspeexdec: return meaningful error codes
libspeexdec: cosmetics: reindent
libspeexdec: decode one frame at a time.
swscale: fix signed shift overflows in ff_yuv2rgb_c_init_tables()
Move timefilter code from lavf to lavd.
mov: add support for hdvd and pgapmetadata atoms
mov: rename function _stik, some indentation cosmetics
mov: rename function _int8 to remove ambiguity, some indentation cosmetics
mov: parse the gnre atom
mp3on4: check for allocation failures in decode_init_mp3on4()
mp3on4: create a separate flush function for MP3onMP4.
mp3on4: ensure that the frame channel count does not exceed the codec channel count.
mp3on4: set channel layout
mp3on4: fix the output channel order
mp3on4: allocate temp buffer with av_malloc() instead of on the stack.
mp3on4: copy MPADSPContext from first context to all contexts.
fmtconvert: port float_to_int16_interleave() 2-channel x86 inline asm to yasm
fmtconvert: port int32_to_float_fmul_scalar() x86 inline asm to yasm
...
Conflicts:
libavcodec/arm/h264dsp_init_arm.c
libavcodec/h264.c
libavcodec/h264.h
libavcodec/h264_cabac.c
libavcodec/h264_cavlc.c
libavcodec/h264_ps.c
libavcodec/h264dsp_template.c
libavcodec/h264idct_template.c
libavcodec/h264pred.c
libavcodec/h264pred_template.c
libavcodec/x86/h264dsp_mmx.c
libavdevice/Makefile
libavdevice/jack_audio.c
libavformat/Makefile
libavformat/flvdec.c
libavformat/flvenc.c
libavutil/pixfmt.h
libswscale/utils.c
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master:
rtp: factorize dynamic payload type fallback
flvdec: Ignore the index if it's from a creator known to be different
cmdutils: move grow_array out of #if CONFIG_AVFILTER
avconv: actually set InputFile.rate_emu
ratecontrol: update last_qscale_for sooner
Fix unnecessary shift with 9/10bit vertical scaling
prores: mark prores as intra-only in libavformat/utils.c:is_intra_only()
prores: return more meaningful error values
prores: improve error message wording
prores: cosmetics: prettyprinting, drop useless parentheses
prores: lowercase AVCodec name entry
Conflicts:
cmdutils.c
libavcodec/proresdec_lgpl.c
tests/ref/lavfi/pixfmts_scale
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* qatar/master: (23 commits)
h264: hide reference frame errors unless requested
swscale: split hScale() function pointer into h[cy]Scale().
Move clipd macros to x86util.asm.
avconv: reindent.
avconv: rescue poor abused start_time global.
avconv: rescue poor abused recording_time global.
avconv: merge two loops in output_packet().
avconv: fix broken indentation.
avconv: get rid of the arbitrary MAX_FILES limit.
avconv: get rid of the output_streams_for_file vs. ost_table schizophrenia
avconv: add a wrapper for output AVFormatContexts and merge output_opts into it
avconv: make itsscale syntax consistent with other options.
avconv: factor out adding input streams.
avconv: Factorize combining auto vsync with format.
avconv: Factorize video resampling.
avconv: Don't unnecessarily convert ipts to a double.
ffmpeg: remove unsed variable nopts
RV3/4 parser: remove unused variable 'off'
add XMV demuxer
rmdec: parse FPS in RealMedia properly
...
Conflicts:
avconv.c
libavformat/version.h
libswscale/swscale.c
tests/ref/fate/lmlm4-demux
Merged-by: Michael Niedermayer <michaelni@gmx.at>
This allows using more specific implementations for chroma/luma, e.g.
we can make assumptions on filterSize being constant, thus avoiding
that test at runtime.
* qatar/master:
swscale: add dithering to yuv2yuvX_altivec_real
rv34: free+allocate buffer instead of reallocating it to preserve alignment
h264: add missing brackets.
swscale: use 15-bit intermediates for 9/10-bit scaling.
Merged-by: Michael Niedermayer <michaelni@gmx.at>
* sws_32bit_integration:
regtests/sws: update checksums for recent changes
sws: dont mess with XInc when the code needing it isnt used
sws: Fix chroma init for 32bit buffers.
swscale: error dithering for 16/9/10-bit to 8-bit.
swscale: fix overflow in 16-bit vertical scaling.
swscale: fix crash in 8-bpc bilinear output without alpha.
swscale: fix 16-bit scaling when output is 8-bits.
sws: fix non native endian 9-15 bit input with 16bit out
sws: disable scale16 when int32 is used
sws: fix rgb -> 16bit
sws: fix uv overwrite in 32bt
sws: fix gray16_1
sws:ix yuv2rgb48_1_c_template()
sws: fix 16/32 bug from merge
swscale: for >8bit scaling, read in native bit-depth.
swscale: fix another yuv range conversion overflow in 16bit scaling. (cherry picked from commit 81cc7d0bd1)
swscale: fix yuv range correction when using 16-bit scaling. (cherry picked from commit e0b8fff6c7)
swscale: implement >8bit scaling support.
Merged-by: Michael Niedermayer <michaelni@gmx.at>
We operated on 31-bits, but with e.g. lanczos scaling, values can
add up to beyond 0x80000000, thus leading to output of zeroes. Drop
one bit of precision fixes this.
