Martin Storsjö aad0e26f93 checkasm: aacpsdsp: Tolerate extra intermediate precision in stereo_interpolate ...

The stereo_interpolate functions add h_step to the values h
BUF_SIZE times. Within the stereo_interpolate C functions, the
values h (h0-h3, h00-h13) are declared as local float variables,
but the compiler is free to keep them in a register with extra
precision.

If the accumulation is rounded to 32 bit float precision after
each step, the less significant bits of h_step end up ignored
and the sum can deviate, affecting the end result more than
the currently set EPS.

By clearing the log2(BUF_SIZE) lower bits of h_step, we make sure
that the accumulation shouldn't differ significantly, regardless
of any extra precision in the accmulating register/variable.

This fixes the aacpsdsp checkasm test when built with clang for
mingw/x86_32.

Signed-off-by: Martin Storsjö <martin@martin.st>

2019-12-18 15:15:29 +02:00

..

aarch64

Merge commit 'd05c9cde0e87c23ca42957646bea483dfc09d6bf'

2017-10-30 21:03:50 -03:00

arm

…

x86

…

.gitignore

…

aacpsdsp.c

checkasm: aacpsdsp: Tolerate extra intermediate precision in stereo_interpolate

2019-12-18 15:15:29 +02:00

af_afir.c

checkasm: af_afir: Use a dynamic tolerance depending on values

2019-12-12 23:57:08 +02:00

alacdsp.c

…

audiodsp.c

…

blockdsp.c

libavcodec/blockdsp : add AVX version

2017-10-03 19:47:37 -03:00

bswapdsp.c

…

checkasm.c

checkasm/vf_eq: add test for vf_eq

2019-09-26 08:10:31 +08:00

checkasm.h

checkasm/vf_eq: add test for vf_eq

2019-09-26 08:10:31 +08:00

exrdsp.c

libavcodec/exr : add x86 SIMD for predictor

2017-10-01 17:35:30 -03:00

fixed_dsp.c

…

flacdsp.c

…

float_dsp.c

checkasm: float_dsp: Scale FLT/DBL_EPSILON sufficiently when comparing

2019-12-11 22:19:45 +02:00

fmtconvert.c

…

g722dsp.c

checkasm: add a g722dsp test

2017-07-13 17:00:19 -03:00

h264dsp.c

Merge commit 'f8abf7d4dfa0504f7f65e4f1fd9d22e01cb371cc'

2019-03-14 16:31:41 -03:00

h264pred.c

…

h264qpel.c

…

hevc_add_res.c

Merge commit 'e00db9f78bb475ed5103364f61892f4e75ef89ba'

2017-10-28 18:18:41 -03:00

hevc_idct.c

checkasm/hevc_idct : update test bit depth from 8 9 and 10 to 8 10 and 12

2018-03-19 00:56:01 -03:00

hevc_sao.c

checkasm/hevc_sao : add hevc_sao for checkasm

2018-03-07 23:53:32 -03:00

huffyuvdsp.c

checkasm : add test for huffyuvdsp add_int16

2017-11-21 09:41:42 +01:00

jpeg2000dsp.c

checkasm/jpeg2000dsp: add test for ict_float

2017-11-20 18:33:57 -03:00

llviddsp.c

checkasm/llviddsp: fix mixed code and declarations

2017-12-10 00:51:35 -03:00

llviddspenc.c

checkasm : add test for losslessvideoencdsp for diff bytes and sub_left_pred

2018-01-28 20:23:16 +01:00

Makefile

checkasm/vf_eq: add test for vf_eq

2019-09-26 08:10:31 +08:00

opusdsp.c

checkasm/opusdsp: declare opus_deemphasis as a function returning a float

2019-09-18 11:42:06 -03:00

pixblockdsp.c

…

sbrdsp.c

checkasm: use declare_func_float() in sbrdsp sum_square test

2017-07-04 23:02:57 -03:00

sw_rgb.c

checkasm/sw_rgb: fix the function declaration warning

2018-05-10 19:28:51 +08:00

synth_filter.c

…

utvideodsp.c

checkasm/utvideo : be more explicit to the WIDTH_PADDED define

2017-12-01 21:08:07 +01:00

v210dec.c

checkasm: add test for v210dec

2019-05-02 19:21:37 +02:00

v210enc.c

…

vf_blend.c

lavfi/tests: Fix 16-bit vf_blend test to avoid memory not aligned to 2 bytes

2018-05-30 02:42:10 +02:00

vf_colorspace.c

…

vf_eq.c

checkasm/vf_eq: add test for vf_eq

2019-09-26 08:10:31 +08:00

vf_gblur.c

checkasm/vf_gblur: add test for horiz_slice simd

2019-06-12 08:54:05 +08:00

vf_hflip.c

checkasm/vf_hflip : add test for vf_hflip byte and short simd

2017-12-13 11:34:29 +01:00

vf_nlmeans.c

tests/checkasm/nlmeans: fix invalid read/write on ii buffer

2018-05-18 19:12:11 +02:00

vf_threshold.c

checkasm/vf_threshold : add test for threshold16

2017-12-09 14:47:13 +01:00

videodsp.c

…

vp8dsp.c

…

vp9dsp.c

…