omse goes from 0.03060703 (which fails for dct-test) to 0.01663750.
This also actually improve the error of decoding the sample generated
by fate-vsynth3-dnxhd1080i-10bit using simple_idct10 to FAANI, which
goes (when resampled to yuv422p) from:
stddev: 0.06 PSNR: 72.28 MAXDIFF: 1
to identical.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The current one, while correct, does not yield the best possible
results. The specificiations suggest another formula, which results
in quality gains in the decoded output from fate tests. This
justifies changing said formula.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
no changes in either standard deviation or PSNR is seen in any of the changed fate
cases
MSE changes from 0.05012422 to 0.04890000
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
* commit 'e816034a5fa131b13c4ad87bb0b5065b4f5697c6':
fate-seek: remove use of gnu make 3.82 only private modifier
fate: move vsynth reference files to their own directory
fate: move fate-acodec reference files to their own dir
configure: avplay now depends on avresample
fate: split dependencies for fate-seek tests
Conflicts:
configure
tests/fate/seek.mak
Merged-by: Michael Niedermayer <michaelni@gmx.at>
