This will allow decoding to md5 and doing a diff comparison to a reference
checksum instead of a fuzzy stddev or oneoff comparison.
Signed-off-by: Mans Rullgard <mans@mansr.com>
The output format is not always the same as the file extension,
which is sometimes required for correct probing. We can avoid
probing by specifying the format since it is already known.
This allows for testing floating-point audio encoders across different
platforms where exact comparisons are unreliable due to float rounding
differences.
Setting CMP=null uses stdout from the test as the diff record in
the report without further inspection. This can be useful when
the exit status of the test itself determines success while the
output is informative.
Signed-off-by: Mans Rullgard <mans@mansr.com>
All tests are run through the fate-run.sh script which already
sets up redirections. Using the outputs set there simplifies
things somewhat.
Signed-off-by: Mans Rullgard <mans@mansr.com>
The diff may provide useful information even if the command was
unsuccessful. The test is still treated as failed in this case.
Originally committed as revision 24353 to svn://svn.ffmpeg.org/ffmpeg/trunk
All tests use the provided helper functions so prepending $target_exec
and using eval is no longer required.
Originally committed as revision 24317 to svn://svn.ffmpeg.org/ffmpeg/trunk
This allows CMP=stddev in test rules. The test passes if the reported
stddev is <= the FUZZ value (default 1).
Originally committed as revision 24289 to svn://svn.ffmpeg.org/ffmpeg/trunk
This adds the 'oneoff' comparison method. It compares two s16le pcm
files with a max difference <= the fuzz factor treated as success.
Default fuzz is 1. On failure, the full tiny_psnr output is printed.
Originally committed as revision 24134 to svn://svn.ffmpeg.org/ffmpeg/trunk
This passes the following additional variables to the test script:
- CMP method to use comparing output with reference, default 'diff'
- REF reference file, default 'tests/ref/fate/${test}'
- FUZZ maximum fuzz factor in comparison, meaning depends on CMP
Originally committed as revision 24133 to svn://svn.ffmpeg.org/ffmpeg/trunk
This adds a "fate" make target which runs the full FATE test suite.
Individual tests can be run with "make fate-$testname".
The location of the FATE test samples must be specified with the
--samples=PATH option to configure.
The tests/fate-update.sh script regenerates the references files and
test list from the online FATE database. These are checked in since
generating them requires non-standard tools.
Originally committed as revision 22552 to svn://svn.ffmpeg.org/ffmpeg/trunk
