mirror of
https://github.com/FFmpeg/FFmpeg.git
synced 2024-11-21 10:55:51 +02:00
286ab878bd
If makeopts_fate is set, these makeopts are used for running the tests instead of the normal makeopts. If it isn't set, the normal makeopts variable is used as before. This is useful if remote testing on a lesser machine where a large number of parallel jobs might be undesireable, while wanting to speed up the build with many parallel processes. Signed-off-by: Martin Storsjö <martin@martin.st>
176 lines
5.1 KiB
Plaintext
176 lines
5.1 KiB
Plaintext
\input texinfo @c -*- texinfo -*-
|
|
|
|
@settitle FATE Automated Testing Environment
|
|
@titlepage
|
|
@center @titlefont{FATE Automated Testing Environment}
|
|
@end titlepage
|
|
|
|
@top
|
|
|
|
@contents
|
|
|
|
@chapter Introduction
|
|
|
|
FATE provides a regression test suite embedded within the Libav build system.
|
|
It can be run locally and optionally configured to send reports to a web
|
|
aggregator and viewer @url{http://fate.libav.org}.
|
|
|
|
It is advised to run FATE before submitting patches to the current codebase
|
|
and provide new tests when submitting patches to add additional features.
|
|
|
|
@chapter Running FATE
|
|
|
|
@section Samples and References
|
|
In order to run, FATE needs a large amount of data (samples and references)
|
|
that is provided separately from the actual source distribution.
|
|
|
|
To inform the build system about the test suite location, pass
|
|
@option{--samples=<path to the samples>} to @command{configure} or set the
|
|
@var{SAMPLES} Make variable or the @var{LIBAV_SAMPLES} environment variable
|
|
to a suitable value.
|
|
|
|
To use a custom wrapper to run the test, pass @option{--target-exec} to
|
|
@command{configure} or set the @var{TARGET_EXEC} Make variable.
|
|
|
|
The dataset is available through @command{rsync}, is possible to fetch
|
|
the current sample using the straight rsync command or through a specific
|
|
@ref{Makefile target}.
|
|
|
|
@example
|
|
# rsync -aL rsync://fate-suite.libav.org/fate-suite/ fate-suite
|
|
@end example
|
|
|
|
@example
|
|
# make fate-rsync SAMPLES=fate-suite
|
|
@end example
|
|
|
|
|
|
@chapter Manual Run
|
|
FATE regression test can be run through @command{make}.
|
|
Specific Makefile targets and Makefile variables are available:
|
|
|
|
@anchor{Makefile target}
|
|
@section FATE Makefile targets
|
|
|
|
@table @option
|
|
@item fate-list
|
|
List all fate/regression test targets.
|
|
|
|
@item fate-rsync
|
|
Shortcut to download the fate test samples to the specified test suite location.
|
|
|
|
@item fate
|
|
Run the FATE test suite (requires the fate-suite dataset).
|
|
@end table
|
|
|
|
@section FATE Makefile variables
|
|
@table @option
|
|
@item V
|
|
Verbosity level, can be set to 0, 1 or 2.
|
|
|
|
@table @option
|
|
@item 0
|
|
show just the test arguments
|
|
|
|
@item 1
|
|
show just the command used in the test
|
|
|
|
@item 2
|
|
show everything
|
|
@end table
|
|
|
|
@item SAMPLES
|
|
Specify or override the path to the FATE samples at make time, it has a
|
|
meaning only while running the regression tests.
|
|
|
|
@item THREADS
|
|
Specify how many threads to use while running regression tests, it is
|
|
quite useful to detect thread-related regressions.
|
|
|
|
@item THREAD_TYPE
|
|
Specify which threading strategy test, either @var{slice} or @var{frame},
|
|
by default @var{slice+frame}
|
|
|
|
@item CPUFLAGS
|
|
Specify a mask to be applied to autodetected CPU flags.
|
|
|
|
@item TARGET_EXEC
|
|
Specify or override the wrapper used to run the tests.
|
|
|
|
@item GEN
|
|
Set to @var{1} to generate the missing or mismatched references.
|
|
|
|
@item HWACCEL
|
|
Specify which hardware acceleration to use while running regression tests,
|
|
by default none is used.
|
|
|
|
@end table
|
|
|
|
@example
|
|
make V=1 SAMPLES=/var/fate/samples THREADS=2 CPUFLAGS=mmx fate
|
|
@end example
|
|
|
|
@chapter Automated Tests
|
|
In order to automatically testing specific configurations, e.g. multiple
|
|
compilers, @command{tests/fate.sh} is provided.
|
|
|
|
This shell script builds Libav, runs the regression tests and prepares
|
|
a report that can be sent to @url{http://fate.libav.org/} or directly
|
|
examined locally.
|
|
|
|
@section Testing Profiles
|
|
The configuration file passed to @command{fate.sh} is shell scripts as well.
|
|
|
|
It must provide at least a @var{slot} identifier, the @var{repo} from
|
|
which fetch the sources, the @var{samples} directory, a @var{workdir} with
|
|
enough space to build and run all the tests.
|
|
Optional submit command @var{fate_recv} and a @var{comment} to describe
|
|
the testing profile are available.
|
|
|
|
Additional optional parameter to tune the Libav building and reporting process
|
|
can be passed.
|
|
|
|
@example
|
|
slot= # some unique identifier
|
|
repo=git://git.libav.org/libav.git # the source repository
|
|
#branch=release/10 # the branch to test
|
|
samples=/path/to/fate/samples
|
|
workdir= # directory in which to do all the work
|
|
fate_recv="ssh -T fate@@fate.libav.org" # command to submit report
|
|
comment= # optional description
|
|
build_only= # set to "yes" for a compile-only instance that skips tests
|
|
|
|
# the following are optional and map to configure options
|
|
arch=
|
|
cpu=
|
|
cross_prefix=
|
|
as=
|
|
cc=
|
|
ld=
|
|
target_os=
|
|
sysroot=
|
|
target_exec=
|
|
target_path=
|
|
target_samples=
|
|
extra_cflags=
|
|
extra_ldflags=
|
|
extra_libs=
|
|
extra_conf= # extra configure options not covered above
|
|
|
|
#make= # name of GNU make if not 'make'
|
|
makeopts= # extra options passed to 'make'
|
|
#makeopts_fate= # extra options passed to 'make' when running tests,
|
|
# defaulting to makeopts above if this is not set
|
|
#tar= # command to create a tar archive from its arguments on
|
|
# stdout, defaults to 'tar c'
|
|
@end example
|
|
|
|
@section Special Instances
|
|
The @var{TARGET_EXEC} option provides a way to run FATE wrapped in
|
|
@command{valgrind}, @command{qemu-user} or @command{wine} or on remote targets
|
|
through @command{ssh}.
|
|
|
|
@section Submitting Reports
|
|
In order to send reports you need to create an @command{ssh} key and send it
|
|
to @email{root@@libav.org}.
|