virtualenv/FFmpeg
1
0
Fork 0
mirror of https://github.com/FFmpeg/FFmpeg.git synced 2025-01-08 13:22:53 +02:00
Code Issues Releases Activity

fate: Add a target for listing failed tests

Browse Source 
If running tests with "make -j<N> fate", the execution will stop
after the first failing test. To get an overview of the whole
test suite, one rather would run "make -k -j<N> fate", which then
again buries the results about what tests actually failed further
up in the console log.

Add a target so one can run "make fate-list-failing", to see a list
of all tests that failed the last time they were executed.

Also add a companion target "fate-clear-reports" which removes all
the old test reports. (When executing a subset of tests, the report
files of all tests that aren't executed stay untouched. This also
allows getting rid of reports for tests that no longer are present
in the testsuite.)

Co-authored-by: Alexander Strasser <eclipse7@gmx.net>

Signed-off-by: Martin Storsjö <martin@martin.st>
This commit is contained in:
Martin Storsjö 2024-11-07 11:31:40 +02:00
parent 4b524649ff
commit 1e76bd2f39
3 changed files with 20 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
doc/build_system.txt
View File

@ -30,6 +30,13 @@ fate
fate-list
List all fate/regression test targets.
fate-list-failing
List the fate tests that failed the last time they were executed.
fate-clear-reports
Remove the test reports from previous test executions (getting rid of
potentially stale results from fate-list-failing).
install
Install headers, libraries and programs.

7
doc/fate.texi
View File

@ -208,6 +208,13 @@ Download/synchronize sample files to the configured samples directory.
@item fate-list
Will list all fate/regression test targets.
@item fate-list-failing
List the fate tests that failed the last time they were executed.
@item fate-clear-reports
Remove the test reports from previous test executions (getting rid of
potentially stale results from fate-list-failing).
@item fate
Run the FATE test suite (requires the fate-suite dataset).
@end table

6
tests/Makefile
View File

@ -313,6 +313,12 @@ $(FATE): $(FATE_UTILS:%=tests/%$(HOSTEXESUF)) | $(FATE_OUTDIRS)
fate-list:
@printf '%s\n' $(sort $(FATE))
fate-list-failing:
@! ls tests/data/fate/*.rep >/dev/null 2>&1 || awk -F: '$$2 != 0 { print "fate-" $$1 }' tests/data/fate/*.rep
fate-clear-reports:
@rm -f tests/data/fate/*.rep
coverage.info: TAG = LCOV
coverage.info:
$(M)lcov -q -d $(CURDIR) -b $(patsubst src%,./,$(SRC_LINK)) --capture | \