If running tests with "make -j<N> fate", the execution will stop after the first failing test. To get an overview of the whole test suite, one rather would run "make -k -j<N> fate", which then again buries the results about what tests actually failed further up in the console log. Add a target so one can run "make fate-list-failing", to see a list of all tests that failed the last time they were executed. Also add a companion target "fate-clear-reports" which removes all the old test reports. (When executing a subset of tests, the report files of all tests that aren't executed stay untouched. This also allows getting rid of reports for tests that no longer are present in the testsuite.) Co-authored-by: Alexander Strasser <eclipse7@gmx.net>
Showing the changes to doc (2 of 3 files). Show all
Unified · Split
| @@ -30,6 +30,13 @@ fate · 29 unmodified lines | |||
| 30 | fate-list | 30 | fate-list |
| 31 | List all fate/regression test targets. | 31 | List all fate/regression test targets. |
| 32 | 32 | ||
| 33 | fate-list-failing | ||
| 34 | List the fate tests that failed the last time they were executed. | ||
| 35 | |||
| 36 | fate-clear-reports | ||
| 37 | Remove the test reports from previous test executions (getting rid of | ||
| 38 | potentially stale results from fate-list-failing). | ||
| 39 | |||
| 33 | install | 40 | install |
| 34 | Install headers, libraries and programs. | 41 | Install headers, libraries and programs. |
| 35 | 42 | ||
| @@ -208,6 +208,13 @@ Download/synchronize sample files to the configured samples directory. · 207 unmodified lines | |||
| 208 | @item fate-list | 208 | @item fate-list |
| 209 | Will list all fate/regression test targets. | 209 | Will list all fate/regression test targets. |
| 210 | 210 | ||
| 211 | @item fate-list-failing | ||
| 212 | List the fate tests that failed the last time they were executed. | ||
| 213 | |||
| 214 | @item fate-clear-reports | ||
| 215 | Remove the test reports from previous test executions (getting rid of | ||
| 216 | potentially stale results from fate-list-failing). | ||
| 217 | |||
| 211 | @item fate | 218 | @item fate |
| 212 | Run the FATE test suite (requires the fate-suite dataset). | 219 | Run the FATE test suite (requires the fate-suite dataset). |
| 213 | @end table | 220 | @end table |