[ << Regression tests ] | [Top][Contents][Index][ ? ] | [ Programming work >> ] | ||
[ < Compiling regression tests ] | [ Up : Regression tests ] | [ Finding the cause of a regression > ] |
9.4 Regtest comparison
Before modified code is committed to master, a regression test comparison must be completed to ensure that the changes have not caused problems with previously working code. The comparison is made automatically upon compiling the regression test suite twice.
-
Before making changes, a baseline should be established by
running:
make test-baseline
- Make your changes, or apply the patch(es) to consider.
- Compile the source with ‘make’ as usual.
-
Check for unintentional changes to the regtests:
make check
After this has finished, a regression test comparison will be available at:
out/test-results/index.html
For each regression test that differs between the baseline and the changed code, a regression test entry will displayed. Ideally, the only changes would be the changes that you were working on. If regressions are introduced, they must be fixed before committing the code.
Note: The special regression test ‘test-output-distance.ly’ will always show up as a regression. This test changes each time it is run, and serves to verify that the regression tests have, in fact, run.
-
If you are happy with the results, then stop now.
If you want to continue programming, then make any additional code changes, and continue.
- Compile the source with ‘make’ as usual.
-
To re-check files that differed between the initial
‘make test-baseline’ and your post-changes
‘make check’, run:
make test-redo
This updates the regression list at ‘out/test-results/index.html’. It does not redo ‘test-output-distance.ly’.
- When all regressions have been resolved, the output list will be empty.
-
Once all regressions have been resolved, a final check should be completed
by running:
make test-clean make check
This cleans the results of the previous ‘make check’, then does the automatic regression comparison again.
[ << Regression tests ] | [Top][Contents][Index][ ? ] | [ Programming work >> ] | ||
[ < Compiling regression tests ] | [ Up : Regression tests ] | [ Finding the cause of a regression > ] |