9.2 Precompiled regression tests

Regression test output

As part of the release process, the regression tests are run for every LilyPond release. Full regression test output is available for every stable version and the most recent development version.

Regression test output is available in HTML and PDF format. Links to the regression test output are available at the developer’s resources page for the version of interest.

The latest stable version of the regtests is found at:

http://lilypond.org/doc/stable/input/regression/collated-files.html

The latest development version of the regtests is found at:

http://lilypond.org/doc/latest/input/regression/collated-files.html

Regression test comparison

Each time a new version is released, the regtests are compiled and the output is automatically compared with the output of the previous release. The result of these comparisons is archived online:

http://lilypond.org/test/

Checking these pages is a very important task for the LilyPond project. You are invited to report anything that looks broken, or any case where the output quality is not on par with the previous release, as described in Bug reports.

Note: The special regression test ‘test-output-distance.ly’ will always show up as a regression. This test changes each time it is run, and serves to verify that the regression tests have, in fact, run.

What to look for

The test comparison shows all of the changes that occurred between the current release and the prior release. Each test that has a significant (noticeable) difference in output is displayed, with the old version on the left and the new version on the right.

Some of the small changes can be ignored (slightly different slur shapes, small variations in note spacing), but this is not always the case: sometimes even the smallest change means that something is wrong. To help in distinguishing these cases, we use bigger staff size when small differences matter.

Staff size 30 generally means "pay extra attention to details". Staff size 40 (two times bigger than default size) or more means that the regtest is about the details.

Staff size smaller than default doesn’t mean anything.

Regression tests whose output is the same for both versions are not shown in the test comparison.

Note: The automatic comparison of the regtests checks the LilyPond bounding boxes. This means that Ghostscript changes and changes in lyrics or text are not found.


LilyPond — Contributor’s Guide v2.18.2 (stable-branch).