14.6.4.4 GOP-PROP 4 - lessons from 2.14
History
A brief history of releases:
| date (YYYY-MM-DD) | version | comment |
|---|---|---|
| 2008-10-28 | 2.11.63 | nobody checking regtests |
| 2008-11-17 | 2.11.64 | |
| 2008-11-29 | 2.11.65 | |
| 2008-12-23 | 2.12.0 | |
| 2009-01-01 | somewhere around here, Graham becomes officially release manager, but Han-Wen still builds the actual releases | |
| 2009-01-01 | 2.12.1 | |
| 2009-01-25 | 2.12.2 | |
| 2009-02-28 | 2.13.0 | |
| 2009-06-01 | 2.13.1 | note jump in time! |
| 2009-06-27 | 2.13.2 | first Graham release? |
| 2009-07-03 | 2.13.3 | |
| 2009-09-09 | Graham arrives in Glasgow, gets a powerful desktop computer, and begins serious work on GUB (sending bug reports to Jan). It takes approximately 100 hours until GUB is stable enough to make regular releases. | |
| 2009-09-24 | 2.13.4 | |
| 2009-10-02 | 2.13.5 | |
| 2009-10-22 | 2.13.6 | |
| 2009-11-05 | 2.13.7 | |
| ... | ||
| 2010-01-13 | 2.12.3 | |
| ... | ||
| 2010-03-19 | 2.13.16 | Bug squad starts doing a few regtest comparisons, but IIRC the effort dies out after a few weeks (BLUE) |
| ... | ||
| 2010-08-04 | 2.13.29 | Phil starts checking regtests (BLUE) |
| ... | ||
| 2011-01-12 | 2.13.46 | release candidate 1 (GREEN) |
| ... | ||
| 2011-05-30 | 2.13.63 | release candidate 7 (GREEN) |
| 2011-06-06 | 2.14.0 |
Carl’s analysis of the bugs
A ‘csv’ spreadsheet is available.
https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00852.html |
lilypond-issues-analysis.csv lilypond-issues-analysis-trim-duplicates.csv
There 148 issues marked with Priority=Critical in the tracker.
I’ve done an analysis, and it looks to me like there was initially a backlog of critical issues that weren’t fixed, and little work was being done to eliminate critical issues.
Somewhere about 2010-08-01, critical issues started to disappear, but occasional new ones appeared.
There were a couple of major changes that introduced unanticipated regressions (new spacing code, beam collision avoidance). These produced more than the expected number of regressions.
It appears to me that we didn’t really get serious about eliminating critical bugs until about 2010-06-15 or so. After that point, the number of critical bugs more-or-less steadily decreased until we got to a release candidate.
Of particular interest, the first release candidate of 2.14 was released on 2011-01-12. Over the next 10 days, about a dozen bugs were reported and fixed. Release candidate 2 came out on 2011-02-09. No surge of bugs occurred with this release. Candidate 3 came out on 2011-03-13; we got 2 bugs per week. Candidate 4 came out on 2011-03-29; 2 new bugs. Candidate 6 came out on 2011-04-07. We got a couple of bugs per week.
Notes, commentary, and opinions
Han-Wen: Overall, I think this cycle took too long Mike: I agree Graham: +1
Discussions
https://lists.gnu.org/archive/html/lilypond-devel/2011-06/msg00797.html https://lists.gnu.org/archive/html/lilypond-devel/2011-07/msg00364.html |