First Stage – Alpha Version
The first stage of our release cycle as always is the release of our Alpha version used by some key partners and customers to adapt their additions to List & Label. The Alpha provides a first metric for the stability of the coming version. This milestone was hit by the end of July. Based on the Alpha feedback we were able to fine-tune some of the features and make it to the public beta program in early September.
By that time, we already had fixed some 500 tickets in the Redmine ticketing system which we’re using as bug management tool. BTW – if you’re also using Redmine make sure to check out the free reporting solution for Redmine we’ve donated to the community a couple of years ago. This is what our current progress looks like for the last 60 days:
As you can see, my will to keep the number of open tickets at or below 100 is a bit pathological ;). This is a fact that’s known by the whole team and everyone acts accordingly. We’re now starting to burn down rapidly to reach the zero-bug-count in time with the end of the beta phase.
Beta Phase – Code Freeze
In the second phase, the beta version is released to all our Enterprise subscribers. This offers ample opportunity to see if our latest bits still work nicely in your own application. We’re getting loads of useful feedback through this channel and even get an opportunity to shape the final touches of newly added features.
During the beta phase we’ve hit the code freeze for UI resources. Translation takes some time and we’re releasing List & Label in a number of languages (English, German, French, Italian, Spanish and Chinese). These “core” languages are being translated by a professional translation service. Other languages are provided by selected partners and customers and may become available shortly after release.
The actual code freeze is just after the end of the beta phase. We only fix killer bugs after this milestone, and only on a peer-peer-review base.
Of course we run an intense and thorough in-house beta test in addition to our external beta program. This internal process is repeated for each sprint in our agile planning process to enable incremental releases at any stage.
Automated Builds and Tests
Manual testing is complemented by automated tests. Both UI and reporting engine are subject to automated testing for most of our nightly builds. The builds and tests are fully automated by our CI server – we use TeamCity for that purpose. We use commercial tools as well as validation tools we’ve created just for this purpose. The main reporting engine tests consist of several hundred reports which are run by the engine and validated against references. Each deviation from the reference files is automatically reported and raises yet another ticket in Redmine. The QA process also includes performance tests to make sure there are no latencies induced by accident. This way we have been able to provide high quality releases with the most recent versions.
This is a live view of our TeamCity project:
As you can see, the last List & Label 26 Core build failed. The reason was a failed regression test that can easily be verified by looking at the build’s artifacts:
The file itself is a “before/after” comparison of the last build’s output vs. the stored reference. To get an impression of the number of tests, here’s a scroll over the results of a typical build:
Wrap-Up
So this is how we’re heading towards our 26th version already. Despite the pandemic craze and everyone still working from home, we’ve been able to stick to our usual processes, although our standups now have a different look and feel:
We’ve come a long way since the initial release of List & Label back in 1992 when the tool was born out of the need for a professional reporting component to use with combit’s CRM applications. Our great team has grown through the years to offer professional services you’ve come to expect from us. I’d like to take this opportunity to thank everyone involved in-house as well as all of our faithful customers. It’s been a pleasure to work with you through the last 25 versions, and I’m looking forward to yet another great release very soon.
How do you release a new software version in your company? We appreciate your comments.
Leading the development at combit as Managing Director. Microsoft .NET enthusiast driving innovation & agile project management. Used to be a physicist in my first life. I love hiking and vanlife.
