Debian STS: Short Term Support

In another of my frequent late-night bouts with insomnia, I started thinking about the intersection of a number of different issues facing Debian today, both from a user point of view and a developer point of view.

Debian has a reputation for shipping “stale” software. Versions in the stable branch are often significantly behind the latest development upstream. Debian’s policy here has been that this is fine, our goal is to ship something stable, not something bleeding edge. Unofficially, our response to users is: If you need bleeding edge software, Debian may not be for you. Officially, we have no response to users who want fresher software.

Debian also has a problem with a lack of manpower. I believe that part of why we have a hard time attracting contributors is our reputation for stale software. It might be worth it for us to consider changes to our approach to releases.

What about running testing?

People who want newer software often look to Debian’s testing branch as a possible solution. It’s tempting, as it’s a dynamically generated release based on unstable, so it should be quite current. In practice, it’s not at all uncommon to find people running testing, and in fact I’m running it right now on the ThinkPad on which this is being typed. However, testing comes with a glaring issue: a lack of timely security support. Security updates must still propagate through unstable, and this can take some time. They can be held up by dependencies, library transitions, or other factors. Nearly every list of “best practices for computer security” lists keeping software up-to-date at or near the top of most important steps to take to safely use networked computer. Debian’s testing branch makes this very difficult, especially when faced with a zero-day with potential for real-world exploit.

What about stable-backports?

Stable backports is both better and worse than testing. It’s better in that it allows you to run a system comprised mainly of packages from the stable branch, which receive updates from the security team in a timely manner. However, it’s worse in that the packages from the backports repository incur an additional delay. The expectation around backports is that a package migrates naturally from unstable to testing, and then requires a maintainer to upload a new package based on the version in testing specifically targeted at stable backports. The migration can potentially be bypassed, and we used to have a mechanism for announcing the availability of security updates for the stable backports archive, but it has gone unused for several years now. The documentation describes a workflow for posting security updates that involves creating a ticket in Debian’s RT system, which is going to be quite foreign to most people. News from mid 2019 suggests that this process might change, but nothing appears to have come of this in over a year, and we still haven’t seen a proper security advisory for stable backports in years.

Looking to LTS for ideas

The Long-Term Support project is an “alternative” branch of Debian, maintained outside the normal stable release infrastructure. It’s stable, and expected to behave that way, but it’s not supported by the stable security team or release team. LTS provides a framework for providing security updates via targeted uploads by a team of interested individuals working outside the structure of the existing stable releases. This project seems to be quite active (how much of this is because at least some members are being paid?), and as of this writing has actually published more security advisories in the past month than the stable security team has published for the current stable branch. This is also interesting in that the software in LTS is quite old, first appearing in a Debian stable release in 2017.

LTS is particularly interesting here as it’s an example an initiative within the Debian community taken specifically to address user needs. For some of our users, remaining on an old release is a perfectly valid thing for them to do, and we recognize this and support them in doing so.

Debian Short-Term Support

So, what would it take to create an “LTS-like” initiative in the other direction? Instead of providing ongoing support for ancient versions of software that previously comprised a stable release, could we build a distribution branch based on something that hasn’t yet reached stable? What would that look like? How would it fit in the existing unstable→testing migration process? What impact would it have on the size of the archive? Would we want a rolling release, or discrete releases? If the latter, how many would we want between proper stable releases?

The security tracker already tracks outstanding issues in unstable and testing, and can even show issues that have been fixed in unstable but haven’t yet propagated to testing.

If we want a rolling release, maybe we could just open up the testing-security repository more broadly? There was once a testing security team, which IIRC was chartered to publish updated packages directly to testing security, along with associated security advisory. Based on the mailing list history, that effort seems to have shut down around the time of the squeeze (Debian 6.0) release in early 2011. Would it be worth resurrecting it? We’ve probably got much of the infrastructure required in place already, since it previously existed.

Personally I’m not really a fan of a pure rolling release. I’d rather see a light-weight release. Maybe a snapshot of testing that gets just a date, not a Toy Story codename. Probably skip building a dedicated installer for it. Upgrade from stable or use a d-i snapshot from testing if needed. This mini release is supported until the next one comes out, maybe 6 or 8 months later. By supported, I mean that the “Short Term Release” team is responsible for it. They can upload security or other critical bug fixes directly to a dedicated repository. When the next STS snapshot is released, packages in the updates repository are either archived, if they’re a lower version than the one in the new mini release, or rebuilt against the new mini release and preserved.

Using some of the same mechanisms as the LTS release, we’d need

1. Something to take the place of oldstable, that is the base release against which updates are released. This could be something that effectively maps to a date snapshot served by http://snapshot.debian.org/. (Snapshot itself could not currently handle the increased load, as I understand it, but conceptually it’s similar.)

2. Something to take the place of the dist/updates apt repository that hosts the packages that are updated.

In theory, if the infrastructure could support hose things, then we could in effect generate a mini release at any time based on a snapshot. I wonder if this could start as something totally unofficial; mirror an arbitrary testing snapshot and provide a place for interested people to publish package updates.

Not a proposal, nor a criticism

To be clear, I don’t really intend this as a proposal; It’s really half-baked. Maybe these ideas have already been considered and dismissed. I don’t know if people would be interested in working on such a project, and I’m not nearly familiar enough with the Debian archive tooling to even make a guess as to how hard it would be to implement much of it. I’m just posting some ideas that I came up with while pondering something that, from my perspective, is an area where Debian is clearly failing to meet the needs of some of our users. We know Debian is a popular and respected Linux distribution, and we know people value our stability. However, we also know that people like running Fedora and Ubuntu’s non-LTS releases. People like Arch Linux. Not just “end-users”, but also the people developing the software shipped by the distros themselves. There are a lot of potential contributors to Debian who are kept away by our unwillingness to provide a distro offering both fresh software and security support. I think that we could attract more people to the Debian community if we could provide a solution for these people, and that would ultimately be good for everybody.

Also, please don’t interpret this as being critical of the release team, the stable security team, or any other team or individual in Debian. I’m sharing this because I think there are opportunities for Debian to improve how we serve our users, not because I think anybody is doing anything wrong.

With all that said, though, let me know if you find the ideas interesting. If you think they’re crazy, you can tell me that, too. I’ll probably agree with you.

Tags: