Changing the Python release cadence

There has been discussion about the release cadence of Python for a couple of years now. The 18-month cycle between major releases of the language is seen by some core developers as causing too much delay in getting new features into the hands of users. Now there are two competing proposals for ways to shorten that cycle, either to one year or by creating a rolling-release model. In general, the steering council has seemed inclined toward making some kind of release-cycle change—one of those Python Enhancement Proposals (PEPs) may well form the basis of Python's release cadence moving forward.

Some history

The idea had undoubtedly been discussed before, but the current push began with a 2018 Python Language Summit presentation on changing to a yearly major release that was given by Łukasz Langa—the release manager for Python 3.8 and 3.9. Roughly a year later, he proposed doubling the release frequency in the first posting of PEP 596 ("Python 3.9 Release Schedule"). That would mean a major Python release every nine months. Around the time that PEP 596 was being discussed, Nick Coghlan created PEP 598 ("Introducing minor feature releases"), which was meant to speed up feature releases; it was an early progenitor of the rolling-release model.

Nine months, as proposed in PEP 596, was too fast of a cycle for many, but the steering council did indicate support for a change of the release cycle; a one-year cycle was seemingly a better fit for most commenters. The council thought that the 3.9 schedule PEP was the wrong place to propose this kind of change, however; it suggested that a new PEP be created to propose the cadence change. To that end, Langa created PEP 602 ("Annual Release Cycle for Python"), returning to his original one-year cadence that was better received than the shorter cycle.

In the discussion of the PEP, Steve Dower suggested an even faster pace of continuous alpha releases. Coghlan was intrigued by Dower's proposal, so they teamed up to create PEP 605 ("A rolling feature release stream for CPython"). When it was posted for discussion, Coghlan noted that he had withdrawn his earlier proposal in favor of the new one.

PEP 605 itself is something of a wall of text, but it is also accompanied by a lengthy design discussion posting as well. As part of the discussion on both PEPs, though, it became clear that there was some confusion about the background and, in particular, what problems they were both trying to solve. That, naturally, led to yet another PEP, which was authored by Langa, Dower, and Coghlan: PEP 607 ("Reducing CPython's Feature Delivery Latency"). Looking at that PEP would seem to be a good starting point for disentangling the various pieces here.

Rationale

PEP 607 lays out a number of reasons for "delivering smaller collections of features to Python’s users more frequently"; it also describes the reasoning for making major releases at roughly the same time of year (every year for PEP 602 and every other year for PEP 605). The size of the batches of features that are delivered is one of the areas that the PEPs are trying to address. Right now, roughly 18 months worth of changes are delivered at once, which increases the chances that there will be problems for users while also complicating the process of tracking down where any problem came from because there are more interacting features to wade through. PEP 602 simply reduces the 18-month window to a 12-month window, while PEP 605 will regularly deliver new features every two months for "a subset of Python’s user base that opts in to running a rolling stream of beta releases".

Then there is the problem of latency between the development of features and their delivery to users. Given that missing a particular major release means that a feature will languish for another 18 months, developers may push changes before they are truly ready. Similarly, features that could be in the hands of users are simply hanging around in the repository unused. Once again, the PEP 602 solution is simply shortening the cycle;

Regularity in the release schedule is helpful to coordinate with other projects (e.g. Linux distributions) and events in the Python world (e.g. PyCon US, core developer sprints), but it is also helpful to regular users—and developers. Rather than consulting a calendar or PEP, the dates of interest will be predictable: major releases in October every year for PEP 602 or August of every other year for PEP 605. Other dates (feature freezes and the like) can be derived from that baseline.

It is also generally the case that major changes in the CPython interpreter or the standard library are not actually tested by users until after a major release. That means the feedback on new features is limited until after they have been solidified, requiring a multi-year deprecation cycle to correct any problems. Both of the PEPs are targeting ways to get feedback earlier in the cycle. PEP 602 would start the alpha period for a new release immediately after the major release (i.e. October of each year), while PEP 605 has a wider scope:

There are, of course, risks when changing the release cadence, some of which both PEPs have considered. Users and distributors may either skip some major releases or may update every time the language does. PEP 602 may cause some of the "skippers" to skip two releases instead of just one, but there is a prolonged deprecation cycle that will hopefully ensure that doing so will not miss out on deprecation warnings. For non-skippers, the support periods will remain roughly the same as they are today under PEP 602 (roughly 18 months of full support plus 42 months of security-fix-only support).

PEP 605 takes a different approach, obviously. Skippers may simply end up moving to the non-skipper bucket since there will be 24 months between major releases. Non-skippers should find that each release will have prolonged full support (roughly 24 months) with a shorter period of bug-fix-only support (36 months) added on. In addition, the idea is that the two-month beta releases will be more widely used, thus tested, which will lead to "more stable final releases with wider ecosystem support at launch".

Meanwhile, those who follow the CPython master branch should see no real impact, though PEP 605 is trying to entice them into switching to a better tested rolling beta stream. Third-party libraries are obviously a major component of the Python ecosystem; these include things like NumPy, SciPy, Django, and all manner of libraries available in the Python Package Index (PyPI). For those, the major pain point is the number of different Python versions that need to be simultaneously supported. This is an area where it would seem that PEP 605 has a clear advantage in that there will be fewer major releases made. The overall support period for a given release is the same in each PEP (60 months), so there will be roughly twice as many active major releases under PEP 602, though it will only be a minor increase from the current situation.

Which is not to say that PEP 605 is without flaws, though. It is a big change from what the Python ecosystem is used to, while PEP 602 is a fairly straightforward shortening of the existing state of affairs. The main complaint in the PEP 602 discussion thread (aside from those that spawned PEP 605) was about the October date, which is poorly situated for Fedora. For the most part, commenters were either in favor or neutral. In fact, most of the comments were about the still-under-discussion ideas that eventually resulted in PEP 605.

Rolling ... or roiling?

In the PEP 605 discussion, things were more mixed. Langa, unsurprisingly, was opposed to the approach. Paul Moore was skeptical that the larger projects in the ecosystem would participate in the rolling releases, leading to simply a slower release cycle. The term "beta" was discussed, as well, with some wondering if it had baggage that would worry potential adopters. But Coghlan said that there may be some confusion about what he and Dowers are aiming for:

PEP 605 proposes that the two-month rolling releases be broken up into alpha releases, which could contain changes to the CPython ABI, and beta releases that would maintain ABI compatibility. That would mean that C-language extensions might need to be reworked and rebuilt for the alpha releases, but should not need to be for the beta releases. The "alpha" and "beta" terms are being used rather differently than most expect. It also leads to a rather unexpected pattern of release numbers, both of which may be contributing to the confusion. For example, the first 3.9 rolling release would presumably be an alpha, thus 3.9.0a1; after that, there would likely be betas for the next few bimonthly releases: 3.9.0b2, 3.9.0b3, ... If there were an ABI breaking change for the fourth release, though, it would be 3.9.0a4.

That unconventional pattern of release versions was considered potentially confusing by several in the discussion. Fedora Python maintainer Miro Hrončok noted that the distribution would have to hack things to make the versions sort correctly. Moore agreed:

The discussion continues as of this writing. At some point, the council will need to decide between the two—or to stay with the status quo. As council member Brett Cannon said in early October, there will be a need to make a decision fairly soon in order to nail down the 3.9 schedule, which effectively started running on October 14 with the release of Python 3.8. There is also an upcoming election for a new steering council, which will "basically freeze up PEP decision making until mid-December". It should be noted that Coghlan is also a member of the council, but seems likely to recuse himself from a decision on a PEP that he has co-authored.

Though the rolling model seems like a huge departure, it does really only affect a self-selected subset of the Python community. The 24-month major release cycle will certainly have more widespread effects, but is not a huge stretch. In fact, while Python has been released on an 18-month cycle for some time, it is documented to be an 18-24-month cycle, so PEP 605 does not really make any changes to that—just to "recent" practice. One might guess that the council will lean toward the least "radical sounding" proposal, thus PEP 602, but one never knows. Langa is the Python 3.9 release manager, which may also be a factor, but certainly PEP 605 and the status quo should not be counted out. It will be interesting to see how it all plays out.