Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 338739 - sys-kernel/vanilla-sources: stabilisation policy
Summary: sys-kernel/vanilla-sources: stabilisation policy
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: High enhancement (vote)
Assignee: Gentoo Kernel Bug Wranglers and Kernel Maintainers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2010-09-25 22:40 UTC by Kerin Millar
Modified: 2011-05-03 15:30 UTC (History)
15 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kerin Millar 2010-09-25 22:40:49 UTC
Firstly, consider that:

  * 2.6.32 is the current long-term stable branch [1]
  * Traditionally, the only other branch that can be expected to receive fixes
    is whichever happens to be current (2.6.35 at the present time) [2]

Given the above, the current situation in terms of which ebuilds are keyworded stable does not seem rational. For instance, these are keyworded stable:

  vanilla-sources-2.6.23.17
  vanilla-sources-2.6.26.7
  vanilla-sources-2.6.27.10
  vanilla-sources-2.6.29.4
  vanilla-sources-2.6.30.9
  vanilla-sources-2.6.31.12
  vanilla-sources-2.6.31.6
  vanilla-sources-2.6.34

Essentially, a swathe of obsolete releases (which are not safe to use in production for security reasons alone) are keyworded stable. 2.6.34 is the latest version keyworded stable, yet the current stable patch release is 2.6.34.7. 2.6.32 is nowhere to be seen, despite being long-term stable, nor is 2.6.35, despite being the most recent target for stable patch releases.

I'm assuming that arch testing will never occur for vanilla-sources releases and I certainly would not expect that. My suggestion for a stabilisation policy is as follows:

  1) All new releases of 2.6.32.x are automatically keyworded stable
  2) Either:
     a) All other releases are keyworded unstable
     b) The latest vanilla-sources release is also automatically keyworded
        stable

This would ensure that any user who chooses to install vanilla-sources will be getting a release which contains the latest bug fixes. I cannot see any logic in having things any other way. Any thoughts?

[1] http://www.kroah.com/log/linux/stable-status-01-2010.html
[2] For instance, new stable patches 2.6.32.22 and 2.6.35.5 were recently
    released simultaneously whereas 2.6.34 now stagnates.
Comment 1 George Kadianakis (RETIRED) gentoo-dev 2010-09-26 23:01:22 UTC
I agree with Kerin's idea on following the Linux Kernel's upstream stabilization steps.
Keeping only 2.6.32 kernels as stable is gonna give vanilla-sources users a secure and stable kernel. 
Concerned users that are looking for cutting-edge kernels can easily append the package to their package.keywords file and receive the latest kernels.

I'm aware that following the LTS release could be considered an overkill by some, but when dealing with core components like the Kernel, I believe that one has to actually offer *stable* products to users that use the stable branch of a distribution. 
Comment 2 Kerin Millar 2010-09-26 23:16:06 UTC
Thanks, George. As it currently stands, those who drop sys-kernel/vanilla-sources in package.keywords are - generally speaking - going to be getting a better deal than those who do not (notwithstanding rare blips such as bug 334341). Such situations raise the question as to whether the distinction between arch and ~arch keywords is correlating at all with the literal stability and degree of production-worthiness of any given release. Without a shadow of a doubt, the 2.6.32.x series constitutes the safest set of kernels in existence. The only other safe choice is typically the stable kernel-du-jour but, as has been seen, there are occasional issues caused by the dissonance between new kernels and pesky old userland (udev is a leopard that is rather too keen on changing its spots also and has been the subject of such bugs in the past).

The bottom line is that I think a user should have the confidence that, upon installing something deemed 'stable', it should, in fact, be stable and safe.
Comment 3 Mike Pagano gentoo-dev 2010-09-29 13:01:44 UTC
I am in agreement here, but what we really need is the arch teams to agree to this or at least voice their opinions on a better solution.

I want Gentoo user's to always be presented with a safe kernel rather than wait for the kernel team to file stabilization bugs and arch teams to stabilize.

This makes sense for the long term support kernel as arch teams have already at least keyworded the version saying it "works" as opposed to newer releases of kernels which sometimes have issues.

Adding arch teams for their input and qa, also. I would like *one* policy for all archs, as opposed to remembering which policy applies to which arch.

Arch teams? 
Comment 4 Anthony Basile gentoo-dev 2010-09-29 22:45:08 UTC
(In reply to comment #0)
> My suggestion for a stabilisation policy
> is as follows:
> 
>   1) All new releases of 2.6.32.x are automatically keyworded stable
>   2) Either:
>      a) All other releases are keyworded unstable
>      b) The latest vanilla-sources release is also automatically keyworded
>         stable

First this is a good policy and I support it.  We may want to think about policies on stabilizing derivative kernels after we settle this one.

There is a dilemma in point #2 --- I'm seeing it with the users of hardened-sources.  Should we keyword stable *only* the latest 2.6.32.x or the latest of each 2.6.x.y branch?  The conservative approach would be only 2.6.32.x.  But there are many features in 2.6.33.x and above which are useful both on servers (eg. drbd) and on workstation (eg. nouveau).

My preference is to mark *all* latest 2.6.x.y that are announced as stable upstream (on kernel.org) as stable for all arches on gentoo.  And mark all older version as unstable.  Users can then locally mask whichever branches they don't want.

If we agree on this, we should also send out a news item so users are aware of the policy and briefly document "how to follow a particular stable branch" on kernel-upgrade.xml.

Comment 5 Tobias Klausmann gentoo-dev 2010-10-01 07:24:04 UTC
While I generally see the point of always having the latest v-s stable, this can cause huge problems for people on off-mainstream arches.

For example, during the last six months there have been no fewer than three occasions where a released kernel did not work correctly on alpha. Two of them were due to restructuring inside the kernel (syscalls going away, IRQ handling being done differently) and one due to userspace apps (udev) expecting behaviour that this arch didn't support at that point in time. In all cases, the kernel built but got stuck on bootup (or, in the udev case, just didn't have networking, which is nearly as bad).

So I'm kind of hesitant to just stabilize all v-s releases. Maybe we can let arches opt out of this if they promise to honor stabilization requests ina speedy way? In the current local root escalation case, alpha would have already stabilized the relevant v-s and g-s if it weren't for the userspace blockers (baselayout, dhcp), but we're happy to test the kernels for 30 days, too.
Comment 6 Kerin Millar 2010-10-01 09:27:20 UTC
> For example, during the last six months there have been no fewer than three
> occasions where a released kernel did not work correctly on alpha. Two of them
> were due to restructuring inside the kernel (syscalls going away, IRQ handling
> being done differently) and one due to userspace apps (udev) expecting
> behaviour that this arch didn't support at that point in time. In all cases,
> the kernel built but got stuck on bootup (or, in the udev case, just didn't
> have networking, which is nearly as bad).

I fully appreciate that these are serious issues (I've been burned by udev in the past). How do you feel about option 1 + 2a, where only the 2.6.32.x releases are fast tracked to stable in keywording terms, with other releases being consumed only at the user's behest via package.keywords? That would be my preference and I think that the 2.6.32 branch has been around long enough to consistitute a known quantity.

Some of the cases that you mention seem like plausible candidates for hard masking (after all, ~arch means testing, not "broken"):

$ tail -n2 arch/sparc/package.mask

# Not yet functional on sparc due to <insert reason>
=sys-kernel/vanilla-sources-2.6.35*
Comment 7 George Kadianakis (RETIRED) gentoo-dev 2010-10-01 09:39:18 UTC
(In reply to comment #6)
> > For example, during the last six months there have been no fewer than three
> > occasions where a released kernel did not work correctly on alpha. Two of them
> > were due to restructuring inside the kernel (syscalls going away, IRQ handling
> > being done differently) and one due to userspace apps (udev) expecting
> > behaviour that this arch didn't support at that point in time. In all cases,
> > the kernel built but got stuck on bootup (or, in the udev case, just didn't
> > have networking, which is nearly as bad).
> 
> I fully appreciate that these are serious issues (I've been burned by udev in
> the past). How do you feel about option 1 + 2a, where only the 2.6.32.x
> releases are fast tracked to stable in keywording terms, with other releases
> being consumed only at the user's behest via package.keywords? That would be my
> preference and I think that the 2.6.32 branch has been around long enough to
> consistitute a known quantity.
> 

The option 1 + 2a is what I believe to be the most sensitive, as well.
Although, Anthony has a point, saying:
> But there are many features in 2.6.33.x and above which are useful
> both on servers (eg. drbd) and on workstation (eg. nouveau).
I think that people who actually want options like DRDB will have no problem picking specific v-s releases via package.keywords.

As a matter of fact, my main issue with the current stabilization policy is that people who are _not_ following security are fetching vulnerable kernels as the defaults. I don't worry that much about savvy users.
Comment 8 Tobias Klausmann gentoo-dev 2010-10-01 09:45:51 UTC
I think having "specially supported" kernels (rather: versions) that then are fast-tracked in stabilization is the sensible thing to do. Aka: I like option 2a.

The question remains: do we want to do this for all the LTS-kernels that upstream produces? Or do we prefer to pick our own set of LTS kernels? In the case of 32/33 I can see reasonable incentive to do the latter.

If we do that (pick whatever we want to support), how do we deal with the need to have all arches (or the relevant ones, whatever that means) agree on the next version to support in this way? I can imagine scenarios where the set of kernel versions all arches agree on is empty (even if we assume the arguing remains reasonable ;))
Comment 9 Kerin Millar 2010-10-01 10:51:17 UTC
> The option 1 + 2a is what I believe to be the most sensitive, as well.

Perhaps, but we have three endorsements here thus far. Let's assume for the sake of argument that no significant objections arise and such a strategy was subsequently agreed upon ...

> The question remains: do we want to do this for all the LTS-kernels that
> upstream produces? Or do we prefer to pick our own set of LTS kernels? In the
> case of 32/33 I can see reasonable incentive to do the latter.

If by "all", you mean each "Y" release in 2.6.X.Y (e.g. a future bump from 2.6.32.23 -> 2.6.32.24) then I would say yes, otherwise we wouldn't really have solved anything. The stable patch queue criteria are fairly strict and there is no scope for any new functionality to be added in this branch, only bug fixes; in particular, security fixes. While there is always the possibility for a mistake to be made, regressions are extremely rare in stable patch bumps. With all due respect, if we leave it to the individual arch teams alone, I think we'll just end up with the same degree of bitrot, merely having moved the goal posts to encompass the 2.6.32 series.
Comment 10 Kerin Millar 2010-10-01 10:52:43 UTC
I forgot to say that, for that to be viable, we'd at least need to establish that there are no known issues with 2.6.32 as it currently stands on any given arch.
Comment 11 Anthony Basile gentoo-dev 2010-10-01 11:08:26 UTC
> The option 1 + 2a is what I believe to be the most sensitive, as well.
> Although, Anthony has a point, saying:
> > But there are many features in 2.6.33.x and above which are useful
> > both on servers (eg. drbd) and on workstation (eg. nouveau).
> I think that people who actually want options like DRDB will have no problem
> picking specific v-s releases via package.keywords.

Having seen the discussion progress, especially the point about off-mainstream arches, I think now that option 2a is the better choice.  Stick to only 2.6.32.x automatically keyworded stable.
Comment 12 Mike Pagano gentoo-dev 2010-10-01 13:15:23 UTC
Short version: 1+ 2a,substituting latest upstream LTS supported version for 2.6.32.X.

My thoughts:

Would this also apply to the previous LTS version which is also still supported at this time? (2.6.27.X). I think that would be ok.

IMO, we should not entertain this recommendation for non upstream LTS kernels. Unsupported by upstream means the possibility of not having needed security fixes in the vanilla kernel. If they stop releasing patches, it just won't be in there.

With 2.6.32.X, or any other LTS kernel version, we at least can expect the backporting of needed security fixes from upstream.  

I do know there have been problems in the past with kernel versions not working on specific arches, so I agree about not auto stabilizing new kernel versions. Usually, a LTS kernel version is chosen after a number of point releases so we should know way beforehand if the version is functional or not on an architecture.

If there are features we think later versions have that would be useful in the LTS kernel, we can attempt to backport or just stabilize the later version using our normal method.
Comment 13 Kerin Millar 2010-10-01 15:50:00 UTC
> Would this also apply to the previous LTS version which is also still supported
> at this time? (2.6.27.X). I think that would be ok.

Well, I don't see why not. There is still activity in its stable queue, with the last release occurring 10 days ago:

http://git.kernel.org/?p=linux/kernel/git/stable/linux-2.6.27.y.git;a=summary

I would expect support for this branch to taper off in the near future - Greg himself hints that it may not be viable for too much longer - but we can cross that bridge when the time comes.

> With 2.6.32.X, or any other LTS kernel version, we at least can expect the
> backporting of needed security fixes from upstream.  

Precisely. Not to mention general bug fixes that are only otherwise making their way into the latest branch.
Comment 14 George Kadianakis (RETIRED) gentoo-dev 2010-10-01 17:11:20 UTC
CCing gregkh - I don't know if he follows kernel@gentoo.org - in case he wants to add anything to the discussion :)
Comment 15 Mike Pagano gentoo-dev 2010-10-04 16:40:15 UTC
Let's involve the arch team leads and see if we can get some agreement to the policy.

Leads as listed in project pages:

alpha:    Blackb|rd
amd64:    hparker
arm:      vapier
hppa:     gmsoft
ia64:     armin76
ppc64:    tgall
ppc:      jer
sh:       ?      
sparc:    armin76
x86:      fauli,maekke

Comment 16 Joe Jezak (RETIRED) gentoo-dev 2010-10-04 18:10:11 UTC
As a PowerPC person, I'd feel fine with point release getting marked stable if the initial version was already marked stable. If we marked 2.6.34 stable, I don't see why a bug fix release like 2.6.34.1 isn't also eligible. Likewise, assuming that we've marked gentoo-sources-2.6.34 stable, vanilla-sources-2.6.34 should also be stable.

I'd be against marking 2.6.35 stable without actual testing on a ppc machine first though.
Comment 17 Mike Pagano gentoo-dev 2010-10-04 19:27:48 UTC
Alpha: Tobias Klausman

<@Blackb|rd> mpagano_: WFM


What works for him is this use case:

kernel version 2.6.XX.Y is stabilized in the normal fashion.
Upstream declares 2.6.XX.Y(+N) a LTS kernel
we auto stabilize 2.6.XX.Y(+N)
Comment 18 Markus Meier gentoo-dev 2010-10-04 21:49:16 UTC
(In reply to comment #16)
> As a PowerPC person, I'd feel fine with point release getting marked stable if
> the initial version was already marked stable. If we marked 2.6.34 stable, I
> don't see why a bug fix release like 2.6.34.1 isn't also eligible. Likewise,
> assuming that we've marked gentoo-sources-2.6.34 stable, vanilla-sources-2.6.34
> should also be stable.
> 
> I'd be against marking 2.6.35 stable without actual testing on a ppc machine
> first though.

That's what I was thinking too when I read this bug. If an arch team marks a kernel version stable (assuming it's vanilla- and gentoo-sources), all following revisions/point-releases can go straight to stable for these arches.
like that, you get LTS kernels and every other upstream (stable) supported kernels stable.
Comment 19 Xake 2010-10-05 20:08:31 UTC
I see one problem with the current discussion, namely the wording:

If (In reply to comment #0)
>   1) All new releases of 2.6.32.x are automatically keyworded stable
>   2) Either:
>      a) All other releases are keyworded unstable
>      b) The latest vanilla-sources release is also automatically keyworded
>         stable

If this would have been:
1) All new releases of an already stabilized LTS are automatically keyworded stable

I would have bought 1+2a at once.

As I see it we else get problems in the following scenario. An exploit is discovered in >=2.6.XX. For a arch 2.6.XX is already marked stable but there never appears a patch for it since it is a bit old and is not a LTS. 2.6.XX+1 is not marked stable (and not even ready to be fasttracked), however a patch gets released for it since it is a that new release.
How to handle? Un-keyword and urge the users to downgrade to latest LTS until the arch team is ready to stabilize 2.6.XX+1?
I would vote for only stabilize LTS to begin with, and if the user or an arch have special reasons to stabilize any release newer then a LTS they have to be up to the task to have a stabilized 2.6.XX+x when 2.6.XX becomes unmaintained.
Comment 20 Christian Faulhammer (RETIRED) gentoo-dev 2010-10-05 23:38:30 UTC
(In reply to comment #15)
> x86:      fauli,maekke

 I totally agree with maekke and josejx here, just to add my two cents.
Comment 21 Kerin Millar 2010-10-06 06:24:15 UTC
> If this would have been:
> 1) All new releases of an already stabilized LTS are automatically keyworded
> stable

Reading between the lines, it should be fairly obvious that this is what I am requesting. 2.6.32 is the current LTS release and therefore the policy would effectively equate with my wording if it were to be adopted. I'm not requesting that the rule only apply to 2.6.32.x forever more. Upon the day that it dies (as 2.6.27.x may do soon), I would suggest that the final release in the branch has its stable keywords revoked - or is package.masked - prior to re-focusing on whichever branch Greg has formally announced as LTS.

As for the "already" part, I would hope that >=2.6.32.24 be immediately stabilised on all arches if the policy is adopted, assuming that there are no compatiblity issues with respect to any particular platform. None have been raised so far.


> As I see it we else get problems in the following scenario. An exploit is
> discovered in >=2.6.XX. For a arch 2.6.XX is already marked stable but there
> never appears a patch for it since it is a bit old and is not a LTS. 2.6.XX+1
> is not marked stable (and not even ready to be fasttracked), however a patch
> gets released for it since it is a that new release.

I don't follow. With the 1+2a combination, only the LTS release would be stable keyworded which, by definition, is receiving the associated fixes. If the branch in question is retired then we can simply move on to the next.
Comment 22 Kerin Millar 2010-10-06 06:39:40 UTC
I think it should also be borne in mind that, while some effort would be required to fully arch test a LTS release to get this policy rolling in the first instance, this effort need only be expended whenever a new branch is designated as LTS (which is to say, not often). 
Comment 23 Xake 2010-10-06 09:27:27 UTC
(In reply to comment #21)
> Reading between the lines, it should be fairly obvious that this is what I am
> requesting. 2.6.32 is the current LTS release and therefore the policy would
> effectively equate with my wording if it were to be adopted. I'm not requesting
> that the rule only apply to 2.6.32.x forever more. Upon the day that it dies
> (as 2.6.27.x may do soon), I would suggest that the final release in the branch
> has its stable keywords revoked - or is package.masked - prior to re-focusing
> on whichever branch Greg has formally announced as LTS.
> 

Yeah, that was what I thought too, but then we have comments like:

(In reply to comment #16)
> If we marked 2.6.34 stable, I
> don't see why a bug fix release like 2.6.34.1 isn't also eligible. Likewise,
> assuming that we've marked gentoo-sources-2.6.34 stable, vanilla-sources-2.6.34
> should also be stable.

And since 2.6.34 is not a LTS (and probably will not be) I was asking myself if everyone really understood that aspect.

> > As I see it we else get problems in the following scenario. An exploit is
> > discovered in >=2.6.XX. For a arch 2.6.XX is already marked stable but there
> > never appears a patch for it since it is a bit old and is not a LTS. 2.6.XX+1
> > is not marked stable (and not even ready to be fasttracked), however a patch
> > gets released for it since it is a that new release.
> 
> I don't follow. With the 1+2a combination, only the LTS release would be stable
> keyworded which, by definition, is receiving the associated fixes. If the
> branch in question is retired then we can simply move on to the next.
> 

Yeah, if only the LTS ever gets stabilized.
But currently we got 2.6.34.y stabilized for gentoo-sources on amd64. Now if there is a exploit before 2.6.35 is stabilized, but after 2.6.34 recieves fixes. And say it is in a rewritten part of the kernel, so you can not just backport a fix from 2.6.35. And say there is a unrelated big bad bug (or like currently, 2.6.35 has to wait for baselayout) that makes it next to impossible to fasttrack 2.6.35 for amd64.
Comment 24 Kerin Millar 2010-10-06 10:09:53 UTC
> And since 2.6.34 is not a LTS (and probably will not be) I was asking myself if
> everyone really understood that aspect.

Ah, I see. Well, quite. 2.6.34 is neither a LTS release nor is it otherwise adequately maintained. I do not think that any instance of this branch should be stable keyworded. As per the second bullet point of my original post, all bets are off when it comes to any non-LTS release, with the exception of that which is current.

Just to clarify my viewpoint, I'm for the following:

  * Latest instance of 2.6.32.x is keyworded stable on all arches (with
    whatever testing is necessary being conducted to make that feasible)
  * Thereafter, all new releases in the 2.6.32.x branch are automatically
    keyworded stable
  * Optionally, we do the same for 2.6.27.x (although I'm not convinced that
    it's worth bothering with)
  * All other releases are ~arch keyworded, without exception, and with
    immediate effect
Comment 25 Anthony Basile gentoo-dev 2010-10-06 10:25:49 UTC
(In reply to comment #24)

>   * All other releases are ~arch keyworded, without exception, and with
>     immediate effect
> 

For clarification, when the latest stable vanilla bumps from 2.6.32.x to 2.6.32.(x+1) we simultaneously mark the .x unstable and .(x+1) stable on all arches.  Correct?  ie. in your statement "all other releases" includes earlier 2.6.32.x's as well as other 2.6.yy branches.

I'm pretty sure that's what you mean since the earlier 2.6.32.x would be the one with possible bugs/exploits etc.
Comment 26 Kerin Millar 2010-10-06 10:49:27 UTC
> For clarification, when the latest stable vanilla bumps from 2.6.32.x to
> 2.6.32.(x+1) we simultaneously mark the .x unstable and .(x+1) stable on all
> arches.  Correct?  ie. in your statement "all other releases" includes earlier
> 2.6.32.x's as well as other 2.6.yy branches.

Actually, I hadn't thought of that but, now that you mention it, that would be fine and dandy as long as the committer doesn't mind incurring that burden. As far as I'm concerned, the most important thing is that the ".(x+1)" release is marked stable so that, when Joe Sixpack invokes emerge vanilla-sources, that's what he gets. The overarching problem as it stands is that he doesn't get a safe kernel without going out of his way to get it.
Comment 27 Mike Pagano gentoo-dev 2010-10-06 12:32:36 UTC
(In reply to comment #24)
> > And since 2.6.34 is not a LTS (and probably will not be) I was asking myself if
> > everyone really understood that aspect.
> 
> Ah, I see. Well, quite. 2.6.34 is neither a LTS release nor is it otherwise
> adequately maintained. 

Note that I will and am currently backporting fixes to 2.6.34. But if you mean not adequately maintained by upstream ,then I agree.

Secondly, if we auto stabilize (for,example) 2.6.32.22 and it contains a severe security fix, we should probably keyword unstable the earlier versions or make them disappear after a few days.

Comment 28 Kerin Millar 2010-10-06 13:12:32 UTC
> Note that I will and am currently backporting fixes to 2.6.34. But if you
> mean not adequately maintained by upstream ,then I agree.

Your backporting work is certainly not being called into question but it has nothing to do with vanilla-sources.

> Secondly, if we auto stabilize (for,example) 2.6.32.22 and it contains a
> severe security fix, we should probably keyword unstable the earlier
> versions or make them disappear after a few days.

Sounds like a good idea to me. Indeed, I sometimes wish that this sort of tree cleaning would occur more often.
Comment 29 Mike Pagano gentoo-dev 2010-10-07 12:44:58 UTC
Vanilla:

kernel version 2.6.XX.Y is stabilized in the normal fashion.
Upstream declares 2.6.XX.Y(+N) a LTS kernel
we auto stabilize 2.6.XX.Y(+N)

Is there any reason to not apply the same rules to gentoo-sources.
We have a use case where a security bug appears, we can't stable 2.6.35 because of baselayout issue, I backported the patch to 2.6.34, but I was slow to ask for stabilization for that one.

If previous versions of gentoo-sources-2.6.34-rX are stable, can we apply the same rules and stable subsequent -rX releases?
Comment 30 Xake 2010-10-07 13:18:50 UTC
(In reply to comment #29)
> Is there any reason to not apply the same rules to gentoo-sources.
> We have a use case where a security bug appears, we can't stable 2.6.35 because
> of baselayout issue, I backported the patch to 2.6.34, but I was slow to ask
> for stabilization for that one.
> 

In this case it would have worked well if this policy was in place. What I am concerned about is if we got into this situation again, but the code in question makes a backports between 2.6.XX and 2.6.YY non-trivial, and you may be not around to make a good fast backport like you did this time.

As it stands it seems like everyone is agreeing on this being a good policy for LTS kenrnels at least. What is required to approve this policy for LTS, and work out the details for other versions later? I mean it maybe is possible some arch team only wants LTSs stabilized to begin with (less work for them, like the hardend kernel team some time ago talked about maybe only support every even release to ease the burden).
Comment 31 Mike Pagano gentoo-dev 2010-10-07 13:57:21 UTC
What is required to approve this policy for LTS, and
> work out the details for other versions later? I mean it maybe is possible some
> arch team only wants LTSs stabilized to begin with (less work for them, like

Waiting to hear from:


amd64:    hparker
arm:      vapier
hppa:     gmsoft
ia64:     armin76
ppc64:    tgall
ppc:      jer
sh:       ?      
sparc:    armin76
Comment 32 Mike Pagano gentoo-dev 2010-10-07 16:13:43 UTC
<hwoarang> mpagano_: i agree ( amd64 )
Comment 33 Greg Kroah-Hartman (RETIRED) gentoo-dev 2010-10-08 03:08:03 UTC
I have no object, this sounds sane to me.
Comment 34 Raúl Porcel (RETIRED) gentoo-dev 2010-10-10 15:52:25 UTC
I agree for arm/ia64/sh/sparc, although i'm not the lead of arm/sh but i'm the one who has tested/stabilized the kernels on those(in the case of arm maekke has done it too)
Comment 35 Raúl Porcel (RETIRED) gentoo-dev 2010-10-10 15:56:05 UTC
Mike,

for ppc jer is NOT the lead. You probably want nixnut, josejx and ranger. For ppc64 you want josejx and ranger as tgall is retired.

hparker is not the team lead for amd64 either, its angelos or kingtaco
Comment 36 nixnut (RETIRED) gentoo-dev 2010-10-11 17:26:28 UTC
(In reply to comment #35)
> Mike,
> 
> for ppc jer is NOT the lead. You probably want nixnut, josejx and ranger. For
> ppc64 you want josejx and ranger as tgall is retired.

See comment #16 for ppc  
(In reply to comment #31)

> Waiting to hear from:
> 
> 
> amd64:    hparker
> arm:      vapier
> hppa:     gmsoft
> ia64:     armin76
> ppc64:    tgall
> ppc:      jer
> sh:       ?      
> sparc:    armin76

See comment #16 for ppc (and ppc64 as well I reckon)

Comment 37 Mike Pagano gentoo-dev 2010-10-14 16:53:50 UTC
> amd64:    angelos, king taco
> hppa:     gmsoft

Just waiting on those two and I think we might be ok.  
Comment 38 Jeroen Roovers gentoo-dev 2010-10-14 17:12:44 UTC
Speaking for HPPA, we used to have our own hppa-sources because

1) kernel patches from parisc-linux -> mainline lagged a lot, but more 
   importantly,
2) Both vanilla-sources and gentoo-sources were broken for HPPA.

Since parisc-linux went git and more directly and even more frequently sends patches that are included in mainline, vanilla-sources has become a lot more stable for us, but it's still not there. With several 2.6.3*'s breaking in the recent past, I could agree on marking point releases stable automatically as in actions 1 + 2a, because if HPPA users need a newer than >=2.6.33 stable, that should be up to us.
Comment 39 Guy Martin (RETIRED) gentoo-dev 2010-10-14 19:24:52 UTC
As Jeroen said, the "1+2a" policy is quite problematic for hppa since there are hppa specific bugs which are solved in later release than the LTS one.

If memory serves, critical XFS bugs were solved since then and also some cache corruption issues were fixed in 2.6.35.

What shall we do about this ?
LTS doesn't seem to be an option as it considers that the current -stable branch is actually 'critical' bug free on all archs which is not the case on hppa.

Instead of rule 1 specifying the LTS branch, I think it would be better to have each arch (or at least hppa) select which branch should be the current stable one.
Of course if security bugs are found in that specific branch, which is not upstream LTS, that arch will have to go for a newer branch.


Comment 40 Kerin Millar 2010-10-14 22:05:27 UTC
> As Jeroen said, the "1+2a" policy is quite problematic for hppa since
> there are hppa specific bugs which are solved in later release than the
> LTS one.

Thanks for commenting; I fully appreciate these concerns. However, I think that your requirements need not be orthogonal to the overall intent of the "1+2a" policy. Rather, it just needs to be carefully worded so that it is not overly discriminatory and, thus, insensitive to the needs of niche arches (taken at face value, it currently is).

Basically, I think that the policy as it is currently depicted may be adhered to as long as it does not stipulate that arch leads/testers are not perfectly free to mark a current release stable for their platform as they see fit.

For instance, if 2.6.35.8 were - hypothetically - to be released and committed tomorrow, I could envisage two scenarios:

1) vanilla-sources-2.6.35.8 is initially committed as ~arch for all platforms
   as per the current wording of the policy. An hppa lead/tester subsequently
   marks it as stable if they deem it appropriate.

2) vanilla-sources-2.6.3.58 is initially committed by someone who is member of
   the hppa team anyway and opts to keyword it stable for hppa at the point of
   committing if they see fit to do so. However, they leave all other platforms
   as ~arch (naturally).

I don't think that this really conflicts with the policy at large. The ultimate goal is to get a stable and safe kernel into the hands of ... well ... everyone. For a large proportion of the userbase, I a LTS release would constitute a fine default choice. However, if a particular arch requires that a current kernel is marked as stable then there's no reason to impede them from doing just that. It would remain their responsibility to test and stabilise it, just as it is now.

Does that sound reasonable?

P.S. One thing that I would ask for - if it is not too much trouble - is that, where a particular kernel version and its predecessors are known to be broken, that the relevant team consider populating package.mask in their profile accordingly. This is just a matter of pragmatism - it stops users from potentially shooting themselves in the foot, and leaves committers of the LTS releases free from the burden of having to remember which arches are not there yet, as jer put it.
Comment 41 Kerin Millar 2010-10-14 22:07:14 UTC
Oops - while it should be evident - "I a LTS release" should have been written "an LTS release". Sorry about that.
Comment 42 Guy Martin (RETIRED) gentoo-dev 2010-10-18 19:28:40 UTC
All of this looks good to me. Hopefully a newer LTS release will come soon so hppa can benefit from this policy :)

Given 2.6.X.Y stable on hppa, would it make sens to stable (or simply keep the keywords) all the newer Y+1 releases ?
I've never seen a such release introducing regression.

As far as I understand, if we implement this policy, on x86, all the stable keywords would be dropped for releases higher than the current LTS (2.6.32.z).
So on hppa we would keep 2.6.35.z stable. Whenever a new point release is out, keeping the existing keywords would just do.

So I would rephrase 1 this way :
 1) New minor releases for LTS or for arch specific current release are automatically marked stable


I think this wording takes all the scenario in account. Any thoughts ?
Comment 43 Christian Faulhammer (RETIRED) gentoo-dev 2010-12-14 10:45:30 UTC
x86 is done and ok with auto-stabilisation.
Comment 44 Mike Pagano gentoo-dev 2010-12-17 14:00:19 UTC
Thanks, Christian. And I apologize for not giving this more attention: work, life, wife, dog, etc. 

Let's see if I can sum this up properly:

For new kernel major point release: 2.6.X
1. Stable request bug is opened and arch teams stabilize as we do today. No auto stabilizing occurs here as only the arch teams can really determine if the kernel is working on their own arch. No change from current policy

For subsequent security related releases of a kernel point release 2.6.X.Y
2. If the kernel team determines a significant security fix is included for a kernel release of 2.6.X.y where 2.6.X or 2.6.X.(Y-1) has already been stabilized per number 1, the kernel team can auto stabilize that specific version
3. Dependant upon the severity of the security bug, (root exploit, minor module) the kernel team will remove stable keywords from earlier versions of the same 2.6.X series within a reasonable timeframe.

I think we own this service to our users to keep them on a secure kernel.

Comment 45 Mike Pagano gentoo-dev 2011-03-21 22:15:22 UTC
I plan on stabilizing gentoo-sources-2.6.36-r7 and vanilla-sources-2.6.36.4 as per above policy, tomorrow.  If anyone has any objections please let me know.

All archs have previous versions of this kernel already stabled.
Comment 46 Mike Pagano gentoo-dev 2011-05-03 15:30:47 UTC
Documented on kernel project page