Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 56405 - development-sources 2.6.7 Modifies Sources
Summary: development-sources 2.6.7 Modifies Sources
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All All
: High normal (vote)
Assignee: Sparc Porters
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2004-07-07 19:57 UTC by Gabriel Devenyi
Modified: 2004-07-16 08:34 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Output of emerge -v =development-sources-2.6.7 (output.bz2,141.76 KB, application/octet-stream)
2004-07-08 08:51 UTC, Gabriel Devenyi
Details
output ^ (output,1.84 MB, text/plain)
2004-07-08 17:04 UTC, Jeremy Huddleston (RETIRED)
Details
Patch development-sources vs untar of linux-2.6.7.tar.bz2 (patch,2.77 KB, patch)
2004-07-08 19:00 UTC, Gabriel Devenyi
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Gabriel Devenyi 2004-07-07 19:57:07 UTC
When trying to apply patches for the vanilla 2.6.7 kernel from the kernel list, patching fails for linux-2.6.7/Makefile. When I extract linux-2.6.7.tar.bz2 manually there are no failures in patching, the ebuild must therefore be somehow modifiying files, causing patches to fail.
Comment 1 Greg Kroah-Hartman (RETIRED) gentoo-dev 2004-07-07 23:29:39 UTC
you are correct, it looks like we are adding a sparc patchset.

Um, sparc people, why is this?  I thought you had your own kernel :)
Comment 2 Joshua Kinard gentoo-dev 2004-07-08 03:10:41 UTC
2.6.x kernel sources were formerly maintained in sparc-dev-sources, but then there was the push to migrate separate arch sources into a single ebuild that all archs shared using kernel-2.eclass, thus s-d-s got deprecated.

Comment 3 Gabriel Devenyi 2004-07-08 04:44:14 UTC
I guess the arch checking must be broken then, since my ~x86 souces get modified on emerge.
Comment 4 Jason Wever (RETIRED) gentoo-dev 2004-07-08 06:46:46 UTC
Basically we apply our own patchset to a) fix problems in the regular kernel for which patches didn't make it upstream in time and b) to make the menu system layout follow how x86 has it setup rather than how it ships by default.  There may be some additional patches in there as well based on a given revision.  wesolows is the guy to talk to (though you may have to ping him on irc or send email to get a response).
Comment 5 Gabriel Devenyi 2004-07-08 06:54:59 UTC
You can patch whatever you want for sparc, but please check your arch checking, since x86 sources are somehow being modified which is causing issues.
Comment 6 Jason Wever (RETIRED) gentoo-dev 2004-07-08 08:31:17 UTC
Can you show us the output of emerge -v =development-sources-2.6.7 to confirm that it is indeed the sparc patch that is effecting you?  Development-sources uses the kernel-2 eclass which is designed to allow each architecture to have it's own patchset.  For this version, x86 may have it's own as well.
Comment 7 Gabriel Devenyi 2004-07-08 08:51:50 UTC
Created attachment 35011 [details]
Output of emerge -v =development-sources-2.6.7

Okay, well I don't seem to notice anything sparc-related, but the sources ARE
modified and don't patch correctly. *sigh* I don't know whats going on.
Comment 8 Jeremy Huddleston (RETIRED) gentoo-dev 2004-07-08 17:00:04 UTC
Comment on attachment 35011 [details]
Output of emerge -v =development-sources-2.6.7

Please always set the mime type properly.
Comment 9 Jeremy Huddleston (RETIRED) gentoo-dev 2004-07-08 17:01:44 UTC
Comment on attachment 35011 [details]
Output of emerge -v =development-sources-2.6.7

for that matter, attatch things as text.  not compressed.
Comment 10 Jeremy Huddleston (RETIRED) gentoo-dev 2004-07-08 17:04:04 UTC
Created attachment 35036 [details]
output ^
Comment 11 Jeremy Huddleston (RETIRED) gentoo-dev 2004-07-08 17:14:55 UTC
Gabriel, it doesn't look like there are any sparc patches applying...

Pleasse do the following:

rm -rf /usr/src/linux-2.6.7
emerge \=sys-kernel/gentoo-dev-sources-2.7.6
mv /usr/src/linux-2.6.7 /usr/src/linux-2.6.7-gentoo-vanilla
cd /usr/src
tar xjf <path to>/linux-2.6.7.tar.bz2
diff -Naur linux-2.6.7 linux-2.6.7-gentoo-vanilla > patch

then show us patch
Comment 12 Gabriel Devenyi 2004-07-08 19:00:51 UTC
Created attachment 35048 [details, diff]
Patch development-sources vs untar of linux-2.6.7.tar.bz2

I'll assume that its supposed to be the linux tarball vs development-sources
since thats the ebuild I have issues with, anywho, here it is.
Comment 13 Jeremy Huddleston (RETIRED) gentoo-dev 2004-07-09 05:21:59 UTC
what patch are you trying to apply that borks?  My guess is that it might be adding an EXTRAVERSION line which changes (whitespace) for the gentoo "version"
Comment 14 Gabriel Devenyi 2004-07-09 05:25:44 UTC
The latest mm, ck, or the staircase scheduler patch all don't work nicely.
Comment 15 Greg Kroah-Hartman (RETIRED) gentoo-dev 2004-07-09 13:32:35 UTC
Well, NO arch specific patches (or any other patches) should be applied
for this package.

If any arch has any specific patches that they want to add to the kernel tree,
then that is what the gentoo-dev-sources package is for, not this development-sources package.

I'm going to rev this package in a few minutes and take out the sparc stuff.
If the sparc developers would be so kind as to use the gentoo-dev-sources kernel, like all other arches do, it would be greatly appreciated, and I would be glad
to work with them (like I do with the amd64, and ppc arches.)
Comment 16 Ciaran McCreesh 2004-07-09 13:52:28 UTC
Can you guys (as in Greg and John) please make up your minds as to which kernel sources we *should* be using? Making our users switch packages every few weeks is hardly ideal...
Comment 17 Keith M Wesolowski (RETIRED) gentoo-dev 2004-07-09 14:11:47 UTC
The development-sources kernel is _SUPPOSED_ to have arch-specific patches as are needed for proper functionality (ie, they are not supposed to add functionality), and are only supposed to be applied when merged _on_that_arch_.  This is what is happening.  This was the whole point of the kernel-2 eclass.  We cannot and will not use the cesspool that is gentoo-dev-sources with its 50000 x86-specific patches thrown in as if they were the most important thing in the world.

This is being referred to devrel.  You don't change other people's arches, especially without asking the arch lead first.  The sparc patchset does not get applied on x86, this bug should be marked INVALID, not FIXED because there is nothing to fix.  The original report clearly shows no patches being applied.
Comment 18 Joshua Kinard gentoo-dev 2004-07-09 14:14:24 UTC
The patch Gabriel posted showing a difference between vanilla 2.6.7 tarball and dev-sources indicates changes to something having to do with documentation.  Looking at the ViewCVS log for kernel-2, I saw this little bit:

Mon Jul 5 00:00:13 2004 UTC (4 days, 21 hours ago) by spock 
Updated the kernel-2 eclass to build & install manual pages when 'doc' USE flag is set.

http://www.gentoo.org/cgi-bin/viewcvs.cgi/eclass/kernel-2.eclass

Mayhaps that has done something?  Spock, any idea whether your change may have inadvertently triggered this?
Comment 19 Greg Kroah-Hartman (RETIRED) gentoo-dev 2004-07-09 14:23:12 UTC
The development kernel is not "supposed" to have such patches, see the 
metadata file for proof of that.  Users are expecting to have a "vanillia
kernel.org" kernel from this package, but that is not what is happening.
I have fixed that bug.

And g-d-s is NOT a cesspool, and does NOT have a load of x86 kernel patches.
Take a look at it today.  All of the patches in it are there for very specific
reasons, and I am not allowing "oh, add this feature", or "add this tunable", or
"add this neat scheduler change" stuff to creap in.

We need a stable, solid, 2.6 kernel package, that we can support, and g-d-s
is that package.

Also, there is already a sparc specific kernel package, why not just use that one?

Or again, I have offered to take your sparc patch into the main g-d-s kernel
as we are trying to standardize on a bare minimum of kernels.
Comment 20 Joshua Kinard gentoo-dev 2004-07-09 14:57:24 UTC
Greg: Originally, development-sources was intended for archs to add their own specific patches to, using kernel-2.eclass.  That was the basis behind its creation and the creation of kernel-2.  I got asked several times about merging 2.6 mips-sources into dev-sources as well, but held off because I felt the number of patches used in 2.6 mips-sources would be too cumbersome for dev-sources.

Technically, sparc is correct that dev-sources was acceptable for sparc-specific patches to go into.  Whoever or whatever changed that policy failed to notify the sparc team, however, thus we have the little problem we have now.
Comment 21 Jeremy Huddleston (RETIRED) gentoo-dev 2004-07-09 17:16:12 UTC
gregkh: Do not remove the sparc specific patches that are in gentoo-dev-sources.  Please come to #gentoo-sparc and talk with those involved on how to best resolve this situation.  I understand your concern about having vanilla sources (vanilla-sources and dev-sources) as a mirror for what is upstream and not maintained by gentoo, but g-d-s is clearly not a viable option for sparc.

Do not do anything rash like removing the sparc patches before we figure out a good solution.

Thank you.
Comment 22 Greg Kroah-Hartman (RETIRED) gentoo-dev 2004-07-09 17:25:38 UTC
I'm not removing any sparc patches from g-d-s.  I don't see any in there :)

irc doesn't work for me on the weekend, email does.
Comment 23 Jeremy Huddleston (RETIRED) gentoo-dev 2004-07-09 18:48:21 UTC
s/gentoo-dev/development/ ;p  Let's keep this on email though... better to have it all in one place ;)
Comment 24 John Mylchreest (RETIRED) gentoo-dev 2004-07-16 04:46:27 UTC
until I get my email sorted I will just quickly post here.
Just wanted to give my 2c as to why everything is how it is, how I feel things should be, and maybe we can then look at technical reasons to change these.

basically, the reasons behind arch_compat patching of sources (with direct relation to development-sources and vanilla-sources) is to ensure that a single package in the tree is capable of building and running on all archs.
Obviously we shouldn't apply sparc patches which fix up all sparc problems on the vanilla sources to all users, hense the requirement above for the conditional arch patching.

amongst the reasons we done this was to reduce the overall number of sources we give to users, and to allow a much more streamlined approach to security updates and release bumps.
I dont want to see the tree getting flooding with <arch>-sources when all they effectively are minimal patchsets applied to development-sources anyways.

The patches which get applied in this manner should be the absolute minimum required to have the kernel boot, and work on that architecture. if addition patches are required/desired then they warrant a new source package.
these patches should ideally be identical across all arches, so we keep the same functionaly across the board. of course there are technical issues here and are dealt with as and when. this is just a general rule of thumb in attempt to create a more stable and consistant feature-set.

So, to sum up.
I think we should still deliver development-sources with arch-compat for broken architectures.

For over 90% of our userbase, there is no difference to them at all (x86), to the others, the difference means the sources work. A distinct advantage over them not working properly I'd say :)

Is there any other reasoning which justifies a change? I'm not opposed to it but it certainly needs to be spoke of.

In the mean time I will aim to update the project page and draft up/write up all the current policies. I also aim for a full meeting of all kernel devs as well to propose what we will be doing for the upcoming 2004.3/.4 releases.

If anyone would like to discuss things further, im all ears :)
Comment 25 Greg Kroah-Hartman (RETIRED) gentoo-dev 2004-07-16 07:44:43 UTC
See your email :)

We've pretty much resolved this already, and the 2.6.8-rc1 version
of d-s do not have any arch patches in it at all.

g-d-s now has full sparc support, and everyone is happy :)
Comment 26 Gabriel Devenyi 2004-07-16 07:46:44 UTC
So yeah, my inital issue still stands, linux-2.6.8-rc1/Makefile gets somehow modified such that patches fail on it.
Comment 27 Ciaran McCreesh 2004-07-16 07:49:05 UTC
No, we're not happy with g-d-s at all.
Comment 28 Jason Wever (RETIRED) gentoo-dev 2004-07-16 07:57:50 UTC
I hate to come across as myself, but indeed this issue is not resolved.  We have not as of yet gotten back to you with a consensus as to whether we find these changes acceptable.
Comment 29 John Mylchreest (RETIRED) gentoo-dev 2004-07-16 08:13:21 UTC
Im happy that g-d-s is now fully functional on sparc, although that isnt the issue at hand.
The issue really is how we manage the sources in the tree, and tbh, the technical reasoning of the sources is secondary to that in this case.

d-s is the vanilla sources for all archs, where vanilla means as minimal as possible.

making g-d-s as cross-arch compatible as possible is certainly desired, but the patchset itself is designed as a gentoo modified desktop kernel. almost all non-x86|ppc users would want "vanilla".

I think the problem here is that some technical reasoning is causing people to miss the basic facts of why they are as they are.
it is a logistical management move for the tree. in that we are condensing sources which are trivially different to each other in a manner which will not effect each other. this brings some big benefits.

We can do this on a per-package basis with similar suiting patchsets. unfortunately in this case, dev- and gentoo-dev are not similar.
Comment 30 Greg Kroah-Hartman (RETIRED) gentoo-dev 2004-07-16 08:20:26 UTC
Ok, sorry, I took a lack of response or complaints as acceptance :)

So, what's wrong with the current g-d-s for the sparc platform?
Please let's take this to email, and not add on to this bug, as I'm about to loose
reliable web access during the kernel summit and OLS traveling and attending.
Comment 31 Greg Kroah-Hartman (RETIRED) gentoo-dev 2004-07-16 08:22:00 UTC
Ok, I think we need to postpone this until the kernel developers can agree
on the different packages, and the directions we want to take with them.

John, any idea when you want to have this discussion?  I'm going to be at
conferences for about the next 3 weeks, so irc chat is not going to work, but
email will.
Comment 32 John Mylchreest (RETIRED) gentoo-dev 2004-07-16 08:29:45 UTC
Ideally sooner than later really. although since you wont be here in a capacity to respond interactively it might be best that we an initial meeting and summarise the minutes to mail on to your good self.
From there we can progress following any input you also have.

I will send an email to kernel@gentoo.org and will arrange for the meeting sometime next week if that is acceptable.

As far as an immediate solution goes, I feel we should continue as we did. dev-sources with arch conditionals, and aim to pull the fixes upstream.

I will be working on the pinning side of config-kernel in the coming few days which will allow us to also interface the masking of kernel major versions for when we move 2.6 into 2.4. 
in that I mean moving gentoo-dev-sources to gentoo-sources and dropping gentoo-dev-sources. development-sources into vanilla-sources, and so on.

I will also be working on migrating the sources in the tree still using kernel.eclass to kernel-2 and working in proper support for wolk.

following this we will be working on implementing the Kbuild changes for koutput and M=.

however all of this can be discussed. I shall send that email now.
Comment 33 Greg Kroah-Hartman (RETIRED) gentoo-dev 2004-07-16 08:34:38 UTC
That sounds fine with me.  And if the time corrisponds with a boring conference
talk, I'll try to attend the irc meeting too :)