Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 121248 - kernel-2 gets git ordering wrong, and git-sources don't actually include git patches
Summary: kernel-2 gets git ordering wrong, and git-sources don't actually include git ...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: Highest major (vote)
Assignee: Gentoo Kernel Bug Wranglers and Kernel Maintainers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2006-02-01 20:07 UTC by Robin Johnson
Modified: 2006-03-08 03:20 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
kernel-2-git-fixes.patch (kernel-2-git-fixes.patch,4.48 KB, patch)
2006-03-08 01:44 UTC, Robin Johnson
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-02-01 20:07:22 UTC
ok, so I wanted the latest git-sources, and to my surpise, I found that they don't actually include any git patch. git-sources-2.6.16_rc1-r4 only gives me 2.6.16-rc1, no git patch at all.

I attempted to fix this, and dug into kernel-2.eclass, however I discovered a major flaw there.
kernel-2 maps _preX to -gitX, however gitX is additive to it's filename PV, not a leadup to it's filename PV.
2.6.15-git1 comes after 2.6.15.
2.6.16-rc1-git1 comes after 2.6.16-rc1.

Now add the kernel-2 mapping pre/git and you get portage's numbering scheme saying that 2.6.16_rc1_pre1 comes BEFORE 2.6.16_rc1 (exact opposite of git).
Comment 1 Greg Kroah-Hartman (RETIRED) gentoo-dev 2006-02-02 13:57:14 UTC
ugh, you are right.  How did this ever work for the -bk snapshots which did
the same thing?

John?
Comment 2 John Mylchreest (RETIRED) gentoo-dev 2006-02-08 09:32:11 UTC
bk snapshots were pulled into an additional variable (same goes with the upstream patches in general, 2.6.14.1 fex) which were applied before anything else. 

I've not had the opportunity after moving to take a look at this though im afriad.
Will do tonight when I get home.
Comment 3 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-02-11 01:04:28 UTC
johnm: bump
Comment 4 John Mylchreest (RETIRED) gentoo-dev 2006-02-11 03:10:30 UTC
this is a culmination of a couple of things.
I'm looking into a nicer alternatice method (perhaps actually utilising -r, but I hate the idea of that) but basically... let me summarise:

the -rc-bk code was originally written totally theoretically, as at the time portage did not support the _rc_pre suffix combination. Please see bug #37406

After looking into the eclass, and the git-sources there were a few little things.
the git-sources set the full patchlist as well as KERNEL_URI but KERNEL_URI will expand totally to contain the neccessary URI's and populate a unipatch_default list to ensure the correct patch order. (this means unipatch-strictorder isnt needed either)

Unfortunately, because of my current situation I cant provide a proper diff, but... as a temporary work-around until we can get access to decent version handling can I suggest this:

at the top of git-sources set:
CKV=${PVR/-r/_pre}

and remove anything but KERNEL_URI from SRC_URI.
if you want, you can then unset STRICTORDER as well.

On top of that, I have actually spotted an eclass bug as well.
Please can you remove the -git* entry from:

kernel-2.eclass, lines 202 & 204  - since $RELEASE will equal -rc2-gitX

please test all that locally. That *should* work around the real issue.
once I've actually witnessed the new version handling code in portage, I will replace the _pre with _p on a -rc-git release, since _pre is a little misleading, and blend the changes into kernel-2 which will then over-come the requirement to set CKV.

CKV is a little hack to totally ignore the downfalls of portage versioning.
Comment 5 John Mylchreest (RETIRED) gentoo-dev 2006-02-11 03:15:09 UTC
also, jsut to comment on your original report Robin, the portage versioning (ie: pre comes before etc) scheme doesnt bare relevance here, since the detect_version stuff in the eclass uses $PV independantly to however portage resolves versioning. Basically, it just uses $PV as a means to work it out *correctly*.

Its important to remember that portage used to detect the kernel version itself, in a totally and unpredictably wrong way, setting $KV. detect_version was a means to do it properly, more granular, and givign the benefit to automate URI changes.
Comment 6 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-03-07 02:44:32 UTC
I gave this a run, and it mostly works - two glitches remaining.
Here's my debug function output, printed at the start of kernel-2_src_unpack.
 * PVR: 2.6.16_rc5-r9
 * CKV: 2.6.16_rc5_pre9
 * OKV: 2.6.15
 * KV: 2.6.16-rc5-git9-r9
 * KV_FULL: 2.6.16-rc5-git9-r9
 * RELEASETYPE: -rc-git
 * RELEASE: -rc5-git9
 * UNIPATCH_LIST_DEFAULT: /dev/shm/portage/git-sources-2.6.16_rc5-r9/distdir/patch-2.6.16-rc5.bz2 /dev/shm/portage/git-sources-2.6.16_rc5-r9/distdir/patch-2.6.16-rc5-git9.bz2 
 * UNIPATCH_LIST_GENPATCHES:  
 * UNIPATCH_LIST: 
 * S: /dev/shm/portage/git-sources-2.6.16_rc5-r9/work/linux-2.6.16-rc5-git9-r9

1. Notice the -r9 on the end of KV/KV_FULL. This is the else fork of the K_PREPATCHED if structure. If K_PREPATCHED is set instead, you get KV= 2.6.16-rc5-git9-git9.

2. The second one is a harder problem. The latest git patches have the following diff header for each file.
--- a/.gitignore
+++ b/.gitignore
unipatch tries various levels of -p, but tries to apply them to ${WORKDIR}, not ${S}, so they always fail. The correct patch level is '-p1 -d ${S}'.
Comment 7 John Mylchreest (RETIRED) gentoo-dev 2006-03-07 05:46:45 UTC
in which case, #1 can be fixed ith K_PREPATCH=yes & CKV=${PVR} instead. which is a nice easy fix.

The second part, I write something to pass per-partch options to unipatch but it hasnt been applied. Its still buggy. I'm sure we can shove some option in there which the ebuild can define :)

I guess its either that, or use sed on the patchset in workdir first, and all kinds of messiness.
Comment 8 John Mylchreest (RETIRED) gentoo-dev 2006-03-07 05:47:51 UTC
in fact, ignoring CKV all together and just using K_PREPATCH will work in this case. Its very rare that the upstream release for what used to be -bk pulls are named after the sources themselves :D
Comment 9 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-03-08 01:26:28 UTC
nope, going for CKV=PVR && PREPATCH=yes makes it even worse.

Here's a review of settings and results.

UNIPATCH_STRICTORDER="yes"
K_NOUSENAME="yes"
K_NOSETEXTRAVERSION="yes"
ETYPE="sources"
CKV="${PVR/-r/_pre}"
inherit kernel-2
detect_version

results in:
 * PVR: 2.6.16_rc5-r9
 * CKV: 2.6.16_rc5_pre9
 * OKV: 2.6.15
 * KV: 2.6.16-rc5-git9-r9
 * KV_FULL: 2.6.16-rc5-git9-r9
 * RELEASETYPE: -rc-git
 * RELEASE: -rc5-git9
 * UNIPATCH_LIST_DEFAULT: /home/gentoo/distfiles/patch-2.6.16-rc5.bz2 /home/gentoo/distfiles/patch-2.6.16-rc5-git9.bz2 
 * UNIPATCH_LIST_GENPATCHES:  
 * UNIPATCH_LIST: 
 * S: /dev/shm/portage/git-sources-2.6.16_rc5-r9/work/linux-2.6.16-rc5-git9-r9
 * KERNEL_URI: mirror://kernel/linux/kernel/v2.6/snapshots/patch-2.6.16-rc5-git9.bz2 mirror://kernel/linux/kernel/v2.6/testing/patch-2.6.16-rc5.bz2 mirror://kernel/linux/kernel/v2.6/linux-2.6.15.tar.bz2
This fetches the correct sources and tarballs, but the directory name is wrong, and the unipatch function fails on the git patches.

and this per your latest suggestion:
K_PREPATCHED="yes"
UNIPATCH_STRICTORDER="yes"
K_NOUSENAME="yes"
K_NOSETEXTRAVERSION="yes"
ETYPE="sources"
CKV="${PVR}"
inherit kernel-2
detect_version

results in:
 * PVR: 2.6.16_rc5-r9
 * CKV: 2.6.16_rc5-r9
 * OKV: 2.6.16
 * KV: 2.6.16-rc5-r9-git9
 * KV_FULL: 2.6.16-rc5-r9-git9
 * RELEASETYPE: -rc-r
 * RELEASE: -rc5-r9
 * UNIPATCH_LIST_DEFAULT:  
 * UNIPATCH_LIST_GENPATCHES:  
 * UNIPATCH_LIST: 
 * S: /dev/shm/portage/git-sources-2.6.16_rc5-r9/work/linux-2.6.16-rc5-r9-git9
 * KERNEL_URI: mirror://kernel/linux/kernel/v2.6/linux-2.6.16.tar.bz2
And it tries to has linux-2.6.16.tar.bz2 

I think the CKV="${PVR/-r/_pre}" solution is closest, but needs some changes in kernel-2 to the PREPATCH code.

I'm going to work on it some more.
Comment 10 John Mylchreest (RETIRED) gentoo-dev 2006-03-08 01:42:13 UTC
thinking about it (not so freshly) this morning I can see what is happening now.
the detect_version uses $PR for appending to the CKV generated kernel version. because its using -r its appending it. However, just for giggles can you try s/-r/_p/ and the same to reflect in the CKV=${PVR/_p/_pre/}

I'll still want to alter the code to handle this nicer for git, but this *should* work for you?

Of course, ignore the patching thing still, im not looking at that issue with the above.
Comment 11 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-03-08 01:42:40 UTC
Ok, I have come up with a solution now, by adding a new variable to kernel-2. K_NOUSEPR - this ensures no fragment using PR is added to EXTRAVERSION.

Firstly, here is the git-sources portion.
UNIPATCH_STRICTORDER="yes"
K_NOUSENAME="yes"
K_NOSETEXTRAVERSION="yes"
K_NOUSEPR="yes"
ETYPE="sources"
CKV="${PVR/-r/-git}"
inherit kernel-2
detect_version

And I'll attach the kernel-2 changes as a patch in a moment for approval before I commit them.
Comment 12 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-03-08 01:44:34 UTC
Created attachment 81671 [details, diff]
kernel-2-git-fixes.patch

Patch that implements K_NOUSEPR and adds debug-print invocations in useful places.
Comment 13 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-03-08 01:45:38 UTC
On a side note, with my patch that fixes the directory name, the unipatch issue goes away.
Comment 14 John Mylchreest (RETIRED) gentoo-dev 2006-03-08 02:05:03 UTC
just a few things..

handle_genpatches:print_all_versioninfo should be removed?
handle_genpatches:if [[ -n "${K_NOUSEPR}" ]]; black, does it really need the : call?
kernel-2_src_unpack:print_all_versioninfo should be removed? Same with the commented return.

Just to clarify, what provides debug-print? is it a portage internal?
The rest of it looks fine. Thanks. Feel free to commit.

I suspect at some point we will change the eclass-based _pre references to something else. WIll wait to see what the viable versioning in the new portage is first though.
Comment 15 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-03-08 03:00:56 UTC
debug-print is part of ebuild.sh (set  ECLASS_DEBUG_OUTPUT=on for direct output, or set ECLASS_DEBUG_OUTPUT to a filename for output to that file).

I've renamed the print_all_variables to something more useful, and removed it from src_unpack, but left it in handle_genpatches.

I'll commit kernel-2 now, and then update git-sources.
Comment 16 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2006-03-08 03:20:27 UTC
Ok, kernel-2 changes commit, and git-sources updated and cleaned up.