Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 383353

Summary: sys-kernel/vanilla-sources-9999: new live git ebuild
Product: Gentoo Linux Reporter: Rob Stoddard <rob>
Component: [OLD] Core systemAssignee: Default Assignee for New Packages <maintainer-wanted>
Status: RESOLVED WONTFIX    
Severity: enhancement CC: kernel
Priority: Low    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---

Description Rob Stoddard 2011-09-17 13:57:03 UTC
I do a lot of Linux kernel development.  I was wondering why none of my systems, which range from large x86 systems to small arm platforms have their kernels in a git.  I would really like to be able to preserve the Linux kernel history, see changes, etc.

Is there a Linux kernel ebuild that IS in a git?  If there is a compelling reason to not have the regular Linux kernel distribution in a git, for example, the .git directory is HUGE (we all know it is), then why not create a special ebuild for it?

Reproducible: Always

Steps to Reproduce:
1. Load Gentoo
2. go to /usr/src/linux
3. type 'git status'
Actual Results:  
pandaboard linux # git status
fatal: Not a git repository (or any of the parent directories): .git
pandaboard linux # ssh build
Last login: Fri Sep 16 15:56:42 CDT 2011 from 192.168.1.102 on pts/4
bloodyaxes ~ # cd /usr/src/linux
bloodyaxes linux # git status
fatal: Not a git repository (or any of the parent directories): .git


Expected Results:  
# On branch master
nothing to commit (working directory clean)


It is possible to foist a kernel git onto the system, however, all of the patches and neato stuff that Gentoo adds will end up as one huge commit that contains no real history.
Comment 1 Matt Turner gentoo-dev 2011-09-18 02:48:58 UTC
I've thought about this as well.

The .git directory in my linux-2.6.git directory is 700+ MiB, and the unpacked sources would be equal to any other unpacked linux tarball. So, having around 10~ linux tarballs in distfiles would be equal to using git, but you get all the history with git.

I think this might be useful to have.
Comment 2 Kevin Pyle 2011-09-18 16:17:31 UTC
Could you explain what you want the Gentoo maintainers to change?  An ebuild is not a suitable mechanism for distributing a git repository, but you could easily keep a linux-2.6.git repository in your home directory to provide historical data.  The Gentoo patches used in the gentoo-sources-* ebuilds are available from the mirrors and follow a standard naming convention, so you could compute their names by scanning the corresponding gentoo-sources-* ebuild.

The kernel ebuilds are in CVS, like the rest of the Portage tree.
Comment 3 Matt Turner gentoo-dev 2011-09-18 17:22:34 UTC
(In reply to comment #2)
> Could you explain what you want the Gentoo maintainers to change?

The proposal is to offer ebuilds that instead of downloading tarballs, ie linux-2.6.39.tar.bz2, pull the sources from git.

> An ebuild is not a suitable mechanism for distributing a git repository

False. Plenty of ebuilds download their sources from a VCS. We call them live ebuilds, but there's nothing that prevents them from grabbing a specific tag from a VCS.
Comment 4 Mike Gilbert gentoo-dev 2011-09-18 21:30:58 UTC
Just passing this to kernel team to see if there is interest.
Comment 5 Kevin Pyle 2011-09-18 21:44:55 UTC
>> An ebuild is not a suitable mechanism for distributing a git repository
> False. Plenty of ebuilds download their sources from a VCS.

Fetching a git repository as a necessary step in retrieving files for a live ebuild is one thing.  Installing the git repository itself, such that "equery files <package>" lists the source controlled files and the .git directory that manages the history, is quite another.  The reporter's "steps to reproduce" seem to indicate he wants the latter.

Unlike normal live ebuilds, a Linux kernel live ebuild would presumably export the sources and stop so that the user could configure and build them as desired.  I have done the equivalent from outside Portage for quite a while.  Clone Linus' repository once, periodically git fetch, and prepare to build with "git archive --format=tar --prefix=linux-git-$(git describe $tip)/ $tip | tar -C $clean_work_area -x -f -".  Given the relative simplicity of those steps, I have trouble thinking this is worth managing through Portage.  Theoretically, such an ebuild could try to apply patches from the genpatches archive of the nearest gentoo-sources, but given the rate of kernel development, I could see having fairly frequent hunk failures (or worse, successful application of patches that are incompatible with recent upstream changes).

> We call them live ebuilds, but there's nothing that prevents them from
> grabbing a specific tag from a VCS.

Yes, I have used them before.  Grabbing a specific tag for a DVCS is probably not that bad, but the devmanual specifically advises against grabbing a static tag for CVS/SVN due to the higher load of having every Gentoo user exporting a tag versus having the maintainer export it once and create a snapshot tar file.
Comment 6 Stratos Psomadakis (RETIRED) gentoo-dev 2011-09-18 22:12:28 UTC
I don't think it makes any sense. 

If you do kernel development, you can keep a local clone of the mainline repo (or whatever repo you work on).

I really can't see how you could 'distribute' the *whole* kernel git repo with a Portage ebuild.

And even if this was possible, the way genpatches are handled now, makes it impossible. So, you would still only have vanilla-sources.
Comment 7 Anthony Basile gentoo-dev 2011-09-19 00:07:42 UTC
(In reply to comment #3)
> (In reply to comment #2)
> > Could you explain what you want the Gentoo maintainers to change?
> 
> The proposal is to offer ebuilds that instead of downloading tarballs, ie
> linux-2.6.39.tar.bz2, pull the sources from git.
> 
> > An ebuild is not a suitable mechanism for distributing a git repository
> 
> False. Plenty of ebuilds download their sources from a VCS. We call them live
> ebuilds, but there's nothing that prevents them from grabbing a specific tag
> from a VCS.

I with psomas on this one.  I know we have ebuilds that download from VCSs but this is not a good idea because an ebuild targets a clearly defined release, a reference point against which we can file bugs, produce patches etc.  Pulling from vcs gives a moving target.  Why would you want an ebuild for that?  Just cd into /usr/src and git clone the first time, then git pull after that.  If you do a lot of kernel development then this is the *least* of your worries.
Comment 8 Matt Turner gentoo-dev 2011-09-19 00:26:36 UTC
True. I guess there's not a lot of value in this.
Comment 9 Rob Stoddard 2011-09-19 14:33:25 UTC
I guess it really got my goat when I downloaded an ebuild for a kernel for the pandaboard, which is clearly a develop-for-kernel type system, only to realize that there was no history associated with the kernel.  I had no way of telling what commits have been applied or anything like that.  I then got to looking at the kernels for my various machines which I don't typically kernel-develop on, and noticed that they didn't have history either.

Downgraded this to "enhancement".
Comment 10 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2011-10-14 22:52:40 UTC
cd /usr/src
git clone ....
mkdir -p /etc/portage/profile/
echo sys-kernel/git-sources-3.1 >>/etc/portage/profile/package.provided

Or whatever source package you want it to say. This will prevent Portage from trying to bring in any kernel sources package when all you want it a custom git tree instead. This is probably the best way, because your git-hosted sources are going to be changing a lot, and you probably want to change them yourself without an upgrade of a hypothetical package causing it's own changes in that tree.

I think NOTABUG.