Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 275476

Summary: sys-fs/aufs2: new ebuild which fixes some issues with the current one and tries to pull the patch set out of git
Product: Gentoo Linux Reporter: jpike <jpike>
Component: New packagesAssignee: Thomas Sachau <tommy>
Status: RESOLVED WONTFIX    
Severity: enhancement CC: della, devurandom, email, j0inty, jarausch, jlec, victor.pelt, weigt.mathias
Priority: High    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: proposed new ebuild (titled 2-9999 in reference to automatic git checkouts)
sorry attached wrong version of patch
latest version of aufs2-9999 ebuild
fixes branch for patch version 31 not existing.
aufs2-9999.ebuild patch for kernel greater equal 2.6.31

Description jpike 2009-06-26 11:45:10 UTC
Hi,

This shows the new ebuild:

http://github.com/tuxjay/nekoverlay/blob/af66e12640ebce76e30748f726ef3a306b0bc4ea/sys-fs/aufs2/aufs2-9999.ebuild

The first problem is that KV_OUT_DIR should be passed to the in preference to aufs2-standalone/aufs2-util, otherwise the auf build scripts won't be able to find .config if the user has build their kernel out of place (as they should). That changes two references from $KV_DIR to ${KV_OUT_DIR:-$KV_DIR}.

This version will also fetch the aufs2 patch appropriate to your kernel from git (and die with a warning if a patch for your kernel does not exist). The build script modifies this patch to remove the silly defines that prevent the modules from being exported. The gentoo patch required you to do the exact same thing manually and upload it to $FILESDIR first.

I have tested this now with/without the patch applied, and both with

Reproducible: Always

Steps to Reproduce:
1. Install aufs2 against out of place kernel build.
2. Install aufs2 for kernel with no matching gentoo patch.
Actual Results:  
1. aufs2 build scripts fail to detect kernel .config
2. aufs2 kernel-patch must be performed manually by user (euch).

Expected Results:  
1. aufs build scripts should detect actual source dir and use appropriate .config
2. patch should be automatically searched for from aufs2 git repository.
Comment 1 jpike 2009-06-26 11:46:45 UTC
That should say $KV_OUT_DIR should be passed in preference to *$KV_DIR to the aufs2-standalone/aufs2-util build scripts.
Comment 2 jpike 2009-06-26 11:48:03 UTC
Created attachment 195811 [details]
proposed new ebuild (titled 2-9999 in reference to automatic git checkouts)
Comment 3 jpike 2009-06-26 11:54:46 UTC
Created attachment 195812 [details]
sorry attached wrong version of patch

replacement for previous aufs2-9999.ebuild
Comment 4 Federico Della Ricca 2009-07-17 19:12:13 UTC
ebuild does not work for me.
It can't find patch file. My _guess_ is that maybe the git repository has changed something.

****************
[...]
Initialized empty Git repository in /dev/shm/portage/sys-fs/aufs2-9999/work/aufs2-9999/.git/
Switched to a new branch 'branch-master'
>>> Unpacked to /dev/shm/portage/sys-fs/aufs2-9999/work/aufs2-9999
Note: moving to 'origin/aufs2-30' which isn't a local branch
If you want to create a new branch from this checkout, you may do so
(now or later) by using -b with the checkout command again. Example:
  git checkout -b <new_branch_name>
HEAD is now at a604541... aufs2 standalone version for linux-2.6.30
 * Patching your kernel...
/dev/shm/portage/sys-fs/aufs2-9999/temp/environment: line 3455: aufs2-standalone-modified.patch: No such file or directory
 * 
 * ERROR: sys-fs/aufs2-9999 failed.
 * Call stack:
 *               ebuild.sh, line   49:  Called src_unpack
 *             environment, line 3455:  Called die
 * The specific snippet of code:
 *               patch --no-backup-if-mismatch --force -p1 -d ${KV_DIR} < $modified_patch > /dev/null || die "could not apply the kernel patch $modified_patch to your kernel";
 *  The die message:
 *   could not apply the kernel patch aufs2-standalone-modified.patch to your kernel
[...]
Comment 5 jpike 2009-07-18 03:36:59 UTC
Hi,

Yes the modification patch is no longer necessary as he fixed his sources. I also think the patch should be installed to the system in share so the user can easily revert the patch.

I'll fix my git repository then upload the new patch.

James
Comment 6 Helmut Jarausch 2009-07-23 09:07:57 UTC
(In reply to comment #5)
> Hi,
> 
> Yes the modification patch is no longer necessary as he fixed his sources. I
> also think the patch should be installed to the system in share so the user can
> easily revert the patch.
> 
> I'll fix my git repository then upload the new patch.
> 

Is it assumed to work with a 2.6.30 kernel, as well? (Here it doesn't)
Thanks,
Helmut.

Comment 7 jpike 2009-07-23 11:42:36 UTC
Hi,

I've just tested it right now on a 2.6.30.2 kernel. I'm attaching a later version of the ebuild.

James
Comment 8 jpike 2009-07-23 11:43:39 UTC
Created attachment 198893 [details, diff]
latest version of aufs2-9999 ebuild
Comment 9 jpike 2009-07-23 11:45:34 UTC
(In reply to comment #6)

> Is it assumed to work with a 2.6.30 kernel, as well? (Here it doesn't)
> Thanks,
> Helmut.

Bear in mind you'll have to rebuild your kernel after this ebuild patches it for the aufs2 kernel module to start working (aufs2 requires some symbols to be exported that by default aren't, the patch is very minor).

If there is still a problem other than this could you pastebin your error somewhere?
Comment 10 Thomas Sachau gentoo-dev 2009-07-30 18:11:58 UTC
This ebuild tries to write to the kernel source dir inside a src_-phase. This will always fail with FEAURES="userpriv". Since i recommend this feature, i wont add this ebuild to the main tree.

You are of course free to use it for youself, but you will also have to deal yourself with the problems you find with it.
Comment 11 jpike 2009-07-30 20:59:08 UTC
(In reply to comment #10)
> This ebuild tries to write to the kernel source dir inside a src_-phase. This
> will always fail with FEAURES="userpriv". Since i recommend this feature, i
> wont add this ebuild to the main tree.
> 
> You are of course free to use it for youself, but you will also have to deal
> yourself with the problems you find with it.

The only alternative to this is to unpack the source in the pkg_setup phase. Would you find this acceptable?

If neither of these options are options are acceptable then an ebuild like this is impossible and points to a deficiency within either the ideology or the ebuild system.

Personally I don't care about some ideological purity if a feature is provided that people need and given that they can be informed of and accept the risk.
Comment 12 Helmut Jarausch 2009-07-31 10:35:43 UTC
Even this version doesn't work with 2.6.30 (gentoo-sources-2.6.30-r4)
Comment 13 jpike 2009-07-31 11:39:26 UTC
(In reply to comment #12)
> Even this version doesn't work with 2.6.30 (gentoo-sources-2.6.30-r4)

And yet it works with vanilla-sources 2.6.30. This is likely due to the aufs2 patches not applying to gentoo-sources.

It's not much use telling me it doesn't work without providing some output so that I can work out why it doesn't work in your case.
Comment 14 Thomas Sachau gentoo-dev 2009-07-31 12:49:01 UTC
(In reply to comment #11)
> (In reply to comment #10)
> > This ebuild tries to write to the kernel source dir inside a src_-phase. This
> > will always fail with FEAURES="userpriv". Since i recommend this feature, i
> > wont add this ebuild to the main tree.
> > 
> > You are of course free to use it for youself, but you will also have to deal
> > yourself with the problems you find with it.
> 
> The only alternative to this is to unpack the source in the pkg_setup phase.
> Would you find this acceptable?
> 
> If neither of these options are options are acceptable then an ebuild like this
> is impossible and points to a deficiency within either the ideology or the
> ebuild system.
> 
> Personally I don't care about some ideological purity if a feature is provided
> that people need and given that they can be informed of and accept the risk.
> 

Unpacking inside pkg_setup is a no-go. Kernel patches should normally not be applied from within an ebuild, especially no checked/tested ones. For local usages, you could either restrict userpriv (not recommended) or drop the kernel-patch useflag, tell the user the path to the patch and require him to apply it by hand, if it is not already applied to the current kernel.
Comment 15 jpike 2009-07-31 13:16:45 UTC
> Unpacking inside pkg_setup is a no-go. Kernel patches should normally not be
> applied from within an ebuild, especially no checked/tested ones. For local
> usages, you could either restrict userpriv (not recommended) or drop the
> kernel-patch useflag, tell the user the path to the patch and require him to
> apply it by hand, if it is not already applied to the current kernel.

Thanks for your advice. Okay then I propose the following changes:

1. I will drop the kernel-patch useflag.
2. I will not apply the patch in the ebuild, but check if it is already applied. If it is not applied I will print out an enotice with a command informing the user of the command they should execute to apply the patch and then fail the ebuild (the command will include the patch to the path in /var/tmp/portage/... if this is acceptable).

This patch is tested in so much that I use it heavily, but I definitely must test this ebuild against gentoo-sources-2.6.30. For the .30* series I've only tested against vanilla-sources.

I'd just go ahead and make the changes but I'd like to know if the ebuild will be considered first.
Comment 16 Federico Della Ricca 2009-08-01 08:47:30 UTC
I was able to make it work with git-sources-2.6.31_rc4-r6 with the following modification to the ebuild:

-git checkout origin/aufs2-${KV_PATCH} ||die "no patch exists for your kernel"
+git checkout origin/aufs2-${KV_PATCH} || git checkout origin/aufs2 ||die "no patch exists for your kernel"

because the support for kernel 2.6.31 is in the master branch and not in a separete branch (i.e. aufs2-31 does not exist)

I had also to modify the file /usr/src/linux/include/asm-generic/int-ll64.h at line 11:

-#include <asm/bitsperlong.h>
+#include <asm-generic/bitsperlong.h>


Hope this helps someone else.
Comment 17 Andreas Schürch gentoo-dev 2009-09-16 07:43:25 UTC
(In reply to comment #16)

> Hope this helps someone else.
> 
For sure it did! :-)

I've just tested it with gentoo-sources-2.6.31 and with your two small adjustments it just worked as expected! :-)

Thanks!
Comment 18 jpike 2009-09-16 17:39:50 UTC
Created attachment 204343 [details]
fixes branch for patch version 31 not existing.

This version includes the fix for branch 31 not existing. The kernel patch is either fixed now or a problem with gentoo-sources as I've tested it fine on vanilla-sources-2.6.31.

Could someone please answer my previous question. Many people use aufs2 in gentoo e.g. sysrescuecd etc so I'm surprised noone from gentoo cares about this ebuild enough to bother with it.
Comment 19 Thomas Sachau gentoo-dev 2009-09-16 19:23:19 UTC
(In reply to comment #18)
> Created an attachment (id=204343) [edit]
> fixes branch for patch version 31 not existing.
> 
> This version includes the fix for branch 31 not existing. The kernel patch is
> either fixed now or a problem with gentoo-sources as I've tested it fine on
> vanilla-sources-2.6.31.
> 
> Could someone please answer my previous question. Many people use aufs2 in
> gentoo e.g. sysrescuecd etc so I'm surprised noone from gentoo cares about this
> ebuild enough to bother with it.
> 

I added the aufs2 ebuild to the main tree, so that statement isnt true. But i have to admit that i dont use aufs2 and that i usually dont use latest avaidable kernel sources, so updates to support the latest kernel are low priority for me.

Can you tell me, why you modify the kernel patch?
Comment 20 jpike 2009-09-17 14:02:36 UTC
I modify the patch as it used to be incorrect. It used to only export the required symbols if a certain preprocessor symbol was defined, but provided no way to define it.

This problem only exists on the 27 and 28 branches I believe, and is fixed now. The modfication to the patch therefore does nothing on recent kernel versions, but it doesn't hurt to leave it in in case the fix hasn't been backported to the older branches.
Comment 21 Mark A Rada 2009-09-19 04:22:24 UTC
(In reply to comment #19)
> (In reply to comment #18)
> > Created an attachment (id=204343) [edit]
> > fixes branch for patch version 31 not existing.
> > 
> > This version includes the fix for branch 31 not existing. The kernel patch is
> > either fixed now or a problem with gentoo-sources as I've tested it fine on
> > vanilla-sources-2.6.31.
> > 
> > Could someone please answer my previous question. Many people use aufs2 in
> > gentoo e.g. sysrescuecd etc so I'm surprised noone from gentoo cares about this
> > ebuild enough to bother with it.
> > 
> 
> I added the aufs2 ebuild to the main tree, so that statement isnt true. But i
> have to admit that i dont use aufs2 and that i usually dont use latest
> avaidable kernel sources, so updates to support the latest kernel are low
> priority for me.
> 
> Can you tell me, why you modify the kernel patch?
> 

Would not compile for me. It starts off fine, then starts to fail around here:

  CC [M]  /var/tmp/portage/sys-fs/aufs2-0_p20090916/work/aufs2-standalone/fs/aufs/vfsub.o
/var/tmp/portage/sys-fs/aufs2-0_p20090916/work/aufs2-standalone/fs/aufs/vfsub.c: In function ‘vfsub_
lookup_hash’:
/var/tmp/portage/sys-fs/aufs2-0_p20090916/work/aufs2-standalone/fs/aufs/vfsub.c:124: error: implicit
 declaration of function ‘lookup_hash’
/var/tmp/portage/sys-fs/aufs2-0_p20090916/work/aufs2-standalone/fs/aufs/vfsub.c:124: warning: assign
ment makes pointer from integer without a cast/var/tmp/portage/sys-fs/aufs2-0_p20090916/work/aufs2-standalone/fs/aufs/vfsub.c: In function ‘vfsub_
splice_to’:
/var/tmp/portage/sys-fs/aufs2-0_p20090916/work/aufs2-standalone/fs/aufs/vfsub.c:493: error: implicit
 declaration of function ‘do_splice_to’
/var/tmp/portage/sys-fs/aufs2-0_p20090916/work/aufs2-standalone/fs/aufs/vfsub.c: In function ‘vfsub_splice_from’:
/var/tmp/portage/sys-fs/aufs2-0_p20090916/work/aufs2-standalone/fs/aufs/vfsub.c:506: error: implicit declaration of function ‘do_splice_from’
make[2]: *** [/var/tmp/portage/sys-fs/aufs2-0_p20090916/work/aufs2-standalone/fs/aufs/vfsub.o] Error 1
make[1]: *** [_module_/var/tmp/portage/sys-fs/aufs2-0_p20090916/work/aufs2-standalone/fs/aufs] Error 2
make[1]: Leaving directory `/usr/src/linux-2.6.31-gentoo'
make: *** [fs/aufs/aufs.ko] Error 2
 *
 * ERROR: sys-fs/aufs2-0_p20090916 failed.
 * Call stack:
 *               ebuild.sh, line   49:  Called src_compile
 *             environment, line 3468:  Called die
 * The specific snippet of code:
 *       emake CC=$(tc-getCC) CONFIG_AUFS_FS=m KDIR=${KV_DIR} || die;
 *  The die message:
 *   (no error message)
 *
 * If you need support, post the topmost build error, and the call stack if relevant.
 * A complete build log is located at '/var/tmp/portage/sys-fs/aufs2-0_p20090916/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sys-fs/aufs2-0_p20090916/temp/environment'.


Comment 22 Federico Della Ricca 2009-09-22 17:01:46 UTC
Also for me the ebuild in Portage (aufs2-0_p20090916) fails in the same manner as comment #21.

Ebuild from comment #18 also fails, but I was able to make it compile with single line patch (see my comment #16)

/usr/src/linux/include/asm-generic/int-ll64.h line 11:

-#include <asm/bitsperlong.h>
+#include <asm-generic/bitsperlong.h>



Comment 23 Steffen 'j0inty' Stollfuß 2009-10-06 11:12:00 UTC
Hi@ll

I found a solution for the kernel release 2.6.31. I tested it here on my x86 machine with the gentoo sources.

http://www.mail-archive.com/aufs-users@lists.sourceforge.net/msg02195.html

I will attach patch for the lastet ebuild for the aufs2-9999 package, too.

Steps to reproduce a successfull compile and installation process:

 1. Unemerge exiting and already aufs2 patched sources. (emerge -avC =gentoo-source-2.6.31-r1)
 2. Backup your .config from /usr/src/linux if neccessary.
 3. Remove the folder from /usr/src (rm -R /usr/src/linux-2.6.31-gentoo-r1)
 4. Install the sources again (emerge -av =gentoo-source-2.6.31-r1)
 5. Let's now generate a new config and prepare the source (make oldconfig && make prepare)
 6. Let the ebuild patching your kernel and it should fail after the compile process with the same output as here http://de.pastebin.ca/1597462
 7. Now compile your kernel, install the modules and reboot your computer
 8. Apply manually the patch from Comment 22 here http://bugs.gentoo.org/show_bug.cgi?id=275476#c22 .
 9. emerge the aufs2 module which should end in this output http://de.pastebin.ca/1597505 here.
 10. Check you dmesg for minimal this line after insert it to the kernel. ([    5.117751] aufs 2-standalone.tree-31-20091005)
 11. HF&GL

My dmesg and emerge --info can you find here under this link:
http://de.pastebin.ca/1597549

regards
j0inty
Comment 24 Steffen 'j0inty' Stollfuß 2009-10-06 11:13:59 UTC
Created attachment 206237 [details, diff]
aufs2-9999.ebuild patch for kernel greater equal 2.6.31
Comment 25 Mathias Weigt 2009-10-07 13:09:15 UTC
(In reply to comment #24)
> Created an attachment (id=206237) [details]
> aufs2-9999.ebuild patch for kernel greater equal 2.6.31
> 

Thanks, that worked for me...
Comment 26 Thomas Sachau gentoo-dev 2009-10-08 17:39:37 UTC
(In reply to comment #22)
> Also for me the ebuild in Portage (aufs2-0_p20090916) fails in the same manner
> as comment #21.
> 
> Ebuild from comment #18 also fails, but I was able to make it compile with
> single line patch (see my comment #16)
> 
> /usr/src/linux/include/asm-generic/int-ll64.h line 11:
> 
> -#include <asm/bitsperlong.h>
> +#include <asm-generic/bitsperlong.h>
> 

You really shouldnt submit bugreports for in-tree ebuilds in bugs open for different things, instead, you should open a different bug.

@jpike: I dont think, that i will add a live ebuild for aufs2 in the near future, i dont have much free time and the live ebuild tends to break and would need additional work and time. Instead, i will leave this bug open for people interested in a live ebuild, as long as people dont use it for bug reports for the in-tree version.
Comment 27 Steffen 'j0inty' Stollfuß 2009-11-26 16:44:43 UTC
Hi,

If you try to export an aufs mountpoint via nfs it fails, because the current ebuild doesn't build the aufs2 module with nfs support.

I made a patch for the current aufs2-9999.ebuild an post it to another bugreport.

http://bugs.gentoo.org/show_bug.cgi?id=294653

Please test that patch and give response if it work successfully or not.

regards
j0inty
Comment 28 Chris Chan 2010-04-06 03:51:49 UTC
This bug also haunts me, in dmesg I see it complaining about various invalid keywords. Here's some info:
__________________________________________________________________________________________
[I] sys-kernel/gentoo-sources
     Installed versions:  2.6.31-r10(2.6.31-r10)!b!s(16:40:14 04/05/10)(-build -symlink)
__________________________________________________________________________________________
[I] sys-fs/aufs2
      Installed versions:  0_p20100405(23:46:38 04/05/10)(kernel-patch kernel_linux)
__________________________________________________________________________________________
~ emerge --info
Portage 2.1.7.17 (default/linux/amd64/10.0/desktop, gcc-4.3.4, glibc-2.10.1-r1, 2.6.31-gentoo-r10 x86_64)
=================================================================
System uname: Linux-2.6.31-gentoo-r10-x86_64-Intel-R-_Core-TM-2_Duo_CPU_T9400_@_2.53GHz-with-gentoo-1.12.13
Timestamp of tree: Mon, 05 Apr 2010 20:45:03 +0000
app-shells/bash:     4.0_p35
dev-lang/python:     2.6.4-r1
dev-util/cmake:      2.6.4-r3
sys-apps/baselayout: 1.12.13
sys-apps/sandbox:    1.6-r2
sys-devel/autoconf:  2.63-r1
sys-devel/automake:  1.10.3
sys-devel/binutils:  2.18-r3
sys-devel/gcc:       4.3.4
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6b
virtual/os-headers:  2.6.30-r1
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=nocona -fomit-frame-pointer -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/X11/xkb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -march=nocona -fomit-frame-pointer -pipe"
DISTDIR="/tmp/dist"
EMERGE_DEFAULT_OPTS="--jobs=2 -q"
FEATURES="assume-digests distlocks fixpackages parallel-fetch parallel_fetch protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch"
GENTOO_MIRRORS="ftp://mirror.csclub.uwaterloo.ca/gentoo-distfiles/ "
LDFLAGS="-Wl,-O1"
MAKEOPTS="-j2"
PKGDIR="/tmp/pkg"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--exclude-from=/etc/portage/rsync_excludes"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/tmp/ram/ebuild"
PORTDIR="/mmc/port"
SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"
USE="amd64 multilib readline" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="intel" 
Unset:  CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTDIR_OVERLAY
______________________________________________________________________________________________________
Comment 29 Thomas Sachau gentoo-dev 2012-07-01 13:45:52 UTC
closing this bug as wontfix, since a live version would cause too much maintainence work. If there are any specific issues with versioned releases, please open a new bug against them.