Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 506666 - Linux >=3.14: Extended attributes cannot be set by non-root users
Summary: Linux >=3.14: Extended attributes cannot be set by non-root users
Status: RESOLVED OBSOLETE
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
: 506936 520042 (view as bug list)
Depends on:
Blocks:
 
Reported: 2014-04-03 14:52 UTC by Tobias Klausmann (RETIRED)
Modified: 2014-12-26 00:26 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
mercurial-2.9.2 build log (build.log,182.16 KB, text/x-log)
2014-04-04 09:54 UTC, Tobias Klausmann (RETIRED)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Tobias Klausmann (RETIRED) gentoo-dev 2014-04-03 14:52:29 UTC
I recently noticed some ebuilds failing when compiling on one of my beefier machines. As it turns out, the difference is using tmpfs. 

Portage seems to use pyxattr to manipulate ACLs during build and/or install phase.

The problem is that non-root users are not allowed to set xattrs on tmpfs, which is not entirely uncommon to use for /var/tmp/portage.

Thus, the build fails on tmpfs with sandboxing.

Removing pyxattr makes things work again, since the alternative (slow) mode fails gracefully.

Unfortunately, some dependencies pull in pyxattr, which then gets used by portage unconditionally, making builds break for everyone with tmpfs. One example is mercurial. Building it will keep breaking until people move away from tmpfs for /var/tmp/portage.
Comment 1 Arfrever Frehtes Taifersar Arahesis 2014-04-03 18:27:48 UTC
Show example failure and output of `emerge --info`.
Comment 2 Tobias Klausmann (RETIRED) gentoo-dev 2014-04-04 09:53:59 UTC
writing byte-compilation script '/var/tmp/portage/dev-vcs/mercurial-2.9.2/temp/python2.7/tmp6321wb.py'
/usr/bin/python2.7 -OO /var/tmp/portage/dev-vcs/mercurial-2.9.2/temp/python2.7/tmp6321wb.py
removing /var/tmp/portage/dev-vcs/mercurial-2.9.2/temp/python2.7/tmp6321wb.py
running install_scripts
creating /var/tmp/portage/dev-vcs/mercurial-2.9.2/image/_python2.7/usr/lib
creating /var/tmp/portage/dev-vcs/mercurial-2.9.2/image/_python2.7/usr/lib/python-exec
creating /var/tmp/portage/dev-vcs/mercurial-2.9.2/image/_python2.7/usr/lib/python-exec/python2.7
copying /var/tmp/portage/dev-vcs/mercurial-2.9.2/work/mercurial-2.9.2-python2_7/scripts/hg -> /var/tmp/portage/dev-vcs/mercurial-2.9.2/image//_python2.7/usr/lib/python-exec/python2.7
changing mode of /var/tmp/portage/dev-vcs/mercurial-2.9.2/image//_python2.7/usr/lib/python-exec/python2.7/hg to 755
running install_egg_info
Writing /var/tmp/portage/dev-vcs/mercurial-2.9.2/image//_python2.7/usr/lib64/python2.7/site-packages/mercurial-2.9.2-py2.7.egg-info
 * python2_7: running distutils-r1_run_phase python_install_all
 * python2_7: running python_doscript contrib/hg-ssh
Traceback (most recent call last):
  File "/usr/lib64/portage/bin/install.py", line 180, in copy_xattrs
    _copyxattr(s, abs_path, exclude=exclude)
  File "/usr/lib64/portage/pym/portage/util/movefile.py", line 129, in _copyxattr
    (_unicode_decode(dest), _unicode_decode(attr)))
OperationNotSupported: Filesystem containing file '/var/tmp/portage/dev-vcs/mercurial-2.9.2/image//etc/mercurial/hgrc.d/cacerts.rc' does not support extended attribute 'system.posix_acl_access'
!!! install: copy_xattrs failed with the following arguments: -m0644 /usr/portage/dev-vcs/mercurial/files/cacerts.rc /var/tmp/portage/dev-vcs/mercurial-2.9.2/image//etc/mercurial/hgrc.d/
!!! doins: /usr/portage/dev-vcs/mercurial/files/cacerts.rc does not exist
 * ERROR: dev-vcs/mercurial-2.9.2::gentoo failed (install phase):
 *   doins failed
 * 
 * If you need support, post the output of `emerge --info '=dev-vcs/mercurial-2.9.2::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=dev-vcs/mercurial-2.9.2::gentoo'`.
 * The complete build log is located at '/var/log/portage/dev-vcs:mercurial-2.9.2:20140404-095159.log'.
 * For convenience, a symlink to the build log is located at '/var/tmp/portage/dev-vcs/mercurial-2.9.2/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/dev-vcs/mercurial-2.9.2/temp/environment'.
 * Working directory: '/var/tmp/portage/dev-vcs/mercurial-2.9.2/work/mercurial-2.9.2'
 * S: '/var/tmp/portage/dev-vcs/mercurial-2.9.2/work/mercurial-2.9.2'
 * QA Notice: file does not exist:
 * 
 * 	doins: /usr/portage/dev-vcs/mercurial/files/cacerts.rc does not exist

>>> Failed to emerge dev-vcs/mercurial-2.9.2, Log file:

>>>  '/var/log/portage/dev-vcs:mercurial-2.9.2:20140404-095159.log'

(full log will be attached in a moment)
# equery l pyxattr
 * Searching for pyxattr ...
[IP-] [  ] dev-python/pyxattr-0.5.2:0

# emerge --info
Portage 2.2.10 (default/linux/amd64/13.0, gcc-4.8.2, glibc-2.19, 3.14.0 x86_64)
=================================================================
System uname: Linux-3.14.0-x86_64-Intel-R-_Core-TM-_i7-2600K_CPU_@_3.40GHz-with-gentoo-2.2
KiB Mem:    16389864 total,  15780604 free
KiB Swap:          0 total,         0 free
Timestamp of tree: Fri, 04 Apr 2014 00:45:01 +0000
ld GNU ld (GNU Binutils) 2.24
distcc 3.1 x86_64-pc-linux-gnu [disabled]
app-shells/bash:          4.2_p46-r1
dev-java/java-config:     2.2.0
dev-lang/python:          2.7.6, 3.3.5, 3.4.0
dev-util/cmake:           2.8.12.2
dev-util/pkgconfig:       0.28-r1
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.4
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.11.6, 1.12.6, 1.14.1
sys-devel/binutils:       2.24-r2
sys-devel/gcc:            4.8.2
sys-devel/gcc-config:     1.8
sys-devel/libtool:        2.4.2
sys-devel/make:           4.0-r1
sys-kernel/linux-headers: 3.14 (virtual/os-headers)
sys-libs/glibc:           2.19
Repositories: gentoo local
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -pipe -march=native"
DISTDIR="/mnt/moep/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs buildsyspkg config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://mirror.switch.ch/ftp/mirror/gentoo/"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j8"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--exclude lost+found"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync5.de.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext X a52 acl alsa amd64 audiofile bash-completion berkdb bmp bzip2 caps cdparanoia cdr cli cracklib crypt cxx dri dts encode exif ffmpeg fftw fortran gdbm gif gtk iconv idn ipv6 jpeg lame lcms lua mad matroska mmap mmx mmxext mng modules mp3 mpeg multilib ncurses nls nptl ogg opengl openmp pam pcre png ppds pulseaudio python qt4 readline recode session sse sse2 ssl svg tcpd theora unicode usb v4l vim vim-pager vim-syntax vlm vorbis wmf xattr xcb xosd xpm xscreensaver xvid zlib" ABI_X86="64" ALSA_CARDS="emu10k1 ice1724" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" ENLIGHTENMENT_MODULES="clock comp conf-applications conf-dialogs conf-display conf-edgebindings conf-interaction conf-intl conf-keybindings conf-menus conf-paths conf-performance conf-randr conf-shelves conf-theme conf-window-manipulation conf-window-remembers cpufreq dropshadow fileman fileman-opinfo ibar ibox illume2 mixer msgbus notification pager quickaccess start syscon systray tasks temperature tiling winlist wizard xkbswitch" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" QEMU_SOFTMMU_TARGETS="i386 x86_64 alpha" QEMU_USER_TARGETS="i386 x86_64 alpha" RUBY_TARGETS="ruby19 ruby20" SANE_BACKENDS="canon microtek microtek2 mustek mustek_usb nec plustek" USERLAND="GNU" VIDEO_CARDS="vesa radeonhd radeon r700 r600" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, USE_PYTHON
Comment 3 Tobias Klausmann (RETIRED) gentoo-dev 2014-04-04 09:54:45 UTC
Created attachment 374238 [details]
mercurial-2.9.2  build log
Comment 4 Arfrever Frehtes Taifersar Arahesis 2014-04-04 21:18:38 UTC
Does this problem occur with Linux 3.13.*?
Comment 5 Tobias Klausmann (RETIRED) gentoo-dev 2014-04-05 10:56:52 UTC
I just tested and it's not present with 3.13.9, but ISTR that it was present with earlier versions of either 3.13 or 3.12
Comment 6 Arfrever Frehtes Taifersar Arahesis 2014-04-07 11:33:52 UTC
Kernel maintainers: It looks like a regression in 3.14.
Comment 7 Michael Palimaka (kensington) gentoo-dev 2014-04-07 11:35:36 UTC
*** Bug 506936 has been marked as a duplicate of this bug. ***
Comment 8 Tobias Klausmann (RETIRED) gentoo-dev 2014-04-07 14:12:00 UTC
I'm not sure it's a regression in the Kernel. AIUI, the error is caused because the user is not root because editing/adding xattr is a venue for DoS.
Comment 9 Arfrever Frehtes Taifersar Arahesis 2014-04-07 18:04:04 UTC
I am able to change system.posix_acl_access attribute as non-root user (even on tmpfs) in 3.13.9. I have not tried 3.14.
Comment 10 Stuart Shelton 2014-04-07 21:15:42 UTC
As per Bug 506936, this happens for me when running as root with PORTAGE_TMPDIR on nfs (options: rw,insecure,async,no_subtree_check,mp=/storage,no_root_squash,anonuid=250,anongid=250 - so 'root' really should be UID 0), but only for dev-util/cmake:

-- Installing: /storage/tmp/portage/dev-util/cmake-2.8.12.2/image/usr/share/cmake/completions/ctest
Traceback (most recent call last):
  File "/usr/lib/portage/bin/install.py", line 180, in copy_xattrs
    _copyxattr(s, abs_path, exclude=exclude)
  File "/usr/lib/portage/pym/portage/util/movefile.py", line 129, in _copyxattr
    (_unicode_decode(dest), _unicode_decode(attr)))
OperationNotSupported: Filesystem containing file '/storage/tmp/portage/dev-util/cmake-2.8.12.2/image//usr/share/vim/vimfiles/ftdetect/cmake.vim' does not support extended attribute 'system.posix_acl_access'
!!! install: copy_xattrs failed with the following arguments: -m0644 /usr/portage/dev-util/cmake/files/cmake.vim /storage/tmp/portage/dev-util/cmake-2.8.12.2/image//usr/share/vim/vimfiles/ftdetect/
!!! doins: /usr/portage/dev-util/cmake/files/cmake.vim does not exist
Comment 11 Stuart Shelton 2014-04-10 22:16:52 UTC
Also affected is sys-apps/sysvinit-2.88-r7:

make: Leaving directory `/storage/tmp/portage/sys-apps/sysvinit-2.88-r7/work/sysvinit-2.88dsf/src'
Traceback (most recent call last):
  File "/usr/lib/portage/bin/install.py", line 180, in copy_xattrs
    _copyxattr(s, abs_path, exclude=exclude)
  File "/usr/lib/portage/pym/portage/util/movefile.py", line 129, in _copyxattr
    (_unicode_decode(dest), _unicode_decode(attr)))
OperationNotSupported: Filesystem containing file '/storage/tmp/portage/sys-apps/sysvinit-2.88-r7/image//etc/init.d/reboot.sh' does not support extended attribute 'system.posix_acl_access'
!!! install: copy_xattrs failed with the following arguments: -m0755 /usr/portage/sys-apps/sysvinit/files/reboot.sh /storage/tmp/portage/sys-apps/sysvinit-2.88-r7/image//etc/init.d/
Traceback (most recent call last):
  File "/usr/lib/portage/bin/install.py", line 180, in copy_xattrs
    _copyxattr(s, abs_path, exclude=exclude)
  File "/usr/lib/portage/pym/portage/util/movefile.py", line 129, in _copyxattr
    (_unicode_decode(dest), _unicode_decode(attr)))
OperationNotSupported: Filesystem containing file '/storage/tmp/portage/sys-apps/sysvinit-2.88-r7/image//etc/init.d/shutdown.sh' does not support extended attribute 'system.posix_acl_access'
!!! install: copy_xattrs failed with the following arguments: -m0755 /usr/portage/sys-apps/sysvinit/files/shutdown.sh /storage/tmp/portage/sys-apps/sysvinit-2.88-r7/image//etc/init.d/


... how can I prevent portage from trying to make use of pyxattr - it's clearly not working as intended?
Comment 12 Arfrever Frehtes Taifersar Arahesis 2014-04-11 14:13:48 UTC
Problem occurs when ${FILESDIR} is on a filesystem supporting ACL, while ${PORTAGE_TMPDIR} (which contains e.g. ${ED}) is on a filesystem not supporting ACL.
Comment 13 Stuart Shelton 2014-04-11 18:24:15 UTC
Not necessarily just this situation - in my case, the $PORTAGE_TMPDIR and $FILESDIR directories are on the same NFS export, and I'm seeing the issue (for a couple of ebuilds, at least).

(In reply to Arfrever Frehtes Taifersar Arahesis from comment #12)
> Problem occurs when ${FILESDIR} is on a filesystem supporting ACL, while
> ${PORTAGE_TMPDIR} (which contains e.g. ${ED}) is on a filesystem not
> supporting ACL.
Comment 14 Stuart Shelton 2014-04-22 12:05:13 UTC
I'm also seeing:

# tune2fs -l /dev/sde2
...
Filesystem features:      has_journal ext_attr resize_inode dir_index filetype needs_recovery extent flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags:         signed_directory_hash
Default mount options:    user_xattr acl
...

Within a chroot() gaol on this ext4 filesystem:

# PORTAGE_TMPDIR="/tmp" FEATURES="-xattr" emerge -ev @system
...
if test -r /tmp/portage/sys-libs/glibc-2.17/image//usr/include/gnu/stubs-hard.h && cmp -s /tmp/portage/sys-libs/glibc-2.17/work/build-default-armv6zk-hardfloat-linux-gnueabi-nptl/stubs.h /tmp/portage/sys-libs/glibc-2.17/image//usr/include/gnu/stubs-hard.h; \
then echo 'stubs.h unchanged'; \
else /tmp/portage/._portage_reinstall_.87P9zt/bin/ebuild-helpers/xattr/install -c -m 644 /tmp/portage/sys-libs/glibc-2.17/work/build-default-armv6zk-hardfloat-linux-gnueabi-nptl/stubs.h /tmp/portage/sys-libs/glibc-2.17/image//usr/include/gnu/stubs-hard.h; fi
rm -f /tmp/portage/sys-libs/glibc-2.17/work/build-default-armv6zk-hardfloat-linux-gnueabi-nptl/stubs.h
/tmp/portage/sys-libs/glibc-2.17/work/build-default-armv6zk-hardfloat-linux-gnueabi-nptl/elf/sln /tmp/portage/sys-libs/glibc-2.17/work/build-default-armv6zk-hardfloat-linux-gnueabi-nptl/elf/symlink.list
 * Unable to trace static ELF: /tmp/portage/sys-libs/glibc-2.17/work/build-default-armv6zk-hardfloat-linux-gnueabi-nptl/elf/sln: /tmp/portage/sys-libs/glibc-2.17/work/build-default-armv6zk-hardfloat-linux-gnueabi-nptl/elf/sln /tmp/portage/sys-libs/glibc-2.17/work/build-default-armv6zk-hardfloat-linux-gnueabi-nptl/elf/symlink.list
rm -f /tmp/portage/sys-libs/glibc-2.17/work/build-default-armv6zk-hardfloat-linux-gnueabi-nptl/elf/symlink.list
make[1]: Leaving directory `/tmp/portage/sys-libs/glibc-2.17/work/glibc-2.17'
Traceback (most recent call last):
  File "/tmp/portage/._portage_reinstall_.87P9zt/bin/install.py", line 180, in copy_xattrs
    _copyxattr(s, abs_path, exclude=exclude)
  File "/tmp/portage/._portage_reinstall_.87P9zt/pym/portage/util/movefile.py", line 129, in _copyxattr
    (_unicode_decode(dest), _unicode_decode(attr)))
OperationNotSupported: Filesystem containing file '/tmp/portage/sys-libs/glibc-2.17/image//usr/lib/systemd/system/nscd.service' does not support extended attribute 'system.posix_acl_access'
!!! install: copy_xattrs failed with the following arguments: -m0644 /usr/portage/sys-libs/glibc/files/nscd.service /tmp/portage/sys-libs/glibc-2.17/image//usr/lib/systemd/system/
!!! doins: /usr/portage/sys-libs/glibc/files/nscd.service does not exist
doins failed
 * ERROR: sys-libs/glibc-2.17::gentoo failed (install phase):
 *   (no error message)
 *
 * Call stack:
 *           ebuild.sh, line   93:  Called src_install
 *         environment, line 3204:  Called eblit-run 'src_install'
 *         environment, line  898:  Called eblit-glibc-src_install
 *   src_install.eblit, line  277:  Called toolchain-glibc_src_install
 *   src_install.eblit, line  194:  Called die
 * The specific snippet of code:
 *                      systemd_dounit "${FILESDIR}"/nscd.service || die

... at the very least, this should be a soft-failure?
Comment 15 Tobias Klausmann (RETIRED) gentoo-dev 2014-04-23 14:58:42 UTC
What changed is that starting with 3.14, non-uid-0 users are not allowed to manipulate xattrs freely -- this is due to a security concern (DoS). Since the install phase does not run as root, the attribute-copy fails.
Comment 16 Tobias Klausmann (RETIRED) gentoo-dev 2014-04-23 14:59:57 UTC
nb: with 3.15.0-rc2 this fails on _all_ filesystems I, so I have to remove pyxattr before merging, say, qemu, then remerge it afterwards.
Comment 17 Arfrever Frehtes Taifersar Arahesis 2014-04-23 15:25:09 UTC
Uninstallation of dev-python/pyxattr makes little sense.
With current default Python version in Gentoo (i.e. 3.3), Portage does not use dev-python/pyxattr, but uses os.*xattr() functions from Python standard library.
Use FEATURES="-xattr" as a workaround.

It does not make sense to disallow users to set extended attributes.
Users need this functionality e.g. for ACL (which is implemented using extended attributes on many filesystems).
Comment 18 Tobias Klausmann (RETIRED) gentoo-dev 2014-04-23 15:57:32 UTC
There seem to be two different ways in which Portage deals with xattr: through pyxattr or calling an external binary. The external binary works, pyxattr does not. 

Unfortunately, I have pyxattr installed as a hard dep of something else, so I can't just remove it.
Comment 19 Arfrever Frehtes Taifersar Arahesis 2014-04-23 16:15:17 UTC
No. There are 3 ways:

- With Python >=3.3 Portage can use os.listxattr(), os.getxattr() and os.setxattr() from standard library.
- With Python <3.3 Portage can use xattr module from dev-python/pyxattr.
- There is fallback to getfattr and setfattr executables.

(In reply to Tobias Klausmann from comment #18)
> The external binary works

Does it succeed to set system.posix_acl_access attribute or does it silently fail (with 0 exit status)?
Comment 20 Tobias Klausmann (RETIRED) gentoo-dev 2014-04-24 12:46:59 UTC
So I have tried these combinations:

1 pyxattr installed, USE="python3 xattr": fails
2 pyxattr installed, USE="python2 xattr": fails
3 pyxattr installed, USE="python2 -xattr": works
4 pyxattr installed, USE="python3 -xattr": works
5 pyxattr removed, USE="python3 -xattr": works
6 pyxattr removed, USE="python2 -xattr": works
7 pyxattr removed, USE="python2 xattr": works
8 pyxattr removed, USE="python3 xattr": fails

And then emerge xdvik. "Fails" means that the merge fails in the install phase, with this message:

make[1]: Leaving directory '/var/tmp/portage/app-text/xdvik-22.87/work/xdvik-22.87/texk/xdvik/tests'
Traceback (most recent call last):
  File "/usr/lib64/portage/bin/install.py", line 180, in copy_xattrs
    _copyxattr(s, abs_path, exclude=exclude)
  File "/usr/lib64/portage/pym/portage/util/movefile.py", line 97, in _copyxattr
    (_unicode_decode(dest), _unicode_decode(attr)))
portage.exception.OperationNotSupported: Filesystem containing file '/var/tmp/portage/app-text/xdvik-22.87/image//usr/share/pixmaps/xdvik.xpm' does not support extended attribute 'system.posix_acl_access'
!!! install: copy_xattrs failed with the following arguments: -m0644 /usr/portage/app-text/xdvik/files/xdvik.xpm /var/tmp/portage/app-text/xdvik-22.87/image//usr/share/pixmaps/

Note that for cases 7 and 8, I removed pyxattr manually after portage was installed. However, #8 did not break with an ImportError, but rather the same "your filesystem does not support xattr" message.

Cases 3-6 work in the sense of not bothering with xattr at all. Case #7 puzzles me, as it _allegedly_ installs files with xattrs -- though I can not find anything on the file that it complains about in the failing case:

# getfacl /usr/share/pixmaps/xdvik.xpm
getfacl: Removing leading '/' from absolute path names
# file: usr/share/pixmaps/xdvik.xpm
# owner: root
# group: root
user::rw-
group::r--
other::r--

Either way, with linux>=3.14, the install phase must run as UID0 if xattrs are to be kept intact. Case #7 seems more like a fluke than intended behavior to me.
Comment 21 Arfrever Frehtes Taifersar Arahesis 2014-04-24 19:06:54 UTC
xattr USE flag of Portage only affects default value of FEATURES in /usr/share/portage/config/make.globals. You can always override it with FEATURES="xattr" or FEATURES="-xattr" in make.conf.

There is no need to test it with Portage. You can run Python commands directly or run getfattr/setfattr and check $? .
Comment 22 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2014-04-24 19:38:20 UTC
Reassigned per Arfrever's request.
Comment 23 Tobias Klausmann (RETIRED) gentoo-dev 2014-04-26 15:08:01 UTC
I don't think this is a kernel bug. The problem with allowing xattr access (note: not file ACL) to non-uid-0 users is a security feature that prevents the functionality to be used in a DoS-like manner.

As I wrote earlier: the correct way to avoid the failure is to run the install phase as UID 0 if xattr/file ACLs are to be kept intact.
Comment 24 Tobias Klausmann (RETIRED) gentoo-dev 2014-04-26 15:17:51 UTC
So here's what works and doesn't:

$ touch somefile
$ getfacl somefile 
# file: somefile
# owner: klausman
# group: klausman
user::rw-
group::r--
other::r--

$ setfacl -m g:wheel:rw- somefile 
$ getfacl somefile 
# file: somefile
# owner: klausman
# group: klausman
user::rw-
group::r--
group:wheel:rw-
mask::rw-
other::r--

$ setfattr -n blargh -v fred somefile 
setfattr: somefile: Operation not supported


What I wonder is whether we actually have any package in the tree that uses xattrs for something outside of Hardened. I personally thing the xdvik ebuild failure I wrote about is a fluke, but I haven't determined the xattr the file carries yet. The only thing that would vaguely make sense is a mime-type set as an xattr.
Comment 25 Jeroen Roovers (RETIRED) gentoo-dev 2014-08-17 16:20:09 UTC
*** Bug 520042 has been marked as a duplicate of this bug. ***
Comment 26 Mike Pagano gentoo-dev 2014-12-18 14:19:04 UTC
Can someone enlighten me how we should direct this bug?
Comment 27 Mike Pagano gentoo-dev 2014-12-22 20:19:01 UTC
Tobias, is there is still something to do here, can you reopen and assign appropiately. Your comment that this is not a kernel bug is helpful that it's not in my bucket, but I don't know whose issue this should be.
Comment 28 Tobias Klausmann (RETIRED) gentoo-dev 2014-12-23 09:35:09 UTC
(In reply to Mike Pagano from comment #27)
> Tobias, is there is still something to do here, can you reopen and assign
> appropiately. Your comment that this is not a kernel bug is helpful that
> it's not in my bucket, but I don't know whose issue this should be.

As noted in comment #15, the problem is that non-UID-0 users can not set file attributes (_not_ ACLs, see comment #24) with kernel >=3.14. This is a feature of the kernel, avoiding a security problem.

I think there are three ways to handle this:

1) make portage discard attributes like these if it detects UID!=0 and kernel>=3.14

2) run this phase as UID0 or have UID0 helper that takes care of the attributes. This helper would also (non-silently!) discard attributes that the target FS can not handle (e.g. ext2 without extended attributes)

3) always wipe all non-ACL attributes (if UID!=0 for the install/merge step)

The way I see it, #3 and #1 are easier to implement, #2 is the (IMO) more correct option, but more complex to build and get right.
Comment 29 Arfrever Frehtes Taifersar Arahesis 2014-12-23 14:27:33 UTC
With FEATURES="userpriv" privileges are dropped only during src_{unpack,prepare,configure,compile,test}, not src_install.

pym/portage/package/ebuild/doebuild.py:
        actionmap = {
"pretend":  {"cmd":ebuild_sh, "args":{"droppriv":0,        "free":1,         "sesandbox":0,         "fakeroot":0}},
"setup":    {"cmd":ebuild_sh, "args":{"droppriv":0,        "free":1,         "sesandbox":0,         "fakeroot":0}},
"unpack":   {"cmd":ebuild_sh, "args":{"droppriv":droppriv, "free":0,         "sesandbox":sesandbox, "fakeroot":0}},
"prepare":  {"cmd":ebuild_sh, "args":{"droppriv":droppriv, "free":0,         "sesandbox":sesandbox, "fakeroot":0}},
"configure":{"cmd":ebuild_sh, "args":{"droppriv":droppriv, "free":nosandbox, "sesandbox":sesandbox, "fakeroot":0}},
"compile":  {"cmd":ebuild_sh, "args":{"droppriv":droppriv, "free":nosandbox, "sesandbox":sesandbox, "fakeroot":0}},
"test":     {"cmd":ebuild_sh, "args":{"droppriv":droppriv, "free":nosandbox, "sesandbox":sesandbox, "fakeroot":0}},
"install":  {"cmd":ebuild_sh, "args":{"droppriv":0,        "free":0,         "sesandbox":sesandbox, "fakeroot":fakeroot}},
"rpm":      {"cmd":misc_sh,   "args":{"droppriv":0,        "free":0,         "sesandbox":0,         "fakeroot":fakeroot}},
"package":  {"cmd":misc_sh,   "args":{"droppriv":0,        "free":0,         "sesandbox":0,         "fakeroot":fakeroot}},
                }
Comment 30 Tobias Klausmann (RETIRED) gentoo-dev 2014-12-24 09:40:02 UTC
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #29)
> With FEATURES="userpriv" privileges are dropped only during
> src_{unpack,prepare,configure,compile,test}, not src_install.
> 
> pym/portage/package/ebuild/doebuild.py:
>         actionmap = {
> "pretend":  {"cmd":ebuild_sh, "args":{"droppriv":0,        "free":1,        
> "sesandbox":0,         "fakeroot":0}},
> "setup":    {"cmd":ebuild_sh, "args":{"droppriv":0,        "free":1,        
> "sesandbox":0,         "fakeroot":0}},
> "unpack":   {"cmd":ebuild_sh, "args":{"droppriv":droppriv, "free":0,        
> "sesandbox":sesandbox, "fakeroot":0}},
> "prepare":  {"cmd":ebuild_sh, "args":{"droppriv":droppriv, "free":0,        
> "sesandbox":sesandbox, "fakeroot":0}},
> "configure":{"cmd":ebuild_sh, "args":{"droppriv":droppriv, "free":nosandbox,
> "sesandbox":sesandbox, "fakeroot":0}},
> "compile":  {"cmd":ebuild_sh, "args":{"droppriv":droppriv, "free":nosandbox,
> "sesandbox":sesandbox, "fakeroot":0}},
> "test":     {"cmd":ebuild_sh, "args":{"droppriv":droppriv, "free":nosandbox,
> "sesandbox":sesandbox, "fakeroot":0}},
> "install":  {"cmd":ebuild_sh, "args":{"droppriv":0,        "free":0,        
> "sesandbox":sesandbox, "fakeroot":fakeroot}},
> "rpm":      {"cmd":misc_sh,   "args":{"droppriv":0,        "free":0,        
> "sesandbox":0,         "fakeroot":fakeroot}},
> "package":  {"cmd":misc_sh,   "args":{"droppriv":0,        "free":0,        
> "sesandbox":0,         "fakeroot":fakeroot}},
>                 }

I re-examined things a bit more in depth.

It seems the original bug was pyxattr trying to set the  system.posix_acl_access xattr (see the build log). This fails on both tmpfs and ext4 with EOPNOTSUPP, no matter whether UID=0 or not.

This is all not making much sense. The only reason for this that I can see is that the _mechanism_ pyxattr used to use/uses is deprecated/broken. I definitely do have CONFIG_TMPFS_XATTR=y on all of my machines.

At any rate, I just now tried the install phases of mercurial 2.8.2 and 3.1.2 and everything worked fine. I guess this was either a fluke or the bug has been fixed (maybe silently).

Either way, let's close this bug for now and re-visit if the failure crops up again.
Comment 31 Arfrever Frehtes Taifersar Arahesis 2014-12-26 00:26:18 UTC
Probably your /usr/portage/dev-vcs/mercurial/files/cacerts.rc had some custom ACL setting when this bug was filed.
Maybe your /var/tmp/portage was on a filesystem without ACL support or ACL support was disabled in kernel.

Setting system.posix_acl_access actually works when value has correct syntax. (I tested on ext4 and tmpfs.)

Example on ext4:

$ rm -f test
$ touch test
$ getfattr -d -m- test
$ getfacl test
# file: test
# owner: Arfrever
# group: Arfrever
user::rw-
group::r--
other::---

$ python3.4 -c 'import xattr; xattr.set("test", "system.posix_acl_access", b"aaa")'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
OSError: [Errno 22] Invalid argument
$ python3.4 -c 'import xattr; xattr.set("test", "system.posix_acl_access", b"\x02\x00\x00\x00\x01\x00\x06\x00\xff\xff\xff\xff\x02\x00\x04\x00\x00\x00\x00\x00\x04\x00\x04\x00\xff\xff\xff\xff\x10\x00\x04\x00\xff\xff\xff\xff \x00\x00\x00\xff\xff\xff\xff")'
$ python3.4 -c 'import xattr; print(xattr.get("test", "system.posix_acl_access"))'
b'\x02\x00\x00\x00\x01\x00\x06\x00\xff\xff\xff\xff\x02\x00\x04\x00\x00\x00\x00\x00\x04\x00\x04\x00\xff\xff\xff\xff\x10\x00\x04\x00\xff\xff\xff\xff \x00\x00\x00\xff\xff\xff\xff'
$ getfattr -d -m- test
# file: test
system.posix_acl_access=0sAgAAAAEABgD/////AgAEAAAAAAAEAAQA/////xAABAD/////IAAAAP////8=

$ getfacl test
# file: test
# owner: Arfrever
# group: Arfrever
user::rw-
user:root:r--
group::r--
mask::r--
other::---

$