Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 413621 - sys-apps/coreutils-8.15 - src_test(): errors setting up rm/deep-2 and du/long-from-unreadable
Summary: sys-apps/coreutils-8.15 - src_test(): errors setting up rm/deep-2 and du/long...
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 1 vote (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords: TESTFAILURE
: 346877 353783 403279 443978 636746 (view as bug list)
Depends on: 548250
Blocks:
  Show dependency tree
 
Reported: 2012-04-26 12:32 UTC by Claes
Modified: 2018-09-14 09:46 UTC (History)
9 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Failing build log. (build.log,521.84 KB, text/plain)
2012-04-26 12:56 UTC, Claes
Details
sandbox-paths.patch (sandbox-paths-413621.patch,861 bytes, patch)
2015-04-30 16:26 UTC, Kent Fredric (IRC: kent\n)
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Claes 2012-04-26 12:32:26 UTC
As far as I can tell, coreutils 8.15 and 8.16 both pass all their tests, despite this the emerge fails with
* ERROR: sys-apps/coreutils-8.15 failed (test phase):
*    make check failed



Reproducible: Always

Steps to Reproduce:
1. emerge =sys-apps/coreutils-8.1{5,6]
Actual Results:  
Emerge fails

Expected Results:  
Emerge success

$ emerge --info
Portage 2.1.10.49 (default/linux/amd64/10.0/desktop/kde, gcc-4.5.3, glibc-2.14.1-r3, 3.2.12-gentoo x86_64)                                                                                                                       
=================================================================                                                                                                                                                                
System uname: Linux-3.2.12-gentoo-x86_64-Intel-R-_Core-TM-_i7-2620M_CPU_@_2.70GHz-with-gentoo-2.0.3                                                                                                                              
Timestamp of tree: Wed, 25 Apr 2012 07:00:01 +0000                                                                                                                                                                               
app-shells/bash:          4.2_p20                                                                                                                                                                                                                                                                                           
dev-java/java-config:     2.1.11-r3                                                                                                                                                                                                                                                                                         
dev-lang/python:          2.7.2-r3, 3.1.4-r3, 3.2.2                                                                                                                                                                                                                                                                         
dev-util/cmake:           2.8.6-r4                                                                                                                                                                                                                                                                                          
dev-util/pkgconfig:       0.26                                                                                                                                                                                                                                                                                              
sys-apps/baselayout:      2.0.3                                                                                                                                                                                                                                                                                             
sys-apps/openrc:          0.9.9.3                                                                                                                                                                                                                                                                                           
sys-apps/sandbox:         2.5                                                                                                                                                                                                                                                                                               
sys-devel/autoconf:       2.13, 2.68                                                                                                                                                                                                                                                                                        
sys-devel/automake:       1.9.6-r3, 1.10.3, 1.11.1                                                                                                                                                                                                                                                                          
sys-devel/binutils:       2.21.1-r1                                                                                                                                                                                                                                                                                         
sys-devel/gcc:            4.4.5, 4.5.3-r2, 4.6.2                                                                                                                                                                                                                                                                            
sys-devel/gcc-config:     1.5-r2                                                                                                                                                                                                                                                                                            
sys-devel/libtool:        2.4-r1                                                                                                                                                                                                                                                                                            
sys-devel/make:           3.82-r1                                                                                                                                                                                                                                                                                           
sys-kernel/linux-headers: 3.1 (virtual/os-headers)                                                                                                                                                                                                                                                                          
sys-libs/glibc:           2.14.1-r3                                                                                                                                                                                                                                                                                         
Repositories: gentoo x-letharion                                                                                                                                                                                                                                                                                            
ACCEPT_KEYWORDS="amd64"                                                                                                                                                                                                                                                                                                     
ACCEPT_LICENSE="* -@EULA skype-eula PUEL AdobeFlash-10.3 google-talkplugin"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -pipe -O2 -ggdb"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt /usr/share/maven-bin-3.0/conf /usr/share/openvpn/easy-rsa /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/apache2-php5.4/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -pipe -O2 -ggdb"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs collision-protect distlocks ebuild-locks fixlafiles news parallel-fetch parallel-install protect-owned sandbox sfperms splitdebug strict test unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS=""
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="sv_SE.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,--hash-style=gnu"
LINGUAS="en_GB"
MAKEOPTS="-j4"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage/letharion"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext X a52 aac acl acpi alsa amd64 bash-completion berkdb bluetooth branding bzip2 cairo cdda cdr cli consolekit cracklib crypt cups cxx dbus declarative dri dts dvd dvdr emboss encode exif fam firefox flac fortran gdbm gdu gif gpm gtk iconv ipv6 jpeg kde kipi lcms ldap libnotify mad mmx mmxext mng modules mp3 mp4 mpeg mudflap multilib ncurses networkmanager nls nptl nptlonly nsplugin ogg opengl openmp pam pango pcre pdf phonon plasma png policykit ppds pppd pulseaudio qt3support qt4 readline sdl semantic-desktop session spell sse sse2 sse3 ssl ssse3 startup-notification svg symlink sysfs tcpd theora tiff truetype udev unicode usb vorbis wxwidgets x264 xcb xcomposite xinerama xml xorg xscreensaver xulrunner xv xvid zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_core authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif socache_shmcb speling ssl status unique_id unixd userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS=";" COLLECTD_PLUGINS=";" ELIBC="glibc" GPSD_PROTOCOLS=";" INPUT_DEVICES="synaptics evdev keyboard mouse" KERNEL="linux" LCD_DEVICES=";" LINGUAS="en_GB" NETBEANS_MODULES="apisupport php websvccommon" PHP_TARGETS="php5-3 php5-4" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="vesa intel nouveau" XTABLES_ADDONS=";"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Comment 1 Claes 2012-04-26 12:56:09 UTC
FEATURES="-test" works around the problem.
Comment 2 Claes 2012-04-26 12:56:49 UTC
Created attachment 310181 [details]
Failing build log.
Comment 3 Jeroen Roovers gentoo-dev 2012-04-26 15:27:17 UTC
deep-2: set-up failure:
ERROR: rm/deep-2
long-from-unreadable: set-up failure: 
ERROR: du/long-from-unreadable
Comment 4 Chris Reffett gentoo-dev Security 2012-11-20 03:30:47 UTC
*** Bug 443978 has been marked as a duplicate of this bug. ***
Comment 5 Leon Schoorl 2013-10-18 16:14:10 UTC
I had sys-apps/coreutils-8.20 fail with the same errors.

When emerged with FEATURES="-usersandbox" all the tests complete successfully.

The test that fail both try to do something with very long paths (longer than PATH_MAX).

I think the problem is that is isn't handled correctly by the sandbox.
Comment 6 eroen 2014-01-08 20:28:15 UTC
I see this with sys-apps/coreutils-8.22 and sys-apps/sandbox-2.6-r1. The errors do not happen with FEATURES=-usersandbox.
Comment 7 Kent Fredric (IRC: kent\n) gentoo-dev 2015-04-30 10:58:55 UTC
Some diagnosis on reproducing the failure on 8.21:

Looking in the test for rm/deep-2, I find 3 interesting errors:

---
+ perl -e 'my $d = "x" x 200; foreach my $i (1..52)' -e '  { mkdir ($d, 0700) && chdir $d or die "$!" }'
Numerical result out of range at -e line 2.
---

This is the "setup failure", I believe.

---
+ chmod -R u+rwx /var/tmp/portage/sys-apps/coreutils-8.21/work/coreutils-8.21/gt-deep-2.sh.1SYU
chmod: cannot read directory '<BIGLONGDIRHERE>': File name too long
---
---
+ rm -rf /var/tmp/portage/sys-apps/coreutils-8.21/work/coreutils-8.21/gt-deep-2.sh.1SYU
rm: cannot remove '<BIGLONGDIRHERE>': Directory not empty
---

The problem dir is 4100 characters long, and has 20x path-parts 200 characters in length: https://gist.github.com/kentfredric/ad533fa2aa225240ea8a#file-gt-deep-sh-txt-L25


long-from-unreadable has the same pattern of failure, with a problem dir 4112 characters long, again, with 20x 200 character path parts.

https://gist.github.com/kentfredric/ad533fa2aa225240ea8a#file-gt-long-from-unreadable-sh-txt-L24


why the perl script can create them, but coreutils can't chmod or remove them, seems like an important thing to work out.

Curiously, it doesn't even look like a _test_ failure, but a failure of the framework to cleanup the test after the test failed to be created.

goto fail ?

:)
Comment 8 Kent Fredric (IRC: kent\n) gentoo-dev 2015-04-30 16:26:55 UTC
Created attachment 402310 [details, diff]
sandbox-paths.patch

Attached patch when applied last to the existing 8.21 patch series, allows sandboxed tests to pass, while not skipping the tests on unsandboxed systems.

It relies on the presence of SANDBOX_ACTIVE to make the tests skip, which I believe suitable for usersandbox/sandbox.

People who want to try this themselves, add the patch to the patch dir, and add the following after the existing epatch statement in src_prepare

epatch "${FILESDIR}"/sandbox-paths.patch

This will allow testing to be expected to pass until bug #548250 is properly resolved and propagated.
Comment 9 SpanKY gentoo-dev 2015-09-28 20:23:41 UTC
*** Bug 403279 has been marked as a duplicate of this bug. ***
Comment 10 SpanKY gentoo-dev 2015-09-28 20:26:25 UTC
*** Bug 353783 has been marked as a duplicate of this bug. ***
Comment 11 SpanKY gentoo-dev 2015-09-28 20:27:06 UTC
*** Bug 346877 has been marked as a duplicate of this bug. ***
Comment 12 Rolf Eike Beer 2017-09-10 18:18:58 UTC
Still happens with 8.26 on hppa.
Comment 13 Thomas Deutschmann gentoo-dev Security 2017-11-06 11:32:07 UTC
Still happening with coreutils-8.28 and sys-apps/sandbox-2.12.
Comment 14 Larry the Git Cow gentoo-dev 2017-11-06 16:22:01 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0edff67f453a7509c08fce538d3da9c947daa433

commit 0edff67f453a7509c08fce538d3da9c947daa433
Author:     Thomas Deutschmann <whissi@gentoo.org>
AuthorDate: 2017-11-06 16:20:43 +0000
Commit:     Thomas Deutschmann <whissi@gentoo.org>
CommitDate: 2017-11-06 16:20:57 +0000

    sys-apps/coreutils: Rev bump to address multiple test failures
    
    - Running test suite with FEATURES=usersandbox is no longer supported
      due to various sandbox issues. (bug #413621, #439574, #629660)
    
    - Due to sandbox failures previously disabled tests
      re-enabled. (bug #415487)
    
    - tests/ls/readdir-mountpoint-inode test failure fixed. (bug #353164)
    
    - "caps" USE flag description more specified. (bug #617702
    
    - Tools/programs/libraries used in test suite now listed in
      DEPEND="test? (...)" which requires re-keywording for some
      architectures.
    
    Bug: https://bugs.gentoo.org/413621
    Bug: https://bugs.gentoo.org/439574
    Bug: https://bugs.gentoo.org/629660
    Bug: https://bugs.gentoo.org/415487
    Closes: https://bugs.gentoo.org/353164
    Closes: https://bugs.gentoo.org/617702
    Package-Manager: Portage-2.3.13, Repoman-2.3.4

 sys-apps/coreutils/Manifest                 |   1 +
 sys-apps/coreutils/coreutils-8.28-r1.ebuild | 210 ++++++++++++++++++++++++++++
 sys-apps/coreutils/metadata.xml             |   1 +
 3 files changed, 212 insertions(+)}
Comment 15 Thomas Deutschmann gentoo-dev Security 2017-11-06 21:37:31 UTC
*** Bug 636746 has been marked as a duplicate of this bug. ***
Comment 16 Thomas Deutschmann gentoo-dev Security 2018-05-21 11:01:00 UTC
*** Bug 636746 has been marked as a duplicate of this bug. ***
Comment 17 Rolf Eike Beer 2018-07-10 19:03:11 UTC
chroot ~ # grep ERROR: /var/tmp/portage/sys-apps/coreutils-8.29-r1/temp/build.log 
# ERROR: 2
ERROR: tests/rm/deep-2
ERROR: tests/du/long-from-unreadable
# ERROR: 2
# ERROR: 0
 * ERROR: sys-apps/coreutils-8.29-r1::gentoo failed (test phase):
chroot ~ # grep ERROR: /var/tmp/portage/sys-apps/coreutils-8.30/temp/build.log 
# ERROR: 2
ERROR: tests/rm/deep-2
ERROR: tests/du/long-from-unreadable
# ERROR: 2
# ERROR: 0
 * ERROR: sys-apps/coreutils-8.30::gentoo failed (test phase):
Comment 18 Kent Fredric (IRC: kent\n) gentoo-dev 2018-09-14 08:05:49 UTC
(In reply to Rolf Eike Beer from comment #17)
> chroot ~ # grep ERROR:
> /var/tmp/portage/sys-apps/coreutils-8.29-r1/temp/build.log 
> # ERROR: 2
> ERROR: tests/rm/deep-2
> ERROR: tests/du/long-from-unreadable
> # ERROR: 2
> # ERROR: 0
>  * ERROR: sys-apps/coreutils-8.29-r1::gentoo failed (test phase):
> chroot ~ # grep ERROR:
> /var/tmp/portage/sys-apps/coreutils-8.30/temp/build.log 
> # ERROR: 2
> ERROR: tests/rm/deep-2
> ERROR: tests/du/long-from-unreadable
> # ERROR: 2
> # ERROR: 0
>  * ERROR: sys-apps/coreutils-8.30::gentoo failed (test phase):

Disable FEATURES="usersandbox" for this package, as instructed.

>  * You are emerging coreutils-8.28 with 'usersandbox' enabled. Expect some test failures or emerge with 'FEATURES=-usersandbox'!

Ideally more than a warning here, might help avoid future bug reports?
Comment 19 Thomas Deutschmann gentoo-dev Security 2018-09-14 09:46:29 UTC
(In reply to Kent Fredric (IRC: kent\n) from comment #18)
> Disable FEATURES="usersandbox" for this package, as instructed.
> 
> >  * You are emerging coreutils-8.28 with 'usersandbox' enabled. Expect some test failures or emerge with 'FEATURES=-usersandbox'!
> 
> Ideally more than a warning here, might help avoid future bug reports?

I have no clear answer for that. We have multiple options:

a) RESTRICT=test

b) Check for FEATURES=userpriv in src_test and skip tests if found

c) Remove (=patch like shown in comment #8) these two tests

Because this is a core package, option "a" isn't a valid option for me -- we have to run tests regularly and a restriction just because of a broken Gentoo sandbox would make this unnecessarily difficult: At the moment, users interested in running tests, can disable that feature just for this package but they cannot work around a general restriction without modifying the ebuild (and even if there would be such an option, I bet we would still get "false positives" just because someone forced tests and missed that warning).

same argument applies to option "c": This is a core package and we need to test this functionality.

Option "b" or maybe sort of option "c" (i.e. conditional removal of these two tests if FEATURES=userpriv) would work in theory. However I don't like that idea: Imagine we want to stabilize a new version and expect that people, at least arch teams, during stabilization, will run these tests. If they forget about that requirement they would mark that package stable because 'all tests passed' but in theory there could be a bug which affects core functionality in a core package.

Maybe we can detect these two failures and don't fail with FEATURES=userpriv. I.e. implement a hack like https://gitweb.gentoo.org/repo/gentoo.git/tree/media-libs/openjpeg/openjpeg-2.3.0.ebuild?id=b4459aa17d4d7b5bf9150a31f4b409733ca5cbb6#n90

But I am not sure if

- it's worth to do something like that

- we even should something like that in general (another hack...)

- see reason for option "b": Maybe if we wouldn't talk about something like 'the mother of all core packages' *SCNR* ... At the moment, false positive rate is very low, we may reconsider this if we start seeing more false positive because people don't read logs before reporting bugs but until then, I think I'd like to keep current 'solution'.