I use test-fail-continue in my FEATURES for running tests but don't get them stopping my merges. The problem is that, after test fails, package is installed but all files in TMPDIR are erased, including build.log file
This is a problem because I need to manually re-emerge the package and disable that FEATURE temporally for getting the full error message
From my point of view, portage should leave build.log file in TMPDIR, even removing the other stuff
Portage 126.96.36.199 (default/linux/amd64/2008.0/desktop, gcc-4.3.2, glibc-2.8_p20080602-r1, 2.6.28-tuxonice-r9 x86_64)
System uname: Linux-2.6.28-tuxonice-r9-x86_64-Intel-R-_Core-TM-2_Duo_CPU_T9300_@_2.50GHz-with-glibc2.2.5
Timestamp of tree: Mon, 06 Apr 2009 18:30:01 +0000
ccache version 2.4 [enabled]
sys-devel/autoconf: 2.13, 2.63
sys-devel/automake: 1.5, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2
CFLAGS="-O2 -pipe -march=native"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-O2 -pipe -march=native"
FEATURES="autoaddcvs ccache collision-protect cvs distlocks fixpackages multilib-strict parallel-fetch protect-owned sandbox sfperms strict test test-fail-continue unmerge-orphans userfetch"
LINGUAS="es es_ES en_US"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTDIR_OVERLAY="/usr/portage/local/layman/sunrise /usr/portage/local/layman/wschlich-testing /usr/local/portage"
USE="X a52 aac acl acpi alsa amd64 applet avahi bash-completion berkdb bluetooth branding bzip2 cairo cdda cddb cdinstall cdr cleartype cli consolekit cracklib crypt css cups daap dbus dell djvu dri dts dvd dvdr dvdread dvi eds emboss encode evo exif fam fat ffmpeg firefox flac fortran fuse gdbm gif git glitz gnome gnome-keyring gpm gstreamer gtk hal iconv ieee1394 imagemagick isdnlog java jpeg jpeg2k kdeenablefinal kdehiddenvisibility kpathsea laptop latex libnotify lm_sensors lyx lzma mad midi mikmod mmx mmxext mono mp3 mpeg mudflap multilib musicbrainz nautilus ncurses network network-cron networkmanager nls nptl nptlonly ntfs nvidia ogg opengl openmp pam pango pch pcre pdf perl png ppds pppd python qt3support qt4 quicktime readline reflection reiserfs scanner sdl session spell spl sse sse2 ssl ssse3 startup-notification subversion svg sysfs t1lib tcpd threads tiff truetype unicode usb v4l2 vcd vorbis webkit xattr xcb xinetd xml xmp xorg xpm xulrunner xv xvid zeroconf zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="es es_ES en_US" USERLAND="GNU" VIDEO_CARDS="nvidia nv"
Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
As a workaround, you can enable PORT_LOGDIR.
Thanks for the tip!
actually, it would be nice if test-fail-continue would keep the entire build dir around on failures so any logs the testsuite might have generated don't get blown away.
(In reply to comment #3)
> actually, it would be nice if test-fail-continue would keep the entire build
> dir around on failures so any logs the testsuite might have generated don't get
> blown away.
Yes, also, after reading make.conf page, I saw the following:
Clean up temporary files after a build failure. This is particularly useful if you have PORTAGE_TMPDIR on tmpfs. If this feature is enabled, you probably also want to enable PORT_LOGDIR in order to save the build log. Both the ebuild(1) command and the noclean feature cause the fail-clean feature to be automatically disabled.
then, from my point of view, portage should simply keep files if any ebuild phase fails and, if somebody disagrees with that behavior, he can use fail-clean. It seems also more consistent than current behavior as, but default, any failure will end with files preserved
Thanks a lot
(In reply to Ryan Hill from comment #3)
> actually, it would be nice if test-fail-continue would keep the entire build
> dir around on failures so any logs the testsuite might have generated don't
> get blown away.
Adapting to this as I just needed to re-run ebuild ... test to get some files were errors are logged for reporting to upstream ;)
I think I've actually changed my mind about this. With the number of packages that fail tests these days, this would fill up a tmpdir pretty fast...
I guess it depends on how often we look for failures and the kind of tmpdir we have. Ideally, we could configure that behavior. I wondered about using "noclean" FEATURE but I didn't as that one would really waste a lot of space as it would keep files for even packages that don't fail tests at all.
Other main advantage of preserving files with test-fail-continue is for cases where different tests fail depending on we updating the package or re-emerging it. For example, I reviewed this yesterday due the long standing rhythmbox failing tests:
1. I read about the failures at /var/log/summary.log
2. I look at its build.log and I see I need to provide a tests.log file that is generated at build dir.
3. I re-emerge the package with ebuild rhythmbox... test to get them
4. The set of failing tests is different (only 1 test failing instead of 4)
5. I need to downgrade it and repeat it.
If I would have got the tests.log file from start it would have been much easier to directly report the issue to upstream :)
This would also be useful when tests are prone to fail but don't always fail, for example with evolution-data-server and its dbus related tests
The test-fail-continue code is buried down inside the EbuildPhase class, but its the EbuildBuild class that needs to be aware of the test failure so that it can tell the EbuildMerge class not to clean the build dir.
We could handle this by having the EbuildBuild class pass a callback down to EbuildPhase, so that it can be notified of the test failure and take appropriate action.
Would that make it possible to suppress the typical "agh, phase failed" wall of error text when test-fail-continue is used? If so I fully endorse it and will be filing a follow-up enhancement req. :)
(In reply to Ryan Hill from comment #9)
> Would that make it possible to suppress the typical "agh, phase failed" wall
> of error text when test-fail-continue is used? If so I fully endorse it and
> will be filing a follow-up enhancement req. :)
That's an entirely separate issue. Anyway, yeah, go ahead and file an enhancement request. What you want is some conditional code inside die, so that it suppresses the usual die message when die is called during src_test and FEATURES=test-fail-continue is enabled.
(In reply to Zac Medico from comment #8)
> The test-fail-continue code is buried down inside the EbuildPhase class, but
> its the EbuildBuild class that needs to be aware of the test failure so that
> it can tell the EbuildMerge class not to clean the build dir.
> We could handle this by having the EbuildBuild class pass a callback down to
> EbuildPhase, so that it can be notified of the test failure and take
> appropriate action.
So, as a hack, maybe we can pass this information "out-of-band" by creating a file ".portage_tests_failed" somewhere in the build directory, then check for it later?