GCC installs despite test errors in GCC - even with FEATURES+=stricter - a single one in libstdc++ and some in libmudflap. There is also a number of warnings regarding missing include files (in libstdc++-v3/include/ext) and matching class members not found (in libstdc++-v3/include/{bits,tr1_impl}). Interested in the corresponding output as well? Portage 2.2_rc20 (!/etc/portage/profile/local-profile, gcc-4.1.2, glibc-2.6.1-r0, 2.6.27.10 i686) ================================================================= System uname: Linux-2.6.27.10-i686-AMD_Athlon-tm-_64_Processor_3500+-with-glibc2.0 Timestamp of tree: Mon, 05 Jan 2009 12:20:01 +0000 distcc 3.0 i686-pc-linux-gnu [disabled] ccache version 2.4 [enabled] app-shells/bash: 3.2_p39 dev-java/java-config: 1.3.7-r1, 2.1.6-r1 dev-lang/python: 2.5.2-r7 dev-python/pycrypto: 2.0.1-r6 dev-util/ccache: 2.4-r7 dev-util/cmake: 2.6.2-r1 sys-apps/baselayout: 1.12.11.1 sys-apps/sandbox: 1.2.18.1-r2 sys-devel/autoconf: 2.13, 2.63 sys-devel/automake: 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.0-r4 sys-devel/libtool: 1.5.26 virtual/os-headers: 2.6.27-r2 ACCEPT_KEYWORDS="x86" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=athlon64 -pipe -mfpmath=sse -msse3 -funit-at-a-time -D_FORTIFY_SOURCE=2 -fstack-protector" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d" CXXFLAGS="-O2 -march=athlon64 -pipe -mfpmath=sse -msse3 -funit-at-a-time -D_FORTIFY_SOURCE=2 -fvisibility-inlines-hidden" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--noconfmem" FEATURES="ccache collision-protect cvs distlocks fixpackages metadata-transfer noinfo parallel-fetch preserve-libs protect-owned sandbox sfperms splitdebug strict stricter test unmerge-orphans userfetch userpriv usersandbox" GENTOO_MIRRORS="ftp://ftp.snt.utwente.nl/pub/os/linux/gentoo http://www.ibiblio.org/pub/Linux/distributions/gentoo" LANG="de_DE.utf8" LDFLAGS="-Wl,-O1 -Wl,--sort-common -Wl,--as-needed -Wl,--hash-style=both" LINGUAS="de" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages' --exclude-from=/etc/portage/rsync_excludes" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="3dnow 3dnowext X aac acl acpi alsa apache2 audiofile berkdb bzip2 cairo caps cddb cdparanoia cdr chroot cli cracklib crypt css cups curl cxx dbus device-mapper dga doc dvd dvdr dvdread emacs encode exif faad fam ffmpeg firefox flac fontconfig fortran gd gif glut gphoto2 gpm graphviz hal hbci iconv idn imagemagick imap ipv6 isdnlog javascript jpeg jpeg2k kde kdehiddenvisibility kipi ladcca lame lcms ldap libnotify lirc lm_sensors lzo mad maildir matroska mbox md5sum mmx mmxext mng mp3 mp4 mpeg mudflap musepack nas ncurses network nfs nls nptl nptlonly ofx ogg openexr opengl openmp pam pcap pcre pdf perl png postgres pppd python qt3 qt3support qt4 quicktime readline reflection rtc ruby samba sasl scanner sdl session slp snmp sox spell spl sse sse2 ssl startup-notification subversion svg svga sysfs tcpd tetex theora threads tiff transcode truetype unicode usb vorbis win32codecs wmf x264 x86 xattr xcb xcomposite xerces xine xml xorg xulrunner xv xvid zlib" ALSA_CARDS="intel8x0" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_alias authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_ftp proxy_http rewrite setenvif so speling status substitute unique_id userdir usertrack vhost_alias" CAMERAS="canon" ELIBC="glibc" INPUT_DEVICES="evdev keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="de" USERLAND="GNU" VIDEO_CARDS="fbdev nvidia vesa" Unset: CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Created attachment 177527 [details] gcc-4.3.2-r2-log-excerpt
toolchain.eclass: gcc_src_test() { cd "${WORKDIR}"/build make -k check || ewarn "check failed and that sucks :(" }
I did see the notice. I still would like to see ebuilds bail out on test errors, even if they are not so important as the e.g. libmudflap ones and I'd prefer broken tests not to be run, instead ignoring test failures at all.
and i'd prefer the testsuites to complete without any failures. but neither of these are realistic.
Of course, but that's beside the point. Ignoring all test failures, because of a couple of known ones, implies that unknown test failures which may be worth to have a look at do not get noticed either.
no version of gcc, just like gdb, has ever been released with a `make check` that passed (except perhaps for gcc-2 way back in the day). no one has ever reviewed the test failures to see what is OK due to how much time it takes. i would certainly encourage someone to dedicate the time to this endeavor, but i'm not going to as i can barely keep up with all other gcc bugs as it is (some might say i'm not even keeping up with those). arch testers are encouraged to compare test results of existing stable versions of gcc/gdb and compare them to proposed stable, but i dont think they do.
*** Bug 265103 has been marked as a duplicate of this bug. ***
It is still the wrong way to do things. It is still a bug. It's hardly "invalid". "By the way, some tests failed, but I'm not telling you which ones" and deleting the build logs is definitely not the "right" behaviour when the user has selected FEATURES=test. What's wrong with test || die "Oops, some tests failed. This is probably normal. Comment the relevant line in gcc_src_test() in /var/tmp/portage/sys-devel/gcc-$VERSION/temp/environment and run ebuild /usr/portage/sys-devel/gcc/gcc-4.3.2-r2.ebuild to continue"?
(In reply to comment #8) > It is still the wrong way to do things. It is still a bug. It's hardly > "invalid". No, it's not a bug. The gcc testsuite is not designed to pass all its tests. It's designed as a regression catching framework. A developer runs the testsuite before and after applying a patch and compares the results. If there are no _new_ regressions, the patch can be applied. Check out the interpreting test results part of http://gcc.gnu.org/install/test.html and the archive of testsuite results at http://gcc.gnu.org/buildstat.html > "By the way, some tests failed, but I'm not telling you which ones" and > deleting the build logs is definitely not the "right" behaviour when the user > has selected FEATURES=test. Which is why we install the full test logs in /usr/share/doc/gcc-${version} for you. > What's wrong with test || die "Oops, some tests failed. This is probably > normal. Comment the relevant line in gcc_src_test() in > /var/tmp/portage/sys-devel/gcc-$VERSION/temp/environment and run ebuild > /usr/portage/sys-devel/gcc/gcc-4.3.2-r2.ebuild to continue"? Because it fails 100% of the time? What would be the point of having a testsuite if the only possible way to install a package is to turn it off?
*** Bug 283070 has been marked as a duplicate of this bug. ***
*** Bug 381077 has been marked as a duplicate of this bug. ***
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=c4eccac296c608e3fafad22d678540ddaab950d7 commit c4eccac296c608e3fafad22d678540ddaab950d7 Author: Sam James <sam@gentoo.org> AuthorDate: 2024-03-17 06:47:09 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-03-23 15:40:52 +0000 toolchain.eclass: don't install all .sum files Just rely on the validate_failures.py manifests instead. These logs get real big real fast. People can save build logs if they want to look at the tests otherwise. Bug: https://bugs.gentoo.org/214345 Bug: https://bugs.gentoo.org/253926 Signed-off-by: Sam James <sam@gentoo.org> eclass/toolchain.eclass | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1d93a491096f1cc0234fcf44458bfec142c213bb commit 1d93a491096f1cc0234fcf44458bfec142c213bb Author: Sam James <sam@gentoo.org> AuthorDate: 2024-03-16 07:00:23 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-03-23 15:40:51 +0000 toolchain.eclass: rework tests more Rework src_test around contrib/testsuite-management/validate_failures.py in gcc.git. This script is being used by the new Linaro CI effort and it appears well-suited to us, as it allows marking expected failures easily. Followup to 9ac3f1cf62b522236ba9efd7e923071c37df1e6d. Bug: https://bugs.gentoo.org/214345 Bug: https://bugs.gentoo.org/253926 Signed-off-by: Sam James <sam@gentoo.org> eclass/toolchain.eclass | 148 ++++++++++++++++++------- sys-devel/gcc/Manifest | 1 + sys-devel/gcc/gcc-10.5.0.ebuild | 1 + sys-devel/gcc/gcc-11.4.1_p20240111.ebuild | 1 + sys-devel/gcc/gcc-11.4.1_p20240208.ebuild | 1 + sys-devel/gcc/gcc-11.4.1_p20240222.ebuild | 1 + sys-devel/gcc/gcc-11.4.1_p20240229.ebuild | 1 + sys-devel/gcc/gcc-11.4.1_p20240307.ebuild | 1 + sys-devel/gcc/gcc-11.4.1_p20240314.ebuild | 1 + sys-devel/gcc/gcc-11.5.9999.ebuild | 1 + sys-devel/gcc/gcc-12.3.1_p20240112.ebuild | 1 + sys-devel/gcc/gcc-12.3.1_p20240209.ebuild | 1 + sys-devel/gcc/gcc-12.3.1_p20240223.ebuild | 1 + sys-devel/gcc/gcc-12.3.1_p20240301.ebuild | 1 + sys-devel/gcc/gcc-12.3.1_p20240308.ebuild | 1 + sys-devel/gcc/gcc-12.3.1_p20240315.ebuild | 1 + sys-devel/gcc/gcc-12.4.9999.ebuild | 1 + sys-devel/gcc/gcc-13.2.1_p20240113-r1.ebuild | 1 + sys-devel/gcc/gcc-13.2.1_p20240210.ebuild | 1 + sys-devel/gcc/gcc-13.2.1_p20240224.ebuild | 1 + sys-devel/gcc/gcc-13.2.1_p20240302.ebuild | 1 + sys-devel/gcc/gcc-13.2.1_p20240309.ebuild | 1 + sys-devel/gcc/gcc-13.2.1_p20240316.ebuild | 1 + sys-devel/gcc/gcc-13.3.9999.ebuild | 1 + sys-devel/gcc/gcc-14.0.1_pre20240218.ebuild | 1 + sys-devel/gcc/gcc-14.0.1_pre20240225.ebuild | 1 + sys-devel/gcc/gcc-14.0.1_pre20240303-r1.ebuild | 1 + sys-devel/gcc/gcc-14.0.1_pre20240310.ebuild | 1 + sys-devel/gcc/gcc-14.0.1_pre20240317.ebuild | 1 + sys-devel/gcc/gcc-14.0.9999.ebuild | 1 + sys-devel/gcc/gcc-8.5.0-r1.ebuild | 1 + sys-devel/gcc/gcc-9.5.0.ebuild | 1 + 32 files changed, 141 insertions(+), 38 deletions(-) https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=abf8e2ee55c52c8ae894e0b3845ea1cebfcfd4e8 commit abf8e2ee55c52c8ae894e0b3845ea1cebfcfd4e8 Author: Sam James <sam@gentoo.org> AuthorDate: 2024-03-16 01:38:45 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-03-23 15:40:50 +0000 toolchain.eclass: install test results as orphaned files in /var/cache/gcc This allows comparison across versions even after they get upgraded, which is useful in itself (and across series), but also for looking into when regressions started if they're reported but started a while ago. Followup to 9ac3f1cf62b522236ba9efd7e923071c37df1e6d. Bug: https://bugs.gentoo.org/214345 Bug: https://bugs.gentoo.org/253926 Signed-off-by: Sam James <sam@gentoo.org> eclass/toolchain.eclass | 32 +++++++++++++++++++++----------- 1 file changed, 21 insertions(+), 11 deletions(-)
