Summary: | sys-devel/gcc fails tests | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | Carsten Lohrke (RETIRED) <carlo> |
Component: | [OLD] Core system | Assignee: | Gentoo Toolchain Maintainers <toolchain> |
Status: | RESOLVED INVALID | ||
Severity: | normal | CC: | please.no.spam.here, something-bz, toralf |
Priority: | High | Keywords: | PullRequest |
Version: | 2008.0 | ||
Hardware: | All | ||
OS: | Linux | ||
See Also: | https://github.com/gentoo/gentoo/pull/35794 | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Attachments: | gcc-4.3.2-r2-log-excerpt |
Description
Carsten Lohrke (RETIRED)
![]() Created attachment 177527 [details]
gcc-4.3.2-r2-log-excerpt
toolchain.eclass: gcc_src_test() { cd "${WORKDIR}"/build make -k check || ewarn "check failed and that sucks :(" } I did see the notice. I still would like to see ebuilds bail out on test errors, even if they are not so important as the e.g. libmudflap ones and I'd prefer broken tests not to be run, instead ignoring test failures at all. and i'd prefer the testsuites to complete without any failures. but neither of these are realistic. Of course, but that's beside the point. Ignoring all test failures, because of a couple of known ones, implies that unknown test failures which may be worth to have a look at do not get noticed either. no version of gcc, just like gdb, has ever been released with a `make check` that passed (except perhaps for gcc-2 way back in the day). no one has ever reviewed the test failures to see what is OK due to how much time it takes. i would certainly encourage someone to dedicate the time to this endeavor, but i'm not going to as i can barely keep up with all other gcc bugs as it is (some might say i'm not even keeping up with those). arch testers are encouraged to compare test results of existing stable versions of gcc/gdb and compare them to proposed stable, but i dont think they do. *** Bug 265103 has been marked as a duplicate of this bug. *** It is still the wrong way to do things. It is still a bug. It's hardly "invalid". "By the way, some tests failed, but I'm not telling you which ones" and deleting the build logs is definitely not the "right" behaviour when the user has selected FEATURES=test. What's wrong with test || die "Oops, some tests failed. This is probably normal. Comment the relevant line in gcc_src_test() in /var/tmp/portage/sys-devel/gcc-$VERSION/temp/environment and run ebuild /usr/portage/sys-devel/gcc/gcc-4.3.2-r2.ebuild to continue"? (In reply to comment #8) > It is still the wrong way to do things. It is still a bug. It's hardly > "invalid". No, it's not a bug. The gcc testsuite is not designed to pass all its tests. It's designed as a regression catching framework. A developer runs the testsuite before and after applying a patch and compares the results. If there are no _new_ regressions, the patch can be applied. Check out the interpreting test results part of http://gcc.gnu.org/install/test.html and the archive of testsuite results at http://gcc.gnu.org/buildstat.html > "By the way, some tests failed, but I'm not telling you which ones" and > deleting the build logs is definitely not the "right" behaviour when the user > has selected FEATURES=test. Which is why we install the full test logs in /usr/share/doc/gcc-${version} for you. > What's wrong with test || die "Oops, some tests failed. This is probably > normal. Comment the relevant line in gcc_src_test() in > /var/tmp/portage/sys-devel/gcc-$VERSION/temp/environment and run ebuild > /usr/portage/sys-devel/gcc/gcc-4.3.2-r2.ebuild to continue"? Because it fails 100% of the time? What would be the point of having a testsuite if the only possible way to install a package is to turn it off? *** Bug 283070 has been marked as a duplicate of this bug. *** *** Bug 381077 has been marked as a duplicate of this bug. *** The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=c4eccac296c608e3fafad22d678540ddaab950d7 commit c4eccac296c608e3fafad22d678540ddaab950d7 Author: Sam James <sam@gentoo.org> AuthorDate: 2024-03-17 06:47:09 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-03-23 15:40:52 +0000 toolchain.eclass: don't install all .sum files Just rely on the validate_failures.py manifests instead. These logs get real big real fast. People can save build logs if they want to look at the tests otherwise. Bug: https://bugs.gentoo.org/214345 Bug: https://bugs.gentoo.org/253926 Signed-off-by: Sam James <sam@gentoo.org> eclass/toolchain.eclass | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=1d93a491096f1cc0234fcf44458bfec142c213bb commit 1d93a491096f1cc0234fcf44458bfec142c213bb Author: Sam James <sam@gentoo.org> AuthorDate: 2024-03-16 07:00:23 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-03-23 15:40:51 +0000 toolchain.eclass: rework tests more Rework src_test around contrib/testsuite-management/validate_failures.py in gcc.git. This script is being used by the new Linaro CI effort and it appears well-suited to us, as it allows marking expected failures easily. Followup to 9ac3f1cf62b522236ba9efd7e923071c37df1e6d. Bug: https://bugs.gentoo.org/214345 Bug: https://bugs.gentoo.org/253926 Signed-off-by: Sam James <sam@gentoo.org> eclass/toolchain.eclass | 148 ++++++++++++++++++------- sys-devel/gcc/Manifest | 1 + sys-devel/gcc/gcc-10.5.0.ebuild | 1 + sys-devel/gcc/gcc-11.4.1_p20240111.ebuild | 1 + sys-devel/gcc/gcc-11.4.1_p20240208.ebuild | 1 + sys-devel/gcc/gcc-11.4.1_p20240222.ebuild | 1 + sys-devel/gcc/gcc-11.4.1_p20240229.ebuild | 1 + sys-devel/gcc/gcc-11.4.1_p20240307.ebuild | 1 + sys-devel/gcc/gcc-11.4.1_p20240314.ebuild | 1 + sys-devel/gcc/gcc-11.5.9999.ebuild | 1 + sys-devel/gcc/gcc-12.3.1_p20240112.ebuild | 1 + sys-devel/gcc/gcc-12.3.1_p20240209.ebuild | 1 + sys-devel/gcc/gcc-12.3.1_p20240223.ebuild | 1 + sys-devel/gcc/gcc-12.3.1_p20240301.ebuild | 1 + sys-devel/gcc/gcc-12.3.1_p20240308.ebuild | 1 + sys-devel/gcc/gcc-12.3.1_p20240315.ebuild | 1 + sys-devel/gcc/gcc-12.4.9999.ebuild | 1 + sys-devel/gcc/gcc-13.2.1_p20240113-r1.ebuild | 1 + sys-devel/gcc/gcc-13.2.1_p20240210.ebuild | 1 + sys-devel/gcc/gcc-13.2.1_p20240224.ebuild | 1 + sys-devel/gcc/gcc-13.2.1_p20240302.ebuild | 1 + sys-devel/gcc/gcc-13.2.1_p20240309.ebuild | 1 + sys-devel/gcc/gcc-13.2.1_p20240316.ebuild | 1 + sys-devel/gcc/gcc-13.3.9999.ebuild | 1 + sys-devel/gcc/gcc-14.0.1_pre20240218.ebuild | 1 + sys-devel/gcc/gcc-14.0.1_pre20240225.ebuild | 1 + sys-devel/gcc/gcc-14.0.1_pre20240303-r1.ebuild | 1 + sys-devel/gcc/gcc-14.0.1_pre20240310.ebuild | 1 + sys-devel/gcc/gcc-14.0.1_pre20240317.ebuild | 1 + sys-devel/gcc/gcc-14.0.9999.ebuild | 1 + sys-devel/gcc/gcc-8.5.0-r1.ebuild | 1 + sys-devel/gcc/gcc-9.5.0.ebuild | 1 + 32 files changed, 141 insertions(+), 38 deletions(-) https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=abf8e2ee55c52c8ae894e0b3845ea1cebfcfd4e8 commit abf8e2ee55c52c8ae894e0b3845ea1cebfcfd4e8 Author: Sam James <sam@gentoo.org> AuthorDate: 2024-03-16 01:38:45 +0000 Commit: Sam James <sam@gentoo.org> CommitDate: 2024-03-23 15:40:50 +0000 toolchain.eclass: install test results as orphaned files in /var/cache/gcc This allows comparison across versions even after they get upgraded, which is useful in itself (and across series), but also for looking into when regressions started if they're reported but started a while ago. Followup to 9ac3f1cf62b522236ba9efd7e923071c37df1e6d. Bug: https://bugs.gentoo.org/214345 Bug: https://bugs.gentoo.org/253926 Signed-off-by: Sam James <sam@gentoo.org> eclass/toolchain.eclass | 32 +++++++++++++++++++++----------- 1 file changed, 21 insertions(+), 11 deletions(-) |