Created attachment 332368 [details, diff] Patch implementing CPU_FEATURES for all qt4-build based packages x11-libs/qt-* ebuild are not featuring any of the standard SIMD instruction sets flags: * mmx * 3dnow * sse * sse[2, 3, 4_1, 4_2, 4a] * ssse3 * avx Additionally, qt build scripts fail to properly detect some of the processors - leading to bugs like https://bugs.gentoo.org/show_bug.cgi?id=436150 I believe that it is in the spirit of Gentoo to provide users the ability to control the target hardware configuration when building the system. Unfortunately, that is not currently the case. This leads to the situation where software fails to run with SIGILL, when one attempts to build it on a processor vastly superior to the running one. The following patch proposes the following changes to apply to all packages using qt4-build: USE flag "cpudetection" which reverts to the current problematic behaviour CPU_FEATURES specific flags which apply correctly to the configure script.
I remember having posted some questions on that in regard to a cross-compiling distcc-setup and the answers back then were: Qt does runtime-auto-detection if CPU-features such as supported SIMD-instructions.
(In reply to comment #0) > Created attachment 332368 [details, diff] [details, diff] > Patch implementing CPU_FEATURES for all qt4-build based packages > > x11-libs/qt-* ebuild are not featuring any of the standard SIMD instruction > sets flags: > > * mmx > * 3dnow > * sse > * sse[2, 3, 4_1, 4_2, 4a] > * ssse3 > * avx > > Additionally, qt build scripts fail to properly detect some of the > processors - leading to bugs like > https://bugs.gentoo.org/show_bug.cgi?id=436150 > This is not true, that bug is invalid, please read it *all* before linking to it. > I believe that it is in the spirit of Gentoo to provide users the ability to > control the target hardware configuration when building the system. > CFLAGS/CXXFLAGS are your friends (see -mno-*). > Unfortunately, that is not currently the case. This leads to the situation > where software fails to run with SIGILL, when one attempts to build it on a > processor vastly superior to the running one. > Do you know of a specific failure case (in qt)? > The following patch proposes the following changes to apply to all packages > using qt4-build: > > USE flag "cpudetection" which reverts to the current problematic behaviour > CPU_FEATURES specific flags which apply correctly to the configure script. (In reply to comment #1) > I remember having posted some questions on that in regard to a > cross-compiling distcc-setup and the answers back then were: Qt does > runtime-auto-detection if CPU-features such as supported SIMD-instructions. Correct, Qt does run-time cpu detection and enables only the appropriate code paths. If you want to disable some cpu-specific features at build-time, please use C(XX)FLAGS.
Created attachment 332504 [details] build.log for the case where -mno-* flags have been provided
(In reply to comment #2) > (In reply to comment #0) > > Created attachment 332368 [details, diff] [details, diff] [details, diff] > > Patch implementing CPU_FEATURES for all qt4-build based packages > > > > x11-libs/qt-* ebuild are not featuring any of the standard SIMD instruction > > sets flags: > > > > * mmx > > * 3dnow > > * sse > > * sse[2, 3, 4_1, 4_2, 4a] > > * ssse3 > > * avx > > > > Additionally, qt build scripts fail to properly detect some of the > > processors - leading to bugs like > > https://bugs.gentoo.org/show_bug.cgi?id=436150 > > > > This is not true, that bug is invalid, please read it *all* before linking > to it. > I will grant you that one, it's still very confusing. > > > I believe that it is in the spirit of Gentoo to provide users the ability to > > control the target hardware configuration when building the system. > > > > CFLAGS/CXXFLAGS are your friends (see -mno-*). See attached build.log. I disabled my eclass-overrides repos.conf setting and building it with regular qt4-build.eclass - for pentium4, on k8-sse3. > > > Unfortunately, that is not currently the case. This leads to the situation > > where software fails to run with SIGILL, when one attempts to build it on a > > processor vastly superior to the running one. > > > > Do you know of a specific failure case (in qt)? > Yes. This is how this entire work started. Try building for -march=pentium4 on anything SSE3+ capable. Some code paths in Qt (creating QMessageBox, for instance) will trigger the SIGILL on pshufb - an SSE3 instruction. > > > The following patch proposes the following changes to apply to all packages > > using qt4-build: > > > > USE flag "cpudetection" which reverts to the current problematic behaviour > > CPU_FEATURES specific flags which apply correctly to the configure script. > Why are we specifically hiding functionality available in the upstream configure script? > (In reply to comment #1) > > I remember having posted some questions on that in regard to a > > cross-compiling distcc-setup and the answers back then were: Qt does > > runtime-auto-detection if CPU-features such as supported SIMD-instructions. > > Correct, Qt does run-time cpu detection and enables only the appropriate > code paths. If you want to disable some cpu-specific features at build-time, > please use C(XX)FLAGS. No, incorrect and incorrect. The C(XX)FLAGS fail to build as demonstrated https://bugs.gentoo.org/attachment.cgi?id=332504 The runtime detection doesn't work correctly either - which is what spawned this entire work.
(In reply to comment #4) > > > > > I believe that it is in the spirit of Gentoo to provide users the ability to > > > control the target hardware configuration when building the system. > > > > > > > CFLAGS/CXXFLAGS are your friends (see -mno-*). > > See attached build.log. > > I disabled my eclass-overrides repos.conf setting and building it with > regular qt4-build.eclass - for pentium4, on k8-sse3. > Uhm well, this looks like a recent regression then... The issue was fixed in bug 336618 but has reappeared in 4.8.4 (maybe earlier?). I'll have a look. > > > > > Unfortunately, that is not currently the case. This leads to the situation > > > where software fails to run with SIGILL, when one attempts to build it on a > > > processor vastly superior to the running one. > > > > > > > Do you know of a specific failure case (in qt)? > > > > Yes. This is how this entire work started. > > Try building for -march=pentium4 on anything SSE3+ capable. Some code paths > in Qt (creating QMessageBox, for instance) will trigger the SIGILL on pshufb > - an SSE3 instruction. > IIRC gcc is allowed to generate any SIMD instruction if not told to avoid them. So if you are cross-compiling you should always specify -mno-$foo for each instruction set $foo not supported by the target machine. > > > > > The following patch proposes the following changes to apply to all packages > > > using qt4-build: > > > > > > USE flag "cpudetection" which reverts to the current problematic behaviour > > > CPU_FEATURES specific flags which apply correctly to the configure script. > > > > Why are we specifically hiding functionality available in the upstream > configure script? > Because it would duplicate functionality available elsewhere (CFLAGS). > > (In reply to comment #1) > > > I remember having posted some questions on that in regard to a > > > cross-compiling distcc-setup and the answers back then were: Qt does > > > runtime-auto-detection if CPU-features such as supported SIMD-instructions. > > > > Correct, Qt does run-time cpu detection and enables only the appropriate > > code paths. If you want to disable some cpu-specific features at build-time, > > please use C(XX)FLAGS. > > No, incorrect and incorrect. > > The C(XX)FLAGS fail to build as demonstrated > https://bugs.gentoo.org/attachment.cgi?id=332504 > This is a bug/regression in our eclass code, as I said above. > The runtime detection doesn't work correctly either - which is what spawned > this entire work. Are you sure the illegal instruction was selected by qt and not inserted by gcc instead?
(In reply to comment #5) > Uhm well, this looks like a recent regression then... The issue was fixed in > bug 336618 but has reappeared in 4.8.4 (maybe earlier?). I'll have a look. Thanks! If you need any help, I'd love to get to the bottom of what ended up wasting a lot of my time. > > Try building for -march=pentium4 on anything SSE3+ capable. Some code paths > > in Qt (creating QMessageBox, for instance) will trigger the SIGILL on pshufb > > - an SSE3 instruction. > > > > IIRC gcc is allowed to generate any SIMD instruction if not told to avoid > them. So if you are cross-compiling you should always specify -mno-$foo for > each instruction set $foo not supported by the target machine. IIRC it's not the case. The -mno-$foo disable the explicit SIMD primitives (the _mm_lddqu_si128 and the likes), while -march=machine controls what the compiler is going to output through autovectorization.. > > Why are we specifically hiding functionality available in the upstream > > configure script? > > > > Because it would duplicate functionality available elsewhere (CFLAGS). Well, I'm not so sure about it. Firstly, I think using -mno-* is wrong - this disables the SIMD primitives, but unless your eclass sets the very same configure flags I propose, the Qt will still build with this support. Unless you build the checks with the CFLAGS (bug 336618), at which point Qt configure will reach the same conclusion automatically since the *compile* phase of those checks will fail. It is a viable workaround, but it is as you noticed, prone to failure. Secondly, I think that the -march=cputype is the more common way of handling restricting gcc SIMD choice - the -mno-* is used to make sure gcc fails when the developer used SIMD primitives manually. Thirdly, SIMD USE flags are commonly accepted throughout other packages as well - mplayer, ffmpeg, etc. They reflect the fact that the upstream has chosen to provide a build-time option to disable support for In fact, it's the first time I saw anyone recommend specifying -mno* CFLAGS. Please, KISS. > > The runtime detection doesn't work correctly either - which is what spawned > > this entire work. > > Are you sure the illegal instruction was selected by qt and not inserted by > gcc instead? I'm quite certain it has to do with -DQT_HAVE_SSE3 -DQT_HAVE_SSSE3 -DQT_HAVE_SSE4_1 -DQT_HAVE_SSE4_2 -DQT_HAVE_AVX build flags. At least it's gone after building it without those through my version of the eclass. If you think it would be helpful, I can reproduce this exact error and provide a backtrace.
(In reply to comment #6) > (In reply to comment #5) > > Uhm well, this looks like a recent regression then... The issue was fixed in > > bug 336618 but has reappeared in 4.8.4 (maybe earlier?). I'll have a look. > > Thanks! If you need any help, I'd love to get to the bottom of what ended up > wasting a lot of my time. > Now fixed in cvs.
where is cvs change for cpudetection? it still failing in some cases, like no-avx supported CPU's, see http://bugs.funtoo.org/browse/FL-380
I think this got eventually fixed in both qt4-build-multilib and qt5-build eclasses. https://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/eclass/qt4-build-multilib.eclass?r1=1.10&r2=1.11 https://gitweb.gentoo.org/proj/qt.git/commit/?id=9843f6b6b71ca7b82c28685105a08a172e767c1c Please reopen if there are still issues.
Created attachment 405586 [details] qtwidgets-5.4.2 build log with incorrect instructions detected
still detection is wrong. Above attached qtwidgets build on athlon-xp box.
(In reply to Oleg from comment #11) > still detection is wrong. Above attached qtwidgets build on athlon-xp box. Please open a separate bug report. It looks like a different issue.
