Bug 571800 – linux-mod.eclass compilation failure with SSE specific flags on x86_64 systems

• Product: Gentoo Linux
• Status: RESOLVED FIXED

[Description Fabio Rossi 2016-01-14 00:29:17 UTC]

Created attachment 422852 [details, diff]
linux-mod.eclass.patch

I was trying to figure out the source of build failure in another bug report, specifically https://bugs.gentoo.org/show_bug.cgi?id=559798#c63

The real error is not the one described in that comment but a more generic gcc compilation failure, in particular 

  error: -mpreferred-stack-boundary=3 is not between 4 and 12

After discussing with the reporter of the problem I have detected the real cause of problem, one or more of his CFLAGS. In the specific case the flags are:

CFLAGS="-O2 -pipe -march=bdver2 -mcx16 -msahf -mno-movbe -mpopcnt -mabm -mlwp -mfma -mfma4 -mxop -mbmi -mno-bmi2 -mtbm -mno-avx2 -msse4.2 -msse4.1 -mlzcnt -mno-rtm -mno-hle -mno-rdrnd -mf16c -mno-fsgsbase -mno-rdseed -mprfchw -mno-adx -mfxsr -mxsave -mno-xsaveopt --param l1-cache-size=16 --param l1-cache-line-size=64 --param l2-cache-size=2048 -mtune=bdver2 -fstack-protector"

I have found a similar gcc error for another kernel module, in the Gentoo bug report https://bugs.gentoo.org/show_bug.cgi?id=492964. The solution over there was to filter out generic SSE related CFLAGS. The same filtering applied to the original problem isn't enough because there are other SSE related flags in the list (I didn't check which one). Viceversa applying the above CFLAGS to the bug #492964 still leads to the same compilation failure. I think it's interesting to read also the gcc bug referenced over there, https://gcc.gnu.org/bugzilla/show_bug.cgi?id=53383. 

My understanding is that in the kernel SSE code is generally disabled/avoided so -mpreferred-stack-boundary=3 is forced on even on 64 bit systems to limit stack usage. This means that using SSE flags is useless. If any user CFLAGS requires the generation of SSE code than gcc would need -mpreferred-stack-boundary=4 leading to a compilation failure as in the two examples reported above.

As seems that SSE flags are almost useless for kernel modules (first comment in the gcc report), the general solution I propose is not to filter out every SSE related gcc option but disable at all SSE code generation appending the -mno-sse option (this must be present, it's not clear to me about the other -mno-sse2, -mno-sse3, etc).

The ideal place for appending the -mno-sse is of course the linux-mod.eclass, here is attached a proposed patch.

[Comment 1 Alex Xu (Hello71) 2016-01-15 15:27:36 UTC]

assigning to kernel-misc, but note that you will likely have to start an RFC on gentoo-dev.

[Comment 2 Mike Pagano 2021-06-09 16:37:46 UTC]

It looks to me that this is fixed in later versions of gcc

Please reopen if I am incorrect

$ touch foo.c
$ gcc -c -mno-sse -mpreferred-stack-boundary=3 -mincoming-stack-boundary=3 foo.c
$ gcc -c -mno-sse -mpreferred-stack-boundary=3 foo.c