Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 261375 (perlxsflags)

Summary: dev-lang/perl and xs: bad CC/CXX/CFLAGS/CPPFLAGS/LDFLAGS handling
Product: Gentoo Linux Reporter: Torsten Veller (RETIRED) <tove>
Component: EclassesAssignee: Gentoo Perl team <perl>
Status: IN_PROGRESS ---    
Severity: normal CC: ago, amit.ugol, arfrever.fta, binki, esigra, flameeyes, itumaykin+gentoo, jstein, kentnl, mgorny, phkbphkbphkb, plevine457, randy-andy-, sam, telans, titanofold
Priority: High Keywords: PullRequest
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://github.com/gentoo/gentoo/pull/30009
https://github.com/Perl/perl5/issues/18939
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on: 378645, 756337, 806619    
Bug Blocks: 59506, 239510, 331933, 723238, 458930, 536460    
Attachments: perl-module.eclass.patch
updated patch
updated patch
updated patch

Description Torsten Veller (RETIRED) gentoo-dev 2009-03-05 22:25:14 UTC
Should we add OPTIMIZE=${CFLAGS} to src_compile?

Needs more investigation.
What are the problems?
What else is missing?


If I understand it correctly, if OPTIMIZE is not set,
perl uses the OPTIMIZE settings when perl was installed.
Comment 1 Matt Turner gentoo-dev 2010-12-21 00:07:59 UTC
Can a Perl dev please take a look at this?
Comment 2 Mario Fetka (geos_one) 2011-08-09 02:38:59 UTC
this is also required for multilib portage
Comment 3 Mario Fetka (geos_one) 2011-08-13 05:39:57 UTC
Created attachment 283109 [details, diff]
perl-module.eclass.patch

this patch also sets some overwrites for some broken makefiles so a multilib compile can work (yes set OPTIMIZE 2 times)
Comment 4 Torsten Veller (RETIRED) gentoo-dev 2011-10-16 07:38:41 UTC
Why do we use optimize and not e.g. ccflags - or both?

I'd like to have all of this documented somewhere.
Comment 5 Nathan Phillip Brink (binki) (RETIRED) gentoo-dev 2013-07-29 02:47:20 UTC
(In reply to Torsten Veller from comment #4)
> Why do we use optimize and not e.g. ccflags - or both?

CCFLAGS: 
http://search.cpan.org/dist/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm#CCFLAGS
“tring that will be included in the compiler call command line between the arguments INC and OPTIMIZE.”

OPTIMIZE:
http://search.cpan.org/dist/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm#OPTIMIZE
“Defaults to -O. Set it to -g to turn debugging on. The flag is passed to subdirectory makes.”

Since, reportedly, MakeMaker defaults to OPTIMIZE=-O *and* OPTIMIZE is on gcc’s command line after CCFLAGS, this would mean any user’s CFLAGS, e.g., -O2, would be overridden by OPTIMIZE. CCFLAGS seems to be intended to be used more like INC (equivalent to CPPFLAGS) and, thus, is intended to store necessary CFLAGS for compiling against external libraries. Since OPTIMIZE is meant to also store flags like “-g”, that is obviously intended to be used by the person building any perl module to append extra options, such as the debugging flags necessary during development of the perl module.

Note that, according to http://search.cpan.org/dist/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm#Using_Attributes_and_Parameters , flags like OPTIMIZE and CCFLAGS should be passed when invoking Makefile.PL like shown in attachment 283109 [details, diff]. It seems that specifying OPTIMIZE=-g may cause MakeMaker to detect that debug mode is requested and, thus, might have side effects.
Comment 6 ewomer 2013-07-29 15:26:14 UTC
This also allowed me to install
 
perl-core/Digest-SHA-5.820.0
virtual/perl-Digest-SHA-5.820.0

on my system so this is definitely a fix.
Comment 7 Andreas K. Hüttel archtester gentoo-dev 2014-06-18 20:57:45 UTC
*** Bug 236200 has been marked as a duplicate of this bug. ***
Comment 8 Andreas K. Hüttel archtester gentoo-dev 2014-06-18 23:11:42 UTC
Created attachment 379234 [details, diff]
updated patch

Well, if you use get_libdir you additionally need to inherit multilib.eclass

This looks promising but badly needs testing before commit.
Comment 9 Andreas K. Hüttel archtester gentoo-dev 2014-06-20 17:14:56 UTC
Created attachment 379328 [details, diff]
updated patch

Let's strip any multilib stuff out for now. Different issue.
Comment 10 Andreas K. Hüttel archtester gentoo-dev 2014-06-21 15:51:16 UTC
Created attachment 379366 [details, diff]
updated patch

Let's also try to address the LDFLAGS pain
Comment 11 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2015-02-13 18:22:21 UTC
07:19:33 <@leont> But overriding lddlflags is something I'd recommend against
07:21:06 <@leont> It tends to contain important stuff
Comment 12 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2015-02-13 19:20:30 UTC
For Module::Build, upstream recommend --extra-compiler-flags and --extra-linker-flags , as these don't clobber any existing defaults provided by perl itself or the .PL itself ( of which, some may be necessary ).

> 07:37:09 <@kentnl> So, imagine theres a XS based dist your code uses a lot of, and you want to squeeze out a few extra cycles by doing a profile-guided compile. This requires adding -fprofile-generate to both cflags and ldflags. How would you go about doing that without a) editing the .PL file by hand, or b) rebuilding perl itself ( or C) doing nasty shit with editing $Config ) ). 
> 07:37:52 <@kentnl> ( Also, I tried doing this with perl itself earlier in the week just for kicks, couldn't get it to build because it failed its own tests in ExtUtils::Constant for some reason I haven't dug into yet )
>07:43:18 <@leont> In MB, --extra-compiler-flags and --extra-linker-flags would be the easiest way
 

However, those values can also be ignored by having the equivalents of those fields specified explicitly in Build.PL, which some modules like https://metacpan.org/pod/ExtUtils::CppGuess do: 

> https://metacpan.org/source/ETJ/ExtUtils-CppGuess-0.08/lib/ExtUtils/CppGuess.pm#L168

> https://metacpan.org/pod/distribution/Module-Build/lib/Module/Build/API.pod#extra_compiler_flags

> https://metacpan.org/pod/distribution/Module-Build/lib/Module/Build/API.pod#extra_compiler_flags1

so overriding optimise and lddflags are both too severe, and may lead to unintentional breakage.

but extra-compiler-flags and extra-linker-flags are somewhat prone to locally being hurt too.


> 07:52:36 <@kentnl> so optimise= # is this extending or obliterrating the defaults?
> 07:54:41 <@leont> obliterrating, but at least in that field I wouldn't expect anything essential
> 07:54:56 <@leont> It's only passed on to the compilation stage, though
> 08:06:32 <@kentnl> so the overall vibe I get is "if you want to stick stuff in after dist-specified flags, there's presently no way of getting that to happen"  ( as oppossed to "what you're trying to do is too crazy and should never be done" )
> 08:07:10  * kentnl knows its a little crazy in the "probably viable but you get to keep the pieces" kind but thats acceptable for this context IME =)
> 08:13:08 <@leont> This is harder than it should be, yet
> 08:13:10 <@leont> yes
> 08:13:31 <@leont> It's also perfectly reasonable to want this, IMO
Comment 13 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2015-02-13 19:57:46 UTC
For Module::Build, I've filed this, with the hope upstream gets nicer =)

https://github.com/Perl-Toolchain-Gang/Module-Build/issues/54
Comment 14 Andreas K. Hüttel archtester gentoo-dev 2015-12-10 23:26:23 UTC
(In reply to Kent Fredric from comment #13)
> For Module::Build, I've filed this, with the hope upstream gets nicer =)
> 
> https://github.com/Perl-Toolchain-Gang/Module-Build/issues/54

Since upstream progress is stalled, deferred for EAPI=7
Comment 15 Andreas K. Hüttel archtester gentoo-dev 2015-12-23 06:24:53 UTC
*** Bug 569266 has been marked as a duplicate of this bug. ***
Comment 16 Andreas K. Hüttel archtester gentoo-dev 2015-12-25 18:06:54 UTC
*** Bug 569662 has been marked as a duplicate of this bug. ***
Comment 17 Andreas K. Hüttel archtester gentoo-dev 2016-12-28 21:23:30 UTC
*** Bug 583306 has been marked as a duplicate of this bug. ***
Comment 18 Andreas K. Hüttel archtester gentoo-dev 2016-12-28 21:41:11 UTC
*** Bug 513866 has been marked as a duplicate of this bug. ***
Comment 19 Andreas K. Hüttel archtester gentoo-dev 2017-01-01 10:20:00 UTC
*** Bug 233354 has been marked as a duplicate of this bug. ***
Comment 20 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-24 12:36:09 UTC
*** Bug 719236 has been marked as a duplicate of this bug. ***
Comment 21 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-24 12:36:17 UTC
*** Bug 719224 has been marked as a duplicate of this bug. ***
Comment 22 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-24 12:36:31 UTC
*** Bug 719228 has been marked as a duplicate of this bug. ***
Comment 23 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-24 12:36:39 UTC
*** Bug 719230 has been marked as a duplicate of this bug. ***
Comment 24 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-24 12:36:44 UTC
*** Bug 719232 has been marked as a duplicate of this bug. ***
Comment 25 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-24 12:36:51 UTC
*** Bug 719234 has been marked as a duplicate of this bug. ***
Comment 26 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-24 12:37:18 UTC
*** Bug 719238 has been marked as a duplicate of this bug. ***
Comment 27 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-30 21:30:59 UTC
*** Bug 719162 has been marked as a duplicate of this bug. ***
Comment 28 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-30 21:31:04 UTC
*** Bug 719164 has been marked as a duplicate of this bug. ***
Comment 29 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-30 21:31:14 UTC
*** Bug 719168 has been marked as a duplicate of this bug. ***
Comment 30 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-30 21:31:19 UTC
*** Bug 719220 has been marked as a duplicate of this bug. ***
Comment 31 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-30 21:31:24 UTC
*** Bug 719222 has been marked as a duplicate of this bug. ***
Comment 32 Jonas Stein gentoo-dev 2020-08-29 07:40:46 UTC
EAPI=7 is in the tree since a while. 
Any news? Is it probably fixed already?
Comment 33 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-08-29 11:43:02 UTC
(In reply to Jonas Stein from comment #32)
> EAPI=7 is in the tree since a while. 
> Any news? Is it probably fixed already?

Upstream sees zero prospect of fixing their nonsense, because nobody there cares about real problems, and prefer to create new ones for us to deal with.

Just today I stumbled on a package that didn't even *respond* to the general approaches that I *typically* use to mitigate CFLAGS handling, such that they trivially tweaked the Makefile.PL in ways I don't understand ( and had been tweaked as so since 2014 ), such that I had to find a different approach to simply make it work. ( See the recent commits I made to dev-perl/File-LibMagic )

That is to say, even specifying OPTIMIZE=${CFLAGS} in src_compile _should_ have worked, but *didnt*.

And additionally, their FUBAR made it so that `make` compiled the .so in all of:

- src_compile
- src_test
- src_install

So naturally, fixing src_compile alone to pass CFLAGS was undone by the recompile in src_test and src_install, which clobbered the .so with a version compiled without the passed CFLAGS.

I'm at a point where I'm just thinking its impossible to generalize, the underlying tooling is too much made of garbage, and due to the nature of it, its prohibitively difficult for us to fix.

So instead, I'm just opting for the harder task, fixing every problem in every perl related ebuild, on a case-by-case basis, and needing to apply case specific hacks to make it work.

So far, to the best of my knowledge, all of dev-perl/A* through to dev-perl/E* now respects user specified CFLAGS, and additionally, has mitigation in place to neuter an LD set in ENV, which upstream tooling is utterly stupid for, because it expects LD to be a *CCLD*, and expects to be able to pass arbitrary CFLAGS to it. ( People who know what they're doing can set CCLD= , and I'd employ toolchain-funcs for such a thing if toolchain-funcs decided to define a usable tc-getCCLD, but such a thing doesn't yet exist )

For the residual flags that need respecting, I need more ways of identifying when they're not respected, as tc-directly, `LD=ld` and `CFLAGS=` respect tests are all I have understanding enough to make packages go "boom" when its done wrong.

To rephrase, I'm open to fixing every single one of these problems, for every perl package, if a reliable way can be made to demonstrate and test the problem sufficient that I can actually be sure I fixed it.

Its a gargantuan task, but achievable incrementally.
Comment 34 Arfrever Frehtes Taifersar Arahesis 2020-09-01 22:39:32 UTC
(In reply to Kent Fredric (IRC: kent\n) from comment #33)
> has mitigation in place to neuter an LD set in ENV, which upstream tooling is
> utterly stupid for, because it expects LD to be a *CCLD*, and expects to be
> able to pass arbitrary CFLAGS to it. ( People who know what they're doing
> can set CCLD= , and I'd employ toolchain-funcs for such a thing if
> toolchain-funcs decided to define a usable tc-getCCLD, but such a thing
> doesn't yet exist )

There is no need for tc-getCCLD() function (or similarly tc-getCCAS()).
Affected ebuilds/eclass can simply have one of the following:
  export LD="$(tc-getCC)"
  export LD="$(tc-getCXX)"
Comment 35 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-09-03 18:00:12 UTC
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #34)
> (In reply to Kent Fredric (IRC: kent\n) from comment #33)
> > has mitigation in place to neuter an LD set in ENV, which upstream tooling is
> > utterly stupid for, because it expects LD to be a *CCLD*, and expects to be
> > able to pass arbitrary CFLAGS to it. ( People who know what they're doing
> > can set CCLD= , and I'd employ toolchain-funcs for such a thing if
> > toolchain-funcs decided to define a usable tc-getCCLD, but such a thing
> > doesn't yet exist )
> 
> There is no need for tc-getCCLD() function (or similarly tc-getCCAS()).
> Affected ebuilds/eclass can simply have one of the following:
>   export LD="$(tc-getCC)"
>   export LD="$(tc-getCXX)"

There is if you want users to be able to set CCLD= in ENV and have it do the right thing.

And yes, I know I could do that, but that's a "workaround", not a "solution".

CC, CXX, LD, and CCLD are all their own concepts, and if a user wants to set one, but not the other, they're entitled to.
Comment 36 Andreas K. Hüttel archtester gentoo-dev 2021-05-01 14:39:49 UTC
*** Bug 731226 has been marked as a duplicate of this bug. ***
Comment 37 Andreas K. Hüttel archtester gentoo-dev 2021-05-01 14:59:47 UTC
*** Bug 737836 has been marked as a duplicate of this bug. ***
Comment 38 Andreas K. Hüttel archtester gentoo-dev 2021-05-02 12:30:48 UTC
*** Bug 747889 has been marked as a duplicate of this bug. ***
Comment 39 Andreas K. Hüttel archtester gentoo-dev 2021-05-04 22:01:39 UTC
*** Bug 762205 has been marked as a duplicate of this bug. ***
Comment 40 Andreas K. Hüttel archtester gentoo-dev 2021-05-09 13:02:31 UTC
*** Bug 747892 has been marked as a duplicate of this bug. ***
Comment 41 Andreas K. Hüttel archtester gentoo-dev 2021-05-13 11:26:44 UTC
*** Bug 756337 has been marked as a duplicate of this bug. ***
Comment 42 Larry the Git Cow gentoo-dev 2021-06-19 19:12:32 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0dd937bace2f074eee910c733ca608627272b60d

commit 0dd937bace2f074eee910c733ca608627272b60d
Author:     Andreas K. Hüttel <dilfridge@gentoo.org>
AuthorDate: 2021-06-18 21:39:33 +0000
Commit:     Andreas K. Hüttel <dilfridge@gentoo.org>
CommitDate: 2021-06-19 19:10:33 +0000

    perl-module.eclass: Add EAPI=8 support
    
    New features: DIST_WIKI and DIST_MAKE
    File permissions are fixed on installation
    
    Bug: https://bugs.gentoo.org/733020
    Bug: https://bugs.gentoo.org/554346
    Bug: https://bugs.gentoo.org/261375
    Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>

 eclass/perl-module.eclass | 89 ++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 76 insertions(+), 13 deletions(-)
Comment 43 Andreas K. Hüttel archtester gentoo-dev 2021-06-23 20:36:59 UTC
In EAPI=8, OPTIMIZE=${CFLAGS} is now set by default for EU::MM distributions.
Comment 44 Larry the Git Cow gentoo-dev 2021-10-26 16:23:40 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=4b0050c7770f26eba5b5e704bc67ffcf82af2052

commit 4b0050c7770f26eba5b5e704bc67ffcf82af2052
Author:     Andreas K. Hüttel <dilfridge@gentoo.org>
AuthorDate: 2021-10-26 16:23:02 +0000
Commit:     Andreas K. Hüttel <dilfridge@gentoo.org>
CommitDate: 2021-10-26 16:23:02 +0000

    dev-lang/perl: Re-keyword -r5
    
    * Set -Dccflags="${CFLAGS}", bug 806619
    * Drop duplicate configure flags, bug 694046
    * Hardcode SINGLE_SLOT for simplification
    
    Closes: https://bugs.gentoo.org/694046
    Closes: https://bugs.gentoo.org/806619
    Bug: https://bugs.gentoo.org/261375
    Package-Manager: Portage-3.0.28, Repoman-3.0.3
    Signed-off-by: Andreas K. Hüttel <dilfridge@gentoo.org>

 dev-lang/perl/perl-5.34.0-r5.ebuild | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
Comment 45 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2021-11-19 12:29:27 UTC
*** Bug 824846 has been marked as a duplicate of this bug. ***
Comment 46 Alec Ari 2022-08-15 23:32:02 UTC
Not sure if related but I had to force LD to either clang or gcc to build dev-perl/Syntax-Keyword-Try and dev-perl/XS-Parse-Keyword:

make.conf: LD="ld.lld"

Every compiler option returns as unknown argument:

ld.lld: error: unknown argument '-fPIC'
ld.lld: error: unknown argument '-fstack-protector-strong'
ld.lld: error: unknown argument '-fstack-clash-protection'
ld.lld: error: unknown argument '-fomit-frame-pointer'
ld.lld: error: unknown argument '-pipe'
ld.lld: error: unknown argument '-U_FORTIFY_SOURCE'
ld.lld: error: unknown argument '-D_FORTIFY_SOURCE=2'
ld.lld: error: unknown argument '-Wl,-O1'
ld.lld: error: unknown argument '-Wl,--as-needed'
ld.lld: error: unknown argument '-fPIC'
ld.lld: error: unknown argument '-fstack-protector-strong'
ld.lld: error: unknown argument '-fstack-clash-protection'
ld.lld: error: unknown argument '-fomit-frame-pointer'
ld.lld: error: unknown argument '-pipe'
ld.lld: error: unknown argument '-U_FORTIFY_SOURCE'
ld.lld: error: unknown argument '-D_FORTIFY_SOURCE=2'
ld.lld: error: unknown argument '-fuse-ld=lld'
ld.lld: error: unknown argument '-Wl,-z,now'
ld.lld: error: unknown argument '-Wl,-z,relro'
ld.lld: error: unknown argument '-Wl,--strip-debug'
Warning: ExtUtils::CBuilder not installed or no compiler detected
Proceeding with configuration, but compilation may fail during Build

My fix:

/etc/portage/package.env: dev-perl/Syntax-Keyword-Try perl_fix.conf
/etc/portage/package.env: dev-perl/XS-Parse-Keyword perl_fix.conf

/etc/portage/env/perl_fix.conf: LD="clang"

This also works if perl_fix.conf has LD="gcc"

LD="ld.bfd" or LD="ld.lld" are broken with those two packages. Perfectly fine system-wide.
Comment 47 Peter Levine 2023-01-15 08:06:31 UTC
Pertinent upstream issue:  https://github.com/Perl/perl5/issues/18939
Comment 48 Larry the Git Cow gentoo-dev 2023-03-11 17:23:28 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6eb964b504c3aef2192403cbf383ae75d46990ff

commit 6eb964b504c3aef2192403cbf383ae75d46990ff
Author:     Alfred Persson Forsberg <cat@catcream.org>
AuthorDate: 2023-03-09 00:21:37 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-03-11 17:23:20 +0000

    perl-module.eclass: Fix linking with ld.lld
    
    If LD is not specified then CCLD will be used as the linker.
    Setting CCLD to CC makes it correctly use LDFLAGS.
    
    Bug: https://bugs.gentoo.org/261375
    Closes: https://bugs.gentoo.org/882373
    Closes: https://bugs.gentoo.org/894608
    Signed-off-by: Alfred Persson Forsberg <cat@catcream.org>
    Closes: https://github.com/gentoo/gentoo/pull/30009
    Signed-off-by: Sam James <sam@gentoo.org>

 eclass/perl-module.eclass | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)