Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 261375 - dev-lang/perl and xs: bad CC/CXX/CFLAGS/CPPFLAGS/LDFLAGS handling
Summary: dev-lang/perl and xs: bad CC/CXX/CFLAGS/CPPFLAGS/LDFLAGS handling
Status: IN_PROGRESS
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Eclasses (show other bugs)
Hardware: All Linux
: High normal with 2 votes (vote)
Assignee: Gentoo Perl team
URL:
Whiteboard:
Keywords:
: 233354 236200 513866 569266 569662 583306 719162 719164 719168 719220 719222 719224 719228 719230 719232 719234 719236 719238 (view as bug list)
Depends on:
Blocks: perl-eclass perl-toolchain 458930 536460
  Show dependency tree
 
Reported: 2009-03-05 22:25 UTC by Torsten Veller (RETIRED)
Modified: 2020-09-03 18:00 UTC (History)
11 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
perl-module.eclass.patch (perl-module.eclass.patch,701 bytes, patch)
2011-08-13 05:39 UTC, Mario Fetka (geos_one)
Details | Diff
updated patch (perl-module.eclass.patch,1.23 KB, patch)
2014-06-18 23:11 UTC, Andreas K. Hüttel
Details | Diff
updated patch (perl-module.eclass.patch,810 bytes, patch)
2014-06-20 17:14 UTC, Andreas K. Hüttel
Details | Diff
updated patch (perl-module.eclass.patch,1.15 KB, patch)
2014-06-21 15:51 UTC, Andreas K. Hüttel
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Torsten Veller (RETIRED) gentoo-dev 2009-03-05 22:25:14 UTC
Should we add OPTIMIZE=${CFLAGS} to src_compile?

Needs more investigation.
What are the problems?
What else is missing?


If I understand it correctly, if OPTIMIZE is not set,
perl uses the OPTIMIZE settings when perl was installed.
Comment 1 Matt Turner gentoo-dev 2010-12-21 00:07:59 UTC
Can a Perl dev please take a look at this?
Comment 2 Mario Fetka (geos_one) 2011-08-09 02:38:59 UTC
this is also required for multilib portage
Comment 3 Mario Fetka (geos_one) 2011-08-13 05:39:57 UTC
Created attachment 283109 [details, diff]
perl-module.eclass.patch

this patch also sets some overwrites for some broken makefiles so a multilib compile can work (yes set OPTIMIZE 2 times)
Comment 4 Torsten Veller (RETIRED) gentoo-dev 2011-10-16 07:38:41 UTC
Why do we use optimize and not e.g. ccflags - or both?

I'd like to have all of this documented somewhere.
Comment 5 Nathan Phillip Brink (binki) (RETIRED) gentoo-dev 2013-07-29 02:47:20 UTC
(In reply to Torsten Veller from comment #4)
> Why do we use optimize and not e.g. ccflags - or both?

CCFLAGS: 
http://search.cpan.org/dist/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm#CCFLAGS
“tring that will be included in the compiler call command line between the arguments INC and OPTIMIZE.”

OPTIMIZE:
http://search.cpan.org/dist/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm#OPTIMIZE
“Defaults to -O. Set it to -g to turn debugging on. The flag is passed to subdirectory makes.”

Since, reportedly, MakeMaker defaults to OPTIMIZE=-O *and* OPTIMIZE is on gcc’s command line after CCFLAGS, this would mean any user’s CFLAGS, e.g., -O2, would be overridden by OPTIMIZE. CCFLAGS seems to be intended to be used more like INC (equivalent to CPPFLAGS) and, thus, is intended to store necessary CFLAGS for compiling against external libraries. Since OPTIMIZE is meant to also store flags like “-g”, that is obviously intended to be used by the person building any perl module to append extra options, such as the debugging flags necessary during development of the perl module.

Note that, according to http://search.cpan.org/dist/ExtUtils-MakeMaker/lib/ExtUtils/MakeMaker.pm#Using_Attributes_and_Parameters , flags like OPTIMIZE and CCFLAGS should be passed when invoking Makefile.PL like shown in attachment 283109 [details, diff]. It seems that specifying OPTIMIZE=-g may cause MakeMaker to detect that debug mode is requested and, thus, might have side effects.
Comment 6 salamanderrake 2013-07-29 15:26:14 UTC
This also allowed me to install
 
perl-core/Digest-SHA-5.820.0
virtual/perl-Digest-SHA-5.820.0

on my system so this is definitely a fix.
Comment 7 Andreas K. Hüttel gentoo-dev 2014-06-18 20:57:45 UTC
*** Bug 236200 has been marked as a duplicate of this bug. ***
Comment 8 Andreas K. Hüttel gentoo-dev 2014-06-18 23:11:42 UTC
Created attachment 379234 [details, diff]
updated patch

Well, if you use get_libdir you additionally need to inherit multilib.eclass

This looks promising but badly needs testing before commit.
Comment 9 Andreas K. Hüttel gentoo-dev 2014-06-20 17:14:56 UTC
Created attachment 379328 [details, diff]
updated patch

Let's strip any multilib stuff out for now. Different issue.
Comment 10 Andreas K. Hüttel gentoo-dev 2014-06-21 15:51:16 UTC
Created attachment 379366 [details, diff]
updated patch

Let's also try to address the LDFLAGS pain
Comment 11 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2015-02-13 18:22:21 UTC
07:19:33 <@leont> But overriding lddlflags is something I'd recommend against
07:21:06 <@leont> It tends to contain important stuff
Comment 12 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2015-02-13 19:20:30 UTC
For Module::Build, upstream recommend --extra-compiler-flags and --extra-linker-flags , as these don't clobber any existing defaults provided by perl itself or the .PL itself ( of which, some may be necessary ).

> 07:37:09 <@kentnl> So, imagine theres a XS based dist your code uses a lot of, and you want to squeeze out a few extra cycles by doing a profile-guided compile. This requires adding -fprofile-generate to both cflags and ldflags. How would you go about doing that without a) editing the .PL file by hand, or b) rebuilding perl itself ( or C) doing nasty shit with editing $Config ) ). 
> 07:37:52 <@kentnl> ( Also, I tried doing this with perl itself earlier in the week just for kicks, couldn't get it to build because it failed its own tests in ExtUtils::Constant for some reason I haven't dug into yet )
>07:43:18 <@leont> In MB, --extra-compiler-flags and --extra-linker-flags would be the easiest way
 

However, those values can also be ignored by having the equivalents of those fields specified explicitly in Build.PL, which some modules like https://metacpan.org/pod/ExtUtils::CppGuess do: 

> https://metacpan.org/source/ETJ/ExtUtils-CppGuess-0.08/lib/ExtUtils/CppGuess.pm#L168

> https://metacpan.org/pod/distribution/Module-Build/lib/Module/Build/API.pod#extra_compiler_flags

> https://metacpan.org/pod/distribution/Module-Build/lib/Module/Build/API.pod#extra_compiler_flags1

so overriding optimise and lddflags are both too severe, and may lead to unintentional breakage.

but extra-compiler-flags and extra-linker-flags are somewhat prone to locally being hurt too.


> 07:52:36 <@kentnl> so optimise= # is this extending or obliterrating the defaults?
> 07:54:41 <@leont> obliterrating, but at least in that field I wouldn't expect anything essential
> 07:54:56 <@leont> It's only passed on to the compilation stage, though
> 08:06:32 <@kentnl> so the overall vibe I get is "if you want to stick stuff in after dist-specified flags, there's presently no way of getting that to happen"  ( as oppossed to "what you're trying to do is too crazy and should never be done" )
> 08:07:10  * kentnl knows its a little crazy in the "probably viable but you get to keep the pieces" kind but thats acceptable for this context IME =)
> 08:13:08 <@leont> This is harder than it should be, yet
> 08:13:10 <@leont> yes
> 08:13:31 <@leont> It's also perfectly reasonable to want this, IMO
Comment 13 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2015-02-13 19:57:46 UTC
For Module::Build, I've filed this, with the hope upstream gets nicer =)

https://github.com/Perl-Toolchain-Gang/Module-Build/issues/54
Comment 14 Andreas K. Hüttel gentoo-dev 2015-12-10 23:26:23 UTC
(In reply to Kent Fredric from comment #13)
> For Module::Build, I've filed this, with the hope upstream gets nicer =)
> 
> https://github.com/Perl-Toolchain-Gang/Module-Build/issues/54

Since upstream progress is stalled, deferred for EAPI=7
Comment 15 Andreas K. Hüttel gentoo-dev 2015-12-23 06:24:53 UTC
*** Bug 569266 has been marked as a duplicate of this bug. ***
Comment 16 Andreas K. Hüttel gentoo-dev 2015-12-25 18:06:54 UTC
*** Bug 569662 has been marked as a duplicate of this bug. ***
Comment 17 Andreas K. Hüttel gentoo-dev 2016-12-28 21:23:30 UTC
*** Bug 583306 has been marked as a duplicate of this bug. ***
Comment 18 Andreas K. Hüttel gentoo-dev 2016-12-28 21:41:11 UTC
*** Bug 513866 has been marked as a duplicate of this bug. ***
Comment 19 Andreas K. Hüttel gentoo-dev 2017-01-01 10:20:00 UTC
*** Bug 233354 has been marked as a duplicate of this bug. ***
Comment 20 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-24 12:36:09 UTC
*** Bug 719236 has been marked as a duplicate of this bug. ***
Comment 21 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-24 12:36:17 UTC
*** Bug 719224 has been marked as a duplicate of this bug. ***
Comment 22 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-24 12:36:31 UTC
*** Bug 719228 has been marked as a duplicate of this bug. ***
Comment 23 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-24 12:36:39 UTC
*** Bug 719230 has been marked as a duplicate of this bug. ***
Comment 24 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-24 12:36:44 UTC
*** Bug 719232 has been marked as a duplicate of this bug. ***
Comment 25 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-24 12:36:51 UTC
*** Bug 719234 has been marked as a duplicate of this bug. ***
Comment 26 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-24 12:37:18 UTC
*** Bug 719238 has been marked as a duplicate of this bug. ***
Comment 27 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-30 21:30:59 UTC
*** Bug 719162 has been marked as a duplicate of this bug. ***
Comment 28 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-30 21:31:04 UTC
*** Bug 719164 has been marked as a duplicate of this bug. ***
Comment 29 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-30 21:31:14 UTC
*** Bug 719168 has been marked as a duplicate of this bug. ***
Comment 30 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-30 21:31:19 UTC
*** Bug 719220 has been marked as a duplicate of this bug. ***
Comment 31 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-04-30 21:31:24 UTC
*** Bug 719222 has been marked as a duplicate of this bug. ***
Comment 32 Jonas Stein gentoo-dev 2020-08-29 07:40:46 UTC
EAPI=7 is in the tree since a while. 
Any news? Is it probably fixed already?
Comment 33 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-08-29 11:43:02 UTC
(In reply to Jonas Stein from comment #32)
> EAPI=7 is in the tree since a while. 
> Any news? Is it probably fixed already?

Upstream sees zero prospect of fixing their nonsense, because nobody there cares about real problems, and prefer to create new ones for us to deal with.

Just today I stumbled on a package that didn't even *respond* to the general approaches that I *typically* use to mitigate CFLAGS handling, such that they trivially tweaked the Makefile.PL in ways I don't understand ( and had been tweaked as so since 2014 ), such that I had to find a different approach to simply make it work. ( See the recent commits I made to dev-perl/File-LibMagic )

That is to say, even specifying OPTIMIZE=${CFLAGS} in src_compile _should_ have worked, but *didnt*.

And additionally, their FUBAR made it so that `make` compiled the .so in all of:

- src_compile
- src_test
- src_install

So naturally, fixing src_compile alone to pass CFLAGS was undone by the recompile in src_test and src_install, which clobbered the .so with a version compiled without the passed CFLAGS.

I'm at a point where I'm just thinking its impossible to generalize, the underlying tooling is too much made of garbage, and due to the nature of it, its prohibitively difficult for us to fix.

So instead, I'm just opting for the harder task, fixing every problem in every perl related ebuild, on a case-by-case basis, and needing to apply case specific hacks to make it work.

So far, to the best of my knowledge, all of dev-perl/A* through to dev-perl/E* now respects user specified CFLAGS, and additionally, has mitigation in place to neuter an LD set in ENV, which upstream tooling is utterly stupid for, because it expects LD to be a *CCLD*, and expects to be able to pass arbitrary CFLAGS to it. ( People who know what they're doing can set CCLD= , and I'd employ toolchain-funcs for such a thing if toolchain-funcs decided to define a usable tc-getCCLD, but such a thing doesn't yet exist )

For the residual flags that need respecting, I need more ways of identifying when they're not respected, as tc-directly, `LD=ld` and `CFLAGS=` respect tests are all I have understanding enough to make packages go "boom" when its done wrong.

To rephrase, I'm open to fixing every single one of these problems, for every perl package, if a reliable way can be made to demonstrate and test the problem sufficient that I can actually be sure I fixed it.

Its a gargantuan task, but achievable incrementally.
Comment 34 Arfrever Frehtes Taifersar Arahesis 2020-09-01 22:39:32 UTC
(In reply to Kent Fredric (IRC: kent\n) from comment #33)
> has mitigation in place to neuter an LD set in ENV, which upstream tooling is
> utterly stupid for, because it expects LD to be a *CCLD*, and expects to be
> able to pass arbitrary CFLAGS to it. ( People who know what they're doing
> can set CCLD= , and I'd employ toolchain-funcs for such a thing if
> toolchain-funcs decided to define a usable tc-getCCLD, but such a thing
> doesn't yet exist )

There is no need for tc-getCCLD() function (or similarly tc-getCCAS()).
Affected ebuilds/eclass can simply have one of the following:
  export LD="$(tc-getCC)"
  export LD="$(tc-getCXX)"
Comment 35 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2020-09-03 18:00:12 UTC
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #34)
> (In reply to Kent Fredric (IRC: kent\n) from comment #33)
> > has mitigation in place to neuter an LD set in ENV, which upstream tooling is
> > utterly stupid for, because it expects LD to be a *CCLD*, and expects to be
> > able to pass arbitrary CFLAGS to it. ( People who know what they're doing
> > can set CCLD= , and I'd employ toolchain-funcs for such a thing if
> > toolchain-funcs decided to define a usable tc-getCCLD, but such a thing
> > doesn't yet exist )
> 
> There is no need for tc-getCCLD() function (or similarly tc-getCCAS()).
> Affected ebuilds/eclass can simply have one of the following:
>   export LD="$(tc-getCC)"
>   export LD="$(tc-getCXX)"

There is if you want users to be able to set CCLD= in ENV and have it do the right thing.

And yes, I know I could do that, but that's a "workaround", not a "solution".

CC, CXX, LD, and CCLD are all their own concepts, and if a user wants to set one, but not the other, they're entitled to.