Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 722090 - sci-mathematics/pari fails to compile
Summary: sci-mathematics/pari fails to compile
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Michael Orlitzky
URL:
Whiteboard:
Keywords: PullRequest
Depends on:
Blocks: 729202
  Show dependency tree
 
Reported: 2020-05-10 09:29 UTC by Agostino Sarubbo
Modified: 2022-09-21 00:29 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log (build.log,37.69 KB, text/plain)
2020-05-10 09:29 UTC, Agostino Sarubbo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Agostino Sarubbo gentoo-dev 2020-05-10 09:29:35 UTC
This is an auto-filled bug because sci-mathematics/pari fails to compile.
The issue was originally discovered on amd64, but it may be reproducible on other arches as well.
If you think that a different summary clarifies the issue better, feel free to change it.
Attached build log and emerge --info.
Comment 1 Agostino Sarubbo gentoo-dev 2020-05-10 09:29:44 UTC
Created attachment 637498 [details]
build.log

build log and emerge --info
Comment 2 François Bissey 2020-05-10 10:02:51 UTC
How very curious. From the attached log
Looking for some tools first ...
...gzip is /bin/gzip
...I could not find cc.
...I could not find gcc.
...I could not find ld.
...perl is /usr/bin/perl
...zcat is /bin/zcat
GNU compatible compiler: gcc version 10.1.0 (Gentoo 10.1.0 p1)
Using mt engine single
Given the previous choices, sizeof(long) is 8 chars.
The internal word representation of a double is not needed (64bit).
==========================================================================
Building for: amd64 running linux (x86-64 kernel) 64-bit version
==========================================================================
C compiler is          x86_64-pc-linux-gnu-gcc -O3 -Wall -fno-strict-aliasing -fomit-frame-pointer  -O2 -pipe -march=native -frecord-gcc-switches  -fPIC
Executable linker is   x86_64-pc-linux-gnu-ld   --export-dynamic -Wl,-O1 -Wl,--as-needed -Wl,--defsym=__gentoo_check_ldflags__=0
Dynamic Lib linker is  x86_64-pc-linux-gnu-gcc  -shared  $(CFLAGS) $(DLCFLAGS) -Wl,-shared,-soname=$(LIBPARI_SONAME) 

The above is using --export-dynamic correctly with ld. However the rest of the line suffers from extra -Wl, which would be useful if the linking command was gcc. When the library is linked the flags for the executable linker are added to the dynamic library linker which is strange in itself but leads to an error since --export-dynamic is not protected by -Wl.

In contrast my logs show
Looking for some tools first ...
...gzip is /bin/gzip
...cc is /usr/bin/cc
...gcc is /usr/bin/gcc
...ld is /usr/bin/ld
...perl is /usr/bin/perl
...zcat is /bin/zcat
GNU compatible compiler: gcc version 9.3.0 (Gentoo 9.3.0 p2)
Using mt engine single
Given the previous choices, sizeof(long) is 8 chars.
The internal word representation of a double is not needed (64bit).
==========================================================================
Building for: amd64 running linux (x86-64/GMP kernel) 64-bit version
==========================================================================
C compiler is          x86_64-pc-linux-gnu-gcc -O3 -Wall -fno-strict-aliasing -fomit-frame-pointer  -march=native -O2 -pipe  -fPIC
Executable linker is   x86_64-pc-linux-gnu-gcc  -O3 -Wall -fno-strict-aliasing -fomit-frame-pointer  -march=native -O2 -pipe  -Wl,--xexport-dynamic -Wl,-O1 -Wl,--as-needed
Dynamic Lib linker is  x86_64-pc-linux-gnu-gcc  -shared  $(CFLAGS) $(DLCFLAGS) -Wl,-shared,-soname=$(LIBPARI_SONAME)

So the main issue is that standard commands are not found and somehow ld is identified as the linker instead of gcc.

Of course the configuration scripts from pari are not the best in the world but here I am wondering if we are more dealing with problems with the setup rather than the build system. I also cannot find bugs for gcc-10 in other distro so far.
Comment 3 Agostino Sarubbo gentoo-dev 2020-05-10 10:16:51 UTC
FTR I have LD="x86_64-pc-linux-gnu-ld"
Comment 4 François Bissey 2020-05-10 10:19:37 UTC
(In reply to Agostino Sarubbo from comment #3)
> FTR I have LD="x86_64-pc-linux-gnu-ld"

That probably explains a lot. I guess I should figure out how to deal with that scenario.
Comment 5 François Bissey 2020-05-10 10:24:41 UTC
(In reply to Agostino Sarubbo from comment #3)
> FTR I have LD="x86_64-pc-linux-gnu-ld"

After checking, that completely explains it. Not a gcc-10 issue. I'll see what I can do to deal with that particular issue.
Comment 6 François Bissey 2020-05-10 10:36:25 UTC
@Agostino: Question
You set LD to x86_64-pc-linux-gnu-ld but you set LDFLAGS to "-Wl,-O1 -Wl,--as-needed -Wl,--defsym=__gentoo_check_ldflags__=0". Doesn't that lead to problems? Or does that mean you expect most packages to ignore your LD setting?
Comment 7 Agostino Sarubbo gentoo-dev 2020-05-13 08:51:26 UTC
(In reply to François Bissey from comment #6)
> @Agostino: Question
> You set LD to x86_64-pc-linux-gnu-ld but you set LDFLAGS to "-Wl,-O1
> -Wl,--as-needed -Wl,--defsym=__gentoo_check_ldflags__=0". Doesn't that lead
> to problems? Or does that mean you expect most packages to ignore your LD
> setting?

this should be fine, because for linking is used CC/CXX instead of LD.

LD is used in particular situations and usually LDFLAGS are not used there
Comment 8 François Bissey 2020-05-13 09:01:31 UTC
I was busy the last two days so I lost sight of this. Pari build system uses the LD variable for its executable linker and DLLD for its dynamic library linker. As such I would say it is an unhappy overlap with one accepted standard variable name.

The best course of action is probably to unset LD in the ebuild, at least at configuration time.
Comment 9 Agostino Sarubbo gentoo-dev 2020-05-13 09:19:11 UTC
(In reply to François Bissey from comment #8)
> I was busy the last two days so I lost sight of this. Pari build system uses
> the LD variable for its executable linker and DLLD for its dynamic library
> linker. As such I would say it is an unhappy overlap with one accepted
> standard variable name.
> 
> The best course of action is probably to unset LD in the ebuild, at least at
> configuration time.

My guess is that they are confusing LD with CC.

CC should be used in the linking command
Comment 10 François Bissey 2020-05-13 09:27:30 UTC
(In reply to Agostino Sarubbo from comment #9)
> (In reply to François Bissey from comment #8)
> > I was busy the last two days so I lost sight of this. Pari build system uses
> > the LD variable for its executable linker and DLLD for its dynamic library
> > linker. As such I would say it is an unhappy overlap with one accepted
> > standard variable name.
> > 
> > The best course of action is probably to unset LD in the ebuild, at least at
> > configuration time.
> 
> My guess is that they are confusing LD with CC.
> 
> CC should be used in the linking command

No, I am fairly sure it is done on purpose as a way to set particular command(s), possibly with options, as their linker. And they have different variables for executable and library because the options would and should be different. Other packages which have internal variables for those things usually name things a bit more carefully.

They could very well have some kind of CI running with 
LD=gcc --foo
DLLD=gcc -shared --bar
for all I know.
Comment 11 Agostino Sarubbo gentoo-dev 2020-06-25 08:49:01 UTC
I can reproduce this bug on AT machine
Comment 12 François Bissey 2020-06-25 08:52:45 UTC
What's the AT machine? 

I was thinking of ending this bug because your settings is a bit like walking around with scissors and we don't protect users against themselves unless it is a vital part of the stack. After that, the only thing that helps users is education.
Comment 13 Agostino Sarubbo gentoo-dev 2020-06-25 09:14:14 UTC
(In reply to François Bissey from comment #12)
> What's the AT machine? 
> 
> I was thinking of ending this bug because your settings is a bit like
> walking around with scissors and we don't protect users against themselves
> unless it is a vital part of the stack. After that, the only thing that
> helps users is education.

I really do not understand your comment.
This package fails to compile. You have the build log and the emerge --info.
To reproduce try to set in your make.conf:

CC="x86_64-pc-linux-gnu-gcc"
CXX="x86_64-pc-linux-gnu-g++"
AR="x86_64-pc-linux-gnu-ar"
CPP="x86_64-pc-linux-gnu-gcc -E"
LD="x86_64-pc-linux-gnu-ld"
NM="x86_64-pc-linux-gnu-nm"
RANLIB="x86_64-pc-linux-gnu-ranlib"
GCOV="x86_64-pc-linux-gnu-gcov"
AS="x86_64-pc-linux-gnu-as"

I reproduced the issue on two different machine so I guess you will be able to reproduce it too.
Comment 14 François Bissey 2020-06-25 09:17:55 UTC
My issue is that setting LD in your default environment rather than letting the package decide is usually unwise. 

Do we have to fix all packages that will not discard that value if it is not appropriate?
Comment 15 Agostino Sarubbo gentoo-dev 2020-06-25 09:29:05 UTC
(In reply to François Bissey from comment #14)
> My issue is that setting LD in your default environment rather than letting
> the package decide is usually unwise. 
> 
> Do we have to fix all packages that will not discard that value if it is not
> appropriate?

Yes, because otherwise we will never be able to use different toolchain like clang/llvm.

If the program confuses LD with CC and so on you can overwrite by setting LD="$(tc-getCC)" and similar.
Comment 16 François Bissey 2020-06-25 09:31:47 UTC
All, right. You think it is better to have 
LD="$(tc-getCC)" 
rather than
unset LD
?
Comment 17 Agostino Sarubbo gentoo-dev 2020-06-25 09:51:15 UTC
(In reply to François Bissey from comment #16)
> All, right. You think it is better to have 
> LD="$(tc-getCC)" 
> rather than
> unset LD
> ?

I'm limiting myself to report what does not work for me.
For what is the better solution is up to you and your proxy-maintainer. Maybe you want ask qa and/or see if there are previous similar cases that end-up with a best practice
Comment 18 Michael Orlitzky gentoo-dev 2020-06-26 12:03:26 UTC
ago, is there a list of the variables that you want to override and their documented purpose? It would help if we had something to show to upstream and say "you shouldn't be using this variable this way."

On another bug, I was able to track down the following:

https://pubs.opengroup.org/onlinepubs/009695399/utilities/make.html
https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
https://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/Preset-Output-Variables.html

None of those mention $LD, however. I do know that the linux kernel uses $LD, and that meson now supports it for the dynamic library linker:

  https://github.com/mesonbuild/meson/pull/6207

But it's meaning still isn't very standard. So to pitch this to pari upstream, I think we need two questions answered:

  1. Is there some place that $LD is documented to mean the dynamic library
     linker, or do we have to rely on the convention from the linux kernel and
     now meson? For a counterexample, I think macOS uses DYLD to refer to the
     dynamic linker.

  2. Is there some other variable that should be used for the static linker,
     in the situations that pari is using $LD currently?

And regardless of the answers to those questions, I think you should add something to the devmanual that explains how we expect these variables to work in Gentoo. If everyone agrees that LD should be the dynamic linker in Gentoo, then we could set e.g. DLLD=$(LD) in the pari ebuild, but that only makes sense if $LD is guaranteed to mean "dynamic linker" in Gentoo. On the other hand, if the meaning of LD is not standardized in Gentoo, then doing that risks introducing as many build failures as it solves.
Comment 19 Agostino Sarubbo gentoo-dev 2020-06-26 12:24:05 UTC
(In reply to Michael Orlitzky from comment #18)
> ago, is there a list of the variables that you want to override and their
> documented purpose? It would help if we had something to show to upstream
> and say "you shouldn't be using this variable this way."

For how the build system works they look for variables. You can see a list of variables in toolchain-funcs.eclass.

Usually in common cases at link time $CC/$CXX is used to link and in rare cases it is used $LD.
So in short people in build system should use CC/CXX unless they know they need $LD

See also as example:
https://stackoverflow.com/questions/21352487/how-to-configure-a-non-standard-linker-for-an-autotooled-build
Comment 20 Michael Orlitzky gentoo-dev 2020-06-26 16:53:32 UTC
(In reply to Agostino Sarubbo from comment #19)
> 
> For how the build system works they look for variables. You can see a list
> of variables in toolchain-funcs.eclass.
> 
> Usually in common cases at link time $CC/$CXX is used to link and in rare
> cases it is used $LD.
> So in short people in build system should use CC/CXX unless they know they
> need $LD

I this case I think we should set LD and DLLD to $CC, but that only addresses half of the problem. You also want your LD environment variable (containing the name of your linker) to be respected, right?

What's unclear to me is why you (and toolchain-funcs.eclass) expect the LD environment variable to refer to the linker. That doesn't appear to be standard anywhere, and so a priori, upstreams are free to use $LD for whatever they want. More importantly, Gentoo users are free to use $LD for whatever they want. If that's the case, then it's hard to always treat $LD as the name of a linker and ensure that ebuilds respect it.

In the list you gave us in comment 13, all of those variables are standard *except* LD and GCOV. So I  understand why you'd expect the others to work. But unless we document somewhere that LD should be the name of a linker in Gentoo, I think you're unjustified (or maybe just optimistic :) ) in expecting ebuilds to respect it.
Comment 21 François Bissey 2020-06-27 08:41:31 UTC
Compiler, binder, linker and assembler, there is a long history down there. I am probably the only one in this conversation with exotic system experience where actually compiler and linker do behave differently and you'll get bitten for assuming GNU style. I am very happy to have decommissioned all my AIX systems :)
HP-UX, solaris, Tru64, those were easier because they were system V while AIX is system IV.

Anyway, the important bit to recognise is that some of those programs increasingly disappear, are made redundant or are getting hidden away. For example the IBM XL compiler on AIX spitted assembly directly making "as" redundant. In fact the AIX engineers only knew about the bugs in "as" because of the report from the people building gcc on AIX.

My point is your compiler is not completely what you think it is. It is literally a collection of subprograms and gcc/g++/gfortran/clang/clang++ are not compilers, they are compiler drivers. That's why we use them as linkers. They are not linkers but they can drive one. It would be very acceptable for a compiler to provide its own linker at this stage. So, direct use of the linker is probably going to become more and more hidden. Yes, it is there but fewer and fewer projects will be making use of it directly. POSIX conventions probably means there will be one for some time, but it could get absorbed by the "compiler".

In the case of pari, LD and DLLD are purely developer tools. They are there so you can experiment with exotic options and possibly in the past, deal with exotic systems like AIX (at the time I retired AIX, pari was segfaulting in interesting ways on it). I don't think there is much to say to pari upstream. Their use is valid but should not be used by normal every day users. We should just configure with LD="" to be on the safe side.
Comment 22 François Bissey 2020-06-27 08:56:24 UTC
PR sent. I added some comments inside the ebuild as well as setting LD="" at configuration time. I checked that it worked with LD="x86_64-pc-linux-gnu-ld" in the environment.
Comment 23 Larry the Git Cow gentoo-dev 2020-06-28 11:46:00 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=623d4ab26471435828adb197d675568b9212af47

commit 623d4ab26471435828adb197d675568b9212af47
Author:     François Bissey <frp.bissey@gmail.com>
AuthorDate: 2020-06-27 08:51:47 +0000
Commit:     Michael Orlitzky <mjo@gentoo.org>
CommitDate: 2020-06-28 11:44:09 +0000

    sci-mathematics/pari: unset LD at configuration time
    
    Package-Manager: Portage-2.3.99, Repoman-2.3.23
    Closes: https://bugs.gentoo.org/722090
    Closes: https://github.com/gentoo/gentoo/pull/16445
    Signed-off-by: François René Pierre Bissey <frp.bissey@gmail.com>
    Signed-off-by: Michael Orlitzky <mjo@gentoo.org>

 sci-mathematics/pari/pari-2.11.4.ebuild | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Comment 24 Michael Orlitzky gentoo-dev 2020-06-28 12:02:35 UTC
FWIW I was able to track down a reference for the LD variable. It's documented for GNU libtool:

  https://www.gnu.org/software/libtool/manual/libtool.html

I will post to the -dev list asking if there's any objection to documenting all of these in the devmanual, in one place.