Created attachment 592474 [details, diff] proposed patch to omit EPREFIX from --with-sysroot Since EAPI 7, econf passes --prefix=$EPREFIX --with-sysroot=${ESYSROOT}. Now libtool does search for libraries in $with_sysroot$prefix, and then replaces $with_sysroot by '=' in dependency_libs in the .la files, ending up with dependency_libs="-L=/usr/lib" even if EPREFIX was not empty. This may cause all sort of strange subsequent linker errors like this one: /prefix/base/usr/i686-pc-linux-gnu/binutils-bin/2.32/ld: /prefix/stack/usr/lib/libicui18n.so: undefined reference to `__divmoddi4@GCC_7.0.0' Reason here was that -L=/usr/lib was on the linkline, coming from some .la file independent of libicui18n, finding /usr/lib/libgcc_s.so on the RHEL 7.6 host system, which does have gcc version 4.8.5 still. Portage patch attached. Thoughts?
In PMS it says to use --with-sysroot ${ESYSROOT:-/} in the "12.3.8 Build commands" section: https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-12800012.3.8
Hijacking this bug for PMS. :) Chewi has been working on an updated wording for PMS that should address this issue. I would suggest that we update the spec first, before working on the implementation.
Created attachment 592560 [details, diff] [PMS] Correct the definition of ESYSROOT as EPREFIX isn't always applicable I believe this was the last version of the proposed PMS patch that was circulated. (@Chewi: Please correct me if there's anything newer.)
I was trying to be mindful of prefix when I did this but looks like I got it wrong. I probably got confused by the toolchain's own --sysroot option, which doesn't have the concept of a prefix. Strange that I didn't hit this problem when I played with prefix later. Your patch looks good. Although it does mention prefix and ESYSROOT, the patch that ulm has attached is not related to this issue.
Wait a second, I think I've misunderstood the problem. I didn't expect --with-sysroot to have any affect on the files produced so that's surprising. Where do you expect the -L flag to point though? Are you expecting it to not be there at all?
In a sense, the fact that it adds = is a good thing. pkg-config does a similar thing using PKG_CONFIG_SYSROOT_DIR but returns the fully adjusted paths rather than relying on the toolchain to expand =. One fairly simple package that writes such .la files is dev-libs/userspace-rcu so I just built that in my prefix. I ended up with: dependency_libs=' -L=/home/chewi/prefix/usr/lib64 -lurcu-common' I believe this is correct. Which .la file had the -L=/usr/lib entry?
(In reply to James Le Cuirot from comment #6) > In a sense, the fact that it adds = is a good thing. Erm, Prefix != Cross: In Prefix there is no such thing like cross compiling (although crossdev may work on top of Prefix), and Prefix does not need SYSROOT: All the build systems do know about /eprefix/usr/lib because of the --prefix=/eprefix/usr configure flag. Prefix is like /usr/local or even /opt, but with portage as the package manager rather than human: Did you ever need any sysroot in --prefix=/usr/local? So any -L= is wrong here (even if that may not break immediately), because all the binaries do natively run on the host kernel. As far as I understand the ld(1) man page, the --with-sysroot configure flag (for libtool) causing the -L= linker flag is designed to be used together with the --sysroot linker flag, and there is no need for any of them in Prefix. > dependency_libs=' -L=/home/chewi/prefix/usr/lib64 -lurcu-common' > > Which .la file had the -L=/usr/lib entry? You need another .la file in another EAPI=7 package depending on this .la file. And while I haven't traced all the libtool scripts in deep, I have also seen dependency_libs=' =/eprefix/usr/lib/libsome.la', which is wrong as well.
(In reply to Michael Haubenwallner from comment #7) > > Erm, Prefix != Cross: In Prefix there is no such thing like cross compiling > (although crossdev may work on top of Prefix), and Prefix does not need > SYSROOT: All the build systems do know about /eprefix/usr/lib because of the > --prefix=/eprefix/usr configure flag. > > Prefix is like /usr/local or even /opt, but with portage as the package > manager rather than human: Did you ever need any sysroot in > --prefix=/usr/local? Steady on, I'm not as familiar with prefix as I'd like but I do know what it is and roughly how it works. crossdev under prefix somewhat works but only because I fixed glibc. :P cross-pkg-config is still broken but I have pending changes for that. > So any -L= is wrong here (even if that may not break immediately), because > all the binaries do natively run on the host kernel. Think of the bigger picture. It is valid do this: ROOT=/foo SYSROOT=/foo EPREFIX=/bar emerge curl That can work from a prefixed or non-prefixed system. Obviously there's no cross-compiling going on here. There could be but it doesn't matter either way. This is saying that we want to build/link against dependencies installed within /foo. .la files within /foo should have paths like /bar/usr/lib rather than /foo/bar/usr/lib because /foo isn't applicable at runtime. How does the toolchain find these files then? It needs the = in -L=/bar/usr/lib. Now that won't work on its own as you need to invoke gcc with --sysroot=/foo. I've found this works best by including the flag in CC but for now, we leave that to the user. > You need another .la file in another EAPI=7 package depending on this .la > file. > And while I haven't traced all the libtool scripts in deep, I have also seen > dependency_libs=' =/eprefix/usr/lib/libsome.la', which is wrong as well. Could you please grep your .la files to find out which one it was? I'd like to reproduce this. I agree that -L=/usr/lib is wrong and I understand why that would break things but -L=/eprefix/usr/lib is okay. On a normal prefixed system when you're not compiling or otherwise setting ROOT, SYSROOT will effectively be / so it will still look in the right place. I can't find any examples of dependency_libs referencing another .la file on my system. Which package was that? I can't find much information about dependency_libs in general but if libtool put that there then I imagine it knows how to expand = when it appears like this.
(In reply to James Le Cuirot from comment #8) > > Steady on, Sorry, didn't want to be abrasive. > > So any -L= is wrong here (even if that may not break immediately), because > > all the binaries do natively run on the host kernel. > > Think of the bigger picture. It is valid do this: > > ROOT=/foo SYSROOT=/foo EPREFIX=/bar emerge curl > > That can work from a prefixed or non-prefixed system. Obviously there's no > cross-compiling going on here. Indeed. I stand corrected. > > You need another .la file in another EAPI=7 package depending on this .la > > file. > Which package was that? This was with some local package not available to the public. However, the relevant difference to lots of Gentoo ebuilds likely is that it does not prune libtool files.
Created attachment 592862 [details, diff] proposed patch-r1 to omit EPREFIX from --with-sysroot Wouldn't libtool (and others) need to support distinct prefix values as well when we want to use distinct EPREFIX values with SYSROOT and ROOT? Anyway, the portage patch can be simplified a little.
(In reply to Michael Haubenwallner from comment #10) > Created attachment 592862 [details, diff] [details, diff] > proposed patch-r1 to omit EPREFIX from --with-sysroot SYSROOT has the trailing slash removed by Portage in all EAPIs because its presence has caused issues. This isn't stated in PMS although I could add it. At the same time, setting --with-sysroot= rather than --with-sysroot=/ means the = doesn't get inserted into -L flags in .la files, losing the benefit I stated above. We've had it this long and no one has raised an issue until now. Your issue seems to stem from the wrong path being used and/or the package itself doing something wrong. I would therefore prefer it to be: --with-sysroot="${SYSROOT:-/}" > Wouldn't libtool (and others) need to support distinct prefix values as well > when we want to use distinct EPREFIX values with SYSROOT and ROOT? No? The concept of a distinct EPREFIX betweeen SYSROOT and ROOT is still in the process of being clarified in PMS (what ulm mentioned earlier) but my intention is to support it, at least to some degree. When a package is merged to SYSROOT, it will be located at SYSROOT's prefix and .la files will also have SYSROOT's prefix. Similarly, when a package is merged to ROOT, it will be located at ROOT's prefix and .la files will also have ROOT's prefix. When you're building packages for ROOT, it should not use any files, .la or otherwise, located within ROOT unless SYSROOT happens to be the same. So .la files found within SYSROOT will reference other files within SYSROOT and that rule is maintained. If you're later running inside the ROOT system and using it to build its own packages, then SYSROOT will be / (same as ROOT) and ROOT's prefix will apply. The old SYSROOT will have no relevance at all.
(In reply to James Le Cuirot from comment #11) > (In reply to Michael Haubenwallner from comment #10) > > Created attachment 592862 [details, diff] [details, diff] [details, diff] > > proposed patch-r1 to omit EPREFIX from --with-sysroot > > SYSROOT has the trailing slash removed by Portage in all EAPIs because its > presence has caused issues. This isn't stated in PMS although I could add it. > > At the same time, setting --with-sysroot= rather than --with-sysroot=/ means > the = doesn't get inserted into -L flags in .la files, losing the benefit I > stated above. We've had it this long and no one has raised an issue until > now. Your issue seems to stem from the wrong path being used and/or the > package itself doing something wrong. I would therefore prefer it to be: > > --with-sysroot="${SYSROOT:-/}" It didn't break yet because we remove most .la files in Gentoo, but leaving .la files alone still should be a valid use case (like when static linking is required). And the single "/" will break Cygwin, which will end up with "${sysroot}${libdir}" being interpreted as "//hostname/path". It didn't break yet because noone uses the empty EPREFIX on Cygwin, AFAICT. But it might work to use "/." instead: --with-sysroot="${SYSROOT:-/.}" > > Wouldn't libtool (and others) need to support distinct prefix values as well > > when we want to use distinct EPREFIX values with SYSROOT and ROOT? > > No? > > The concept of a distinct EPREFIX betweeen SYSROOT and ROOT is still in the > process of being clarified in PMS (what ulm mentioned earlier) but my > intention is to support it, at least to some degree. > > When a package is merged to SYSROOT, it will be located at SYSROOT's prefix > and .la files will also have SYSROOT's prefix. Similarly, when a package is > merged to ROOT, it will be located at ROOT's prefix and .la files will also > have ROOT's prefix. > > When you're building packages for ROOT, it should not use any files, .la or > otherwise, located within ROOT unless SYSROOT happens to be the same. So .la > files found within SYSROOT will reference other files within SYSROOT and > that rule is maintained. I'm not deep enough into sysroot usage, so maybe it could work, but: Isn't a libdir (and similar) value found in SYSROOT potentially encoded as RUNPATH (and similar) into the binaries installed into ROOT, where it would be wrong then? With "and similar" I'm thinking of any configured (or detected) "path where to locate a dependent package" that is compiled in for runtime, but also used at build time already to locate the package.
(In reply to Michael Haubenwallner from comment #12) > (In reply to James Le Cuirot from comment #11) > > > --with-sysroot="${SYSROOT:-/}" > > And the single "/" will break Cygwin, which will end up with > "${sysroot}${libdir}" being interpreted as "//hostname/path". > It didn't break yet because noone uses the empty EPREFIX on Cygwin, AFAICT. > > But it might work to use "/." instead: --with-sysroot="${SYSROOT:-/.}" Hmm, do you mean a networked UNC path? That would indeed be a problem. I vaguely recall Cygwin but I'm not familiar with how Prefix applies there. I know that / isn't really C:\ or somewhere like that but I'm still not sure what you mean by the empty EPREFIX. You still need the standard Cygwin system to bootstrap from, right? I guess /. would work though and it doesn't end up in the .la files anyway. > I'm not deep enough into sysroot usage, so maybe it could work, but: > Isn't a libdir (and similar) value found in SYSROOT potentially encoded as > RUNPATH (and similar) into the binaries installed into ROOT, where it would > be wrong then? > With "and similar" I'm thinking of any configured (or detected) "path where > to locate a dependent package" that is compiled in for runtime, but also > used at build time already to locate the package. RUNPATH is only used at runtime and binaries are never copied from SYSROOT to ROOT. Static libraries from SYSROOT may be linked into ROOT binaries but they don't use RUNPATH. I just checked my pkg-config files and have found just 4 instances of -Wl,-rpath,/usr/lib64. This is potentially an issue but they all come from the family of libraries used by Samba. I would argue that this flag should be omitted if it is one of the default locations searched by the dynamic loader. At the same time, I don't claim that this is perfect. I'm also aware of some paths being recorded in headers though I think few of them are used in practise. The only distinct SYSROOT case I'm going out of my way to support is the bootstrapping of new systems. If you're cross-bootstrapping a brand new system into an empty directory, you need to get the kernel and libc headers from somewhere!
(In reply to James Le Cuirot from comment #13) > (In reply to Michael Haubenwallner from comment #12) > > (In reply to James Le Cuirot from comment #11) > > > > > --with-sysroot="${SYSROOT:-/}" > > > > And the single "/" will break Cygwin, which will end up with > > "${sysroot}${libdir}" being interpreted as "//hostname/path". > > It didn't break yet because noone uses the empty EPREFIX on Cygwin, AFAICT. > > > > But it might work to use "/." instead: --with-sysroot="${SYSROOT:-/.}" > > Hmm, do you mean a networked UNC path? That would indeed be a problem. Exactly. Actually, per some standard I don't remember a pointer to, ///path has to be equal to /path, but //path is allowed to be distinct from /path. > I vaguely recall Cygwin but I'm not familiar with how Prefix applies there. On a non-Linux kernel host (MacOS, Solaris, Cygwin, etc.), Prefix uses the host libc, which we call a "guest" Prefix. On a Linux kernel host only, by default we install glibc inside Prefix, which we call "standalone" Prefix or "RAP" (RAP Ain't Prefix) then. But this can be disabled - which is my use case. > I know that / isn't really C:\ or somewhere like that Think of Cygwin / like a chroot into the Cygwin install dir, e.g. C:\cygwin, with a host drive C:\ being mapped in as /cygdrive/c/. > but I'm still not sure what you mean by the empty EPREFIX. The empty EPREFIX actually is non-Prefix, as in plain Gentoo OS. If I understand correctly, EAPI/PMS doesn't distinguish Prefix from non-Prefix - there is just EPREFIX that may be empty. > You still need the standard Cygwin system to bootstrap from, right? Yes. But while it's not my goal, it should be possible per PMS to create a Gentoo Cygwin OS (empty EPREFIX), IMO. > I guess /. would work though and it doesn't end up in the .la files anyway. Not sure if it does work as expected, but at least it doesn't break here any more: I do see both -L/prefix/usr/lib and -L=/prefix/usr/lib in dependency_libs now, but maybe this is because not every package involved is EAPI 7 yet. > > > I'm not deep enough into sysroot usage, so maybe it could work, but: > > Isn't a libdir (and similar) value found in SYSROOT potentially encoded as > > RUNPATH (and similar) into the binaries installed into ROOT, where it would > > be wrong then? > > With "and similar" I'm thinking of any configured (or detected) "path where > > to locate a dependent package" that is compiled in for runtime, but also > > used at build time already to locate the package. > > RUNPATH is only used at runtime and binaries are never copied from SYSROOT > to ROOT. Static libraries from SYSROOT may be linked into ROOT binaries but > they don't use RUNPATH. For creating final binaries (for ROOT): Because of your "it should not use any files ... located within ROOT" in comment#11, do I understand correctly that even shared libaries and .la files should be found in SYSROOT? But then, libtool may encode the SYSROOT based /prefix/usr/lib from .la file's libdir as RUNPATH into the final binary, no? > I just checked my pkg-config files and have found just 4 instances of > -Wl,-rpath,/usr/lib64. This is potentially an issue but they all come from > the family of libraries used by Samba. I would argue that this flag should > be omitted if it is one of the default locations searched by the dynamic > loader. Agreed this should be omitted, but I don't think it's a real issue. > At the same time, I don't claim that this is perfect. I'm also aware of some > paths being recorded in headers though I think few of them are used in > practise. The only distinct SYSROOT case I'm going out of my way to support > is the bootstrapping of new systems. If you're cross-bootstrapping a brand > new system into an empty directory, you need to get the kernel and libc > headers from somewhere! Agreed, but this is about the empty EPREFIX, no? While I'm just unsure about additional settings (like CFLAGS=--sysroot=$SYSROOT) being required here, I still fail to imagine additional settings when there's a different EPREFIX used within SYSROOT compared to ROOT.
Has this been fixed by the following commit? https://gitweb.gentoo.org/proj/pms.git/commit/?id=039b450670c4b7c632c0c4b52f3f4dfed3e7ae15 Or are there any issues remaining?
Looking at this again following the recent prefix changes with a much stronger knowledge of prefix, I'm still struggling to wrap my head around it. I suspect it may still be a problem. I've still never hit it, but .la files are very rare now. A solid reproduction that I can try would help a lot.
