So, this is a bit of an odd use-case, but I get the feeling I may not be the only person looking into this. The theory on gentoo prefix/rap is that it is more or less identical to using gentoo, no matter what the host os/distro is, right? Well, I've been putting it through the hoops, and have found sys-devel/crossdev behaves oddly under prefix. If you install crossdev into a prefixed gentoo, and emerge a toolchain, it works fine up until that point; however, if you actually install a package (sys-libs/zlib in my example here), the EPREFIX variable is used twice in the install path. For a prefix at /home/hanetzer/gentoo, installing sys-libs/zlib with the x86_64-w64-mingw32-emerge wrapper results the following list of files, relative to the real root of the system. /home/hanetzer/gentoo/usr/x86_64-w64-mingw32/home/hanetzer/gentoo/usr /home/hanetzer/gentoo/usr/x86_64-w64-mingw32/home/hanetzer/gentoo/usr/bin /home/hanetzer/gentoo/usr/x86_64-w64-mingw32/home/hanetzer/gentoo/usr/bin/zlib1.dll /home/hanetzer/gentoo/usr/x86_64-w64-mingw32/home/hanetzer/gentoo/usr/include /home/hanetzer/gentoo/usr/x86_64-w64-mingw32/home/hanetzer/gentoo/usr/include/zconf.h /home/hanetzer/gentoo/usr/x86_64-w64-mingw32/home/hanetzer/gentoo/usr/include/zlib.h /home/hanetzer/gentoo/usr/x86_64-w64-mingw32/home/hanetzer/gentoo/usr/lib /home/hanetzer/gentoo/usr/x86_64-w64-mingw32/home/hanetzer/gentoo/usr/lib/libz.dll.a /home/hanetzer/gentoo/usr/x86_64-w64-mingw32/home/hanetzer/gentoo/usr/lib/pkgconfig /home/hanetzer/gentoo/usr/x86_64-w64-mingw32/home/hanetzer/gentoo/usr/lib/pkgconfig/zlib.pc And the normal portage sync and config files /var/{cache,db,lib,tmp} end up at /home/hanetzer/gentoo/usr/x86_64-w64-mingw32/home/hanetzer/gentoo/var
Created attachment 511938 [details] x86_64-w64-mingw32-emerge with bash debugging turned on.
Created attachment 511940 [details] x86_64-w64-mingw32-emerge --info
Created attachment 511942 [details] prefix's emerge --info
Also, this mingw-w64 toolchain w/crossdev under prefix is dependent on https://github.com/gentoo/gentoo/pull/6675, but I can confirm that when a mingw-w64 crossdev toolchain is created outside of a prefix with my changes to dev-util/mingw64-runtime sys-libs/zlib is emerged properly, with the following file list relative to /: /usr/x86_64-w64-mingw32/usr /usr/x86_64-w64-mingw32/usr/bin /usr/x86_64-w64-mingw32/usr/bin/zlib1.dll /usr/x86_64-w64-mingw32/usr/include /usr/x86_64-w64-mingw32/usr/include/zconf.h /usr/x86_64-w64-mingw32/usr/include/zlib.h /usr/x86_64-w64-mingw32/usr/lib64 /usr/x86_64-w64-mingw32/usr/lib64/libz.dll.a /usr/x86_64-w64-mingw32/usr/lib64/pkgconfig /usr/x86_64-w64-mingw32/usr/lib64/pkgconfig/zlib.pc The path duplication occurs somewhere in the interaction between prefix and cross-emerge.
Same problem with a musl crossdev toolchain under gentoo prefix; the file list is as follows: /home/hanetzer/gentoo/usr/x86_64-gentoo-linux-musl/home/hanetzer/gentoo/lib /home/hanetzer/gentoo/usr/x86_64-gentoo-linux-musl/home/hanetzer/gentoo/lib/libz.so.1 -> libz.so.1.2.11 /home/hanetzer/gentoo/usr/x86_64-gentoo-linux-musl/home/hanetzer/gentoo/lib/libz.so.1.2.11 /home/hanetzer/gentoo/usr/x86_64-gentoo-linux-musl/home/hanetzer/gentoo/usr /home/hanetzer/gentoo/usr/x86_64-gentoo-linux-musl/home/hanetzer/gentoo/usr/include /home/hanetzer/gentoo/usr/x86_64-gentoo-linux-musl/home/hanetzer/gentoo/usr/include/zconf.h /home/hanetzer/gentoo/usr/x86_64-gentoo-linux-musl/home/hanetzer/gentoo/usr/include/zlib.h /home/hanetzer/gentoo/usr/x86_64-gentoo-linux-musl/home/hanetzer/gentoo/usr/lib /home/hanetzer/gentoo/usr/x86_64-gentoo-linux-musl/home/hanetzer/gentoo/usr/lib/libz.so /home/hanetzer/gentoo/usr/x86_64-gentoo-linux-musl/home/hanetzer/gentoo/usr/lib/pkgconfig /home/hanetzer/gentoo/usr/x86_64-gentoo-linux-musl/home/hanetzer/gentoo/usr/lib/pkgconfig/zlib.pc
Had a look at it a few days ago. The issue boils down to one major issue: toolchain ebuilds do not distinct between host's EPREFIX and target's EPREFIX and use (or don't) ${EPREFIX} in both contexts. A simple example: host system is prefix (EPREFIX=/foo), target chroot is unprefixed (EPREFIX=). toolchain.eclass is assuming this setup as valid: PREFIX=${TOOLCHAIN_PREFIX:-${EPREFIX}/usr} econf ... --prefix="${PREFIX}" ... # ok econf ... --with-sysroot="${PREFIX}"/${CTARGET} but not the toolchain-glibc.eclass: econf ... --prefix="${EPREFIX}/usr" # ths is not correct, should be --prefix="/usr" to cross-libc case There is a few things we can do here: 1. clean up ebuilds to assume --prefix="/usr" when installed through cross-* ebuilds. it's needed only for: - cross-* libc packages - cross-* headers packages 2. extend ebuilds to accept --prefix="${TARGET_EPREFIX}/usr" to allow cross-compiling from one EPREFIX system to another. It's a very minor extension. it's needed only for: - cross-* libc packages - cross-* headers packages - cross-* gcc packages 3. convert crossdev to not use cross-* libc and cross-* headers packages at all and use cross-emerge instead. changes will be needed for: - cross-* gcc packages (they rely on probing of cross-{libs,headers}) Not all of them are required to fix this bug but I think are nice cleanups. All of the changes needed are changes in toolchain@ packages.
(In reply to Sergei Trofimovich from comment #6) > Had a look at it a few days ago. The issue boils down to one major issue: > toolchain ebuilds do not distinct between host's EPREFIX and target's > EPREFIX and use (or don't) ${EPREFIX} in both contexts. > > A simple example: > > host system is prefix (EPREFIX=/foo), target chroot is unprefixed (EPREFIX=). > > toolchain.eclass is assuming this setup as valid: > > PREFIX=${TOOLCHAIN_PREFIX:-${EPREFIX}/usr} > econf ... --prefix="${PREFIX}" ... # ok > > econf ... --with-sysroot="${PREFIX}"/${CTARGET} > > but not the toolchain-glibc.eclass: > > econf ... --prefix="${EPREFIX}/usr" # ths is not correct, should be > --prefix="/usr" to cross-libc case > > There is a few things we can do here: > 1. clean up ebuilds to assume --prefix="/usr" when installed through cross-* > ebuilds. > it's needed only for: > - cross-* libc packages > - cross-* headers packages This is the easiest and over-simplified way. > 2. extend ebuilds to accept --prefix="${TARGET_EPREFIX}/usr" to allow > cross-compiling from one EPREFIX system to another. It's a very minor > extension. > it's needed only for: > - cross-* libc packages > - cross-* headers packages > - cross-* gcc packages We had this discussion on #gentoo-prefix and at that time the variable has been decided to be "TPREFIX". > 3. convert crossdev to not use cross-* libc and cross-* headers packages at > all and use cross-emerge instead. > > changes will be needed for: > - cross-* gcc packages (they rely on probing of cross-{libs,headers}) > > Not all of them are required to fix this bug but I think are nice cleanups. > All of the changes needed are changes in toolchain@ packages. I liked this approach best. But the last time I mixed cross-emerge glibc and cross-*/gcc, it is then hard to express dependency in different PORTAGE_CONFIGROOT. We can take one step further and have cross-emerge take care of the cross gcc as well.
> I liked this approach best. But the last time I mixed cross-emerge glibc and > cross-*/gcc, it is then hard to express dependency in different > PORTAGE_CONFIGROOT. > We can take one step further and have cross-emerge take care of the cross gcc > as well. While I also think the last option is best, I would suggest against having cross-emerge do the cross-*/gcc as well. Reason being, you need an actual cross compiler on the host in order to cross-emerge libc and others, and if the cross compiler is installed into the cross-root, it won't be ideal, as it would be the, for example, only x86_64 binaries in an otherwise arm64 root. We should emerge cross-*/gcc, and cross-emerge libc & headers, and later on you can cross-emerge a native (to the cross-target) sys-devel/gcc.
(In reply to hanetzer from comment #8) > > I liked this approach best. But the last time I mixed cross-emerge glibc and > > cross-*/gcc, it is then hard to express dependency in different > > PORTAGE_CONFIGROOT. > > > We can take one step further and have cross-emerge take care of the cross gcc > > as well. > > While I also think the last option is best, I would suggest against having > cross-emerge do the cross-*/gcc as well. Reason being, you need an actual > cross compiler on the host in order to cross-emerge libc and others, and if > the cross compiler is installed into the cross-root, it won't be ideal, as > it would be the, for example, only x86_64 binaries in an otherwise arm64 > root. > > We should emerge cross-*/gcc, and cross-emerge libc & headers, and later on > you can cross-emerge a native (to the cross-target) sys-devel/gcc. It is hard to express dependency in different PORTAGE_CONFIGROOT. I don't think portage supports this for the moment.
EAPI=7 could have given us better options to move forward.
(In reply to Benda Xu from comment #10) > EAPI=7 could have given us better options to move forward. Agreed, but that's not going to hit any of the low-level base-system packages for quite some time if we use the usual metric of adoption of a new eapi into them, which is a pity, as its the packages in that set/project/whatever are the ones which would benefit most from it.
(In reply to hanetzer from comment #11) > (In reply to Benda Xu from comment #10) > > EAPI=7 could have given us better options to move forward. > > Agreed, but that's not going to hit any of the low-level base-system > packages for quite some time if we use the usual metric of adoption > of a new eapi into them, which is a pity, as its the packages in that > set/project/whatever are the ones which would benefit most from it. If we can prove that EAPI=7 toolchain can help a lot on this issue, it will be a good motivation for the toolchain team to accept our patch to bump to EAPI=7.
Minor comment on: > > 2. extend ebuilds to accept --prefix="${TARGET_EPREFIX}/usr" to allow > > cross-compiling from one EPREFIX system to another. It's a very minor > > extension. > > it's needed only for: > > - cross-* libc packages > > - cross-* headers packages > > - cross-* gcc packages > > We had this discussion on #gentoo-prefix and at that time the variable has > been decided to be "TPREFIX". EAPI=7 apparently defines EPREFIX as target prefix and host prefix is $BROOT: https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-670008 Not sure if it's enough for toolchain ebuilds. Maybe.
A few thoughts even if I don't use crossdev myself: Especially when talking about cross compiling, using the words "build", "host" and "target" requires special care to reduce confusion. For least confusion IMHO we should stick to toolchain's CBUILD, CHOST and CTARGET meanings. (In reply to Sergei Trofimovich from comment #13) > EAPI=7 apparently defines EPREFIX as target prefix and host prefix is $BROOT: > https://dev.gentoo.org/~ulm/pms/head/pms.html#x1-670008 EAPI 7 defines BROOT as the prefix for the "CBUILD" system, and EPREFIX as the prefix for the "CHOST" system. > Not sure if it's enough for toolchain ebuilds. Maybe. During emerge of cross-* packages, I doubt they already should know anything about the prefix (or root) value used for any "CTARGET" system they subsequently shall be used to compile for. Instead, the prefix value for what the cross-toolchain calls "CTARGET" system will be available no earlier than during cross-emerge of sys-libs/zlib for example. Then, the CTARGET's prefix value is in ${EPREFIX}, because zlib's "CHOST" system now is equal to cross-toolchain's "CTARGET" system. And for completion, zlib's "CBUILD" system is equal to cross-toolchain's "CHOST" system, and cross-toolchain can find it's own prefix value in ${BROOT} now.
You're all very confused indeed! This has nothing to do with how the cross-* packages are built or installed or anything to do with the toolchain at all. The only thing wrong with crossdev is that cross-emerge doesn't set EPREFIX to be blank when it hasn't otherwise been set. Portage therefore defaults to the build system prefix. If you look at cross-emerge, you'll see that it does explicitly hardcode EPREFIX to match the build system prefix but it doesn't export the variable, it's only used to set SYSROOT. We should rename this variable to something else to allow the exported EPREFIX to be handled differently. That's not the end of the story though because there's another issue in Portage itself. Even if crossdev did the right thing, you'd still get the very same result. Watch what happens if you build natively while simply changing ROOT. > % ROOT=/home/chewi/myroot2 emerge -p sys-libs/zlib > [ebuild N] sys-libs/zlib-1.2.11-r2 to /home/chewi/myroot2/home/chewi/prefix/ In this case, EPREFIX defaults to the build system prefix as it does with crossdev now. Perhaps that's not unreasonable in a native context. So let's set EPREFIX to be blank. > % EPREFIX= ROOT=/home/chewi/myroot2 emerge -p sys-libs/zlib > [ebuild N] sys-libs/zlib-1.2.11-r2 to /home/chewi/myroot2/home/chewi/prefix/ Oh. Portage is treating blank the same as unset. That's a bug. I've tried to find the offending line but no luck yet. We can at least set it to something else. > % EPREFIX=/foo ROOT=/home/chewi/myroot2 emerge -p sys-libs/zlib > [ebuild N] sys-libs/zlib-1.2.11-r2 to /home/chewi/myroot2/foo/ That leads me to a workaround. We can cheat by setting EPREFIX to /. > % EPREFIX=/ ROOT=/home/chewi/myroot2 emerge -p sys-libs/zlib > [ebuild N] sys-libs/zlib-1.2.11-r2 to /home/chewi/myroot2/ Victory! We could have crossdev do this but we really should fix Portage instead.
I found the offending line. Here's the Portage fix: https://github.com/gentoo/portage/pull/434
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/proj/portage.git/commit/?id=cfc3ca6647235d8a3fbe719b9d447863a75deeb6 commit cfc3ca6647235d8a3fbe719b9d447863a75deeb6 Author: James Le Cuirot <chewi@gentoo.org> AuthorDate: 2019-07-07 13:18:54 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2019-07-08 06:37:15 +0000 emerge: Don't treat empty EPREFIX or PORTAGE_CONFIGROOT as unset If a prefix user wanted to build within a ROOT but without a prefix, they previously had to set EPREFIX=/ rather than EPREFIX= as the latter was simply treated as unset. Also applies to ROOT and SYSROOT but probably makes no difference to these as they are blank by default. This should be safe to do as all these variables get normalised anyway. Bug: https://bugs.gentoo.org/642604 Closes: https://github.com/gentoo/portage/pull/434 Signed-off-by: James Le Cuirot <chewi@gentoo.org> Signed-off-by: Zac Medico <zmedico@gentoo.org> lib/_emerge/actions.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=909c967e7480e2477e40172bab5817b31ea200f0 commit 909c967e7480e2477e40172bab5817b31ea200f0 Author: Zac Medico <zmedico@gentoo.org> AuthorDate: 2019-07-11 03:45:08 +0000 Commit: Zac Medico <zmedico@gentoo.org> CommitDate: 2019-07-11 04:03:07 +0000 sys-apps/portage: Bump to version 2.3.69 #642604 handle empty EPREFIX, ROOT, SYSROOT, etc settings #689072 default repo.conf sync-openpgp-keyserver to hkps://keys.gentoo.org in order to prevent key poisoning #689506 default repos.conf sync-webrsync-verify-signature for USE=rsync-verify Bug: https://bugs.gentoo.org/642604 Bug: https://bugs.gentoo.org/683434 Bug: https://bugs.gentoo.org/689072 Bug: https://bugs.gentoo.org/689506 Package-Manager: Portage-2.3.69, Repoman-2.3.16 Signed-off-by: Zac Medico <zmedico@gentoo.org> sys-apps/portage/Manifest | 1 + sys-apps/portage/portage-2.3.69.ebuild | 260 +++++++++++++++++++++++++++++++++ 2 files changed, 261 insertions(+)
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ea4cbdc9159c0ebbd29d4062bbb314393a8cc32f commit ea4cbdc9159c0ebbd29d4062bbb314393a8cc32f Author: James Le Cuirot <chewi@gentoo.org> AuthorDate: 2019-07-12 21:13:47 +0000 Commit: James Le Cuirot <chewi@gentoo.org> CommitDate: 2019-07-21 13:30:54 +0000 sys-libs/glibc: Fix handling of ${EPREFIX} when building cross-glibc It was duplicating the prefix in the form ${EPREFIX}/usr/${CTARGET}/${EPREFIX}. This also fixes the kernel header version check, which was broken for native prefixed builds. Bug: https://bugs.gentoo.org/642604 Closes: https://github.com/gentoo/gentoo/pull/12435 Package-Manager: Portage-2.3.69, Repoman-2.3.13 Signed-off-by: James Le Cuirot <chewi@gentoo.org> sys-libs/glibc/glibc-2.29-r2.ebuild | 32 ++++++++++++++++++++++---------- sys-libs/glibc/glibc-9999.ebuild | 32 ++++++++++++++++++++++---------- 2 files changed, 44 insertions(+), 20 deletions(-)
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/proj/crossdev.git/commit/?id=cd2aa636cc5d0476bff29d50fbaab7591b48903e commit cd2aa636cc5d0476bff29d50fbaab7591b48903e Author: James Le Cuirot <chewi@gentoo.org> AuthorDate: 2019-07-09 20:58:41 +0000 Commit: James Le Cuirot <chewi@gentoo.org> CommitDate: 2019-07-21 13:35:32 +0000 cross-emerge: Default to using no prefix On non-prefixed systems, cross-emerge installs to /usr/${CHOST} by default. On prefixed systems, this default effectively becomes ${BROOT}/usr/${CHOST}/${BROOT}, which is unexpected and makes little sense. The first BROOT originates from the ROOT setting in the cross make.conf. The second BROOT is the prefix that Portage is configured to use by default. We therefore need to avoid the second BROOT by overriding Portage with a blank EPREFIX value. Note that a bug in Portage itself means that this is ineffective on versions before 2.3.69 but it's no worse than it was before either. For users who do want to set their own EPREFIX, the PORTAGE_CONFIGROOT default has been updated to ${SYSROOT}${EPREFIX} as the prefixed location is required for this variable. This is despite man emerge suggesting otherwise! Closes: https://bugs.gentoo.org/642604 Signed-off-by: James Le Cuirot <chewi@gentoo.org> wrappers/cross-emerge | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
Created attachment 600584 [details, diff] glibc-prefix-crossdev.patch Unfortunately commit ea4cbdc9159c0ebbd29d4062bbb314393a8cc32f has introduced a regression that cross-*/glibc can no longer be built on Prefix. Please find the patch to fix it. commit 9ac94ffb4efcb3d6f6b5e6d947e4b6fcab5bf154 (HEAD -> crossdev) Author: Benda Xu <heroxbd@gentoo.org> Date: Wed Dec 25 17:28:55 2019 +0800 sys-libs/glibc: cross-*: pass EPREFIX to --with-headers This is a follow up of ea4cbdc9159c0ebbd29d4062bbb314393a8cc32f. Otherwise when building cross toolchain on Gentoo Prefix, configure cannot find the needed linux-headers. Package-Manager: Portage-2.3.79, Repoman-2.3.18 Signed-off-by: Benda Xu <heroxbd@gentoo.org> This only affects the Prefix systems, and I intend to commit if no objections appear in a week.
(In reply to Benda Xu from comment #21) > Created attachment 600584 [details, diff] [details, diff] > glibc-prefix-crossdev.patch > > Unfortunately commit ea4cbdc9159c0ebbd29d4062bbb314393a8cc32f has introduced > a regression that cross-*/glibc can no longer be built on Prefix. > > Please find the patch to fix it. > > commit 9ac94ffb4efcb3d6f6b5e6d947e4b6fcab5bf154 (HEAD -> crossdev) > > Author: Benda Xu <heroxbd@gentoo.org> > Date: Wed Dec 25 17:28:55 2019 +0800 > > sys-libs/glibc: cross-*: pass EPREFIX to --with-headers > > > This is a follow up of ea4cbdc9159c0ebbd29d4062bbb314393a8cc32f. > > > Otherwise when building cross toolchain on Gentoo Prefix, configure > cannot > find the needed linux-headers. > > Package-Manager: Portage-2.3.79, Repoman-2.3.18 > Signed-off-by: Benda Xu <heroxbd@gentoo.org> > > > This only affects the Prefix systems, and I intend to commit if no > objections appear in a week. Feel free to push the change. And please keep the bug open. I'd like to address #comment6/[3.] as part of it.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=7c1ace82e82ecebd13ac180bb0d1e6d82258e99b commit 7c1ace82e82ecebd13ac180bb0d1e6d82258e99b Author: Benda Xu <heroxbd@gentoo.org> AuthorDate: 2019-12-25 09:28:55 +0000 Commit: Benda Xu <heroxbd@gentoo.org> CommitDate: 2019-12-25 10:57:58 +0000 sys-libs/glibc: cross-*: pass EPREFIX to --with-headers This is a follow up of ea4cbdc9159c0ebbd29d4062bbb314393a8cc32f. Otherwise when building cross toolchain on Gentoo Prefix, configure cannot find the needed linux-headers. Bug: https://bugs.gentoo.org/642604 Package-Manager: Portage-2.3.79, Repoman-2.3.18 Signed-off-by: Benda Xu <heroxbd@gentoo.org> sys-libs/glibc/glibc-2.30-r3.ebuild | 4 ++-- sys-libs/glibc/glibc-9999.ebuild | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-)
is there still things we'd like to do for this bug?
Given the more recent work I did with prefix on the toolchain and the testing I did around that, I think we're good here.
thanks a lot!
