It appears that the USE flags passed through the USE env var are ignored: * Error: circular dependencies: (virtual/acl-0-r2:0/0::gentoo, ebuild scheduled for merge) depends on (sys-apps/acl-2.2.53:0/0::gentoo, ebuild scheduled for merge) (runtime) (sys-devel/gettext-0.19.8.1:0/0::gentoo, ebuild scheduled for merge) (buildtime) (virtual/acl-0-r2:0/0::gentoo, ebuild scheduled for merge) (buildtime) It might be possible to break this cycle by applying any of the following changes: - sys-devel/gettext-0.19.8.1 (Change USE: -acl) - sys-apps/acl-2.2.53 (Change USE: -nls) This hack enable me to start the merging process: @@ -1555,7 +1555,7 @@ # cmake has some external dependencies which require autoconf, etc. # unless we only build the buildtool, bug #603012 - echo "dev-util/cmake -server" >> "${ROOT}"/tmp/etc/portage/package.use + echo -e "dev-util/cmake -server\nsys-devel/gettext -acl\ndev-lang/perl -gdbm -berkdb\nsys-apps/help2man -nls\ndev-util/pkgconfig internal-glib" >> "${ROOT}"/tmp/etc/portage/package.use for pkg in ${compiler_stage1} ; do # <glibc-2.5 does not understand .gnu.hash, use A similar situation as been reported by other users on irc.
Created attachment 551778 [details] extract from build.log detailed informations (just tested with LATEST_TREE_YES=1). One can see that the env USE variable is being ignored.
This is a troubleshooting attempt. The prefix is to be installed in /scratch/huber/test_prefix. Stage 2 was successful. When using the stage2 portage file, the USE env variable is not always taken into account: the command USE="-acl -berkdb -fortran -gdbm -git -libcxx -nls -pcre-python bootstrap clang internal-glib ssl" PYTHONPATH=/scratch/huber/test_prefix/tmp/usr/lib/portage/pym /scratch/huber/test_prefix/tmp/usr/bin/emerge --root-deps -vp sys-apps/portage --prefix="/scratch/huber/test_prefix" ends up complaining about circular dependencies * Error: circular dependencies: (sys-devel/gettext-0.19.8.1:0/0::gentoo, ebuild scheduled for merge) depends on (virtual/acl-0-r2:0/0::gentoo, ebuild scheduled for merge) (buildtime) (sys-apps/acl-2.2.53:0/0::gentoo, ebuild scheduled for merge) (runtime) (sys-devel/gettext-0.19.8.1:0/0::gentoo, ebuild scheduled for merge) (buildtime) It might be possible to break this cycle by applying any of the following changes: - sys-apps/acl-2.2.53 (Change USE: -nls) - sys-devel/gettext-0.19.8.1 (Change USE: -acl) On the other hand, removing EPREFIX (or --prefix) make the circular dependencies disappear: USE="-acl -berkdb -fortran -gdbm -git -libcxx -nls -pcre-python bootstrap clang internal-glib ssl" PYTHONPATH=/scratch/huber/test_prefix/tmp/usr/lib/portage/pym /scratch/huber/test_prefix/tmp/usr/bin/emerge --root-deps -vp sys-apps/portage ends up finding a way to merge portage, with the useflags specified in USE taken into account. It appears there are some instances where the USE env variable is taken into account: $ USE="-acl -berkdb -fortran -gdbm -git -libcxx -nls -pcre-python bootstrap clang internal-glib ssl" PYTHONPATH=/scratch/huber/test_prefix/tmp/usr/lib/portage/pym emerge --root-deps -vp sys-apps/acl --prefix="/scratch/huber/test_prefix" These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild N ] sys-apps/attr-2.4.48-r3::gentoo USE="-debug -nls -static-libs" 457 KiB [ebuild N ] sys-apps/acl-2.2.53::gentoo USE="-nls -static-libs" 513 KiB
@heroxbd: do you have an idea about that --prefix argument to emerge?
(In reply to Fabian Groffen from comment #3) > @heroxbd: do you have an idea about that --prefix argument to emerge? It is equivalent to setting EPREFIX. I speculate this bug is a regression of portage.
(In reply to Benda Xu from comment #4) > (In reply to Fabian Groffen from comment #3) > > @heroxbd: do you have an idea about that --prefix argument to emerge? > > It is equivalent to setting EPREFIX. I speculate this bug is a regression > of portage. Sorry, I used --prefix rather than EPREFIX to try to make the command as easy to read as possible. What's the best way forward? Try to use an older prefix-portage or try to port he patches to the newest portage? I tried doing the latter, but had to resort to create the patch first from the latest prefix-portage. Are the patches hosted somewhere?
As of today, I see the same thing when I use bootstrap-prefix.sh from commit 1f1ddc839a4d7053477a4aef3a6e78b2356f2b87 I try to install the prefix on SLES 11
I have the same problem both on a 64bit platform and a 32bit platform: Ubuntu 64b bootstrapping Gentoo Prefix error: * Error: circular dependencies: (sys-devel/automake-1.16.1-r1:1.16/1.16::gentoo, ebuild scheduled for merge) depends on (dev-lang/perl-5.26.2:0/5.26::gentoo, ebuild scheduled for merge) (buildtime) (sys-libs/db-6.0.35-r1:6.0/6.0::gentoo, ebuild scheduled for merge) (buildtime_slot_op) (sys-devel/automake-1.16.1-r1:1.16/1.16::gentoo, ebuild scheduled for merge) (buildtime) It might be possible to break this cycle by applying the following change: - dev-lang/perl-5.26.2 (Change USE: -berkdb) Gentoo (in a Docker container) 32b bootstrapping Gentoo Prefix error: * Error: circular dependencies: (sys-apps/acl-2.2.53:0/0::gentoo, ebuild scheduled for merge) depends on (sys-devel/gettext-0.19.8.1:0/0::gentoo, ebuild scheduled for merge) (buildtime) (virtual/acl-0-r2:0/0::gentoo, ebuild scheduled for merge) (buildtime) (sys-apps/acl-2.2.53:0/0::gentoo, ebuild scheduled for merge) (runtime) It might be possible to break this cycle by applying the following change: - sys-apps/acl-2.2.53 (Change USE: -nls) Its interesting that the cycles are not the same.
Created attachment 556114 [details] Last 500 lines of stage 3 bootstrap on a 64 bit system
Created attachment 556116 [details] Last 500 lines of stage 3 bootstrap on a 32 bit system
I think modifying bootstrap-prefix.sh in this way: pkgs=( virtual/os-headers sys-apps/portage ) #USE="ssl" \ USE="ssl -gdbm -berkdb -nls internal-glib" \ emerge_pkgs "" "${pkgs[@]}" || return 1 Workarounds the problem to at least keep going on. (The interesting line is the USE line where I added a few flags that were failing until the emerge of portage seems to be going through).
hmm, thanks. It seems we'd have to force install some more packages to avoid these dependency cycles.
What I tried failed. I don't know why it went thru a bit more. Any idea how I can get it bootstrapped meanwhile it gets fixed? Any specific older version of bootstrap-prefix.sh?
(In reply to Sammy Pfeiffer from comment #12) > What I tried failed. I don't know why it went thru a bit more. > > Any idea how I can get it bootstrapped meanwhile it gets fixed? Any specific > older version of bootstrap-prefix.sh? In the bug report, I provided a quick fix for that.
(In reply to Olivier Huber from comment #13) > > Any idea how I can get it bootstrapped meanwhile it gets fixed? Any specific > > older version of bootstrap-prefix.sh? > > In the bug report, I provided a quick fix for that. Thank you for the pointer, I missed it (I read stage1 in the line and got confused, as I was already on stage3). I still can't bootstrap it. This may be unrelated to this specific issue but I ended up with perl-5.26.2 not emerging. (With this error already reported: https://bugs.gentoo.org/670836 ). What may be related to the issue was that I thought the best option would be mask perl to use the latest stable (5.24.3-r1). So I added (well, created) to EPREFIX/etc/portage/package.mask >dev-lang/perl-5.24.3-r1 and ran bootstrap-prefix.sh again, but I got this error: python: no python-exec wrapped executable found in /tmp/gentoo/tmp/usr/lib/python-exec. * Bootstrapping Gentoo prefixed portage installation using * host: x86_64-pc-linux-gnu * prefix: /tmp/gentoo * ready to bootstrap stage3_log python: no python-exec wrapped executable found in /tmp/gentoo/tmp/usr/lib/python-exec. ... (this line repeats 8 times) /tmp/gentoo/tmp/usr/lib/python-exec. USE=-acl -berkdb -fortran -gdbm -git -libcxx -nls -pcre-python bootstrap clang internal-glib ssl PKG=sys-apps/portage python: no python-exec wrapped executable found in /tmp/gentoo/tmp/usr/lib/python-exec. So I went a bit further in the log of the failed bootstrap and found something else that may be relevant: * Messages for package dev-lang/python-exec-2.4.6: ... (big info text about overwriting) * Detected file collision(s): * * /tmp/gentoo/tmp/usr/bin/python2 * /tmp/gentoo/tmp/usr/bin/idle * /tmp/gentoo/tmp/usr/bin/python2-config * /tmp/gentoo/tmp/usr/bin/python-config * /tmp/gentoo/tmp/usr/bin/pydoc * /tmp/gentoo/tmp/usr/bin/python * /tmp/gentoo/tmp/usr/bin/2to3 * * Searching all installed packages for file collisions... * * Press Ctrl-C to Stop * * None of the installed packages claim the file(s). Note that my EPREFIX is /tmp/gentoo
I believe it is installing into the wrong directory (/tmp/gentoo/tmp/ like in stage 2 as opposed to /tmp/gentoo/ like in stage 3 should be) for the very same root cause as this bug report about emerge ignoring environment variables and reading the wrong make.conf. Considering things are really unstable until the bootstraping is done, the method I used around a week ago involves bootstraping an old tree, without this bug present, and upgrading the tree after bootstraping. However, it won't be so smooth. 1. wget the usual latest bootstrap script and chmod. 2. Comment out anything that may update the tree: * Line 539: `is-rap && LATEST_TREE_YES=1` * Line 1822 to 1828: `[[ ( ! -e ${PORTDIR}/.unpacked ) && $((nowdate - (60 * 60 * 24))) -lt ${treedate} ]] || \` to `fi` 3. Run the bootstrap script like normal. 4. You will hit bug 581940 during stage3 emerging portage; when that happens, `ln -s openrc-run $EPREFIX/sbin/runscript`, re-execute bootstrap 5. You will hit a circular dependency, same as the 32 bit one on comment 7, during stage3 with emerge -u system; when that happens, edit $EPREFIX/etc/portage/make.conf to set USE="-nls", re-execute bootstrap 6. The bootstraping script should successfully finish. 7. Undo the USE="-nls" workaround from step 5 and emerge -avuDN world 8. Manually change the main tree repo name from gentoo_prefix to gentoo, and sync url to `rsync://rsync.gentoo.org/gentoo-portage` by editing $EPREFIX/usr/share/portage/config/repos.conf and $EPREFIX/usr/portage/profiles/repo_name 9. emerge --sync and emerge -avuDN world 10. You will hit bug 671464 and fail to emerge =app-admin/metalog-20181125; when that happens, either emerge =app-admin/metalog-3-r2 or sys-devel/autoconf-archive, re-execute emerge 11. Undo the /sbin/runscript workaround from step 4 12. I suggest emerge -e world to update portage's data of the repo name which each package is installed from (gentoo_prefix -> gentoo) 13. Do the usual stuffs after an upgrade, such as emerge -ac and eclean-dist These are all the workarounds I can remember I used. (In reply to Sammy Pfeiffer from comment #7) > Its interesting that the cycles are not the same. At least in the case of 64 bit, both cycles exist. It;s just a matter of which one portage complains about first. (In reply to Olivier Huber from comment #2) > On the other hand, removing EPREFIX (or --prefix) make the circular > dependencies disappear: I think this will just end up merging into $EPREFIX/tmp like in the case of stage 2.
(In reply to YiFei Zhu from comment #15) Thanks for sharing your worksaround and your experience. I've documented mine at https://pad.crans.org/p/gentoo-prefix However, I feel like we should keep this bug about the portage bug. The IRC channe #gentoo-prefix is better to talk about the workarounds and how to get things in a better shape. We are in need also of reporting bugs. > (In reply to Olivier Huber from comment #2) > > On the other hand, removing EPREFIX (or --prefix) make the circular > > dependencies disappear: > > I think this will just end up merging into $EPREFIX/tmp like in the case of > stage 2. This was just to highlight the root cause being the EPREFIX env var (or --prefix flag). It was NOT intended as a fix the issue. This is clearly a portage bug (regression?) that would be better looked at by the portage dev team.
This is a different issue... but now bootstrap-prefix.sh stops for me earlier with the case of this bug: https://bugs.gentoo.org/672042 Which, if I try to bootstrap again, it goes thru... and then I end up in this issue of the circular dependencies.
Also, find a Docker image where you can inspect the system on the circular dependencies state: docker pull awesomebytes/gentoo_prefix_circular_dependencies docker run -it awesomebytes/gentoo_prefix_circular_dependencies /bin/bash If you do ./bootstrap-prefix.sh (and answer /tmp/gentoo as the EPREFIX) you'll find yourself in the circular dependencies error. I'm nextly exploring the workarounds from https://pad.crans.org/p/gentoo-prefix Expect to find extra images to play soon.
So I applied the workaround: --- bootstrap-prefix.sh 2018-10-18 11:24:38.000000000 +0200 +++ bootstrap-prefix.sh 2018-10-15 11:40:28.031622999 +0200 @@ -1555,7 +1555,7 @@ # cmake has some external dependencies which require autoconf, etc. # unless we only build the buildtool, bug #603012 - echo "dev-util/cmake -server" >> "${ROOT}"/tmp/etc/portage/package.use + echo -e "dev-util/cmake -server\nsys-devel/gettext -acl\ndev-lang/perl -gdbm -berkdb\nsys-apps/help2man -nls\ndev-util/pkgconfig internal-glib" >> "${ROOT}"/tmp/etc/portage/package.use for pkg in ${compiler_stage1} ; do # <glibc-2.5 does not understand .gnu.hash, use And now you can find yourself in a machine with the state after failing with both the errors: - [bug 670836] while compiling perl, missing errno ${EPREFIX}/tmp/usr/include/errno.h - [no bug] while trying to resume the bootstrap process, the error "no python-exec wrapped executable" pops up Just do: docker pull awesomebytes/gentoo_prefix_errno_error_and_no_python docker run -it awesomebytes/gentoo_prefix_errno_error_and_no_python /bin/bash The next image I'm creating contains the fixes for the errno and the "no python-exec wrapped executable" from https://pad.crans.org/p/gentoo-prefix applied. We'll see where it hangs.
The next error found is: In file included from perl.h:1138, from dump.c:29: /tmp/gentoo/tmp/usr/include/errno.h:48:10: fatal error: bits/types/error_t.h: No such file or directory #include <bits/types/error_t.h> ^~~~~~~~~~~~~~~~~~~~~~ (happens a bunch of times) Which I reported in this bug: https://bugs.gentoo.org/672116 You can find a box in that exact state doing: docker pull awesomebytes/gentoo_prefix_latest_image:26 docker run -it awesomebytes/gentoo_prefix_latest_image:26 /bin/bash
(In reply to Sammy Pfeiffer from comment #20) > The next error found is: > > In file included from perl.h:1138, > from dump.c:29: > /tmp/gentoo/tmp/usr/include/errno.h:48:10: fatal error: > bits/types/error_t.h: No such file or directory > #include <bits/types/error_t.h> > ^~~~~~~~~~~~~~~~~~~~~~ > > (happens a bunch of times) > > Which I reported in this bug: https://bugs.gentoo.org/672116 > > You can find a box in that exact state doing: > > docker pull awesomebytes/gentoo_prefix_latest_image:26 > docker run -it awesomebytes/gentoo_prefix_latest_image:26 /bin/bash That can be workarounded with: mkdir -p ${EPREFIX}/tmp/usr/include/bits/types && cp ${EPREFIX}/usr/include/bits/types/error_t.h ${EPREFIX}/tmp/usr/include/bits/types After that you get: Can't figure out your cwd! Which can be workarounded with: cp ${EPREFIX}/bin/pwd ${EPREFIX}/tmp/bin/pwd And then perl fails building... From https://pad.crans.org/p/gentoo-prefix (I also updated it) this makes perl go thru...: rm -f ${EPREFIX}/tmp/usr/bin/perl ln -s /usr/bin/perl ${EPREFIX}/tmp/usr/bin/perl But it fails later on: perl-core/File-Temp-0.230.400-r1::gentoo perl: error while loading shared libraries: libperl.so.5.26: cannot open shared object file: No such file or directory I'm lost here, I don't know what is causing this, and I can't think of more workarounds. You can get in the machine in the state previous to the commands of this comment, from my previous comment.
Docker image with the specific state I just reported (after perl workarounds, but failing in perl-core/File-Temp-0.230.400-r1::gentoo perl: error while loading shared libraries: libperl.so.5.26: cannot open shared object file: No such file or directory): docker pull awesomebytes/gentoo_prefix_latest_image:28 docker run -it awesomebytes/gentoo_prefix_latest_image:28 /bin/bash
Doing all the bunch of workarounds from https://pad.crans.org/p/gentoo-prefix Now I get to: * running emerge -u system But that fails again with circular dependencies, which I just applied the same workaround we did for ${EPREFIX}/tmp. Then the next error I have no clue what is going on. I reported it here: https://bugs.gentoo.org/672212 ERROR:root:Failed to copy file: _parsed_options=Namespace(group=-1, mode=420, owner=-1, preserve_timestamps=False), source=b'README', dest_dir=b'/tmp/gentoo/var/tmp/portage/sys-apps/which-2.21/image/tmp/gentoo/usr/share/doc/which-2.21' Traceback (most recent call last): File "/tmp/gentoo/usr/lib/portage/python3.6/doins.py", line 209, in run copyfile(source, dest) File "/tmp/gentoo/usr/lib64/python3.6/site-packages/portage/util/file_copy/__init__.py", line 30, in _optimized_copyfile _file_copy(src_file.fileno(), dst_file.fileno()) PermissionError: [Errno 1] Operation not permitted * ERROR: sys-apps/which-2.21::gentoo failed (install phase): * dodoc failed If I workaround this package, which-2.21, the next package fails with the same dodoc failed error. (in my case the next package is sys-apps/net-tools-1.60_p20180626073013). Anyone has any idea?
(In reply to Sammy Pfeiffer from comment #23)> > If I workaround this package, which-2.21, the next package fails with the > same dodoc failed error. (in my case the next package is > sys-apps/net-tools-1.60_p20180626073013). > > Anyone has any idea? I've been suggested to try df -i To check if I was left without any available inodes. I haven't been able to replicate this problem again. I have successfully bootstrapped Gentoo Prefix with all the hacks from https://pad.crans.org/p/gentoo-prefix I have created a release with a full Gentoo Prefix amd64 available here (with instructions): https://github.com/awesomebytes/gentoo_prefix_ci_test/releases/tag/18.11.30-1 A 32b version is being produced (hopefully).
And a fully bootstrapped Gentoo Prefix 32bit/x86: https://github.com/awesomebytes/gentoo_prefix_ci_test_32b/releases/tag/18.12.01-1
Here i (In reply to Sammy Pfeiffer from comment #17) > This is a different issue... but now bootstrap-prefix.sh stops for me > earlier with the case of this bug: > > https://bugs.gentoo.org/672042 > > Which, if I try to bootstrap again, it goes thru... and then I end up in > this issue of the circular dependencies. Hi Sam, Thank you for your input. But please do not pollute this bug as memo of your workarounds irrelevant to this discussion. Benda
(In reply to Fabian Groffen from comment #3) > @heroxbd: do you have an idea about that --prefix argument to emerge? Hi Fabian, here is what I have found: I can confirm Ollvier's observation: portage 2.3.40.3-prefix does not respect USE environment variable during cross-prefix. Neither portage 2.3.40-r1 nor portage 2.3.51 from the master branch has this bug. USE works for both while not in cross-prefix mode. There must be a regression with the prefix branch of portage. I haven't tested older versions of portage-prefix yet. It could be easier for you to reproduce this bug on a prefix-rpath installation with portage-2.3.40.3-prefix by, for example: EPREFIX=/tmp USE='-nls' emerge -1pv attr Yours, Benda
*** This bug has been marked as a duplicate of bug 655414 ***
