Created attachment 790271 [details] emerge.log XZ compressed I realise some packages depend on others before they can be rebuilt But it feels like after the initial batch of virtual packages are built, there are a lot that go serially Attaching an emerge world --empty with a -j100 which hopefully highlights the issue
<sam_> FireBurn: it'd be interesting if you could tell me if it still happens with --implicit-system-deps=n in EMERGE_DEFAULT_OPTS
>>> Installing (59 of 1483) media-libs/fdk-aac-2.0.2::gentoo >>> Installing (113 of 1483) app-text/scdoc-1.11.2-r1::gentoo >>> Installing (120 of 1483) acct-group/libreoffice-0::gentoo >>> Emerging (162 of 1483) acct-user/libreoffice-0::gentoo >>> Installing (117 of 1483) app-shells/quoter-4.2::gentoo >>> Installing (118 of 1483) app-text/libexttextcat-3.4.6::gentoo >>> Installing (124 of 1483) app-misc/editor-wrapper-4-r1::gentoo >>> Installing (119 of 1483) app-text/libnumbertext-1.0.10::gentoo >>> Installing (125 of 1483) virtual/libintl-0-r2::gentoo >>> Emerging (163 of 1483) sys-libs/timezone-data-2022a::gentoo >>> Installing (123 of 1483) sci-libs/suitesparseconfig-5.4.0::gentoo >>> Installing (122 of 1483) sys-devel/ucpp-1.3.5::gentoo >>> Installing (128 of 1483) acct-group/man-0-r1::gentoo >>> Emerging (164 of 1483) acct-user/man-1-r1::gentoo >>> Installing (129 of 1483) app-text/manpager-1::gentoo >>> Installing (130 of 1483) acct-group/ssmtp-0::gentoo >>> Installing (121 of 1483) dev-util/cppunit-1.15.1-r3::gentoo >>> Installing (127 of 1483) virtual/libiconv-0-r2::gentoo >>> Emerging (165 of 1483) sys-fs/dosfstools-4.2::gentoo >>> Installing (131 of 1483) acct-group/mail-0-r1::gentoo >>> Emerging (166 of 1483) acct-user/mail-0-r1::gentoo >>> Emerging (167 of 1483) acct-user/postmaster-0-r1::gentoo >>> Installing (111 of 1483) dev-lang/go-1.18.3::gentoo >>> Installing (126 of 1483) dev-util/pkgconf-1.8.0-r1::gentoo >>> Emerging (168 of 1483) virtual/pkgconfig-2-r1::gentoo >>> Installing (133 of 1483) acct-group/rtkit-0-r1::gentoo >>> Emerging (169 of 1483) acct-user/rtkit-0-r1::gentoo >>> Installing (132 of 1483) app-emulation/wine-gecko-2.47.2::gentoo >>> Installing (134 of 1483) dev-util/opencl-headers-2022.05.18::gentoo >>> Installing (13 of 1483) net-analyzer/sslscan-2.0.15::gentoo >>> Installing (135 of 1483) dev-lang/python-exec-2.4.9::gentoo >>> Installing (136 of 1483) acct-user/messagebus-0-r1::gentoo >>> Installing (139 of 1483) sys-devel/gcc-config-2.5-r1::gentoo >>> Installing (138 of 1483) sys-devel/binutils-config-5.4.1::gentoo >>> Installing (137 of 1483) games-util/esteam-0.20220522::steam-overlay >>> Installing (141 of 1483) acct-user/systemd-timesync-0-r1::gentoo >>> Installing (142 of 1483) acct-user/systemd-network-0-r1::gentoo >>> Installing (143 of 1483) acct-user/minidlna-0::gentoo >>> Installing (144 of 1483) acct-user/root-0-r1::gentoo >>> Emerging (170 of 1483) acct-user/nobody-0::gentoo >>> Installing (145 of 1483) acct-user/polkitd-0-r1::gentoo >>> Installing (153 of 1483) acct-user/systemd-coredump-0-r1::gentoo >>> Installing (147 of 1483) acct-user/systemd-resolve-0-r1::gentoo >>> Jobs: 145 of 1483 complete, 1 running Load avg: 3.8, 11.8, 17.0 Pretty sure it's getting the same issue
>>> Emerging (206 of 1483) virtual/acl-0-r2::gentoo >>> Installing (206 of 1483) virtual/acl-0-r2::gentoo >>> Emerging (207 of 1483) virtual/yacc-0::gentoo >>> Installing (207 of 1483) virtual/yacc-0::gentoo >>> Emerging (208 of 1483) virtual/perl-File-Spec-3.840.0::gentoo >>> Installing (208 of 1483) virtual/perl-File-Spec-3.840.0::gentoo >>> Emerging (209 of 1483) virtual/perl-ExtUtils-MakeMaker-7.640.0::gentoo >>> Installing (209 of 1483) virtual/perl-ExtUtils-MakeMaker-7.640.0::gentoo >>> Emerging (210 of 1483) virtual/perl-Encode-3.170.0::gentoo >>> Installing (210 of 1483) virtual/perl-Encode-3.170.0::gentoo >>> Emerging (211 of 1483) virtual/perl-Carp-1.520.0-r2::gentoo >>> Installing (211 of 1483) virtual/perl-Carp-1.520.0-r2::gentoo >>> Emerging (212 of 1483) virtual/perl-Exporter-5.770.0::gentoo >>> Installing (212 of 1483) virtual/perl-Exporter-5.770.0::gentoo >>> Emerging (213 of 1483) virtual/perl-Scalar-List-Utils-1.620.0::gentoo >>> Installing (213 of 1483) virtual/perl-Scalar-List-Utils-1.620.0::gentoo >>> Emerging (214 of 1483) virtual/tmpfiles-0-r3::gentoo >>> Installing (214 of 1483) virtual/tmpfiles-0-r3::gentoo >>> Emerging (215 of 1483) virtual/libcrypt-2::gentoo >>> Installing (215 of 1483) virtual/libcrypt-2::gentoo >>> Emerging (216 of 1483) virtual/libudev-232-r7::gentoo >>> Installing (216 of 1483) virtual/libudev-232-r7::gentoo >>> Emerging (217 of 1483) virtual/udev-217-r5::gentoo
The above is all output from and emerge --empty world -j100
emerge -j has been broken for years here It sometimes (randomly, rarely) does several package in //, but most of the time it does them serially, using <5% of cpu, almost no RAM, unless a package has log of files to compile (then make -j does its jobs). I triple checked all possible causes, and failed to understand why. It's not because of dependancies, and it's not because of -l
Whenever I am doing a world rebuild, I precalculate all deps and feed the list to emerge --oneshot --nodeps --keep-going --jobs=n
Can you provide mode details ? How do you 'precalculate' ?
I use emerge --pretend to make the list and then I use --nodeps for the actual merge. This way the dependency calculation does not get in the way of parallelization.
