Many times over the years I have encountered packages that fail to build due to dependency issues caused from race conditions from --jobs and MAKEOPTS greater than -j1 Often the packages build without problems when --jobs is disabled and MAKEOPTS=-j1 However building an entire system with MAKEOPTS=-j1 is becoming somewhat impossible these days. The updates are coming faster than slower systems can complete a sequential build! Is there a mechanism in portage to tell it to retry a build when it fails, but with some customized options. For me that would simply be to disable --jobs and append MAKEOPTS="-j1" to the end of whatever the current MAKEOPTS are. Reproducible: Always Steps to Reproduce: 1. Grab a new stage3 2. Set MAKEOPTS="-j16" 3. eselect profile default/linux/amd64/17.1/desktop/plasma 4. emerge --jobs x11-base/xorg-x11 Actual Results: As of 2021-05-22 the media-libs/mesa-20.3.5 package is marked stable and yet it always fails to build unless you set MAKEOPTS="-j1" I run into this kind of problem all the time and I am starting to get tired of posting bugs that become stale in this bug tracker (through no fault of the great people here). I totally understand that developer resources are limited and often the bugs are resolved by a new package being "eventually" unmasked and deprecating the problematic ones. Expected Results: If portage could just temporarily switch to a slower performance but more reliable build mode when a package fails, retry, then resume with the original config that would save a ton of effort, and reduce the number of time-wasting bugs.
It'd be worth actually reporting the bugs for these packages which fail with parallel build. This feels like kind of a niche option given that often, the failure reason won't be parallel build-related.
(Sorry, I see you did say you've reported them in the past -- I just didn't recall one for Mesa, and packages are really different in the causes.)
The one for mesa just happened on a new system I am installing right now. I switched to MAKEOPTS="-j1" and it's still installing packages at this point now. I haven't submitted that bug yet because I really need to get that system up and running first and I was able to bypass the build error by setting MAKEOPTS="-j1". Once I get that system done I can re-build it again and submit the mesa bug. I just really need to get it up and running first, and it takes days to start from a stage3 and get xorg running on a lower power system. Each time emerge fails it can cost many hours of build-time to finish the install. I have 64GB of ecc ram on this system and the error is reproducible which is why I suspected it was a dependency race condition.
(In reply to Nathan Shearer from comment #3) > The one for mesa just happened on a new system I am installing right now. I > switched to MAKEOPTS="-j1" and it's still installing packages at this point > now. I haven't submitted that bug yet because I really need to get that > system up and running first and I was able to bypass the build error by > setting MAKEOPTS="-j1". > > Once I get that system done I can re-build it again and submit the mesa bug. > I just really need to get it up and running first, and it takes days to > start from a stage3 and get xorg running on a lower power system. Each time > emerge fails it can cost many hours of build-time to finish the install. > > I have 64GB of ecc ram on this system and the error is reproducible which is > why I suspected it was a dependency race condition. FWIW, our Mesa maintainer is also a developer upstream and is therefore pretty good at debugging things like this. I’m sorry it’s happening so often though… I honestly can’t remember the last time it happened to me. This isn’t a fix but might it be worth producing binary packages if you’re setting up systems so often?
(In reply to Sam James from comment #4) > This isn’t a fix but might it be worth producing binary packages if you’re > setting up systems so often? It's funny you mention that, because I've been thinking of ways to do exactly that in some way. Sometimes I also need an older package that was removed from portage or something with conflicting use flags and the only way to get it installed is to build it manually. I've even thought about making some kind of an overlay that just contains all the debian/ubuntu packages so I can rapidly grab one of those and get the job done ;)
