Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 420459 - emerge --keep-going should leave behind --resume state for skipped/failed builds
Summary: emerge --keep-going should leave behind --resume state for skipped/failed builds
Status: UNCONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Enhancement/Feature Requests (show other bugs)
Hardware: All Linux
: Normal enhancement
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 373807
  Show dependency tree
 
Reported: 2012-06-10 02:03 UTC by Robert White
Modified: 2022-03-05 23:42 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Robert White 2012-06-10 02:03:10 UTC
When doing large or complext builds, or rebuilding world, --keep-going is very helpful. But when a build keeps going passed a failed component it may also skip a number of dependents when it recomputes the dependenceis.

So the /var/tmp/portage directory will retain a directory for every failed package but there is no record of the packages skipped outright.

It would be Better™ if "emerge --keep-going whatever" recorded all the failed and skipped packages and left that behind as a state for "emerge --resume" so that build breakage could be addresed with the various utilities (*-updater perl-cleaner, revdep-rebuild, etc) and actions (dispatch-conf, reboot, etc); after which point an "emerge --resume" could go back and get the sragglers.

Lather-Rinse-Repeat of course since the resume pass will likely need to be resumed more than once.

Reproducible: Always
Comment 1 Zac Medico gentoo-dev 2012-09-04 01:18:31 UTC
(In reply to comment #0)
> It would be Better™ if "emerge --keep-going whatever" recorded all the
> failed and skipped packages and left that behind as a state for "emerge
> --resume" so that build breakage could be addresed with the various
> utilities (*-updater perl-cleaner, revdep-rebuild, etc)

The plan is to replace tools like perl-cleaner and revdep-rebuild with automatic rebuilds in EAPI 5. See bug #192319.

> and actions
> (dispatch-conf, reboot, etc); after which point an "emerge --resume" could
> go back and get the sragglers.

So, you want it to keep a record of skipped builds so it can go back an repeat them. Is this just for things like `emerge -e @world`, so that you can make sure that you've rebuilt everything? If so, maybe it's better to use a time-based set like DateSet introduced in bug #433704.
Comment 2 Robert White 2012-09-04 06:17:26 UTC
No.

Typical use case. Machine that hasn't been upgraded in a month or so. Do "emerge --sync", then "emerge --update --deep --newuse --keep-going @world".

The later spins for a while. Then starts in with 1 of 248.

You come back later and it says working on 33 of 87.

You let it finish.

Now, how many of the 248 were skipped when whatever problem it was was cut down to the last 87? We don't know. We may be able to scroll back and do some math if our scroll buffer is big enoug.

Now nine times out of ten you can just do a revdep-rebuild and then reissue the command and re-wait for the results after re-waiting for the dependency computation and things will work out. This is the typical missed deep depencency. I don't expect to have emerge look throuh all the build.log files to figure out that /usr/lib/libsomeobscurething.so can not load /lib/libsomeotherobscurething.so and do a backtrack.

It would be nice if you could just "emerge --resume", just like you could if you -didn't- say keep going and it woudl start off with the package that broke in the first place without waiting for a dependency computation etc.

I'd just like to tool to dump the difference between the two lists (remaining X jobs from original list vs new X-N jobs it will resume) somewhwere convenient for me/it to use in a resume.

(long dependency lists, it's not just for emerge -e @world any more 8-)
Comment 3 Zac Medico gentoo-dev 2012-09-06 19:04:53 UTC
(In reply to comment #2)
> Now, how many of the 248 were skipped when whatever problem it was was cut
> down to the last 87? We don't know. We may be able to scroll back and do
> some math if our scroll buffer is big enoug.

With default settings, they should all be logged in /var/log/portage/elog/summary.log. If I encounter a case like this, I typically just run `emerge -uDN @world` again, after masking broken packages if necessary. That pulls in all the packages that were skipped, so i don't really need the list of skipped packages for anything.

> Now nine times out of ten you can just do a revdep-rebuild and then reissue
> the command and re-wait for the results after re-waiting for the dependency
> computation and things will work out. This is the typical missed deep
> depencency. I don't expect to have emerge look throuh all the build.log
> files to figure out that /usr/lib/libsomeobscurething.so can not load
> /lib/libsomeotherobscurething.so and do a backtrack.

BTW, this is one of the things that automatic rebuilds with EAPI 5 (bug #192319) is supposed to help with. The idea is that the relevant packages will be rebuilt before they have a chance to trigger a build failure due to an SONAME change.

> It would be nice if you could just "emerge --resume", just like you could if
> you -didn't- say keep going and it woudl start off with the package that
> broke in the first place without waiting for a dependency computation etc.
> 
> I'd just like to tool to dump the difference between the two lists
> (remaining X jobs from original list vs new X-N jobs it will resume)
> somewhwere convenient for me/it to use in a resume.
> 
> (long dependency lists, it's not just for emerge -e @world any more 8-)

I suppose we can add some way to do that (though I wouldn't need to use it myself, as explained earlier).