Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 460712 - sys-apps/portage - Emerge system and small packages before world and large optional ie Chromium, Firefox, etc
Summary: sys-apps/portage - Emerge system and small packages before world and large op...
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Enhancement/Feature Requests (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Portage team
Depends on:
Reported: 2013-03-07 18:13 UTC by William L. Thomson Jr.
Modified: 2022-06-12 18:55 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---

emerge package list (emerge,1.22 KB, text/plain)
2013-03-07 18:54 UTC, William L. Thomson Jr.
emerge package list (emerge.txt,16.08 KB, text/plain)
2013-03-07 18:56 UTC, William L. Thomson Jr.
output of emerge -uDNkpv system (emerge_system.txt,18.23 KB, text/plain)
2013-03-12 20:11 UTC, William L. Thomson Jr.

Note You need to log in before you can comment on or make changes to this bug.
Description William L. Thomson Jr. 2013-03-07 18:13:55 UTC
This is not really specific to any one version of portage, or any package you might emerge. In a nutshell I am tired of having my entire system update stop because I run out of disk space on a large package that prevents others from merging that are either dependencies or more core packages. Though at times it happens as a result of parallel compilations. Which tends to block and prevent many other smaller packages that are more necessary to the system from being merged. At times I have had to mask packages to update my system, and then come back to update those packages last, Chromium, Firefox, kernel sources, and other larger packages. Not sure exactly what can be done to correct this. Here are some ideas, no particular order

1. If a package is not required to be merged before another, then it would be nice if the smallest packages are merged first, and largest last. This even includes kernel sources, those really do not need to be installed, rarely a dependency issue, but if no dependency, then closer to the end.

2. Along the same lines, which I assume takes place now but does not seem to always be the case, packages in system are always updated before world.

3. Optional packages are merged after dependencies. I do not want to see chromium or firefox coming in before say kde packages/dependencies, or other things that are more core to the system/desktop than others.

Also more of a PMS thing or something, its funny how some of the game packages will not merge if you do not have enough disk space either tmp or in the system. But many other packages that likely more use do not have such. Might be time to start requiring packages to have a variable that discloses how much space is needed. So emerges will fail before they start if there is not adequate resources, like games and others do not. Excuse me if that was already added to PMS haven't been following development.
Comment 1 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2013-03-07 18:28:47 UTC
Why not emerge them yourself in the order that you want them?

First emerge @system, then emerge @world and then emerge your set which has the packages that you want to emerge last; for this last one, you can pass -j1 such that it runs one big package at a time.

Some people will want to do the opposite of what you suggest.
Comment 2 Zac Medico gentoo-dev 2013-03-07 18:48:39 UTC
You can use the --exclude option in order to temporarily exclude packages.

Automated comparisons based on "size" can be tricky, since it's possible for packages to have large distfiles containing lots of things that don't need to be compiled.
Comment 3 William L. Thomson Jr. 2013-03-07 18:53:53 UTC
I will try with the exclude option, but again that does not correct illogical order, udev/portage emerging after a many other updates, I will attach an example.

But again as for packages I look to games, I recently was emerging urban terror. I was very impressed how it failed because of not enough tmp and installed disk space. Thus if its just a distfile issue, would be more a tmp issue, and not install. So its two sizes, size needed to unpack/compile, and size on disk. I do not see where it could hurt for packages to provide something along those lines. Given some packages do need considerable resources of both, or one or the other.

Again all just suggestions, I know implementation would not be so easy. Just saying its a bit crazy at times now.
Comment 4 William L. Thomson Jr. 2013-03-07 18:54:39 UTC
Created attachment 341254 [details]
emerge package list
Comment 5 William L. Thomson Jr. 2013-03-07 18:56:12 UTC
Created attachment 341256 [details]
emerge package list
Comment 6 David Carlos Manuelda 2013-03-07 20:39:09 UTC
Well...actually I would suggest a thing more in this matter:

#1 mentions specifically new kernel emerge, and that's the one that is problematic:

If you have a very long list with kernel *and* some kernel modules (nvidia, virtualbox....), and you have symlink USE is very annoying having the whole list failing because modules fail to build.

So I'd suggest to postpone new kernel install the more it can, specially if it have kernel modules in list also.

It is not a critical issue though, but would improve usability a lot.
Comment 7 William L. Thomson Jr. 2013-03-12 20:11:58 UTC
Created attachment 341844 [details]
output of emerge -uDNkpv system

You can see packages that I would think belong more to world than to the system. I would think non-optional, non-user install packages to be part of system, gcc, glibc, bash, etc. Not sure why things such as gtk, kde anything, samba, and other things are considered part of my system and not part of world. Firebird is definitely part of world and not system. This again shows its quite difficult to even update in pieces at times, no way to avoid installing some apps. Like in my case, I rather not deal with the libxml2/qtwebkit issue, but I can't get around that simply by updating my system then world. Which shows that doing what is suggested in comment #1 is impossible at times. Not to mention order of installing smaller before larger. Which falls along the lines of doing easy things before hard, which is usually the rule of thumb for anything. Not sure anyone would want to emerge a large package before a small one, or a bunch of smaller ones. Not to mention most times smaller packages are dependencies of larger ones.

Anyway the update order logic is just crazy, order in which packages are merged. The package list of updates here is all over the place. Pulling in world packages for system update.
Comment 8 Mike Gilbert gentoo-dev 2013-06-19 05:02:28 UTC
(In reply to William L. Thomson Jr. from comment #7)

You are pulling in direct and indirect dependencies of packages in @system; that's what the -D/--deep flags does.

Several packages in @system have optional dependencies on libraries like GTK. Once GTK or Qt is in your depgraph, it tends to explode rather quickly.
Comment 9 Henry paradiz 2022-06-01 07:58:36 UTC
It would be nice if portage looked at merge times from previous merges and sorted the merge list to prioritize faster merging packages first while still honoring the dependency tree properly (allowing a slower merge to go in first if it's a dependency for other things).
Comment 10 Charles Nérot 2022-06-12 18:54:57 UTC
Maybe a feature that will prevent this could be :
- add a file like /etc/portage/package.hudge
- every package listed in this file will only build one by one by portage

So everybody can chose which package should never be compiled in parallel with other on his system.