| Summary: | app-portage/portage-utils does not fetch dependencies | ||
|---|---|---|---|
| Product: | Portage Development | Reporter: | Zdenek Behan <rain> |
| Component: | Tools | Assignee: | solar (RETIRED) <solar> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | vapier |
| Priority: | High | Keywords: | InVCS |
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: | Fetch binary packages recursively | ||
|
Description
Zdenek Behan
2009-04-17 01:10:33 UTC
OK, since no reply or rejection of this bug has happened, i dug into the problem a bit deeper, and it is obviously pretty simple. The basic qmerge utility logic is that commandline arguments are being parsed, then the function pkg_fetch() is called, which basically takes all non-getopt arguments, treats them as package atoms, checks/fetches the binary package, and calls pkg_merge() on each of them. This is where the main issue is, because pkg_merge() then calls itself directly recursively on all of the packages dependencies, but does not even attempt to fetch them. The call stack looks like pkg_fetch()->pkg_merge()->pkg_merge()->..., whereas any other but the first pkg_merge() is doomed to fail unless the packages have been manually downloaded prior to running this. My fix works very easilly. Since pkg_fetch() has the arguments handy for calling pkg_merge() i split pkg_fetch() into 1) pkg_process() which takes the original arguments, and 2) pkg_fetch() which takes the same arguments as pkg_merge(). New pkg_process() keeps the logic of taking every commandline argument, resolving the atom, doing some checks, and then calls new pkg_fetch(). New pkg_fetch() is basically the for-cycle content of original function that receives a package atom as an argument, checks/fetches the binary package, and calls pkg_merge(). The above described is a mere refactoring. The function had a bad name and was unnecessarily huge anyway. But it makes possible for a simple fix, changing the pkg_merge() recursion into indirect, always calling pkg_fetch() first. So the call stack looks like this: pkg_process()->pkg_fetch()->pkg_merge()->pkg_fetch()->pkg_merge()->... It is really not much of a change, hopefully breaks nothing, and certainly fixes the described problem. Created attachment 188756 [details, diff]
Fetch binary packages recursively
And the patch
Taking bug. Uhhhhh, sorry, I think I reported this to the wrong target :/. I mean, tools-portage@gentoo.org sounds almost like utils-portage@gentoo.org. Besides, it said something like Portage Utils <tools-portage@gentoo.org>. '- Refactor fetching logic (Zdenek Behan <behanz[@]seznam[dot]cz>). bug 266455' /var/cvsroot/gentoo-projects/portage-utils/qmerge.c,v <-- qmerge.c new revision: 1.83; previous revision: 1.82 Zdenek, Thanks for the patch. If you feel motivated there is much more refactoring to be done. Mianly in the area of better handling/parsing of the rdeps that could be done. Ideally the tool would build a deptree before merging the first package vs it's linear approach now. Well, yes, that's what I wanted to point out. That if you "qmerge package itsdependency", then itsdependency will be merged twice, which is quite unnecessary. But this kind of issue would take a lot more than a simple fix. I can look into it, but my motivation for the patch was more practical than that I would be going about being bored looking for things to fix. I was working on deploying gentoo to series of identical embedded boxes which are just too slow to build all the stuff themselves (emerge system takes just over a day in the more faster cases), and rebuilding a stage3 tarball periodically to avoid having to overcome too steep system updates seemed like too much of a hassle. Instead, I built binary packages for everything, packaged statically linked busybox and portage-utils into a tiny "binary bootstrap" tarball, which has just over few megabytes large, will not need to be updated for a _long_ time, and even the update is pretty much trivial and straightfoward (USE=static ROOT=whereto/ emerge busybox portage-utils), and which can be used to download all of portage and its dependencies very smoothly and using a tiny script, with nearly no requirements for the host system (just wget and chroot, really). This bug was the only issue in the way. |
