The current output of emerge --pretend --fetchonly (also applies to --fetch-all-uri to a lesser degree) is only of limited use to people who wish to use it for alternative source downloading (usually on a different computer). Firstly, it lists a source even if it is successfully downloaded. Secondly, and more importantly, it prints neither the target file name, nor its size and digest(s). This further complicates successful fetching. I previously used a rather complicated script to alleviate these problems, but now I found a better solution, presented in my patch. With this patch, fetch does not list any URIs for files that are present in DISTDIR and their checksums are verified. It also adds a new settings variable, PORTAGE_FETCH_LISTHEADER, that can be specified, like FETCHCOMMAND, either as an environment variable in the shell, or in /etc/make.conf. The contents of this variable are prepended to the space-separated list of URIs for the file. In this string, the following variables get expanded: - file: the target file name (may be different from the url name) - size: the size in bytes - RMD160, SHA256, SHA1: all digests from the manifest - nl, tab: convenience variables for inserting a newline or tab The variable defaults to "${nl}", matching current emerge output separating uli lists by blank lines (but still just listing missing files). I have also written a sample download script, utilising this functionality. Reproducible: Always Additionally, I looked into a few bug reports regarding these matters, which are all tracked by bug #377365 (among others). Some of them may be closed, either by this patch or a simple WONTFIX. I do not wish to pimpspam this patch on all of them, so I comment on them here, hoping the competent developers will sort them out: Bugs #293827 and #293830 from the same reporter, also bug #358923 only wish to list one URI per file. This can be readily solved by standard utilities, ex. : emerge -pfq ... | cut -f 1 (easy) emerge -pfq ... | sed '/^\s*$/d;s/\s.*//' (better) emerge -pfq ... | tr ' ' '\n' | sed '/^\s*$/d' | wget -nc -i - (best, note -nc) This does not even require my patch (although my solution presented in a later attachment/comment is superior). Bug #109304, especially comment 6, were an inspiration for skipping verifiably fetched files without any more options to portage.
Created attachment 284269 [details, diff] portage-fetch_listheader.patch This patch does not address listing the same file for multiple packages. This can be solved bu simply piping through sort -u, with a caveat that the URI list may be AFAIR in a randomized order. Alternatively, one may use something like this: PORTAGE_FETCH_LISTHEADER="${file}${tab}${size}${tab}" emerge -pfq ... | sort -uk 1,2 This patch does not handle documentation. Also, it does not register PORTAGE_FETCH_LISTHEADER anywhere (which may be a requirement in portage).
Created attachment 284271 [details] Sample downloader script For lack of a better venue, I present a sample downloader script utilising the patch. It requires a custom setting for PORTAGE_FETCH_LISTHEADER, either in make.conf or on the commandline. Simplest usage: Target machine: PORTAGE_FETCH_LISTHEADER="${file}${tab}${size}${tab}${SHA256}${tab}" emerge -pfq -uDn --with-bdeps=y @world | sort -uk 1,1 >/media/drive/wishlist Fetcher machine: cd /media/drive; KEEP_GOING=y bash getem
Bad mistake in the examples I gave: the value of PORTAGE_FETCH_LISTHEADER must use escaped \$ signs both in make.conf and on the command line of course. That is: PORTAGE_FETCH_LISTHEADER="\${file}\${tab}\${size}\${tab}\${SHA256}\${tab}"
