Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 629762 - Request --fetchlater and --fetchnow options for emerge
Summary: Request --fetchlater and --fetchnow options for emerge
Status: UNCONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Enhancement/Feature Requests (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 377365
  Show dependency tree
 
Reported: 2017-09-03 12:54 UTC by donaldway
Modified: 2017-09-03 17:43 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description donaldway 2017-09-03 12:54:49 UTC
I would like to see us be able to do something like...

emerge --fetchlater vim > fetch.log

...and then...

cat fetch.log | emerge --fetchnow /mnt/mydistfiles

The idea is that there are occasions where you can't/won't connect to the internet to satisfy an emerge.  Say you have a PC at home and the internet isn't working, but you have a laptop you can take to an internet cafe.  Or, you want to build a gentoo system that remains airgapped at all times for security.

The fetchlater option would simply output the list of packages required in order to satisfy an emerge.  This would be run on the airgapped machine, and then the resulting list could be copied via USB stick to the networked machine, and given as input to emerge with the fetchnow option and a location where to put retrieved files.

This idea was discussed at some length in the following Gentoo forum thread:

https://forums.gentoo.org/viewtopic-t-1068066-start-0-postdays-0-postorder-asc-highlight-.html

One criticism is that you could accomplish the very same thing by processing emerge --fetchonly --pretend output with something like awk or sed, but if you look at the problem more closely that proves to be unsuitable.  First, the airgapped machine may not be in the position to know what the value of the GENTOO_MIRRORS variable is on the networked machine, which figures mightily in determining which servers to access when attempting to retrieve the files.  

Complicating this is that portage will often (always?) return many URIs for a package so the logic necessary to recreate portage's behavior here becomes nontrivial for an airgapped machine to specify via something like a bash script.

More importantly, the desired files may be already sitting on the networked machines distfiles directory, in which case, fetching is neither necessary nor desired.

Another criticism partly addresses the above by noting that wget can be instructed to ignore fetches if the file already exists, but even this is complicated if you consider that you wouldn't necessarily want files downloaded for another machine to occupy space in the networked machine's distfiles directory.  It isn't just a question of saving space as there is eclean and the like, but how do you instruct eclean to discard files downloaded for another system while preserving possibly older versions of packages downloaded for that system (as you'd want to do if you desire to restore your system to an earlier state)?  And I see no option in wget that lets you skip downloading a file based on its existence in some other directory.

Giving portage a list of ebuilds and a destination directory should leave the state of the networked machine unaffected.  Files that already exist can simply be copied by portage over to the destination directory.  Files that need to be downloaded can be downloaded using the proper set of mirrors as set on the networked machine.  Admittedly, this turns the emerge command into a sort of glorified wget agent but as portage is in possession of all of the data necessary to make this work correctly and completely, it doesn't seem that far outside of its primary mission.

Gentoo is an ideal distro for the security-minded user.  Security is a battle that has many, many fronts.  The easier we make it to administer these machines in a secure manner, the more secure these systems become, and no strategy for security has a better cost/benefit ratio than airgapping, even if it isn't always a viable option.

Thank you for your consideration.