Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 470002 - sys-apps/portage: support PORTAGE_SSH_OPTS to provide arguments to ssh and sftp
Summary: sys-apps/portage: support PORTAGE_SSH_OPTS to provide arguments to ssh and sftp
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks: 468524
  Show dependency tree
 
Reported: 2013-05-15 20:54 UTC by Jason A. Donenfeld
Modified: 2016-01-22 11:48 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jason A. Donenfeld archtester Gentoo Infrastructure gentoo-dev Security 2013-05-15 20:54:16 UTC
The summary says it all. Portage fetches sftp and ssh binary packages using a non-customizable command. This makes usage of private keys very cumbersome, not to mention proper placement of known-hosts files and other related ssh functionality. Adding a SSH_FETCH_ARGUMENTS which would be merged into the arguments for sftp and ssh for binary fetching would be quite nice.

Reproducible: Always
Comment 1 Zac Medico gentoo-dev 2013-05-15 22:46:52 UTC
So both ssh and sftp commands support all of the same arguments? Maybe it's better to have a separate variable for each one?
Comment 2 Jason A. Donenfeld archtester Gentoo Infrastructure gentoo-dev Security 2013-05-16 00:45:12 UTC
AFAIK, they support the same options.
Comment 3 Paul Varner (RETIRED) gentoo-dev 2013-05-16 15:25:57 UTC
The port options may be different depending upon what command you are using

ssh use '-p' - lowercase 'p'
scp/sftp use '-P' - uppercase 'P'
Comment 4 Zac Medico gentoo-dev 2013-05-16 16:35:01 UTC
Another thing to consider is that the binhost fetch code has 2 different parts which behave differently:

1) Fetching of the Packages file can close the connection early if the TIMESTAMP header has a value equal to the cached copy. The is optimized to avoid downloading the the whole (megabytes) of file when it has not been modified since it was previously downloaded. The relevant code parses the PORTAGE_BINHOST URI and generates ssh arguments as follows:

	if port is not None:
		port_args = ['-p', "%s" % (port,)]
	proc = subprocess.Popen(['ssh'] + port_args + \
		[user_passwd + host, '--', 'cat', path],
		stdout=subprocess.PIPE)

2) Meanwhile, fetching of the actual binary packages uses this command:

FETCHCOMMAND_SSH="bash -c \"x=\\\${2#ssh://} ; host=\\\${x%%/*} ; port=\\\${host##*:} ; host=\\\${host%:*} ; [[ \\\${host} = \\\${port} ]] && port=22 ; exec rsync --rsh=\\\"ssh -p\\\${port}\\\" -avP \\\"\\\${host}:/\\\${x#*/}\\\" \\\"\\\$1\\\"\" rsync \"\${DISTDIR}/\${FILE}\" \"\${URI}\""

It seems like we should be able to merge the proposed SSH_FETCH_ARGUMENTS value into the --rsh part of FETCHCOMMAND_SSH, and we can also merge it into the generated arguments in part (1). Does everyone think that will work?
Comment 5 Jason A. Donenfeld archtester Gentoo Infrastructure gentoo-dev Security 2013-05-16 17:33:48 UTC
(In reply to comment #4)
> It seems like we should be able to merge the proposed SSH_FETCH_ARGUMENTS
> value into the --rsh part of FETCHCOMMAND_SSH, and we can also merge it into
> the generated arguments in part (1). Does everyone think that will work?

Perfect.
Comment 6 Jason A. Donenfeld archtester Gentoo Infrastructure gentoo-dev Security 2013-05-16 17:36:43 UTC
(In reply to comment #3)
> The port options may be different depending upon what command you are using

Ahh, right, this. Shucks. At least there's "-oPort=2222", which works on all of them.
Comment 7 Zac Medico gentoo-dev 2013-05-17 01:43:22 UTC
If sftp doesn't have a way to pipe the file like ssh has, then there's no point having a separate kind of fetch optimized for the Packages file. So for sftp, we can include the SSH_FETCH_ARGUMENTS variable in the default FETCHCOMMAND_SFTP setting, and use that command for both the Packages file and the binary packages.
Comment 8 Zac Medico gentoo-dev 2013-05-17 02:10:24 UTC
Only use FETCHCOMMAND_SFTP for sftp support:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=75a524e9a53ef4dc941947cb99d34fdf1ccab6e4

Note that the current default FETCHCOMMAND_SFTP looks like this:

FETCHCOMMAND_SFTP="bash -c \"x=\\\${2#sftp://} ; host=\\\${x%%/*} ; port=\\\${host##*:} ; host=\\\${host%:*} ; [[ \\\${host} = \\\${port} ]] && port=22 ; exec sftp -P \\\${port} \\\"\\\${host}:/\\\${x#*/}\\\" \\\"\\\$1\\\"\" sftp \"\${DISTDIR}/\${FILE}\" \"\${URI}\""

We can update it to insert SSH_FETCH_ARGUMENTS after the port arguments.
Comment 10 Zac Medico gentoo-dev 2013-05-19 00:24:59 UTC
This is fixed in 2.1.12.1 and 2.2.0_alpha176.