if I set PORTAGE_BINHOST="sftp://user:passwd@mymachine" portage fails saying sftp is not a valid transport. We do not use ftp on our site, so it would be really useful to be able to use sftp as a transport Reproducible: Always
Created attachment 120600 [details, diff] use the paramiko library to add sftp support for PORTAGE_BINHOST One thing missing is that RESUMECOMMAND is used to fetch the package, but the default RESUMECOMMAND (defined in /etc/make.globals) uses wget which unfortunately doesn't support sftp. Do you know of a program that will support sftp in addition to http and ftp? You can write a script which checks the protocol and calls a different program if necessary, but that's not quite ideal. I suppose we could implement the fetch using python libraries and just not use RESUMECOMMAND at all, but that's not really ideal either.
(In reply to comment #1) > Created an attachment (id=120600) [edit] > use the paramiko library to add sftp support for PORTAGE_BINHOST > > One thing missing is that RESUMECOMMAND is used to fetch the package, but the > default RESUMECOMMAND (defined in /etc/make.globals) uses wget which > unfortunately doesn't support sftp. Do you know of a program that will support > sftp in addition to http and ftp? You can write a script which checks the > protocol and calls a different program if necessary, but that's not quite > ideal. I suppose we could implement the fetch using python libraries and just > not use RESUMECOMMAND at all, but that's not really ideal either. Well, fetch() supports protocol specific variables like FETCHCOMMAND_SFTP and so on, so could use those to be consistent.
Zac: any reason not to commit that patch?
I think the patch is fine but it's incomplete until we fix binarytree.gettbz2() to share code with portage.fetch() to enable FETCHCOMMAND_SFTP support.
Created attachment 123003 [details, diff] portage-2.1.3_rc5 patch for paramiko + {FETCHCOMMAND,RESUMECOMMAND}_SFTP This patch should provide complete sftp support.
Created attachment 123020 [details, diff] portage-2.1.3_rc5 patch for paramiko + {FETCHCOMMAND,RESUMECOMMAND}_SFTP This version fixes a couple bugs. It seems to work fine.
Created attachment 123021 [details] python script for use in {FETCHCOMMAND,RESUMECOMMAND}_SFTP I had trouble finding a good sftp client program so I wrote this little python script that uses the paramiko library. The options are similar to wget: $ sftp.py --help usage: sftp.py [OPTION]... [URL]... options: -h, --help show this help message and exit -O FILE, --output-document=FILE save uri to FILE -c, --continue continue getting a partially-downloaded file -b BLOCKSIZE, --blocksize=BLOCKSIZE block size used for read/write -q, --quiet disable output The following settings in make.conf will work: FETCHCOMMAND_SFTP="sftp.py -O \${DISTDIR}/\${FILE} \${URI}" RESUMECOMMAND_SFTP="sftp.py -c -O \${DISTDIR}/\${FILE} \${URI}"
This has been released in 2.1.3_rc6.
In portage-2.1.9.12 there's ssh and sftp support for the newer binhost protocol: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=d9c4a7dec291f3f57296a1bf249887fa950bbcc1 http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=9065f86cf61ae69968e7d1155d7ecb89688c037c http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=a0c2fa6dd742a273d328238604594a78107581db