Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 625246 - sys-apps/portage: binhost https If-Modified-Since header support
Summary: sys-apps/portage: binhost https If-Modified-Since header support
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Enhancement/Feature Requests (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks: 627566 668302
  Show dependency tree
 
Reported: 2017-07-15 21:02 UTC by ge1ger
Modified: 2020-09-06 18:36 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description ge1ger 2017-07-15 21:02:50 UTC
PORTAGE_BINHOST="http://example.com" emerge -G package

When the above is ran, http://example.com/Packages is redownloaded every time emerge is ran, even if it's the same exact file.

Is it possible to have emerge save this to a specific location and check timestamps to see if it needs to be redownloaded?

A workaround is possible by editing this file: 
/usr/lib/python3.4/site-packages/portage/dbapi/bintree.py
Comment 1 Zac Medico gentoo-dev 2017-07-15 21:54:52 UTC
For http protocol, we already support the If-Modified-Since header:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=e06cb6d66db37ac7ab77acf65038b1f770c13c96

For https protocol, If-Modified-Since support has been disabled since bug 469888:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=b5969af9f575e4e4b669f44e76ad01f0dbc2dd27
Comment 2 ge1ger 2017-07-15 22:38:22 UTC
Will date-checking work for https binhosts in the future?



There's the following solution for having cache in https:

https://p.sicp.me/bCETD.bash

FETCHCOMMAND_HTTP="curlcache.sh \"\${URI}\" \"\${DISTDIR}/\${FILE}\""
Comment 3 ge1ger 2017-07-15 22:43:55 UTC
Correction:

FETCHCOMMAND_HTTPS="curlcache.sh \"\${URI}\" \"\${DISTDIR}/\${FILE}\""
Comment 4 Zac Medico gentoo-dev 2017-07-16 23:18:57 UTC
Since python 2.7.9/3.4.3, python libraries perform ssl certificate and hostname checks by default:

https://docs.python.org/2/library/httplib.html#httplib.HTTPSConnection
https://docs.python.org/3/library/http.client.html#http.client.HTTPSConnection

This means that we can apply our existing If-Modified-Since support to https, as long as the python version is new enough.
Comment 5 Zac Medico gentoo-dev 2017-07-19 07:28:00 UTC
I've got a patch here:

https://github.com/zmedico/portage/tree/bug_625246