Summary: | sys-apps/portage-2.3.51-r1 - proceeds to sync with an outdated rsync mirror when timestamp.chk comparison fails (locale may trigger strptime ValueError) | ||
---|---|---|---|
Product: | Portage Development | Reporter: | Fab <netbox253> |
Component: | Unclassified | Assignee: | Portage team <dev-portage> |
Status: | CONFIRMED --- | ||
Severity: | normal | CC: | christian, esigra, M4rkusXXL |
Priority: | Normal | ||
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 240187 | ||
Attachments: | emerge --sync from old mirror |
Description
Fab
2018-11-30 09:08:36 UTC
I also ran into this once again. Was wondering about the many downgrades. According to the mirrorstats https://mirrorstats.gentoo.org/rsync/ there are quite some mirrors behind. I also think, that portage should reject to downgrade the tree and print a prominent error, also mentioning the used mirror. (In reply to Markus from comment #1) > I also think, that portage should reject to downgrade the tree and print a > prominent error, also mentioning the used mirror. Normally, comparison of the local and remote metdata/timestamp.chk files should prevent syncing with an older version of the repository. However, the if either the local or remote metdata/timestamp.chk file is missing the there's no protection. Could that be the cause of your problem? For reference, the timestamp comparison logic is here: https://gitweb.gentoo.org/proj/portage.git/tree/lib/portage/sync/modules/rsync/rsync.py?h=portage-2.3.51#n657 Created attachment 557386 [details]
emerge --sync from old mirror
Sync is not prevented. Any ideas?
I added a debug print: servertimestamp and timestamp are both always 0 at that point. For a good and for a bad mirror. outdated: # dir /usr/portage/metadata/timestamp* -rw-r--r-- 1 root root 29 2018-12-02 13:09 /usr/portage/metadata/timestamp -rw-r--r-- 1 root root 32 2018-12-02 13:15 /usr/portage/metadata/timestamp.chk -rw-r--r-- 1 root root 78 2018-12-02 13:09 /usr/portage/metadata/timestamp.commit -rw-r--r-- 1 root root 43 2018-12-02 13:09 /usr/portage/metadata/timestamp.x current: # dir /usr/portage/metadata/timestamp* -rw-r--r-- 1 root root 29 2018-12-09 10:39 /usr/portage/metadata/timestamp -rw-r--r-- 1 root root 32 2018-12-09 10:45 /usr/portage/metadata/timestamp.chk -rw-r--r-- 1 root root 78 2018-12-09 10:09 /usr/portage/metadata/timestamp.commit -rw-r--r-- 1 root root 43 2018-12-09 10:39 /usr/portage/metadata/timestamp.x Catching the exception in line 652: ValueError: time data 'Sun, 02 Dec 2018 12:15:01 +0000' does not match format '%a, %d %b %Y %H:%M:%S +0000' That being said, the follwing works as expected: LC_ALL=C emerge --sync So it needs to be worked around the locale dependency. Different ideas come to my mind: 1) Set locale to "C" before all "strptime" calls and restore afterwards. (Can can messy to catch every case that is locale dependend but shouldnt. Threadsafe?) 2) Get timestamp from metadata of file. (Dont know if this is guaranteed?) 3) Use an universial timestamp without a locale dependency. E.g. "%Y-%m-%d %H:%M:%S" or just "seconds since the epoch". (More work, as the whole interface around the timestamp file changes.) this one seems to have hit me during the past week, as rsync5.de.gentoo.org is outdated since about early Nov. 14.. (<- yes, I've already contacted the maintainer about the mirror issue) however, is this bug being actually worked on to globally fix this in Portage..? We certainly want to skip sync with the mirror if timestamp.chk is corrupt. If timestamp.chk is missing entirely, we might need a repos.conf flag to control the behavior, for example we could use a setting like sync-rsync-check-timestamp = false to make it ignore the timestamp.chk file. |