Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 829085 - emerge --sync does not update portage timestamp for last sync
Summary: emerge --sync does not update portage timestamp for last sync
Status: UNCONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: AMD64 Linux
: Normal enhancement (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-12-13 12:44 UTC by DanF
Modified: 2021-12-23 23:21 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description DanF 2021-12-13 12:44:47 UTC
I run `eix-sync` and noticed that I get " * Last emerge --sync was 59d 12h 2m 26s ago." which I found strange as I knew for a fact I had updated my checkout. I `strace`d emerge to find `/usr/portage/metadata/timestamp.x` is not updated on `eix-sync`. Here's a reproduction of the behaviour.

###########################################################
root@localhost:~# cat /usr/portage/metadata/timestamp.x
1634258101 Fri 15 Oct 2021 12:35:01 AM UTC
root@localhost:~# eix-sync
 * Running emerge --sync
>>> Syncing repository 'gentoo' into '/usr/portage'...
/usr/bin/git fetch origin
Already up to date.
=== Sync completed for gentoo
 * Updating DTDs ...                                                                                                                                                                                        [ ok ]
 * Updating GLSAs ...                                                                                                                                                                                       [ ok ]
 * Updating news items ...                                                                                                                                                                                  [ ok ]
 * Updating projects.xml ...                                                                                                                                                                                [ ok ]
>>> Syncing repository 'localrepo' into '/var/db/repos/localrepo'...
/usr/bin/git fetch origin
Already up to date.
=== Sync completed for localrepo

Action: sync for repo: gentoo, returned code = 0
Action: sync for repo: localrepo, returned code = 0


 * Main gentoo tree does not appear to have changed: exiting
 * Use -a or set have_changed=: in a ! hook to override this check
 * Time statistics:
     3 seconds for syncing
     3 seconds total
root@localhost:~# cat /usr/portage/metadata/timestamp.x
1634258101 Fri 15 Oct 2021 12:35:01 AM UTC
root@localhost:~# emerge portage -1 -pv
 * Last emerge --sync was 59d 12h 2m 26s ago.

These are the packages that would be merged, in order:

Calculating dependencies... done!
[binary   R    ] sys-apps/portage-3.0.28-r1-1::gentoo  USE="(ipc) native-extensions rsync-verify (xattr) -apidoc -build -doc -gentoo-dev (-selinux) -test" PYTHON_TARGETS="python3_8 python3_9 (-pypy3) (-python3_10)" 0 KiB

Total: 1 package (1 reinstall, 1 binary), Size of downloads: 0 KiB
###########################################################

I posted the output of an `eix-sync` run after I had previously run it that same day (also to reduce the size of what I post here).

Ideally `eix-sync` would also update the timestamp once it finishes so `emerge` doesn't complain that the repo hasn't been updated when it has been, in fact, updated.
Comment 1 Martin Väth 2021-12-14 16:12:27 UTC
Not related with eix. Whichever tool/script/hooks you use to sync the main portage tree is responsible for this. This is not a task for the eix-sync script.
If you want, you can write a manual hook for eix-sync, but I would not recommend it.
Comment 2 DanF 2021-12-22 22:32:13 UTC
The sync is done over git from https://github.com/gentoo-mirror/gentoo.git (`eix-sync` calls `emerge --sync`). To alleviate any concerns about `eix-sync` I ran `emerge --sync` directly and got the same result.

###############################################################
# emerge --sync                                                                                                                                                                                                   
>>> Syncing repository 'gentoo' into '/usr/portage'...                                                                                                                                                            
/usr/bin/git fetch origin                                                                                                                                                                                         
remote: Enumerating objects: 14, done.                                                                                                                                                                            
remote: Counting objects: 100% (14/14), done.                                                                                                                                                                     
remote: Compressing objects: 100% (4/4), done.                                                                                                                                                                    
remote: Total 14 (delta 9), reused 14 (delta 9), pack-reused 0                                                                                                                                                    
Unpacking objects: 100% (14/14), 3.04 KiB | 29.00 KiB/s, done.                                                                                                                                                    
From https://github.com/gentoo-mirror/gentoo
   0e77efef07bb..c1a26281fe2e  stable     -> origin/stable
   0e77efef07bb..c1a26281fe2e  master     -> origin/master
Updating 0e77efef07bb..c1a26281fe2e
Fast-forward
 dev-ada/libadalang/libadalang-22.0.0.ebuild  | 4 +++-
 metadata/md5-cache/dev-ada/libadalang-22.0.0 | 6 +++---
 metadata/timestamp.chk                       | 2 +-
 3 files changed, 7 insertions(+), 5 deletions(-)
=== Sync completed for gentoo
 * Updating DTDs ...                                                                                                                                                                                        [ ok ]
 * Updating GLSAs ...                                                                                                                                                                                       [ ok ]
 * Updating news items ...                                                                                                                                                                                  [ ok ]
 * Updating projects.xml ...                                                                                                                                                                                [ ok ]
###############################################################

In the git checkout there's a `timestamp.chk` which gets updated on `emerge --sync` but `timestamp.x` is the file that `emerge` is looking for (based on the `strace`) for last updated timestamp. I've updated to `sys-apps/portage-3.0.28-r1-1::gentoo` and I still have the same behaviour.

###############################################################
# emerge portage -1 -pv                                                                                                                                                                           
 * Last emerge --sync was 68d 20h 57m 17s ago. 
###############################################################

This is the layout of the metadata dir.
###############################################################
# ls -lash /usr/portage/metadata/
total 2.6M
   0 drwxr-xr-x   9 portage portage  333 Dec 22 21:30 .
 12K drwxr-xr-x 179 portage portage 8.0K Dec 22 20:44 ..
4.0K -rw-r--r--   1 portage portage  237 Jan 28  2021 .gitignore
4.0K -rw-r--r--   1 portage portage  896 Feb  2  2019 AUTHORS
4.0K -rw-r--r--   1 portage portage 3.5K Oct 15 00:39 Manifest.gz
   0 drwxr-xr-x   2 portage portage  189 Oct 15 00:39 dtd
948K drwxr-xr-x   2 portage portage 916K Oct 15 00:39 glsa
   0 drwxr-xr-x   2 portage portage  196 Oct 15 00:39 install-qa-check.d
4.0K -rw-r--r--   1 portage portage 2.3K Oct 21 15:08 layout.conf
 12K drwxr-xr-x 171 portage portage 8.0K Dec 22 20:44 md5-cache
 12K drwxr-xr-x  83 portage portage 8.0K Nov 28 22:07 news
1.5M -rw-r--r--   1 portage portage 1.5M Dec 22 20:44 pkg_desc_index
124K -rw-r--r--   1 portage portage 121K Dec 22 20:44 projects.xml
4.0K -rw-r--r--   1 portage portage 2.3K Aug  8 23:39 qa-policy.conf
   0 drwxr-xr-x   2 portage portage   72 Oct 15 00:39 repoman
4.0K -rw-r--r--   1 portage portage   32 Oct 15 00:39 timestamp
4.0K -rw-r--r--   1 portage portage   32 Dec 22 21:23 timestamp.chk
4.0K -rw-r--r--   1 portage portage   78 Oct 15 00:39 timestamp.commit
4.0K -rw-r--r--   1 portage portage   43 Oct 15 00:39 timestamp.x
   0 drwxr-xr-x   2 portage portage  150 Oct 15 00:39 xml-schema
###############################################################

Also in the gentoo mirror only the timestamp.chk is present https://github.com/gentoo-mirror/gentoo/tree/stable/metadata - perhaps there was a change in what got sync'ed into that repo that caused this?
Let me know if there's anything else I can try.
Comment 3 DanF 2021-12-23 23:21:27 UTC
After some more digging through the code I found that for both webrsync https://github.com/gentoo/portage/blob/8851a437c7edc7ae01ba04dbff42b27e1f36b37b/bin/emerge-webrsync#L147 and delta-webrsync https://github.com/gentoo/portage/blob/8851a437c7edc7ae01ba04dbff42b27e1f36b37b/misc/emerge-delta-webrsync#L228 if the timestamp is not present it defaults to 0 regardless of when the last sync was done, which is most likely an error.

This is also referenced in https://github.com/gentoo/portage/blob/master/lib/portage/sync/old_tree_timestamp.py#L62 which gives the "Last emerge --sync was" message, but only if the file is present. `old_tree_timestamp()` is also used in https://github.com/gentoo/portage/blob/0e9e12aadb889766d61c0561b9723e71542d43e6/lib/_emerge/actions.py#L128-L129 but in this case the missing `timestamp.x` file would not have a default value (like webrsync and delta-webrsync).

It seems `timestamp.chk` https://github.com/gentoo/portage/blob/8851a437c7edc7ae01ba04dbff42b27e1f36b37b/bin/egencache#L1171-L1172 is used for `rsync` but does not have the same format as `timestamp.x`.

Desired workflow update: `emerge --sync` should update `timestamp.x` on each successful run.