Newer portage versions delete the contents of the metadata cache in /var/cache/edb/. Now when doing a sync and the timestamps on the server and in the local repository are the same. The metadata is not regenerated and thus missing. This leads in long dependency calculations when issuing emerge for the first time. Is it possible to generate the cache even when the timestamps are identically, or deleting the cache after checking for the timestamps?
(In reply to comment #0) > Newer portage versions delete the contents of the metadata cache in > /var/cache/edb/. I should clarify this a bit, when the repository is synced the metadata cache is deleted before the actual syncing starts.
No comments on this, or is this just a low priority issue?
I think you're looking for the --metadata option. Here are the docs from `man emerge`: Transfers metadata cache from ${PORTDIR}/metadata/cache/ to /var/cache/edb/dep/ as is normally done on the tail end of an rsync update using emerge --sync. This process populates the cache database that portage uses for pre-parsed lookups of package data. It does not populate cache for the overlays listed in PORTDIR_OVERLAY. In order to generate cache for overlays, use --regen. In versions of portage >=2.1.5 the --metadata action is totally unnecessary unless the user has enabled FEATURES="metadata-transfer" in make.conf(5).
(In reply to comment #3) > I think you're looking for the --metadata option. Thanks for your reply! Has this option been the default before and turned off in newer versions of portage?
Yeah, it has changed in version 2.1.5. Here is the relevant part from the release notes: * The metadata-transfer feature is now disabled by default. This disables the "Updating Portage cache" routine that used to run at the tail end of each `emerge --sync` operation. If you use something like the sqlite module and want to keep all metadata in that format alone (useful for querying), enable FEATURES="metadata-transfer" in make.conf. You should also enable FEATURES="metadata-transfer" if you have any eclasses from PORTDIR_OVERLAY that override eclasses from PORTDIR (in this case, you may have disabled a relevant warning message by setting PORTAGE_ECLASS_WARNING_ENABLE="0" in make.conf).
Sorry for the spam, it was entirely my fault. This all happens because eix-sync is removing /var/cache/edb/dep [1]. I thought i have tested this with plain emerge --sync and got the same, but it works fine when i tested it again today as portage does not remove the metadata at the beginning of a sync. So when using eix-sync and the timestamp on the server is the same as on the local machine portage stops and does not recreate the portage cache. This is probably a bug in eix as it should first check for the timestamps and if they are the same it should not delete the depcache. By the way i use stable portage 2.1.4.4 so the disabled metadata-transfer does not affect me, though i should have given more info when filing the bug to avoid this misunderstanding. Just out of curiosity why is this feature disabled by default in versions >=2.1.5. In the man page which you mentioned above you say it is unnecessary. Does it now use the ${PORTDIR}/metadata/cache/ directly? [1] snippet from the eix manpage: portage-2.1 and portage-2.1.1 doesn’t remove the old dep-cache, thus eix would always find packages that are not in portage anymore. To circumvent this, eix-sync deletes the old cache (rm -rf /var/cache/edb/dep/*).
(In reply to comment #6) > Just out of curiosity why is this feature disabled by default in versions > >=2.1.5. In the man page which you mentioned above you say it is unnecessary. > Does it now use the ${PORTDIR}/metadata/cache/ directly? That's right. > So when using eix-sync and the timestamp on the server is the same as on the > local machine portage stops and does not recreate the portage cache. This is > probably a bug in eix as it should first check for the timestamps and if they > are the same it should not delete the depcache. OK, reassigning to eix maintainers.
I am eix maintainer, but I don't know how to solve this problem: Since eix-sync must delete /var/cache/edb *before* emerge --sync is called, there is no way to tell whether the timestamps *will* be up-to-date. However, have I understood correctly, that all new portage version will take care about cleaning up /var/cache/edb themselves? In this case, I think that I will change the default behavior of eix-sync to -R (i.e. to not remove /var/cache/edb at all) in the next eix-release, since only a few people will use a cache method of eix which needs the /var/cache/edb content, and moreover, those who do will have no problems with new portage versions. (And if they use older ones, they can still put -r into /etc/eix-sync.conf)
(In reply to comment #8) > However, have I understood correctly, that all new portage version will take > care about cleaning up /var/cache/edb themselves? Actually, it's gotten worse. Before, with FEATURES=metadata-transfer enabled, it called the equivalent of emerge --metadata with should have cleaned up stale entries in /var/cache/edb. Now, with FEATURES=metadata-transfer disabled by default, it doesn't even do that. As an alternative to cleaning up /var/cache/edb directly, you could call `emerge --regen -q`. It shouldn't take much time, as long as all the cache is valid (normally it is). The --regen action will remove stale cache in /var/cache/edb/dep for all currently configured repositories (those in PORTDIR and PORTDIR_OVERLAY). However it's possible that some stale cache will still be left behind from repositories that may have been previously configured (changed PORTDIR or PORTDIR_OVERLAY settings). Maybe it's still suitable for your purposes though.
(In reply to comment #9) > it called the equivalent of emerge --metadata I suppose that the eix cache method which accesses /var/cache/edb directly was probably only used by people who have non-permanent (e.g. temporarily net-mounted) overlays or non-permanent or non-existing /usr/portage/metadata like e.g. the folks from gentoo-alt. These people have to set FEATURES=metadata-transfer anyway (or will let eix-sync call emerge --metadata or even emerge --regen). So I guess it is safe enough if I just remove any treatment of /var/cache/edb and any calling of emerge --metadata from the eix-sync defaults and document for these people how they can setup eix-sync anyway that these things are executed.
With metadata-transfer feature being now disabled by default and portage not really needing /var/cache/edb/ besides for overlays the initial dependency calculation is not slowed down anymore. By now I consider this as a non issue so feel free to close this bug.
Closing this for now as the problem is gone.
