Bug 619400 – [R_Overlay] metadata and rsync

• Product: Gentoo Linux
• Status: RESOLVED WORKSFORME

[Description Alessandro Barbieri 2017-05-22 20:32:13 UTC]

This repositiry can be synces with rsync or git but is huge and I can't afford to rebuild often the metadata (rsync) and I don't want the history (git). It is possible to add metadata to rsync in the same way as used in the main repository?

[Comment 1 Jonas Stein 2017-05-22 21:02:01 UTC]

Alessandro, please do not use a bug report for questions or discussions.
You will find the best help available on our support channels https://www.gentoo.org/support/

[Comment 2 Alessandro Barbieri 2017-05-22 21:05:54 UTC]

Is not a question, is a request to add metadata to rsync

[Comment 3 Jonas Stein 2017-05-22 21:14:18 UTC]

Sorry, I think I misunderstood you... This was a feature request.

[Comment 4 Tim Harder 2017-05-23 11:24:56 UTC]

Can't you just shallow clone from https://github.com/gentoo-mirror/R_Overlay? Pretty much all public overlays registered for use with layman should be available under the gentoo-mirror org on github with metadata pre-generated.

[Comment 5 Benda Xu 2017-05-23 12:32:56 UTC]

Thanks, one way out is to automatically get rid of outdated ebuilds.

BTW, the metadata are generated in the overlay.  Which URI are you using?

@Tim, thanks for the pointer.  I was not aware of the overlay mirrors.

[Comment 6 Alessandro Barbieri 2017-05-23 12:48:36 UTC]

I'm using it and I think that shallow clone is the default now
https://bugs.gentoo.org/show_bug.cgi?id=552814, but sometimes git do dirty things and does not want to pull and so I have to manually clean etc.
see https://github.com/vaeth/portage-postsyncd-mv/issues/1

Switching to rsync is cleaner but metadata is not served similar as the main repo.

with rsync the default one in layman
with git https://github.com/gentoo-mirror/R_Overlay.git

don't remember if the metadata is served with rsync but I remember that using it led to rebuild often the cache locally

[Comment 7 Alessandro Barbieri 2017-05-23 21:03:45 UTC]

I confirm that both methods provides metadata, but not the same metadata ...

for example I just cloned and synced the two repos and

diff -r R_Overlay/ roverlay/

spits out alot of things for example:

diff -r R_Overlay/metadata/md5-cache/sci-BIOC/CopyNumber450kData-1.6.0 roverlay/metadata/md5-cache/sci-BIOC/CopyNumber450kData-1.6.0
11,12c11
< _eclass_exported_funcs=pkg_postinst:R-packages src_unpack:R-packages src_prepare:R-packages src_compile:R-packages src_install:R-packages
< _eclasses_=R-packages 973a1b065237e8f79f6e6123ef7a0ff6        epatch  8233751dc5105a6ae8fcd86ce2bb0247        estack  43ddf5aaffa7a8d0482df54d25a66a1f        eutils 9c113d6a64826c40154cad7be15d95ea ltprune 2770eed66a9b8ef944714cd0e968182e        multilib        97f470f374f2e94ccab04a2fb21d811e        toolchain-funcs 6eb35f81556258a4bc9182ad3dfd58ee
---
> _eclasses_=R-packages 973a1b065237e8f79f6e6123ef7a0ff6        epatch  8233751dc5105a6ae8fcd86ce2bb0247        epunt-cxx       f8073339d152f56626493f43d8ba4691       estack   43ddf5aaffa7a8d0482df54d25a66a1f        eutils  e7491451996e8b54c3caeb713285b0fb        ltprune 2770eed66a9b8ef944714cd0e968182e        multilib        0236be304ee52e7f179ed2f337075515        toolchain-funcs 6eb35f81556258a4bc9182ad3dfd58ee

[Comment 8 Benda Xu 2017-05-24 05:29:55 UTC]

Hi, I am out of ideas why this difference occur.  It might come from the difference in the ways metadata are generated in rsync and git.

The Overlay team is maintaining the git R overlay.  @Overlay team, could you please remind me how the metadata are generated in the gentoo github mirror of overlays?

On the rsync side, it is basically an `egencache --update`.

[Comment 9 Michał Górny 2017-05-24 06:33:34 UTC]

pmaint regen --use-local-desc --pkg-desc-index R_Overlay

[Comment 10 Alessandro Barbieri 2017-05-31 15:31:03 UTC]

Since is working fine I'll close for now.