Recently I switched my binpkg setup from 'FEATURES="getbinpkg"+ssh' to '"--usepkg"+autofs+sshfs' because I wanted bi-directional access to the remote binpkgs (If I do compile things on my laptop then I want to upload those binpkgs to the same binhost). I noticed a *massive* performance decrease in the new setup compared the old one, even though the underlying mechanism for transferring the files is still ssh. With getbinpkg+ssh emerge would only hang for a bit after dependency resolution and before starting the merge while it was fetching the binpkgs from the remote binhost. Now with usepkg+sshfs emerge hangs for several minutes immediately after executing any "emerge ...." command (e.g. emerge --search is also effected, but eix is not), I don't know what it is doing but it is definitely accessing things in /var/cache/binpkgs because unmounting that directory solves the hanging problem. I have a hunch that the problem is that with usepkg emerge is somehow reading every single xpak in /var/cache/binpkgs, whereas with getbinpkg it relies on the Packages file. In any case, something is causing emerge to be way less efficient in terms of accessing the binpkgs with --usepkg compared to --getbinpkg. This kinda breaks having the binpkg directory on a NAS. Since even something as simple as an "emerge --search" now takes several minutes whereas eix can still accomplish the same result in less than a second. And it shouldn't be necessary because at this stage (right after pressing enter) there is no need to access any file in /var/cache/binpkgs apart from the Packages file right? Would love to help out with debugging the problem further, but I'm not sure where to start.
> I have a hunch that the problem is that with usepkg emerge is somehow reading > every single xpak in /var/cache/binpkgs, whereas with getbinpkg it relies on > the Packages file. In any case, something is causing emerge to be way less > efficient in terms of accessing the binpkgs with --usepkg compared to > --getbinpkg. Using lsof I confirmed my hunch. After simply calling "emerge" (no arguments or anything) I can see the emerge process slowly access every single xpak in /var/cache/binpkgs. Calling "emerge --ignore-default-opts" (which ignores --usepkg) immediately returns the --help as expected. Why does emerge do this? I thought the whole point of the Packages file was to avoid having to access every file individually?
You can enable FEATURES=pkgdir-index-trusted to solve this issue.
(In reply to Zac Medico from comment #2) > You can enable FEATURES=pkgdir-index-trusted to solve this issue. Out of interest, why not enable this by default? I can't speak for everyone, but all of my Gentoo environments that consume binpkgs can trust their binhost.
(In reply to Zac Medico from comment #2) > You can enable FEATURES=pkgdir-index-trusted to solve this issue. Thank You! This is exactly what I'm looking for. Emerge is way faster with this feature enabled (a couple of seconds vs several minutes). I'll add a note about this FEATURE to this wiki page[1] because I'd imagine that other people with PKGDIR on NFS/sshfs/samba will also want to enable this. [1] https://wiki.gentoo.org/wiki/Binary_package_guide#NFS_exported
(In reply to John Helmert III from comment #3) > (In reply to Zac Medico from comment #2) > > You can enable FEATURES=pkgdir-index-trusted to solve this issue. > > Out of interest, why not enable this by default? I can't speak for everyone, > but all of my Gentoo environments that consume binpkgs can trust their > binhost. When FEATURES=pkgdir-index-trusted is not enabled, the behavior is to automatically scan for packages that have been manually added or removed. When FEATURES=pkgdir-index-trusted is enabled, it's necessary to run `emaint --fix binhost` if any packages have been manually added or removed. I think it would probably be fine to enable FEATURES=pkgdir-index-trusted, though there is some possibility that it would surprise some people that manually add or remove packages without running `emaint --fix binhost`.
