Hi all, With the recent rise of vendor tarballs for popular languages (Golang, NPM, Rust, etc), we're in a situation where rather than the occasional user needing a particularly large patch hosted by proxy it's not uncommon to receive queries from a contributor about hosting relatively large files. Currently we recommend that users upload vendor tarballs to a git forge (or anywhere else that can provide a stable URI), however a git forge not _really_ a suitable place to host binary files. I'm interested in seeing what solutions can be implemented that don't have significant overhead for Gentoo / the infra team. I'm imagining a single 'pool' where we'd enforce a structure that mirrors of ::gentoo/::guru, (i.e. `category/package/dist.file`), enforced via policy or technical means to enable easy (and automated) pruning of removed package artefacts. Some form of S3 (self-hosted or otherwise) behind a CDN + regular mirroring seems reasonable at a glance, handing out access key + secret key isn't particularly burdensome. I'm not particularly set on a particular technology as long as the barrier to entry is low, and continuing usage is straightforward once you have been provisioned. A manual process where users request file hosting access via a bug (or as part of their GURU / Proxy Maint bug?) would seem reasonable.
I have a strong preference (and obvious bias) for any solution to not include use of a proprietary, US-based cloud. There are free (libre+beer) solutions to consider (e.g. libre-centric hosts like opendev.org, but that might be overkill for this alone). Even beyond this though, this seems like a moderation and security nightmare. Are you going to inspect the packages uploaded to a *gentoo official* s3 account? Who is legally responsible if someone adds a backdoor to a dependency in some nonobvious way? At least having these downloaded from an unofficial location keeps Gentoo at arms length from a potential bad actor.
> Even beyond this though, this seems like a moderation and security nightmare. I agree, which is what I wanted to touch on with my own comments. I think there’s a lot of benefit to something like this, both as a GURU contributor and as someone looking in from the sidelines, but it’s also true that this would be basically an opaque monolith. I’m not sure what the -right- way to handle it is, nor am I comfortable suggesting one (don’t want to backseat drive too much :) but I think there’s probably a middle ground to go for between granting access on joining GURU and becoming a Gentoo developer. The rough area I’d go for (with regards to GURU, where I’m a little more familiar) would be for people who’ve made previous contributions and have shown to be actively maintaining packages. Well short of guru-trusted, but -a- bar to be cleared, at least.
