Currently this is a bunch of interlinked bash+Python scripts that handle all repositories from repositories.xml, with some extra hooks specific to gentoo.git. Current cronjob interval: 20 minutes (might be reasonable to lower it if hardware can handle it) Dependencies: pkgcore (with non-upstreamable patches), PyGithub Moderate CPU usage with spikes on major changes (cache generation) Disk space: ~1.5T on btrfs at the moment (includes 3 copies of all repos) Expect lot of network traffic (I have 8/12G RX/TX for 2 days uptime) Description: That's a bunch of interlinked scripts that, in order: 1. sync all repos from repositories.xml, 2. do minimal QA checks (valid repo_name, masters), 3. rsync repos from syncing copies to working copies, 4. generate cache for all repos that synced successfully, 5. create new repos / removes old repos for the mirrors via GitHub API, 6. rsync working copies to mirror repos, commit and push. This process takes 6-7 minutes currently but I think it's mostly due to poor network latency of the current host. The service runs fully unprivileged, and doesn't require any specific host setup. However, it requires running patched pkgcore since we need access to some special properties (like various sources of repository name) + upstream slacks horribly at accepting patches. I am using virtualenv to keep the local copy so far. Scripts: https://github.com/mgorny/repo-mirror-ci (repos.job is central) pkgcore branch (I keep rebasing it locally, so might need a push): https://github.com/mgorny/pkgcore/tree/bonedaddy
