Bug 223989 – Portage Peer-2-Peer support

• Product: Portage Development
• Status: RESOLVED DUPLICATE

[Description ml 2008-05-28 18:54:58 UTC]

Hi,

Gentoo has a huge distribution infrastructure, but depends on voluntarily provided Rsync servers.

Using a P2P infrastructure by adding a P2P client to portage would improve distribution capacity/bandwith a lot and reach the next step in community support (each user spends bandwith).


PROs:

- higher distribution bandwith capacities
- more independence of bandwith capacity from volunteers
- more independence of bandwith costs
- no central nodes, which can be DoS attacked
- like Rsync reduced traffic by checksumming/hashes


CONs:
- little more complexity


I'd suggest to use a slim Bittorrent client with VHT support. That way Gentoo can control efficient use of bandwith by trackers and VHT keeps the P2P distribution system working in case the trackers fail (e.g. DoS attack).

Additionally the current Rsync system can backup the P2P system (e.g. problems with firewalls, ISPs, ...)


Best regards,

Renne


Reproducible: Always

Steps to Reproduce:



Expected Results:  
Very high performance ;)

[Comment 1 Marius Mauch (RETIRED) 2008-05-28 19:26:19 UTC]

P2P systems are usually designed for single-file transfers and therefore would not be a substitute for rsync (syncing a directory tree, only transfer changes).

If you're referring to distfile mirrors, this has been handled in bug #8468

*** This bug has been marked as a duplicate of bug 8468 ***

[Comment 2 ml 2008-05-28 19:38:23 UTC]

1.) Bittorrent metafiles can handle complete directory structures with multiple files
2.) I want to provide everything with P2P (stages, Portage snapshots, distfiles, GRPs, ...)

[Comment 3 Marius Mauch (RETIRED) 2008-05-28 22:04:12 UTC]

Well, after discussing this a bit with our infra group it doesn't seem very beneficial compared to the current system, so there isn't much interest in it from our side. Maybe when we've refactored the sync code to be more modular instead of the current mess we can revisit this, but at the moment we aren't really interested in it.
Of course you could provide a patch to add support for additional sync mechanisms and we'd review and probably apply it, as long as it doesn't affect existing functionality in a negative way (IOW: has to be strictly optional). However for the most part my comment from bug #8468 still stands that this is something to be handled via external commands and $FETCHCOMMAND.