In my gentoo system, 150k of 340k files are due to portage (over 40%). This is just an enourmous number or tiny files to store in the filesystem and over time they seem to get spread out across the drive, leading to progressively slower performance of emerge sync and search operations. Reproducible: Always Steps to Reproduce: 1. emerge sync Actual Results: Lots and lots of files. Expected Results: At least /usr/portage/*-* files should be in a single file. It would also be good if /var/db/pkg/* and /var/cache/edb/dep/* were also in each in one file. Suppose the portage files are stored in a normal .zip file without compression: * Sync: Probably less overhead on sync servers since sending the complete file list and per file rsync overhead is avoided. Order of magnitude faster update on client side, since file access is the slow part. Still only sends only the xdelta of what changed if they are stored the same way in the .zip each time. * Smaller footprint on client: 168 mb portage on ext3 vs 111 mb portage.zip with no compression (on reiser 4 it is still 128 mb for the separate file), 120k fewer files. * Random access: unzip portage.zip sys-devel/make/* is basically instant (it doesn't scan the whole archive like tar would since zip stores an index at the end). * Python support: python has standard libraries for reading a .zip file, so the current code (to index the portage metadata, etc) should be able to read from the portage.zip with relatively few changes. * Still user friendly: simply have 'emerge search', etc use /usr/portage if present, or portage.zip otherwise. User can simple "unzip portage.zip" and make whatever local changes they want just like now.
or much much LATER unless you are willing to supply a patch for portage, emerge, repoman, gentoolkit, gentoolkit-dev, porthole and most of the other apps in app-portage. Until portage offers a way for all apps to get access to all possible information without going directly, it's simply not realistic. Until most apps are updated to use portage, it's simply not nice.
