Due to a corporate firewall that blocks rsync, I rely on emerge-webrsync and the portage snapshots to update. I have one "master" machine which runs emerge-webrsync to update it's portage tree; and a number of other machines that then use "emerge sync" against the master (which is running rsyncd). The issue is that whatever mechanism is being used to generate the portage snapshot tarball causes each file in the tarball to be timestamped with the tarball creation date/time - this is clearly obvious if I use "tar jxf portage-<date>.tar.bz2 This, in turn, means that the local "rsync" processes (called from both emerge-webrsync and emerge sync) that compare this regenerated tree against the "local" tree (/usr/portage) see EVERY SINGLE FILE has changed, and thus causes the entire tree to be rebuilt. This is not very efficient, especially when performed across multiple pcs. Reproducible: Always Steps to Reproduce: 1. emerge-webrsync. Observe that the ntire portage tree is replaced by the rsync command (fairly obvious from viewing the output). Expected Results: The snapshot generation process sbould ideally keep the date/time stamps that are in the tree that is normally used by "emerge rsync" users - whilst I'd still have to download at 15Mb tarball, at least the local rsync process would be faster.
I made a slight adjustment to the script that creates the snapshots -- we'll see if that helps. If it doesn't, here's the relevant guts of the script. You are welcome to submit patches that gives it the behavior you want: (the change I made was adding the -p option to the /bin/cp command) ------------------------------------- # # create a copy of the gentoo-x86 repository to a temp location # /bin/mkdir -p ${TEMP} /bin/cp -p -R ${MASTER} ${TEMP}/portage /bin/chown -R 250:250 ${TEMP}/portage cd ${TEMP} # # create the tarball and move it to the right location # tar --exclude=CVS -cjf ${FILENAME} portage /usr/bin/md5sum ${FILENAME} > ${FILENAME}.md5sum /bin/mv ${FILENAME} ${FILENAME}.md5sum ${UPLOAD} -------------------------------------
(Sorry for delay in responding - I didn't get a Bugailla notification of the update to this bug) Hi Kurt, Thanks for that change - it does exactly what I'd like it to, and works really well. Nice to see rsync taking 15 seconds instead of 15 minutes per machine....
someone removed the -p option in this script...not sure who or why. I put it back, added a big note asking folks not to change it without talking to me and chattr +i'd the file. --kurt
after a discussion in IRC on the nuances of cp, it sounds like -a may be a safer option that still provides the desired behavior of preserving timestamps. So...I changed -p to -a. Just noting that here in case something changes in the future.
