like this: 142500 <+CIA-30> angelos * r206 proj/xfce/xfce4-svn/xfce-base/xfconf/ (. ChangeLog Manifest metadata.xml xfconf-9999.ebuild): 142500 <+CIA-30> Initial import of xfconf 142500 <+CIA-30> (Portage version: 2.2_rc1/svn/Linux 2.6.25-gentoo-r4 x86_64) 142500 <+CIA-30> angelos * r207 proj/xfce/xfce4-svn/xfce-base/xfconf/Manifest: 142500 <+CIA-30> Initial import of xfconf 142504 <+CIA-30> (Portage version: 2.2_rc1/svn/Linux 2.6.25-gentoo-r4 x86_64) 142505 <+CIA-30> (Signed Manifest commit) even though there are no changed headers on svn and therefore no need to commit the Manifest file twice
On CVS you don't see this happening because the Manifest only commit message is ignored. overlays unfortunately doesn't, resulting in huge commit spam. Originally I had a hack around this, but for the sake of being generic, we removed the hack when it was pushed into the trunk.
Well yeah, but I'm not complaining about the spam - there's just no need to commit twice, the Manifest will be the same before and after the commit, so it could also be signed on the first commit with the second commit being removed for svn.
you're assuming that no svn keyword expansion is happening ... i think having repoman parse svn and try and guess whether this will happen is the wrong way to go. i imagine the right answer here is scm agnostic: have portage recompute the manifest and only commit if it's changed.
Well, the hack I did initially was to have repoman commit the Manifest in the first commit, then repoman did regenerate it, and svn would see nothing had changed. Because in the general case repoman skips committing the Manifest in the first batch, you get two commits, always. In the svn case you can look for the keyword expansion property on to be committed files, or just leave it as is, take the extra commit, and make sure the cia scripts ignore that commit.
except with svn (and actually cvs), you can control which keywords get expanded. existence of a property doesnt not mean the file actually contains the keywords. so this could actually see use with cvs as well, just on a very tiny scale in the gentoo-x86 tree
I guess that the decision to not commit the Manifest in the first commit, is to half the number of revisions on the Manifest files in the (CVS) tree. However, doing so does not affect anything, IMO. What we could do is commit Manifest with the first commit, and only exclude it from the first commit based on a simple heuristic, that probably covers 90-99% of the gentoo-x86 commits. If the heuristic fails, the tree is still perfectly fine, see above. The simple heuristic would be to check for any known file that usually has a keyword (*.ebuild?) and is modified if keyword expansion is enabled. If so, exclude Manifest from the first commit. In gentoo-x86/CVS this should result in Manifest being excluded from the first commit on most commits, since the ebuilds are touched. It could optimise a single commit in case a patch or init.d file is modified without touching an ebuild. In such case the first commit includes the Manifest, and the regenerated Manifest won't differ, and cause CVS not to commit anything in the second commit. For most SVN repositories the same would happen even for changed ebuilds since no keywords expansion is active on them. Makes sense for anyone? I guess I could even give this a shot to implement it, if people like the idea.
I gave this a shot in r11211
... and the off-by-one fix in r11213
Created attachment 161840 [details, diff] repoman commit manifest patch Here are the extracted revisions that make up my patch. I've been using it with SVN for a while now on ebuilds without any keyword expansion (typically those I've imported from CVS) and ebuilds with (Prefix only ebuilds), and repoman correctly commits in one time, or with the Manifest afterwards. I haven't checked it on CVS. If anyone would like to volunteer...?
Created attachment 161843 [details, diff] final repoman commit manifest patch uhm... correct patch with all revisions in it...
Thanks, I've committed your patch with a couple modification: * For clarity, use separate "no_expansion" variable for cvs bin blobs. * Fix svn keyword parsing to properly handle multiple keywords delimited by newlines.
This is fixed in 2.2_rc6.
This is again an issue. After some debugging, it seems that svn status doesn't list Manifest (although it should have been regenerated according to the messages), resulting in mymanifests being empty, hence it won't get committed in the first go. Further testing reveals that doing on a clean dir: % echo >> the.ebuild && repoman manifest && repoman commit will result in the Manifest committed in the first commit, whereas % echo >> the.ebuild && repoman commit will result in the Manifest not being seen by svn status, although repoman says it regenerated the Manifest.
(In reply to comment #13) > will result in the Manifest not being seen by svn status, although repoman says > it regenerated the Manifest. This should be fixed in svn r11692. The paths of the auto-added Manifests were stored in the "mynew" list and they were not being moved to the "mymanifests" list like they should have been.
my sync run with 2.2.00.11694-prefix shows double commits, so I'm affraid this didn't fix it
I think this is fixed in 2.2_rc13 (fix merged from prefix branch r11755). Can somebody confirm?
I made a possible fix in -r11755, but I can't confirm since Portage is broken in Prefix, getting somehow a list passed to a thing that should be a single string: File "/Library/Gentoo//usr/lib/portage/pym/portage/dbapi/vartree.py", line 945, in getSoname raise KeyError("%s not in object list" % obj) KeyError: "['/Library/Gentoo/usr/lib/libxerces-c.28.0.dylib', '/Library/Gentoo/usr/lib/libxerces-c.28.dylib'] not in object list" It worked for me in when I was hacking live repoman, so I assume we can close it.
(In reply to comment #17) > File "/Library/Gentoo//usr/lib/portage/pym/portage/dbapi/vartree.py", line > 945, in getSoname > raise KeyError("%s not in object list" % obj) > KeyError: "['/Library/Gentoo/usr/lib/libxerces-c.28.0.dylib', > '/Library/Gentoo/usr/lib/libxerces-c.28.dylib'] not in object list" That's not really a list, it's an instance of LinkageMap._LibGraphNode which is supposed to represent a preserved library. When LinkageMap.rebuild() is called, it's supposed to call scanelf on all your preserved libraries. It seems like LinkageMap.rebuild() has neglected to call scanelf on libxerces-c.28.dylib for some reason. > It worked for me in when I was hacking live repoman, so I assume we can close > it. Ok, reopen later if necessary.
(In reply to comment #18) > That's not really a list, it's an instance of LinkageMap._LibGraphNode which is > supposed to represent a preserved library. When LinkageMap.rebuild() is called, > it's supposed to call scanelf on all your preserved libraries. It seems like > LinkageMap.rebuild() has neglected to call scanelf on libxerces-c.28.dylib for > some reason. I was missing _LibGraphNode in LinkageMapMachO, now it works again, thanks. > > It worked for me in when I was hacking live repoman, so I assume we can close > > it. > > Ok, reopen later if necessary. Confirmed to work.
Somewhat related to this bug, I was contacted on irc by someone reporting separate Manifest commits (not double). See r862 and r863: http://code.google.com/p/gentoo-china-overlay/source/list It would be nice if we could detect when a separate commit is not necessary (when there is no keyword expansion in the header) and commit the Manifest together with all of the other files.
(In reply to comment #20) > Somewhat related to this bug, I was contacted on irc by someone reporting > separate Manifest commits (not double). See r862 and r863: I'm told that it already does single commits when signing is not enabled, so I guess that problem is triggered by FEATURES=sign.
Correct. The manifest in above mentioned commit is signed, and repoman takes a two-way path for that. I have no idea if that can be done in one go easily.
