At the moment emerge runs scripts from /etc/portage/postsync.d/ after each repo gets synced. My proposal is to run those scripts only once after all repos are done syncing. Reproducible: Always
Hmm, I have the new plugin-sync branch almost ready for final review and merge into portage master. Could you please get a checkout of that branch and test run it. There have been a number of changes to the syncing. It would be nice to know if it too behaves the same. Easiest way to install it would be to modify the 9999 ebuild, set EGIT_BRANCH='plugin-sync'. I have some testing to do after rebasing it on the updated master, then push it.
(In reply to Brian Dolbec from comment #1) > Could you please get a checkout of that branch and test run it. There have > been a number of changes to the syncing. It would be nice to know if it too > behaves the same. It behaves the same, because in pym/portage/emaint/modules/sync/sync.py it calls sync_manager.sync for each repo, and in pym/portage/sync/controller.py you can see that the post_sync hook is called for each sync call (with repo.sync_uri passed as the first argument to the hook).
Ok, I've looked over that new code. I remember some people stating they have post-sync hooks for certain repos to perform some actions on it like modifying times... So, we need to keep the individual post-sync hooks operational. What those people do is read the url value passed to it and act only if it is the desired url value. What I think we could do is easily add a final sync_manager.perform_post_sync_hook('PORTAGE_SYNC_HOOK_FINAL') call from the emaint sync controller after the last repo is synced. In that way someones post-sync hook(s) just need to check the value of the url passed in and exit or act on it if it is the url string it is looking for. It will mean the post-sync hook is called every time a repo is synced, but the user hook scripts need to decide if they should run for themselves depending on the argv value passed it. That will keep the most flexibility for everyone without adding complexity to portage code.
(In reply to Brian Dolbec from comment #3) > So, we need to keep the individual post-sync hooks operational. What those > people do is read the url value passed to it and act only if it is the > desired url value. Unfortunately, at the moment nothing is passed to the script. Is that a bug?
(In reply to grey dot from comment #4) > Unfortunately, at the moment nothing is passed to the script. Is that a bug? That would be the fault of the /etc/portage/bin/post_sync hook that is installed by portage-utils. It calls the scripts in postsync.d without passing the arguments that it was given. Something like this should do the trick: for f in /etc/portage/postsync.d/* ; do [ -x "${f}" ] && "${f}" "@" done
(In reply to Zac Medico from comment #5) > [ -x "${f}" ] && "${f}" "@" That should be "$@", of course.
(In reply to Zac Medico from comment #6) > (In reply to Zac Medico from comment #5) > > [ -x "${f}" ] && "${f}" "@" > > That should be "$@", of course. Great. Than(In reply to Zac Medico from comment #6) > (In reply to Zac Medico from comment #5) > > [ -x "${f}" ] && "${f}" "@" > > That should be "$@", of course. Great, thank you.
I remember more about the post-sync hooks checking the info passed it. It was with layman when I was coding that functionality. Only that info was somewhat configurable. OK, I've committed a final 'PORTAGE_SYNC_HOOK_FINAL' post_sync call after all selected repos have been synced. It is in the plugin-sync branch of portage which I am finalizing before merge back into portage as the new sync system. commit: 21788efe20f2bd5a142ab4cd0518052449fce1db This will allow the hooks to read the url value passed in for fine grain control of their actions. In this way a post-sync hook can operate only when it's url is passed in, or exit if it is not. This also allows for applications like eix to watch for the 'PORTAGE_SYNC_HOOK_FINAL' url to only run once after all repos have been synced. cc'ing eix developer. Are there any other app-portage apps installing hooks? Mike, Could you please patch portage-utils to pass on the url it is passed, perhaps bump current builds with that patch, plus modify it for the next release. I have done Zac's modification to the post_sync script and this feature works correctly.
cc'ing Mike
I think it would be better to set both URL and destination folder (e.g. `/usr/overlays/sunrise') in environment. AFAIK it's not there now.
Just since I was CC'd as eix maintainer: eix is not using postsync.d Actually, eix-sync would lose some functionality if it would use postsync.d (instead of wrapping portage). Most notably, the script could then not deal with changes in the eix cachefile format, because when postsync.d is executed, the old portage tree is no longer accessible for the case that the eix cachefile needs to be regenerated in a new format. I know that some people do not care about these restrictions and want to call eix-update in postsync.d anyway. This is fine, and eix-sync attempts to detect this case to avoid duplicate calls of eix-update, but since it is only a trivial one-liner, I see no reason to provide such a "script" in the eix tarball.
Thank you for your input Martin. It should be possible to add the repo location as well. I'll look into it.
(In reply to Brian Dolbec from comment #8) if this is stuff we want portage to handle, then let's drop the portage-utils hook entirely and let portage run things directly ? i've always thought the portage-utils hook was a bit hacky and it really should all be in portage.
(In reply to SpanKY from comment #13) > (In reply to Brian Dolbec from comment #8) > > if this is stuff we want portage to handle, then let's drop the > portage-utils hook entirely and let portage run things directly ? i've > always thought the portage-utils hook was a bit hacky and it really should > all be in portage. You won't get any arguments against that from me. Now, what is the best way to migrate it? 1)Take over ownership of /etc/portage/bin/posy_sync. 2) Rewrite the post_sync python function to spawn the scripts it finds in /etc/portage/postsync.d/ and drop the post_sync script that it spawns now? 3) A combination of both, let a /etc/portage/bin/post_sync script from (1) override the python builtin function created in (2) if it exists? Essentially the existing behavior.
(In reply to Brian Dolbec from comment #14) implement (2), and in the version of portage that includes that support, add a blocker against older portage-utils (while also bumping portage-utils to no longer install the script). portage-utils keeps these in $FILESDIR, so you don't need me to update the git repo and roll a new release. just update the src_install in the ebuild to stop installing the files.
Created attachment 385580 [details] sample postsync.d script
OK, I've updated and rebased the plugin-sync branch of portage's git repo onto the latest master. I've also completed a native python postsync.d controller to call any executable found in /etc/portage/postsync.d. Also it now passes the repo name and sync-uri in that order to each post-sync script it finds. I've attached a test postsync script I used for testing. It shows how a script can be tailored to run for only certain repos or times. It can be tested directly from the checkout or modify the 9999 ebuild, setting EGIT_BRANCH='plugin-sync' to install it.
If the spec's not final yet, I'd note that it would be useful to pass the repository path there too.
i've migrated portage-utils to repo.postsync.d starting with 0.63
Fixed in 2.2.16.
https://gitweb.gentoo.org/proj/portage.git/commit/?id=a3991f7727be0eb973aad1b120d248a0739be453
