Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 839777 - Add more portage hooks
Summary: Add more portage hooks
Status: UNCONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Enhancement/Feature Requests (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2022-04-21 06:41 UTC by Pascal Jäger
Modified: 2023-12-14 08:59 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
portage_emergerc_functions.patch (portage_emergerc_functions.patch,958 bytes, patch)
2022-04-26 20:31 UTC, Pascal Jäger
Details | Diff
new functions for registering hooks (emergerc-functions.sh,381 bytes, application/x-shellscript)
2022-04-26 20:32 UTC, Pascal Jäger
Details
example emergerc (emergerc,121 bytes, text/plain)
2022-04-26 20:33 UTC, Pascal Jäger
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Pascal Jäger 2022-04-21 06:41:21 UTC
Right now Portage has already bashrc to hook into the ebuild process. However, initially this was thought to manipulate environment variables (I think so, hence the name?) for a single package build. 
Bashrc is sourced several times DURING the ebuild process. However it is not sourced BEFORE and AFTER the ebuild process. 

My proposed feature would be to source bashrc more times and change a environment variable (e.g. EMERGE_STATE) to let the users control with code is called. (Or add some more functions that get called in a given situation for that instead of doing it with the environment variable. (Like there is pre_pkg_setup() and such)

I already did that for my own purpose by changing the file actions.py. This is very rudimentary, I just added `subprocess.run(["/etc/portage/emergerc",], shell=True)` into line 522 of actions.py and added the bash script emergerc accordingly.
https://github.com/gentoo/portage/blob/e893f4fc12eb618318b1945ce7a05a94fb1ea1b4/lib/_emerge/actions.py#L522
My purpose for that is to make a btrfs snapshot every time before I start emerging an update.

There are several situations that come to mind, where a user could benefit from a source of bashrc:
- every time emerge starts
- after dependencies are successfully calculated
- when there is a dependency conflict
- after the prompt "Would you like to merge these packages?" (if --ask) right before emerge starts working with the first package
- before --sync, after --sync (although there is postsync.d for that)
- when emerge installed all packages successfully
- when emerge failed with one package
Comment 1 Pascal Jäger 2022-04-21 20:30:21 UTC
I could probably implement this myself similar to the existing register_success_hook function and make a PR on portages GitHub. 
I just wonder if there is any interest in this?
Comment 2 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2022-04-24 14:39:24 UTC
I don't think I have any objections in principle. Thanks!
Comment 3 Pascal Jäger 2022-04-26 20:31:00 UTC
Created attachment 774945 [details, diff]
portage_emergerc_functions.patch

put in /etc/portage/patches/sys-apps/portage-3.0.30
Comment 4 Pascal Jäger 2022-04-26 20:32:46 UTC
Created attachment 774948 [details]
new functions for registering hooks

put in /usr/lib/portage/python3.9 (or whatever python version portage uses for you)
Comment 5 Pascal Jäger 2022-04-26 20:33:42 UTC
Created attachment 774951 [details]
example emergerc

put in /etc/portage/
Comment 6 Pascal Jäger 2022-04-26 20:44:40 UTC
I cant really figure out how to use the existing MiscFunctionsProcess.py and the AbstractEbuildProcess.py. The misc-functions.sh they are using are made to be sources during an ebuild process it seems. 
Maybe someone with more experience with portage can have a look at that. 

I made this patch and ebuild-functions.sh to emulate the same behavior like the register_die_hook function, but tbh I am not very happy with that myself. This starts a new /bin/sh-process with its own environment variables. So it wont have all the goodies from emerge to do scripting with. On the other hand I don't even know if there are any important environment variables from emerge in the moment that script is sources in currently.

If that is acceptable in any way, I would put the starting of that process on top of actions.py and source emergerc, then search for some more useful moments in actions.py and add hook functions for that moments.
Comment 7 Pascal Jäger 2022-08-21 12:53:33 UTC
It took a while, because I forgot about this. However, here’s the PR for portage that does this:

https://github.com/gentoo/portage/pull/894
Comment 8 Mike Gilbert gentoo-dev 2022-11-16 21:32:23 UTC
It seems like you are proposing a solution without clearly defining the problem(s) you want to solve.

The interface implemented in the PR only passes a single piece of data to the user-controlled script. That doesn't give much data to the script, and I'm having trouble envisioning how this would really be used in practice.
Comment 9 Pascal Jäger 2022-11-17 17:53:13 UTC
It all started with the need to run timeshift-autosnap before a @world update. 
But I thought since there is also one hook for one special case already, instead of adding another single hook for my use case let’s solve this for many use cases at once. 

So, yes, only the hook that is called right before portage starts emerging the first package solves an actual problem I have. The other hooks are for users that want to automate functionality with portage. 
As with bashrc that could be anything. For example we are using bashrc for Sams clang14-clang15 comparison hook right now, I also use it to rebuild my initramfs in the case of a package update of one particular package. 
I am sure there are plenty of other uses of the bashrc hook and so there are for the emergerc hooks. 

I know not all of this will work for the current state of the emergerc hooks, but take a look at this thread on Reddit where users of Arch Linux present their pacman hooks, for some inspiration of what people come up with if they have the basic functionality: https://www.reddit.com/r/archlinux/comments/dsnu81/hear_ye_archers_share_your_pacman_hooks/

For more possibilities we would have to pass more information to the hooks, I also asked for suggestions in the PR
Comment 10 John Helmert III archtester Gentoo Infrastructure gentoo-dev Security 2022-11-17 18:23:33 UTC
I don't see why this would necessitate a solution within Portage itself, when you could just do:

whatever-command && emerge -vuDN @world

Or any other shell construct to run multiple commands sequentially
Comment 11 Pascal Jäger 2022-11-17 22:04:10 UTC
That would run the command even when portage stops after dependency calculations because of e.g. a slot conflict or when I decide not to merge the packages in case of —ask.

I admit that is not a big deal, but in this case timeshift-autosnap makes a limited amount of snapshots. If it reaches that limit it will delete the oldest snapshot when creating a new one. Before I had the hook I did this with the && approach and I often had 3 useless snapshots after dealing with some conflict and no useful snapshot from before any update at all.

I could think of an other use case. User runs a headless server that is updated every week or so via cron. In case this goes wrong, the hook could be used to trigger an e-mail.
Comment 12 John Helmert III archtester Gentoo Infrastructure gentoo-dev Security 2022-11-17 22:09:35 UTC
Ok, then how about:

emerge -pvuDN @world && snapshot && emerge -vuDN @world

You'll "waste" a little bit of time maybe, but this way you'll only get snapshots when the dependency calculations are successful.
Comment 13 Pascal Jäger 2022-11-18 08:42:49 UTC
Yes, I would have to do this also every time I emerge a package. I was also thinking of writing a wrapper script around emerge that does this. But I thought a integrated solution is better. 

I can see that this becomes way more useful when more data is passed into the hook. 
I was thinking about passing
- the list of packages that are going to be emerged, so users can execute actions depending on that
- the flags, so for example I don't want to make a snapshot when -1 is passed to emerge
- if there are any news items
Comment 14 Pascal Jäger 2022-11-20 13:46:44 UTC
I did that for the list of packages and the flags, but I ditched the idea with the news. Some users maybe don't want portage to check the news all the time.