Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 482666

Summary: [Future EAPI] pkg_needrebuild() to replace smart-live-rebuild
Product: Gentoo Hosted Projects Reporter: Michał Górny <mgorny>
Component: PMS/EAPIAssignee: PMS/EAPI <pms>
Status: CONFIRMED ---    
Severity: enhancement CC: adjudicatordarren, ahipp0, eschwartz93, esigra, eugene.shalygin, gentoo, kingjon3377, mail, mike, pacho, rafallalik, tsmksubc
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=182028
https://bugs.gentoo.org/show_bug.cgi?id=481434
https://github.com/pkgcore/pkgcore/pull/315
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 174380    

Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-08-27 12:54:14 UTC
(better name appreciated :P)

What smart-live-rebuild does is basically calling VCS commands to check the version of the remote repository and compare it to the previously-built revision/hash/whatever. I'd like to add a pkg_* phase in a future EAPI that would allow moving most of it to the ebuilds.

The pkg_liveuptodate() phase would be disjoint from other phases, much like pkg_config(). At the same time, it will be run in ebuild environment alike pkg_prerm(), so that it could access the E* variables set by the VCS eclass.

The phase would be exported by our VCS eclasses and perform tasks similar to those that are performed within smart-live-rebuild currently. Something like:

  git-2_pkg_liveuptodate() {
    local remote=$(git ls-remote --heads ${EGIT_REPO_URI} ${EGIT_BRANCH})

    [[ ${remote} == ${EGIT_VERSION} ]]
  }

(note: the code is oversimplified)

That is, the function would return 0 (true) if the package is up-to-date, and 1 otherwise. When doing a @live-rebuild run, portage would call this phase function to determine which packages need to be rebuilt.

Having an explicit PMS-defined phase would allow not only to easily integrate this into PMs but also provide ebuilds that hack VCS-es a clear way to become friends with smart-live-rebuild. It will also allow other kinds of live ebuilds (e.g. eclass-manpages) to get rebuilt properly.
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-08-31 20:57:00 UTC
Thinking about it, it should probably go for the opposite logic. That is, pkg_liveneedsrebuild() or something like that, so that we wouldn't return 0 on failure to reach remote :).
Comment 2 Ulrich Müller gentoo-dev 2013-08-31 22:26:57 UTC
I suggest that the function has three possible return values:
   0 = needs rebuild
   1 = doesn't need rebuild
   2 = some trouble occured

(In reply to Michał Górny from comment #1)
> pkg_liveneedsrebuild() or something like that,

pkg_outofdate()?
Comment 3 Zac Medico gentoo-dev 2015-02-11 02:09:59 UTC
*** Bug 539698 has been marked as a duplicate of this bug. ***
Comment 4 Ulrich Müller gentoo-dev 2015-10-24 08:32:30 UTC
(In reply to Ulrich Müller from comment #2)
> I suggest that the function has three possible return values:
>    0 = needs rebuild
>    1 = doesn't need rebuild
>    2 = some trouble occured

I withdraw the last one, as doesn't fit into the systematic of other phase functions. The function should just die if it cannot determine if a rebuild is needed (e.g. if it cannot contact remote).
Comment 5 Ulrich Müller gentoo-dev 2015-10-24 08:44:22 UTC
At what point would this be called? After pkg_pretend, but before pkg_setup?
Comment 6 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-10-24 08:54:08 UTC
(In reply to Ulrich Müller from comment #5)
> At what point would this be called? After pkg_pretend, but before pkg_setup?

I was thinking of making it completely disjoint. The initial idea was to call it when determining the contents of @live-rebuild set, pretty much how smart-live-rebuild works right now. If we added some '--live-rebuild y' option, then it would probably be called during dependency calc.
Comment 7 Ulrich Müller gentoo-dev 2015-10-24 16:49:28 UTC
(In reply to Michał Górny from comment #6)
> I was thinking of making it completely disjoint. The initial idea was to
> call it when determining the contents of @live-rebuild set, pretty much how
> smart-live-rebuild works right now. If we added some '--live-rebuild y'
> option, then it would probably be called during dependency calc.

So it would be similar to pkg_config, with respect to dependencies. The problem is that it won't be guaranteed that any build time dependencies (like dev-vcs/git) are satisfied at this point.

So either git would have to be a runtime dependency for any affected package (which is somewhat ugly), or we would need a new dependency type.
Comment 8 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-10-24 17:00:23 UTC
I'd say this is strictly related to src_fetch() dependency handling. I don't really see putting some intermediate install in middle of dependency resolution, so I'd guess PM would need to keep appropriate dependencies installed.
Comment 9 Ulrich Müller gentoo-dev 2015-10-24 17:14:53 UTC
Then I don't understand it. I thought the PM would call the pkg_needrebuild phase function of the _installed_ package? (For a package that isn't installed the question if it needs to be rebuilt doesn't arise.)
Comment 10 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-10-24 18:51:12 UTC
(In reply to Ulrich Müller from comment #9)
> Then I don't understand it. I thought the PM would call the pkg_needrebuild
> phase function of the _installed_ package? (For a package that isn't
> installed the question if it needs to be rebuilt doesn't arise.)

Yes, but if it has extra dependencies needed to run the phase, then you either need to ensure that they aren't cleaned, or install them before running the phase... which would mean having an unintended install run in action that isn't supposed to install stuff.
Comment 11 Zac Medico gentoo-dev 2015-10-24 19:06:37 UTC
The package manager could check if DEPEND (possibly FETCHDEPEND if we add a new variable) is satisfied for relevant packages, and if not then bail out and advise the user to run a command that installs the relevant dependencies (`emerge --noreplace --with-bdeps=y @live` or something like that).
Comment 12 Zac Medico gentoo-dev 2019-09-08 20:40:28 UTC
*** Bug 693660 has been marked as a duplicate of this bug. ***
Comment 13 Ulrich Müller gentoo-dev 2021-08-31 14:00:51 UTC
(In reply to Michał Górny from comment #0)
> (better name appreciated :P)

Definitely needed, because pkg_needrebuild contains "ebuild" as a substring and therefore is a reserved name.