Bug 481434 – [Future EAPI] src_fetch() for fetching VCS sources

• Product: Gentoo Hosted Projects
• Status: CONFIRMED ---

[Description Michał Górny 2013-08-17 10:00:30 UTC]

Initially suggested here:

http://thread.gmane.org/gmane.linux.gentoo.devel/79780

Possibly controversial, discussion required.

[Comment 1 Michał Górny 2013-08-27 08:56:44 UTC]

As an extension of this, we could add a few vcs_* fake-phases that VCS eclasses could define for smart-live-rebuild.

[Comment 2 Michał Górny 2014-02-28 09:30:02 UTC]

Oh, two more questions here:

1. do we want default SRC_URI fetching implicit or explicit via default_src_fetch?

2. do we want some explicit Manifest functions or just implicit checking of everything that was in SRC_URI?

3. I think we ought to support something like SRC_URI="foo.tar.bz2" with foo.tar.bz2 being fetched via src_fetch() and portage doing Manifest for it afterwards.

[Comment 3 Ulrich Müller 2015-10-24 08:50:15 UTC]

Seems that pkg_fetch would be a better name here? src_* phases are typically run in a sandboxed environment.

Also, at what point of the sequence would the function be called? Certainly it should be called after pkg_needrebuild (see bug 482666, assuming that would be accepted too)? Before pkg_setup?

[Comment 4 Michał Górny 2015-11-21 13:50:15 UTC]

(In reply to Ulrich Müller from comment #3)
> Seems that pkg_fetch would be a better name here? src_* phases are typically
> run in a sandboxed environment.

I'm not convinced here. pkg_* are run for binary package installs while this is going to only happen for source builds. Besides, I think src_fetch() should be partially sandboxed (i.e. except for network-sandbox and DISTDIR access).

> Also, at what point of the sequence would the function be called? Certainly
> it should be called after pkg_needrebuild (see bug 482666, assuming that
> would be accepted too)? Before pkg_setup?

Well, it certainly should be done before src_unpack(), and I think we should allow being it run independently of ebuild process (i.e. for parallel fetching). My initial idea assumed that src_fetch() will have no access to WORKDIR, only DISTDIR, and will not pass any data through to the ebuild run following it.

[Comment 5 Ulrich Müller 2016-05-18 07:12:50 UTC]

*** Bug 249086 has been marked as a duplicate of this bug. ***

[Comment 6 Martin Baute 2016-05-18 07:20:01 UTC]

Nice to see that, about ten years after people started encountering this problem,

[Comment 7 Ulrich Müller 2017-03-28 14:35:15 UTC]

(In reply to Michał Górny from comment #4)
> > Seems that pkg_fetch would be a better name here? src_* phases are typically
> > run in a sandboxed environment.
> 
> I'm not convinced here. pkg_* are run for binary package installs while this
> is going to only happen for source builds. Besides, I think src_fetch()
> should be partially sandboxed (i.e. except for network-sandbox and DISTDIR
> access).

Looking at it from the POV of our recent clarification of variables' scope, what variables from the ebuild environment would the function have to access, that are defined either only in src_* or only in pkg_* phases? git-r3.eclass seems to access DISTDIR and also WORKDIR, but the latter only for checkout (which I presume would be called from src_unpack).

However, we just decided that DISTDIR must have a consistent value, so it would a temporary dir. So do we need a new variable, equivalent to PORTAGE_REAL_DISTDIR?

[Comment 8 Adjudicator Darren 2021-05-13 15:45:47 UTC]

Presumably FEATURES=userfetch or FEATURES=-userfetch would need to be respected, would that mean fetch(git clone/pull,wtw) as portage:portage respectively as root:root ?

[Comment 9 Ulrich Müller 2022-02-17 21:11:46 UTC]

*** Bug 833567 has been marked as a duplicate of this bug. ***