Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 481434

Summary: [Future EAPI] src_fetch() for fetching VCS sources
Product: Gentoo Hosted Projects Reporter: Michał Górny <mgorny>
Component: PMS/EAPIAssignee: PMS/EAPI <pms>
Status: CONFIRMED ---    
Severity: normal CC: adjudicatordarren, admwiggin, chewi, esigra, flippynelle, O01eg, simon.vanderveldt+gentoo
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=182028
https://bugs.gentoo.org/show_bug.cgi?id=482666
https://bugs.gentoo.org/show_bug.cgi?id=737136
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 174380    

Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-08-17 10:00:30 UTC
Initially suggested here:

http://thread.gmane.org/gmane.linux.gentoo.devel/79780

Possibly controversial, discussion required.
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-08-27 08:56:44 UTC
As an extension of this, we could add a few vcs_* fake-phases that VCS eclasses could define for smart-live-rebuild.
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-02-28 09:30:02 UTC
Oh, two more questions here:

1. do we want default SRC_URI fetching implicit or explicit via default_src_fetch?

2. do we want some explicit Manifest functions or just implicit checking of everything that was in SRC_URI?

3. I think we ought to support something like SRC_URI="foo.tar.bz2" with foo.tar.bz2 being fetched via src_fetch() and portage doing Manifest for it afterwards.
Comment 3 Ulrich Müller gentoo-dev 2015-10-24 08:50:15 UTC
Seems that pkg_fetch would be a better name here? src_* phases are typically run in a sandboxed environment.

Also, at what point of the sequence would the function be called? Certainly it should be called after pkg_needrebuild (see bug 482666, assuming that would be accepted too)? Before pkg_setup?
Comment 4 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-11-21 13:50:15 UTC
(In reply to Ulrich Müller from comment #3)
> Seems that pkg_fetch would be a better name here? src_* phases are typically
> run in a sandboxed environment.

I'm not convinced here. pkg_* are run for binary package installs while this is going to only happen for source builds. Besides, I think src_fetch() should be partially sandboxed (i.e. except for network-sandbox and DISTDIR access).

> Also, at what point of the sequence would the function be called? Certainly
> it should be called after pkg_needrebuild (see bug 482666, assuming that
> would be accepted too)? Before pkg_setup?

Well, it certainly should be done before src_unpack(), and I think we should allow being it run independently of ebuild process (i.e. for parallel fetching). My initial idea assumed that src_fetch() will have no access to WORKDIR, only DISTDIR, and will not pass any data through to the ebuild run following it.
Comment 5 Ulrich Müller gentoo-dev 2016-05-18 07:12:50 UTC
*** Bug 249086 has been marked as a duplicate of this bug. ***
Comment 6 Martin Baute 2016-05-18 07:20:01 UTC
Nice to see that, about ten years after people started encountering this problem,
Comment 7 Ulrich Müller gentoo-dev 2017-03-28 14:35:15 UTC
(In reply to Michał Górny from comment #4)
> > Seems that pkg_fetch would be a better name here? src_* phases are typically
> > run in a sandboxed environment.
> 
> I'm not convinced here. pkg_* are run for binary package installs while this
> is going to only happen for source builds. Besides, I think src_fetch()
> should be partially sandboxed (i.e. except for network-sandbox and DISTDIR
> access).

Looking at it from the POV of our recent clarification of variables' scope, what variables from the ebuild environment would the function have to access, that are defined either only in src_* or only in pkg_* phases? git-r3.eclass seems to access DISTDIR and also WORKDIR, but the latter only for checkout (which I presume would be called from src_unpack).

However, we just decided that DISTDIR must have a consistent value, so it would a temporary dir. So do we need a new variable, equivalent to PORTAGE_REAL_DISTDIR?
Comment 8 Adjudicator Darren 2021-05-13 15:45:47 UTC
Presumably FEATURES=userfetch or FEATURES=-userfetch would need to be respected, would that mean fetch(git clone/pull,wtw) as portage:portage respectively as root:root ?