Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 311101

Summary: Unification of variables used within VCS eclasses
Product: Gentoo Linux Reporter: Michał Górny <mgorny>
Component: EclassesAssignee: Gentoo Quality Assurance Team <qa>
Status: CONFIRMED ---    
Severity: enhancement CC: arfrever, dberkholz, esigra, hattya, jlec, jmbsvicetto, nelchael, qnikst, sam, ulm, vapier, zlin
Priority: High Keywords: Tracker
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=182028
https://bugs.gentoo.org/show_bug.cgi?id=497798
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on: 343355, 343581, 392019, 410465, 410467, 410469, 410471    
Bug Blocks: 307743, 310899    

Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2010-03-24 10:35:56 UTC
Currently, various SCM eclasses differ very much in the subset of features
and control variables implemented. The idea is to establish a single subset
of such variables and rules for all SCM eclasses to follow, and maybe even
develop a common scm.eclass which would be sourced by other SCM eclasses.

Variables suggest by me:

a) Common variables - the variables which would have to be used by various SCM
eclasses as default/fallback values.

1. ESCM_DISTDIR (defaulting to PORTAGE_ACTUAL_DISTDIR/PORTDIR)
    - an alternate parent dir to all SCM stores. It would be useful if user
      would like to use an small file-inefficient filesystem for main DISTDIR
      or rsync it with other machine (where SCM files are not as important
      as the tarballs are).

2. ESCM_OFFLINE (most eclasses use it already)
    - a common switch to easily switch off all network interaction.

3. ESCM_LIVE_FAIL_IF_REPO_NOT_UPDATED (similar to the one in git.eclass)
    - a common switch to force unpack() phase to fail if no updates were found
      during the pull/update.

b) Common eclass-specific variables - these ones should allow user to override
above variables for single SCM.

1. E*_STORE_DIR (defaulting to ${ESCM_DISTDIR}/*-src)
    - already used by few eclasses, allowing user to change the location where
      SCM-specific clones are stored.

2. E*_OFFLINE (defaulting to ${ESCM_OFFLINE})
    - allowing user to override global 'offline switch'. Thus, it should also
      support setting 'false' value to enable network interaction for single
      SCM.

3. E*_LIVE_FAIL_...
    - another override for the global one.

4. E*_REPO_URI
    - the URI to the main repository. It might be extended to support multiple
      URIs.

5. E*_REVISION
    - explicit expected-revision/tag specification, preferably along with
      implicit one (e.g. in ESVN_REPO_URI) deprecation. This would allow
      applications to easily distinguish between 'real' live ebuilds
      and snapshot ones fetching directly from the repo.

c) Common export variables - these ones should be exported by SCM eclass
and stored in environment.bz2 after successful emerge.

1. E*_VERSION (or _REVISION, or ...)
    - the version/revision to which the package was updated. This would be
      useful to determine whether the current repo is newer than one used when
      merging package.

2. E*_WC_PATH
    - the absolute path to the last-used clone dir (i.e. ${E*_STORE_DIR}/sth)
      and thus the most probable location to perform further updates in.

d) Other:

1. ESCM_CUSTOM_FETCH
    - this one is not directly related to eclasses but for use of ebuild
      authors. Setting this in an ebuild should notice applications that
      the ebuild does use custom fetching procedures (i.e. fetches
      from multiple repositories in a manner not supported directly
      by the eclass) and thus external applications should not try to update
      the repository themselves.
Comment 1 Samuli Suominen (RETIRED) gentoo-dev 2010-03-24 11:02:11 UTC
Sounds like you should post this to gentoo-dev mailinglist instead.
Comment 2 Justin Lecher (RETIRED) gentoo-dev 2010-03-24 11:02:41 UTC
I CCed all maintainers of the SCM eclasses. Could someone please assign the bug to the right adresse.
Comment 3 Christian Faulhammer (RETIRED) gentoo-dev 2010-03-24 11:09:12 UTC
I will take the bug for now, but really better start a discussion on the -dev mailling list.
Comment 4 Ulrich Müller gentoo-dev 2011-02-07 07:12:34 UTC
(In reply to comment #0)
> a) Common variables - the variables which would have to be used by various
> SCM eclasses as default/fallback values.

Coming back to this. As was pointed out in the thread in -dev (almost a year ago), these variables should better be named EVCS_*, because SCM is ambiguous with the scheme language. The category is called dev-vcs too.

> 1. ESCM_DISTDIR (defaulting to PORTAGE_ACTUAL_DISTDIR/PORTDIR)
>     - an alternate parent dir to all SCM stores. It would be useful if user
>       would like to use an small file-inefficient filesystem for main DISTDIR
>       or rsync it with other machine (where SCM files are not as important
>       as the tarballs are).

> 1. E*_STORE_DIR (defaulting to ${ESCM_DISTDIR}/*-src)
>     - already used by few eclasses, allowing user to change the location where
>       SCM-specific clones are stored.

If we introduce E*_DISTDIR, then it doesn't make sense for E*_STORE_DIR to be an absolute path. It should be relative to E*_DISTDIR. But probably it is too late to change it, therefore I've chosen _not_ to implement E*_DISTDIR in the updated bzr.eclass.

> 3. ESCM_LIVE_FAIL_IF_REPO_NOT_UPDATED (similar to the one in git.eclass)
>     - a common switch to force unpack() phase to fail if no updates were
>       found during the pull/update.

Good idea, but I see some problems with implementation:
- What to do if the package is rebuilt because of a changed USE flag, or
  because the previous compilation failed? How can the eclass distinguish that
  case from a normal rebuild of the live package? Maybe a new package manager
  variable is needed?
- For this to work properly, one would like to compare the current revision of
  the repo with the revision of the last successfully built package. Comparing
  revisions before and after update (as currently implemented in git.eclass)
  doesn't look right to me.

> 1. E*_VERSION (or _REVISION, or ...)
>     - the version/revision to which the package was updated. This would be
>       useful to determine whether the current repo is newer than one used
>       when merging package.

Probably shouldn't be called E*_REVISION because of the variable with the same name in b) 5.
Comment 5 Ulrich Müller gentoo-dev 2012-04-01 18:50:02 UTC
*** Bug 410249 has been marked as a duplicate of this bug. ***
Comment 6 Ulrich Müller gentoo-dev 2012-09-09 07:44:15 UTC
Maybe it's time to summarise what has been implemented and what not.

> a) Common variables - the variables which would have to be used by various
> SCM eclasses as default/fallback values.

> 2. ESCM_OFFLINE

EVCS_OFFLINE implemented by all VCS eclasses.

> 1. ESCM_DISTDIR
> 3. ESCM_LIVE_FAIL_IF_REPO_NOT_UPDATED

Not implemented by any eclass.

> b) Common eclass-specific variables - these ones should allow user to
> override above variables for single SCM.

> 1. E*_STORE_DIR
> 2. E*_OFFLINE
> 4. E*_REPO_URI
> 5. E*_REVISION

Implemented.
- cvs.eclass, darcs.eclass miss E*_STORE_DIR, E*_REPO_URI.
- cvs.eclass, darcs.eclass, git-2.eclass miss E*_REVISION.

> 3. E*_LIVE_FAIL_...

Not implemented.

> c) Common export variables - these ones should be exported by SCM eclass
> and stored in environment.bz2 after successful emerge.

> 1. E*_VERSION (or _REVISION, or ...)

Implemented.
- cvs.eclass, git-2.eclass have E*_VERSION.
- bzr.eclass has EBZR_REVNO.
- darcs.eclass has EDARCS_PATCHCOUNT.
- mercurial.eclass has HG_REV_ID.
- subversion.eclass has ESVN_WC_REVISION.

> 2. E*_WC_PATH

Not implemented generally (only in subversion.eclass).

> d) Other:

> 1. ESCM_CUSTOM_FETCH

Not used at all in the tree.
Comment 7 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-09-14 09:19:40 UTC
Sorry for getting back at it so late. I'm really doing too many things at once ;).

(In reply to comment #6)
> > 1. ESCM_DISTDIR

What was the problem with it, again? I think it still could benefit some people, although modern VCS-es have less ugly checkouts than CVS and subversion have.

> > 3. ESCM_LIVE_FAIL_IF_REPO_NOT_UPDATED
> 
> Not implemented by any eclass.

Let's bury this. smart-live-rebuild is much better solution.

> > b) Common eclass-specific variables - these ones should allow user to
> > override above variables for single SCM.
> 
> > 1. E*_STORE_DIR
> > 2. E*_OFFLINE
> > 4. E*_REPO_URI
> > 5. E*_REVISION
> 
> Implemented.
> - cvs.eclass, darcs.eclass miss E*_STORE_DIR, E*_REPO_URI.

*_STORE_DIR should be easy to fix. *_REPO_URI not sure about.

> - cvs.eclass, darcs.eclass, git-2.eclass miss E*_REVISION.

That one should probably have a better name (more suited for DVCS?).

> > c) Common export variables - these ones should be exported by SCM eclass
> > and stored in environment.bz2 after successful emerge.
> 
> > 1. E*_VERSION (or _REVISION, or ...)
> 
> Implemented.
> - cvs.eclass, git-2.eclass have E*_VERSION.
> - bzr.eclass has EBZR_REVNO.
> - darcs.eclass has EDARCS_PATCHCOUNT.
> - mercurial.eclass has HG_REV_ID.
> - subversion.eclass has ESVN_WC_REVISION.
> 
> > 2. E*_WC_PATH
> 
> Not implemented generally (only in subversion.eclass).

This one I don't need anymore with modern VCS-es as well. But I wouldn't mind something like this being exported. Again, a better name would be useful.
Comment 8 Ulrich Müller gentoo-dev 2012-09-14 10:03:25 UTC
(In reply to comment #7)
> > > 1. ESCM_DISTDIR
> 
> What was the problem with it, again? I think it still could benefit some
> people, although modern VCS-es have less ugly checkouts than CVS and
> subversion have.

See comment #4 above. Everyone has E*_STORE_DIR as an absolute path, but it would need to be relative to E*_DISTDIR if the latter was implemented.
Comment 9 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-09-14 14:13:02 UTC
(In reply to comment #8)
> (In reply to comment #7)
> > > > 1. ESCM_DISTDIR
> > 
> > What was the problem with it, again? I think it still could benefit some
> > people, although modern VCS-es have less ugly checkouts than CVS and
> > subversion have.
> 
> See comment #4 above. Everyone has E*_STORE_DIR as an absolute path, but it
> would need to be relative to E*_DISTDIR if the latter was implemented.

Well, I phrased myself wrong. I meant I didn't understand the issue.

My original proposal was about doing something like:

  : ${EGIT_DISTDIR:=${EVCS_DISTDIR:-${DISTDIR}}
  : ${EGIT_STORE_DIR:=${EGIT_DISTDIR}/git-src}
  : ${EGIT_DIR:=${EGIT_STORE_DIR}/${EGIT_PORJECT}}

As far as I understand, this shouldn't make introduce any problems, and existing checkouts would work as they do now.
Comment 10 Ulrich Müller gentoo-dev 2012-09-14 15:49:27 UTC
(In reply to comment #9)
> My original proposal was about doing something like:
> 
>   : ${EGIT_DISTDIR:=${EVCS_DISTDIR:-${DISTDIR}}

Shouldn't it be set to ${EVCS_DISTDIR:-${PORTAGE_ACTUAL_DISTDIR:-${DISTDIR}}}?

>   : ${EGIT_STORE_DIR:=${EGIT_DISTDIR}/git-src}
>   : ${EGIT_DIR:=${EGIT_STORE_DIR}/${EGIT_PORJECT}}
> 
> As far as I understand, this shouldn't make introduce any problems, and
> existing checkouts would work as they do now.

Too many variables, for my taste. A global EVCS_DISTDIR might have some uses, but why would we need a separate E*_DISTDIR for each eclass?
Comment 11 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-09-14 17:36:22 UTC
(In reply to comment #10)
> (In reply to comment #9)
> > My original proposal was about doing something like:
> > 
> >   : ${EGIT_DISTDIR:=${EVCS_DISTDIR:-${DISTDIR}}
> 
> Shouldn't it be set to
> ${EVCS_DISTDIR:-${PORTAGE_ACTUAL_DISTDIR:-${DISTDIR}}}?

Ah yes, the DISTDIR magic. We should get something for that dir in PMS probably.

> >   : ${EGIT_STORE_DIR:=${EGIT_DISTDIR}/git-src}
> >   : ${EGIT_DIR:=${EGIT_STORE_DIR}/${EGIT_PORJECT}}
> > 
> > As far as I understand, this shouldn't make introduce any problems, and
> > existing checkouts would work as they do now.
> 
> Too many variables, for my taste. A global EVCS_DISTDIR might have some
> uses, but why would we need a separate E*_DISTDIR for each eclass?

I agree; it was just for readability.