Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 447140 - Please allow ebuilds (eclasses) to prohibit metadata updates in special cases
Summary: Please allow ebuilds (eclasses) to prohibit metadata updates in special cases
Status: CONFIRMED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core - Ebuild Support (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-12-13 18:47 UTC by Michał Górny
Modified: 2012-12-16 10:44 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-12-13 18:47:19 UTC
I'd like to add a hidden (developer) feature to python-r1 allowing us to enable additional PYTHON_TARGETS using an environment variable. This sometimes results in portage updating caches using those values and therefore 'binding' the system to them.

Therefore, I'd like to ask for a private portage variable or a RESTRICT key which would prohibit metadata generation in cases when our hidden variable is used, e.g.

  if [[ ${PYTHON_COMPAT_OVERRIDE} ]]; then
    RESTRICT=metadata
    # ...
  fi
Comment 1 Zac Medico gentoo-dev 2012-12-13 19:27:15 UTC
Something like this would really have to be part of PMS.
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-12-13 19:29:46 UTC
(In reply to comment #1)
> Something like this would really have to be part of PMS.

I disagree. Considering it's an internal feature dedicated to developers, and developers are expected to use portage, I think it'd be enough for portage-specific feature.

But I don't mind adding that kind of RESTRICT to PMS. Of course, it's a question of the code itself violating PMS heavily which I doubt we should explain all the way there.
Comment 3 Zac Medico gentoo-dev 2012-12-13 19:35:19 UTC
(In reply to comment #2)
> I disagree. Considering it's an internal feature dedicated to developers,
> and developers are expected to use portage, I think it'd be enough for
> portage-specific feature.

I'm not really interested in supporting such a portage-specific feature. However, if it's documented by PMS then I would be happy to.
Comment 4 Zac Medico gentoo-dev 2012-12-13 21:10:03 UTC
(In reply to comment #0)
> Therefore, I'd like to ask for a private portage variable or a RESTRICT key
> which would prohibit metadata generation in cases when our hidden variable
> is used, e.g.
> 
>   if [[ ${PYTHON_COMPAT_OVERRIDE} ]]; then
>     RESTRICT=metadata
>     # ...
>   fi

It seems like you'd need the package manager to be explicitly aware of your PYTHON_COMPAT_OVERRIDE variable (or equivalent), so it could compare the cached value to the current value, and automatically regenerate the cache if they differ. You wouldn't need a RESTRICT setting for this, or if you have one then it should be set unconditionally.
Comment 5 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-12-13 21:57:34 UTC
(In reply to comment #4)
> (In reply to comment #0)
> > Therefore, I'd like to ask for a private portage variable or a RESTRICT key
> > which would prohibit metadata generation in cases when our hidden variable
> > is used, e.g.
> > 
> >   if [[ ${PYTHON_COMPAT_OVERRIDE} ]]; then
> >     RESTRICT=metadata
> >     # ...
> >   fi
> 
> It seems like you'd need the package manager to be explicitly aware of your
> PYTHON_COMPAT_OVERRIDE variable (or equivalent), so it could compare the
> cached value to the current value, and automatically regenerate the cache if
> they differ. You wouldn't need a RESTRICT setting for this, or if you have
> one then it should be set unconditionally.

Hmm, you are correct indeed. I thought so much about protecting the cache that I forgot it won't be useful in practice.

So let's get back to square one. How about a metadata file or layout.conf entry which would specify the variables which should prevent use of cache?
Comment 6 Zac Medico gentoo-dev 2012-12-13 22:07:00 UTC
(In reply to comment #5)
> So let's get back to square one. How about a metadata file or layout.conf
> entry which would specify the variables which should prevent use of cache?

I think it's better if you can configure it separately for each package, rather than do it for the whole repo in layout.conf.

How about if we add a new constant metadata variable where you can list the names of these other variables that need to be watched. When cache is generated, the values of each of these watched variables would also be stored in the cache entry, so that they can be compared with later values and trigger regeneration when they change.
Comment 7 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-12-13 22:10:48 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > So let's get back to square one. How about a metadata file or layout.conf
> > entry which would specify the variables which should prevent use of cache?
> 
> I think it's better if you can configure it separately for each package,
> rather than do it for the whole repo in layout.conf.
> 
> How about if we add a new constant metadata variable where you can list the
> names of these other variables that need to be watched. When cache is
> generated, the values of each of these watched variables would also be
> stored in the cache entry, so that they can be compared with later values
> and trigger regeneration when they change.

If it's stacked, I don't mind. I'm just not sure if it's worth changing the cache format, considering that I don't think there will be many variables like that.

But I'm not sure if it should generate metadata. Invalidate the current one and force getting one directly from ebuild yes, but writing it just to rewrite it back to the correct one doesn't seem good to me.
Comment 8 Zac Medico gentoo-dev 2012-12-13 22:16:01 UTC
(In reply to comment #7)
> But I'm not sure if it should generate metadata. Invalidate the current one
> and force getting one directly from ebuild yes, but writing it just to
> rewrite it back to the correct one doesn't seem good to me.

Considering that emerge --dynamic-deps is enabled by default, it could be a severe performance problem for dependency calculations if we don't write cache for these ebuilds.
Comment 9 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-12-13 22:31:42 UTC
(In reply to comment #8)
> (In reply to comment #7)
> > But I'm not sure if it should generate metadata. Invalidate the current one
> > and force getting one directly from ebuild yes, but writing it just to
> > rewrite it back to the correct one doesn't seem good to me.
> 
> Considering that emerge --dynamic-deps is enabled by default, it could be a
> severe performance problem for dependency calculations if we don't write
> cache for these ebuilds.

No, no. I didn't mean not writing it. I meant 'not writing it if the relevant envvar is set' (which should be unset by default).
Comment 10 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-12-16 09:37:26 UTC
Alternatively, could we at least get some kind of --no-cache option?
Comment 11 Zac Medico gentoo-dev 2012-12-16 10:44:33 UTC
Considering that emerge --dynamic-deps is enabled by default, a --no-cache option will probably trigger severe performance problems.