Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 156308 - update make.conf.example to delete confcache feature
Summary: update make.conf.example to delete confcache feature
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: All Linux
: High trivial (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks:
 
Reported: 2006-11-26 07:45 UTC by Giuseppe Ricupero
Modified: 2007-11-05 01:16 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Giuseppe Ricupero 2006-11-26 07:45:39 UTC
As above."/etc/make.conf.example" yet contains FEATURE=confcache as an option. 
But, for what i know, confcache has been declared "broken by design" and removed from portage.
Comment 1 Paul Gilbert 2007-01-09 01:39:59 UTC
(In reply to comment #0)
> As above."/etc/make.conf.example" yet contains FEATURE=confcache as an option. 
> But, for what i know, confcache has been declared "broken by design" and
> removed from portage.
> 

(I'm not sure if this is the right place to post this. Please let me know if there is someplace better.)

Before writing confcache out of the documentation it might be worth trying to use it to turn on configure's own cache. I think this can be done fairly simply by
1/ setting CC (for some reason configure seems to want this when using a cache),eg
  export CC=i586-pc-linux-gnu-gcc

2/ setting CONFIG_SITE to point to a file configure executes, eg
 export CONFIG_SITE=/var/tmp/portage/config_site

3/ creating the file, eg /var/tmp/portage/config_site, with the 2 lines

LDset=${LDFLAGS+LD}
export cache_file=`echo /var/tmp/portage/config.cache.C$CFLAGS.CXX$CXXFLAGS.$LDset$LDFLAGS | sed "s/ //g"`

This sets different cache files depending on the settings of CFLAGS, CXXFLAGS, and LDFLAG. It is complicated a little bit by the fact that configure seems to distinguish a non-existent LDFLAGS from LDFLAGS set with a null value.

I am not sure what the problem was with confcache, so perhaps I am missing some fatal flaw, but the scheme above seems to mostly work. It may need some fine tuning for host alias or a couple of other things configure complained about in a handful of ebuilds. I had to --resume --skipfirst a few time in emerge -ev world. (I usually have to do this a lot more just for the ebuilds that choke with distcc.)

It would be a pity to give up on configure caching. When using distcc and some fast machines to help a slow machine, the majority of time is spent running configure.
Comment 2 Marius Mauch (RETIRED) gentoo-dev 2007-01-10 07:47:39 UTC
Brian, I assume you've investigated the idea behind comment #1 before writing confcache. Would be nice to get your opinion on that before we're going to waste time.
Comment 3 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2007-01-10 08:29:35 UTC
Personally the code in portage to use confcache still exists; I don't see any reason to remove the functionality at this point (see for example, AUTOUSE which still exists and works if enabled).  dev-util/confcache has been removed from the tree, but I don't see a point in removing it from portage...
Comment 4 Brian Harring (RETIRED) gentoo-dev 2007-01-10 18:12:01 UTC
(In reply to comment #1)
> (In reply to comment #0)
> > As above."/etc/make.conf.example" yet contains FEATURE=confcache as an option. 
> > But, for what i know, confcache has been declared "broken by design" and
> > removed from portage.
> > 
> 
> (I'm not sure if this is the right place to post this. Please let me know if
> there is someplace better.)
> 
> Before writing confcache out of the documentation it might be worth trying to
> use it to turn on configure's own cache. I think this can be done fairly simply
confcache _was_ using configures own cache.

> by
> 1/ setting CC (for some reason configure seems to want this when using a
> cache),eg
>   export CC=i586-pc-linux-gnu-gcc
> 
> 2/ setting CONFIG_SITE to point to a file configure executes, eg
>  export CONFIG_SITE=/var/tmp/portage/config_site
> 
> 3/ creating the file, eg /var/tmp/portage/config_site, with the 2 lines
> 
> LDset=${LDFLAGS+LD}
> export cache_file=`echo
> /var/tmp/portage/config.cache.C$CFLAGS.CXX$CXXFLAGS.$LDset$LDFLAGS | sed "s/
> //g"`
> 
> This sets different cache files depending on the settings of CFLAGS, CXXFLAGS,
> and LDFLAG. It is complicated a little bit by the fact that configure seems to
> distinguish a non-existent LDFLAGS from LDFLAGS set with a null value.
No need.  Those flags aren't that much of an issue...

> I am not sure what the problem was with confcache, so perhaps I am missing some
> fatal flaw,
confcache (caching in general of configure) is reliant on the configure script not being daft; confcache died off due the the fact their are a decent # of pkgs that have daft configure's, and getting them RESTRICT=confcache was rather problematic (namely tracking down the original poisoner).

> but the scheme above seems to mostly work.
Keep using it; the old restricts are still there for known poisoners, but all it takes is one daft pkg to start causing problems elsewhere.


Re: confcache... punt it.
Comment 5 Paul Gilbert 2007-01-10 21:41:14 UTC
(In reply to comment #0)
> As above."/etc/make.conf.example" yet contains FEATURE=confcache as an option. 
> But, for what i know, confcache has been declared "broken by design" and
> removed from portage.
> 

(In reply to comment #4)
> (In reply to comment #1)
> > (In reply to comment #0)
> > > As above."/etc/make.conf.example" yet contains FEATURE=confcache as an option. 
> > > But, for what i know, confcache has been declared "broken by design" and
> > > removed from portage.
> > > 
> > 
> > (I'm not sure if this is the right place to post this. Please let me know if
> > there is someplace better.)
> > 
> > Before writing confcache out of the documentation it might be worth trying to
> > use it to turn on configure's own cache. I think this can be done fairly simply
> confcache _was_ using configures own cache.
> 
> > by
> > 1/ setting CC (for some reason configure seems to want this when using a
> > cache),eg
> >   export CC=i586-pc-linux-gnu-gcc
> > 
> > 2/ setting CONFIG_SITE to point to a file configure executes, eg
> >  export CONFIG_SITE=/var/tmp/portage/config_site
> > 
> > 3/ creating the file, eg /var/tmp/portage/config_site, with the 2 lines
> > 
> > LDset=${LDFLAGS+LD}
> > export cache_file=`echo
> > /var/tmp/portage/config.cache.C$CFLAGS.CXX$CXXFLAGS.$LDset$LDFLAGS | sed "s/
> > //g"`
> > 
> > This sets different cache files depending on the settings of CFLAGS, CXXFLAGS,
> > and LDFLAG. It is complicated a little bit by the fact that configure seems to
> > distinguish a non-existent LDFLAGS from LDFLAGS set with a null value.
> No need.  Those flags aren't that much of an issue...

The issue is only that configure throws an error and stops if it finds
an environment variable that is newly set or set differently from the
cache.  I am almost finished an emerge -ev world on another machine, and
have found a few more instances of these problems.  I have been trying
to accommodate them by defining a new cache file for different variable
settings, but another alternative might be to use the failures as a scan
for cleaning up the ebuilds. (On a system with about 600 packages I am
getting about a dozen cache files, and I think I should only really need
two or three.)

> 
> > I am not sure what the problem was with confcache, so perhaps I am missing some
> > fatal flaw,
> confcache (caching in general of configure) is reliant on the configure script
> not being daft; confcache died off due the the fact their are a decent # of
> pkgs that have daft configure's, and getting them RESTRICT=confcache was rather
> problematic (namely tracking down the original poisoner).

Do you mean that the ebuild or the configure is poisoning, or the person
responsible?  I think it might be a fairly simple modification of my
scheme to get an indication of packages that are setting spurious or
unnecesssary variables.  I don't know about tracking down the people.

> 
> > but the scheme above seems to mostly work.
> Keep using it; the old restricts are still there for known poisoners, but all
> it takes is one daft pkg to start causing problems elsewhere.
> 
> 
> Re: confcache... punt it.
> 

Comment 6 Brian Harring (RETIRED) gentoo-dev 2007-01-10 23:17:01 UTC
(In reply to comment #5)
> (In reply to comment #0)
> > As above."/etc/make.conf.example" yet contains FEATURE=confcache as an option. 
> > But, for what i know, confcache has been declared "broken by design" and
> > removed from portage.
> > 
> 
> (In reply to comment #4)
> > (In reply to comment #1)
> > > (In reply to comment #0)
> > > > As above."/etc/make.conf.example" yet contains FEATURE=confcache as an option. 
> > > > But, for what i know, confcache has been declared "broken by design" and
> > > > removed from portage.
> > > > 
> > > 
> > > (I'm not sure if this is the right place to post this. Please let me know if
> > > there is someplace better.)
> > > 
> > > Before writing confcache out of the documentation it might be worth trying to
> > > use it to turn on configure's own cache. I think this can be done fairly simply
> > confcache _was_ using configures own cache.
> > 
> > > by
> > > 1/ setting CC (for some reason configure seems to want this when using a
> > > cache),eg
> > >   export CC=i586-pc-linux-gnu-gcc
> > > 
> > > 2/ setting CONFIG_SITE to point to a file configure executes, eg
> > >  export CONFIG_SITE=/var/tmp/portage/config_site
> > > 
> > > 3/ creating the file, eg /var/tmp/portage/config_site, with the 2 lines
> > > 
> > > LDset=${LDFLAGS+LD}
> > > export cache_file=`echo
> > > /var/tmp/portage/config.cache.C$CFLAGS.CXX$CXXFLAGS.$LDset$LDFLAGS | sed "s/
> > > //g"`
> > > 
> > > This sets different cache files depending on the settings of CFLAGS, CXXFLAGS,
> > > and LDFLAG. It is complicated a little bit by the fact that configure seems to
> > > distinguish a non-existent LDFLAGS from LDFLAGS set with a null value.
> > No need.  Those flags aren't that much of an issue...
> 
> The issue is only that configure throws an error and stops if it finds
> an environment variable that is newly set or set differently from the
> cache.
As I said... this is *not* an issue.

Confcache was more then just portage flipping on a --cache flag; it *tracked* those vars on it's own, and tracked the file access that lead to specific cache decisions (note you're not accounting for that angle also- the fact headers influence cache values).

Vars changing is not a problem; can invalidate the cache (wipe and restart), or rewrite the cache on the fly (what confcache eventually did); still doesn't make it any easier to spot cache 'poisoners', which is what the problem actually was.

> I am almost finished an emerge -ev world on another machine, and
> have found a few more instances of these problems.  I have been trying
> to accommodate them by defining a new cache file for different variable
> settings, but another alternative might be to use the failures as a scan
> for cleaning up the ebuilds.

Failure scan is more then a bit tricky; requires verifying that all pkgs don't poison each other- it's possible for the cache poisoning to *not* break the configure/compilation, but result in funky runtime results.


> > > I am not sure what the problem was with confcache, so perhaps I am missing some
> > > fatal flaw,
> > confcache (caching in general of configure) is reliant on the configure script
> > not being daft; confcache died off due the the fact their are a decent # of
> > pkgs that have daft configure's, and getting them RESTRICT=confcache was rather
> > problematic (namely tracking down the original poisoner).
> 
> Do you mean that the ebuild or the configure is poisoning, or the person
> responsible?
pkg's actual configure (folks being stupid about adding the restrict didn't help matters).

> I think it might be a fairly simple modification of my
> scheme to get an indication of packages that are setting spurious or
> unnecesssary variables.  I don't know about tracking down the people.

No offense intended, but you're talking about the basic beginning of confcache- there is a lot of stuff required (tracking files influencing cache values) that is required with the scheme your initially suggesting before you get to what confcache was actually doing, but it still doesn't address the issue that effectively killed confcache.

Personally, not inclined to label confcache 'broken by design'- fixing as going was intended, problem is the tool needs mangling to better enable detection of poisoners, and it's an ongoing maintenance pita spotting them.
Comment 7 Jakub Moc (RETIRED) gentoo-dev 2007-11-05 01:16:01 UTC
No such thing in /etc/make.conf.example any more. Closing.