Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 211529 - [Future EAPI] have econf run ./configure with --disable-dependency-tracking
Summary: [Future EAPI] have econf run ./configure with --disable-dependency-tracking
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Hosted Projects
Classification: Unclassified
Component: PMS/EAPI (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: PMS/EAPI
URL:
Whiteboard: in-eapi-4
Keywords:
Depends on: 269765
Blocks: future-eapi 273620
  Show dependency tree
 
Reported: 2008-02-26 14:36 UTC by Tristan Heaven (RETIRED)
Modified: 2011-02-25 21:46 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
elfutils-0.140-build.log (elfutils-0.140-build.log,28.26 KB, text/plain)
2009-04-14 15:37 UTC, Tristan Heaven (RETIRED)
Details
proposed PMS wording change (0001-Fix-disable-dependency-tracking-b0rkage-bug-211529.patch,1.00 KB, patch)
2010-12-31 15:03 UTC, Ulrich Müller
Details | Diff
proposed PMS wording change (0001-Fix-disable-dependency-tracking-b0rkage-bug-211529.patch,1.18 KB, patch)
2011-01-08 16:34 UTC, Ulrich Müller
Details | Diff
proposed PMS wording change (0001-Fix-disable-dependency-tracking-b0rkage-bug-211529.patch,1.07 KB, patch)
2011-01-11 21:46 UTC, Ulrich Müller
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tristan Heaven (RETIRED) gentoo-dev 2008-02-26 14:36:26 UTC
There are a lot of ebuilds with econf --disable-dependency-tracking, so it would make sense to do this by default.
Comment 1 SpanKY gentoo-dev 2008-03-02 10:56:20 UTC
just need to double check autoconf-2.13 configure scripts ...
Comment 2 Tristan Heaven (RETIRED) gentoo-dev 2009-04-14 14:18:09 UTC
I have found one package that fails to build with --disable-dependency-tracking 

dev-libs/elfutils

elf_version.c:91: fatal error: opening dependency file .deps/elf_version.Tpo: No such file or directory
compilation terminated.

econf --enable-dependency-tracking fixes that
Comment 3 Ciaran McCreesh 2009-04-14 14:26:26 UTC
Are you absolutely totally sure that's not a more general parallelisation bug?
Comment 4 Tristan Heaven (RETIRED) gentoo-dev 2009-04-14 15:37:26 UTC
Created attachment 188317 [details]
elfutils-0.140-build.log
Comment 5 Ciaran McCreesh 2009-04-14 15:43:40 UTC
Ah, I see it. The Makefile.am's doing something funky:

%.os: %.c %.o
»·······if $(COMPILE.os) -c -o $@ -fpic -DPIC -DSHARED -MT $@ -MD -MP \
»·······  -MF "$(DEPDIR)/$*.Tpo" `test -f '$<' || echo '$(srcdir)/'`$<; \
»·······then cat "$(DEPDIR)/$*.Tpo" >> "$(DEPDIR)/$*.Po"; \
»·······     rm -f "$(DEPDIR)/$*.Tpo"; \
»·······else rm -f "$(DEPDIR)/$*.Tpo"; exit 1; \
»·······fi

I'd say this is a special case that needs overriding by hand, rather than something blocking the idea.
Comment 6 Mart Raudsepp gentoo-dev 2009-04-23 17:39:56 UTC
SpanKY - you changed the title to include --enable-fast-install in it. Where did that originate or come from? I see zero discussion about it here.
Comment 7 Martin Väth 2009-06-29 15:29:31 UTC
Making --disable-dependency-tracking the default is a very bad idea, because
it interferes with the silent rules of automake-1.11 (the rules are
then no more silent). You can verify this e.g. for eix-0.16.2.
Although this may be considered a bug of automake, it is the current state of
things, and probably many packages will use automake-1.11.
Comment 8 Martin Väth 2009-09-28 08:32:10 UTC
Just a note in case you try with newer versions of eix:
With newer versions of eix, the rules remain silent, since I have written a hack
in configure.in which *enables* dependency tracknig if silent rules are used.
Of course, this is only a hack to make it work with EAPI=3, and other
packages will not have implemented it.
Comment 9 Diego Elio Pettenò (RETIRED) gentoo-dev 2010-12-31 12:19:18 UTC
Dependency tracking is something that is handled by automake, not autoconf, so adding it without any test at all is going to create a bunch of false positive in QA warnings regarding to unrecognized options.

Before adding it, econf should check whether there is a Makefile.am or GNUmakefile.am, and if not, avoid appending it at all. It's not 100% fool proof, but it should make it much better manageable than what has been done now.

BTW, making rules non silent? That's pretty much a positive side, not a negative one; you want build logs to give info, not hide it.
Comment 10 Petteri Räty (RETIRED) gentoo-dev 2010-12-31 12:47:10 UTC
(In reply to comment #9)
> 
> Before adding it, econf should check whether there is a Makefile.am or
> GNUmakefile.am, and if not, avoid appending it at all. It's not 100% fool
> proof, but it should make it much better manageable than what has been done
> now.
> 

It's already part of EAPI 4.
Comment 11 Ulrich Müller gentoo-dev 2010-12-31 12:54:33 UTC
(In reply to comment #10)
> It's already part of EAPI 4.

Yes, and it's a major screwup, because ebuilds have no way to remove the option, other than changing from econf to configure. I wonder how this feature could make it into EAPI 4. Obviously nobody has tested it thoroughly.

IMO, it's still time to change it. But it should be done immediately, before any EAPI 4 ebuilds are allowed in the main tree.
Comment 12 Petteri Räty (RETIRED) gentoo-dev 2010-12-31 13:21:33 UTC
(In reply to comment #11)
> (In reply to comment #10)
> > It's already part of EAPI 4.
> 
> Yes, and it's a major screwup, because ebuilds have no way to remove the
> option, other than changing from econf to configure. I wonder how this feature
> could make it into EAPI 4. Obviously nobody has tested it thoroughly.
> 

Do we have any other alternatives besides these?

1) Make Portage filter dependency-tracking from its QA warnings - no PMS changes needed for EAPI 4. Do changes proposed by Diego for EAPI 5.

2) Release a new EAPI with the changes Diego proposed
Comment 13 Ulrich Müller gentoo-dev 2010-12-31 14:20:43 UTC
(In reply to comment #12)
> 1) Make Portage filter dependency-tracking from its QA warnings - no PMS
> changes needed for EAPI 4. Do changes proposed by Diego for EAPI 5.

That would sweep the problem under the rug. And you can't filter the original configure warning.

> 2) Release a new EAPI with the changes Diego proposed

We have too many EAPIs already, for my taste. Why can't the mistake that is in EAPI 4 simply be corrected? It is not to late for that. (But if we discuss it for a month it will be ...)

3) Change all ebuilds where this is a problem to use configure directly,
   instead of the broken econf. I won't be afraid to do that for my packages
   if it doesn't get fixed otherwise.
Comment 14 Pacho Ramos gentoo-dev 2010-12-31 14:45:26 UTC
(In reply to comment #13)
> > 2) Release a new EAPI with the changes Diego proposed
> 
> We have too many EAPIs already, for my taste. Why can't the mistake that is in
> EAPI 4 simply be corrected? It is not to late for that. (But if we discuss it
> for a month it will be ...)
> 

I don't know much about PMS but, from my point of view, I would vote for this second option, if I don't misremember, portage versions supporting eapi4 were released a few hours ago, then, maybe that version could be hardmasked letting this to be fixed properly.
Comment 15 Ulrich Müller gentoo-dev 2010-12-31 15:03:10 UTC
Created attachment 258532 [details, diff]
proposed PMS wording change

The current wording is also contradictory, because it says: "Calls the program's ./configure script. This is designed to work with GNU Autoconf-generated scripts." Which is no longer true with --disable-dependency-tracking.

Proposed change of wording is attached, please comment.
Comment 16 Ciaran McCreesh 2011-01-01 00:54:00 UTC
Exherbo's been doing this for ages... It *has* had substantial testing, and it doesn't cause problems. Configure scripts that don't support it simply ignore the option with a little easily-filterable message.
Comment 17 Ulrich Müller gentoo-dev 2011-01-01 02:16:04 UTC
(In reply to comment #16)
> Exherbo's been doing this for ages... It *has* had substantial testing, and
> it doesn't cause problems. Configure scripts that don't support it simply
> ignore the option with a little easily-filterable message.

So you pass --d-d-t to configure unconditionally and filter the warning? Is it guaranteed that configure scripts will output only a warning for an unknown option, and won't abort with an error (i.e. the behaviour that one would get with --enable-option-checking=fatal)?
Comment 18 Ciaran McCreesh 2011-01-01 02:22:56 UTC
Exherbo doesn't even bother filtering the warning, since it's purely cosmetic. Autoconf scripts just ignore options they don't use.

Some non-autoconf scripts break on it, but I'm only aware of one package whose build system's hand-written configure is close enough to autoconf that econf (with all the fancy options we're already passing) works with it anyway...
Comment 19 Ulrich Müller gentoo-dev 2011-01-01 02:50:09 UTC
Portage will throw a QA notice at the user:

 * QA Notice: Unrecognized configure options:
 *
 * 	configure: WARNING: unrecognized options: --disable-dependency-tracking
 * 	configure: WARNING: unrecognized options: --disable-dependency-tracking

So I think that _not_ filtering this is not really an option. Otherwise users will report bugs for these false positives.
Comment 20 Ciaran McCreesh 2011-01-01 03:00:22 UTC
Isn't that just a Portage quality-of-implementation issue, not a PMS thing?
Comment 21 Ulrich Müller gentoo-dev 2011-01-01 10:36:34 UTC
(In reply to comment #20)
> Isn't that just a Portage quality-of-implementation issue, not a PMS thing?

I'd still say that it's a bad spec if it causes warnings.

Comment 22 Petteri Räty (RETIRED) gentoo-dev 2011-01-01 11:11:16 UTC
(In reply to comment #21)
> (In reply to comment #20)
> > Isn't that just a Portage quality-of-implementation issue, not a PMS thing?
> 
> I'd still say that it's a bad spec if it causes warnings.
> 

This is the reason I suggested filtering out the warning now and then fixing the spec for the next version.
Comment 23 Zac Medico gentoo-dev 2011-01-03 01:57:50 UTC
> *      configure: WARNING: unrecognized options: --disable-dependency-tracking

This warning is filtered in portage-2.1.9.28:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=f3785f651bff9febc0bb3be4f0ee8e4d732495d5
Comment 24 Ulrich Müller gentoo-dev 2011-01-08 16:34:59 UTC
Created attachment 259298 [details, diff]
proposed PMS wording change

The ECONF_SOURCE variable should be honoured, updated wording accordingly.

So, looks like the possible options are:
1) Only filter the warning for now, and reiterate for EAPI 5, as suggested by
   Petteri in comment #12.
2) Change the spec as in attached patch.
3) Call ./configure --help | grep disable-dependency-tracking to determine if
   the option is available (another idea from Diego).
4) Remove --disable-dependency-tracking from EAPI 4 entirely.
Comment 25 Ulrich Müller gentoo-dev 2011-01-11 21:46:23 UTC
Created attachment 259575 [details, diff]
proposed PMS wording change

(In reply to comment #24)
> 3) Call ./configure --help | grep disable-dependency-tracking to determine if
>    the option is available (another idea from Diego).

This has been accepted by the council. New wording is attached.
Comment 26 Zac Medico gentoo-dev 2011-01-12 16:26:49 UTC
(In reply to comment #25)
> Created an attachment (id=259575) [details]
> proposed PMS wording change

This change is included in portage-2.1.9.31 and 2.2.0_alpha15.
Comment 27 Fabian Groffen gentoo-dev 2011-01-12 16:28:39 UTC
> 3) Call ./configure --help | grep disable-dependency-tracking to determine if
>   the option is available (another idea from Diego).

instead of grepping through the output of configure --help for --disable-dependency-tracking, isn't it a more friendly approach to just export enable_dependency_tracking to "no"?
Comment 28 Ulrich Müller gentoo-dev 2011-01-12 21:56:10 UTC
(In reply to comment #27)
> instead of grepping through the output of configure --help for
> --disable-dependency-tracking, isn't it a more friendly approach to just
> export enable_dependency_tracking to "no"?

Hm, but then the option would appear neither in config.status nor in any log file. Also some packages record their configure options, so that they can be included with bug reports.

What is so bad about calling configure --help?
Comment 29 Fabian Groffen gentoo-dev 2011-01-12 22:02:30 UTC
it takes a noticable time on slower systems
Comment 30 Brian Harring gentoo-dev 2011-01-14 03:53:54 UTC
(In reply to comment #29)
> it takes a noticable time on slower systems

Quantify that...

Bearing in mind we're talking about a configure cycle, and making; both of which aren't the lightest of things.
Comment 31 Ulrich Müller gentoo-dev 2011-01-16 08:48:43 UTC
(In reply to comment #30)
> > it takes a noticable time on slower systems
> 
> Quantify that...

For x11-libs/gtk+, on a 500 MHz Pentium 3:

   # time ./configure --help | grep -q disable-dependency-tracking
   real    0m0.313s
   user    0m0.168s
   sys     0m0.140s

> Bearing in mind we're talking about a configure cycle, and making; both of
> which aren't the lightest of things.

   # time ./configure
   real    0m28.071s
   user    0m11.177s
   sys     0m6.880s
Comment 32 Fabian Groffen gentoo-dev 2011-01-16 10:03:49 UTC
On a PowerPC G5, 2GHz:

% time bash ./configure --help | grep -q disable-dependency-tracking
        0.70 real         0.11 user         0.30 sys

of course running configure takes way longer, even though it doesn't even finish:

% time bash ./configure > /dev/null
configure: error: Package requirements (cairo-xlib >= 1.6) were not met:

No package 'cairo-xlib' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables CAIRO_BACKEND_CFLAGS
and CAIRO_BACKEND_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
       36.96 real         5.75 user        14.10 sys


Almost a second for the check.  Same numbers for a Sparc T1 system.
Comment 33 Brian Harring gentoo-dev 2011-01-16 10:28:13 UTC
(In reply to comment #32)
> On a PowerPC G5, 2GHz:
> 
> % time bash ./configure --help | grep -q disable-dependency-tracking
>         0.70 real         0.11 user         0.30 sys

> 
> of course running configure takes way longer, even though it doesn't even
> finish:

<snip>

> Almost a second for the check.  Same numbers for a Sparc T1 system.

.7s != 1s, that's some pretty extreme rounding claiming that.  For a failed configure (meaning it exited earlier than it would've), this is ~1.8% slower via your timings; said percentile sucks as a state since as said, the configure didn't actually complete.

Basically, yes, it's there, but no, it doesn't matter; guessing
1) you didn't compensate for the undue hit the first configure invocation gets due to cold cache,
2) your timing here is of the configure process alone, rather than the pipe group (technically it could result in the timing being lower than it should), although it'll be lost in the noise.

For reference, forced hot cache for a core2 locked at 800mhz for php, .241 cumulative, while the configure ran to completion is 59.85s.  So the worst case, where it searches all output (php doesn't have dep tracking) is an additional .4% to the runtime.

Hell, even diffball (which has dep tracking and is a simple configure), the timing is .183s for --help, 22.8s for full configure.  ~.8%

Thing is... this approach works reliably.  Adding less than a percentile to the runtime really isn't hugely concerning to me in light of that...
Comment 34 Fabian Groffen gentoo-dev 2011-01-16 10:57:09 UTC
(In reply to comment #33)
> .7s != 1s, that's some pretty extreme rounding claiming that.  For a failed
> configure (meaning it exited earlier than it would've), this is ~1.8% slower
> via your timings; said percentile sucks as a state since as said, the configure
> didn't actually complete.
> 
> Basically, yes, it's there, but no, it doesn't matter; guessing
> 1) you didn't compensate for the undue hit the first configure invocation gets
> due to cold cache,

I did, the first timings were way worse, which will actually be what one will be seeing most of the time.

> 2) your timing here is of the configure process alone, rather than the pipe
> group (technically it could result in the timing being lower than it should),
> although it'll be lost in the noise.
> 
> For reference, forced hot cache for a core2 locked at 800mhz for php, .241
> cumulative, while the configure ran to completion is 59.85s.  So the worst
> case, where it searches all output (php doesn't have dep tracking) is an
> additional .4% to the runtime.

Yes, but PowerPC and Sparc CPUs aren't too fast in context switching, so fork/execs are very expensive.  This is the reason it takes about a minute before configure even starts to run due to all the stuff portage does before that.

> runtime really isn't hugely concerning to me in light of that...

point taken, and I agree that having --disable-dependency-tracking in the configure invocation is worth spending the time on it to figure out if it works.
Comment 35 Ulrich Müller gentoo-dev 2011-01-18 11:00:05 UTC
Fixed, as far as --disable-dependency-tracking is concerned: <http://git.overlays.gentoo.org/gitweb/?p=proj/pms.git;a=commit;h=109bc792c16a00d5061048bc93509fc19aa2b286>

I suggest to open a new bug for --enable-fast-install (if it's still wanted).