Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 379497 - econf: auto append --disable-silent-rules
Summary: econf: auto append --disable-silent-rules
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Hosted Projects
Classification: Unclassified
Component: PMS/EAPI (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: PMS/EAPI
URL:
Whiteboard: in-eapi-5
Keywords:
Depends on:
Blocks: future-eapi
  Show dependency tree
 
Reported: 2011-08-16 23:58 UTC by Andrew John Hughes
Modified: 2012-09-23 06:47 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andrew John Hughes 2011-08-16 23:58:19 UTC
Many autotools packages now seem to default to using silent rules so that the actual compiler commands are not displayed during the build.  While this may make things look a bit prettier, it makes it a lot hard to debug things when they go wrong.  On a number of occasions recently, I've had failing ebuilds where I've had to go into the build tree, change AM_DEFAULT_VERBOSITY up to 1 and then rebuild to actually see what went wrong.

Might it be worth fixing the econfigure rule to automatically add --disable-silent-rules?  Fedora have also been discussing doing this; see

http://lists.fedoraproject.org/pipermail/devel/2011-August/155411.html
Comment 1 Ulrich Müller gentoo-dev 2011-08-21 18:15:00 UTC
We'd first have to check if configure supports the option, otherwise there would be a spurious warning. See bug 211529 comment 25.
Comment 2 Diego Elio Pettenò (RETIRED) gentoo-dev 2011-08-21 20:01:24 UTC
I would reso WONTFIX, mostly because appending it without checking for its presence would cause _again_ the same issue we had with --disable-dependency-check.

IMHO it's not much of a use to force it, it can be overridden on the make command line anyway:

make V=1
Comment 3 Andrew John Hughes 2011-09-01 22:46:19 UTC
Assuming you know about that option.

You're also then building in a different environment to that of the ebuild.
Comment 4 SpanKY gentoo-dev 2011-09-02 03:41:21 UTC
it's not hard to check the configure script to see if it supports the option
Comment 5 Ulrich Müller gentoo-dev 2011-09-02 07:22:48 UTC
(In reply to comment #4)
> it's not hard to check the configure script to see if it supports the option

... especially since we already run configure --help:
<http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=c2682228be270158966e99596b18d0f4eccca54a>
It wouldn't be a problem to parse its output for the presence of both options.
Comment 6 SpanKY gentoo-dev 2011-09-02 14:29:06 UTC
indeed.

conf_help=$(./configure --help)

case ${conf_help} in
 *--disable-silent-rules*) set -- --disable-silent-rules "$@";;
esac
case ${conf_help} in
 *--disable-dependency-tracking*) set -- --disable-dependency-tracking "$@";;
esac
Comment 7 Brian Harring gentoo-dev 2011-09-08 23:35:35 UTC
(In reply to comment #6)
> indeed.
> 
> conf_help=$(./configure --help)
> 
> case ${conf_help} in
>  *--disable-silent-rules*) set -- --disable-silent-rules "$@";;
> esac
> case ${conf_help} in
>  *--disable-dependency-tracking*) set -- --disable-dependency-tracking "$@";;
> esac

Any reason to go the "see if configure supports it and append it if so" versus diego's suggested env setting?
Comment 8 SpanKY gentoo-dev 2011-09-09 00:08:01 UTC
parsing the help output works all the time.  adding random V=1 to MAKEOPTS is known to break builds which use V for something else.  adding the option to EXTRA_ECONF means it only works for the few users who opt into that, and it gets added to configure scripts which don't actually support it.
Comment 9 Brian Harring gentoo-dev 2011-09-09 00:19:48 UTC
I was thinking more thus AM_DEFAULT_VERBOSITY=1 then make V=1 (pardon, andrew's suggestion rather than diegos), but grok the point.

Whatever the result, I could see mandating this in a specific EAPI, but for pkgcore I'd be looking at enabling it for all eapi's unless someone can think of a failure case...
Comment 10 SpanKY gentoo-dev 2011-09-09 01:17:56 UTC
adding AM_DEFAULT_VERBOSITY=0 to default make probably wouldn't have any random side effects like V=1.  but i guess we would add that to the `emake` helper rather than default src_compile() ?

in that case, i imagine we'd get the same result.  i'd just prefer the econf approach as we're already parsing the output ...
Comment 11 Julian Ospald 2012-08-04 19:45:52 UTC
(In reply to comment #6)
> indeed.
> 
> conf_help=$(./configure --help)
> 
> case ${conf_help} in
>  *--disable-silent-rules*) set -- --disable-silent-rules "$@";;
> esac
> case ${conf_help} in
>  *--disable-dependency-tracking*) set -- --disable-dependency-tracking "$@";;
> esac

Can we apply that retroactively so that this issue will be solved above per package level?
Comment 12 Zac Medico gentoo-dev 2012-08-04 21:20:45 UTC
(In reply to comment #11)
> Can we apply that retroactively so that this issue will be solved above per
> package level?

Yes, it's now in Portage git:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=1cc39de72ac5311db748341ef9183586556719d9
Comment 13 Ulrich Müller gentoo-dev 2012-08-05 05:59:57 UTC
(In reply to comment #12)
> (In reply to comment #11)
> > Can we apply that retroactively so that this issue will be solved above per
> > package level?

IMHO, such change of configure options need an EAPI bump. So this should be applied for EAPI 5 but not retroactively.

> Yes, it's now in Portage git:
> 
> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;
> h=1cc39de72ac5311db748341ef9183586556719d9

Can you revert this please?
Comment 14 Zac Medico gentoo-dev 2012-08-05 07:36:09 UTC
(In reply to comment #13)
> IMHO, such change of configure options need an EAPI bump. So this should be
> applied for EAPI 5 but not retroactively.

Does it break stuff? If so, then that would be a good reason to tie it to EAPI. Otherwise, it should be safe to apply retroactively.
Comment 15 Julian Ospald 2012-08-05 08:37:43 UTC
(In reply to comment #13)
> (In reply to comment #12)
> > (In reply to comment #11)
> > > Can we apply that retroactively so that this issue will be solved above per
> > > package level?
> 
> IMHO, such change of configure options need an EAPI bump. So this should be
> applied for EAPI 5 but not retroactively.
> 
> > Yes, it's now in Portage git:
> > 
> > http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;
> > h=1cc39de72ac5311db748341ef9183586556719d9
> 
> Can you revert this please?

Doug requested to apply this retroactively, otherwise it is not above per package level and must be solved by eapi-bump for every single package.
Comment 16 Ciaran McCreesh 2012-08-05 12:58:18 UTC
Not every single package. Just those affected by it.
Comment 17 Julian Ospald 2012-08-05 13:07:32 UTC
(In reply to comment #16)
> Not every single package. Just those affected by it.

erm... obviously.

Judging from the gentoo developers feedback they don't want to fix it per-package, but on eclass or PM level.

The bug bloat on the tracker was more than expected, so it's not just a few packages. Otherwise I would have to reopen most of them and start filing more...
Comment 18 Ciaran McCreesh 2012-08-05 13:14:52 UTC
The fix is to make packages that have build systems that do this use EAPI 5 when they're bumped.
Comment 19 Julian Ospald 2012-08-05 13:19:34 UTC
(In reply to comment #18)
> The fix is to make packages that have build systems that do this use EAPI 5
> when they're bumped.

This is not a consistent fix, some packages may never get bumped again and some packages/maintainers use a lower EAPI on purpose.

it ends up this way:
- keep portage fix EAPI agnostic
- start filing bugs against all affected packages again
Comment 20 Julian Ospald 2012-08-05 13:24:11 UTC
(In reply to comment #19)
> it ends up this way:
> - keep portage fix EAPI agnostic
> - start filing bugs against all affected packages again

as in: either keep portage fix EAPI agnostic or start filing bugs against all affected packages again
Comment 21 Ciaran McCreesh 2012-08-05 13:27:24 UTC
(In reply to comment #19)
> This is not a consistent fix, some packages may never get bumped again

And those package that never get bumped are unlikely to get bumped to a newer auto* that enables silent rules by default. So we're down to the small number of packages that get bumped once, before we have EAPI 5, and then never again afterwards.

> and some packages/maintainers use a lower EAPI on purpose.

For the small number of legitimate cases, if any of them are affected by this, they can pass in the necessary argument themselves.

> it ends up this way:
> - keep portage fix EAPI agnostic
> - start filing bugs against all affected packages again

I think you're missing the point of having EAPIs here. Portage needs to have this change reverted, since it violates the spec.
Comment 22 Julian Ospald 2012-08-05 13:34:25 UTC
(In reply to comment #21)
> (In reply to comment #19)
> > This is not a consistent fix, some packages may never get bumped again
> 
> And those package that never get bumped are unlikely to get bumped to a
> newer auto* that enables silent rules by default. So we're down to the small
> number of packages that get bumped once, before we have EAPI 5, and then
> never again afterwards.

You assume that packages with lower EAPI are mostly outdated. That is not correct.

> 
> > and some packages/maintainers use a lower EAPI on purpose.
> 
> For the small number of legitimate cases, if any of them are affected by
> this, they can pass in the necessary argument themselves.

Using a lower EAPI is always legitimate. The numbers are not so small as you might think... I was already in the process of filing those bugs.

> 
> > it ends up this way:
> > - keep portage fix EAPI agnostic
> > - start filing bugs against all affected packages again
> 
> I think you're missing the point of having EAPIs here. Portage needs to have
> this change reverted, since it violates the spec.

Which one?
Comment 23 Ciaran McCreesh 2012-08-05 13:37:20 UTC
(In reply to comment #22)
> You assume that packages with lower EAPI are mostly outdated. That is not
> correct.

No, I assume that the window between packages starting to use silent rules, and EAPI 5 being available, is small enough that most packages that were bumped inside that period will also be bumped afterwards.

> Using a lower EAPI is always legitimate.

...for key packages that can't use a newer EAPI, yes. Otherwise, no.

> > I think you're missing the point of having EAPIs here. Portage needs to have
> > this change reverted, since it violates the spec.
> 
> Which one?

The arguments passed by econf are specified by PMS.
Comment 24 Julian Ospald 2012-08-05 13:40:17 UTC
(In reply to comment #23)
> (In reply to comment #22)
> The arguments passed by econf are specified by PMS.

Then PMS has to be fixed.
Comment 25 Ciaran McCreesh 2012-08-05 13:49:24 UTC
(In reply to comment #24)
> (In reply to comment #23)
> > (In reply to comment #22)
> > The arguments passed by econf are specified by PMS.
> 
> Then PMS has to be fixed.

Again, you don't understand the point of having EAPIs.
Comment 26 Julian Ospald 2012-08-05 13:49:55 UTC
(In reply to comment #25)
> (In reply to comment #24)
> > (In reply to comment #23)
> > > (In reply to comment #22)
> > > The arguments passed by econf are specified by PMS.
> > 
> > Then PMS has to be fixed.
> 
> Again, you don't understand the point of having EAPIs.

And you did not bring up any technical reason.
Comment 27 Ciaran McCreesh 2012-08-05 13:54:59 UTC
The point of EAPIs is that they're *fixed* standards against which developers develop. Changes affect only new EAPIs, so that developers don't have to worry about whether the behaviour of their ebuild will change after it's written.

But this isn't the place for this discussion.
Comment 28 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-08-05 18:42:02 UTC
If PMS prohibits package manager-specific extensions, then I believe PMS is broken.
Comment 29 Ciaran McCreesh 2012-08-05 18:50:08 UTC
Uhm, this isn't an extension. It's a behaviour change to a function visible to ebuilds.

An example of an extension is allowing users to supply additional arguments to econf on a per-package basis. This isn't that.
Comment 30 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-08-05 18:57:32 UTC
(In reply to comment #29)
> Uhm, this isn't an extension. It's a behaviour change to a function visible
> to ebuilds.

It is an extension.
Comment 31 Ciaran McCreesh 2012-08-05 19:03:30 UTC
(In reply to comment #30)
> (In reply to comment #29)
> > Uhm, this isn't an extension. It's a behaviour change to a function visible
> > to ebuilds.
> 
> It is an extension.

No, it's really not.
Comment 32 Ryan Hill (RETIRED) gentoo-dev 2012-08-05 20:05:20 UTC
I fail to see how this changes the behaviour of the ebuild.  In fact, the change is being made to preserve the status quo as far as what the user experiences.  

When the behaviour of the tools outside of our control changes our from under us, we need to be able to compensate.  We gain absolutely nothing by forcing this to go through the EAPI happydance.
Comment 33 Ciaran McCreesh 2012-08-05 20:29:34 UTC
You fail to see how adding an argument to econf changes the behaviour of an ebuild? Well if it doesn't change the behaviour of an ebuild, why are we doing it?
Comment 34 Ciaran McCreesh 2012-08-05 20:37:55 UTC
Also, doesn't the Portage patch turn on --help parsing for everyone, even though we know breaks some packages with slightly odd configure scripts?
Comment 35 Julian Ospald 2012-08-05 20:42:22 UTC
(In reply to comment #34)
> Also, doesn't the Portage patch turn on --help parsing for everyone

No, only for econf which expects autotools.
Comment 36 Ulrich Müller gentoo-dev 2012-08-05 20:46:04 UTC
(In reply to comment #34)
> Also, doesn't the Portage patch turn on --help parsing for everyone, even
> though we know breaks some packages with slightly odd configure scripts?

Right, there are some hand-crafted configure scripts around that may do unexpected things. I know of at least one case where unknown options are ignored.

It would be less problematic if the change was applied retroactively to EAPI 4 only (where configure --help is already called).
Comment 37 Ciaran McCreesh 2012-08-05 20:50:58 UTC
(In reply to comment #35)
> (In reply to comment #34)
> > Also, doesn't the Portage patch turn on --help parsing for everyone
> 
> No, only for econf which expects autotools.

I don't follow what you mean by "expects autotools". Please explain. As I understand it, before the patch, EAPI 0 packages which either call econf or use the defaults have their ./configure run once if it exists, whether or not they use autotools. After the patch, they'll have their ./configure run twice, which we know has caused problems with some packages previously.
Comment 38 Julian Ospald 2012-08-05 20:57:28 UTC
(In reply to comment #37)
> (In reply to comment #35)
> > (In reply to comment #34)
> > > Also, doesn't the Portage patch turn on --help parsing for everyone
> > 
> > No, only for econf which expects autotools.
> 
> I don't follow what you mean by "expects autotools".

an autotools generated configure script where we expect certain options to be existent anyway

The handcrafted configure scripts I know of in the tree call ./configure instead of econf, because the unknown options already bork them.

How would the patch calling --help and redirecting error output change this?

That is an ebuild bug. Econf is not meant for handcrafted configure scripts and it will break anyway if they bump it to a newer EAPI when I request it because of the verbose bug.
Comment 39 Ciaran McCreesh 2012-08-05 21:10:15 UTC
(In reply to comment #38)
> > I don't follow what you mean by "expects autotools".
> 
> an autotools generated configure script where we expect certain options to
> be existent anyway
> 
> The handcrafted configure scripts I know of in the tree call ./configure
> instead of econf, because the unknown options already bork them.

The canonical example to the contrary used to be mplayer, which at one point worked with econf right up until we started being too clever with it...
 
> How would the patch calling --help and redirecting error output change this?

We know we *used* to have packages which used "almost" autoconf-like configure scripts, which still used econf due to defaults. Are you certain there are no such packages?

> That is an ebuild bug. Econf is not meant for handcrafted configure scripts
> and it will break anyway if they bump it to a newer EAPI when I request it
> because of the verbose bug.

No, it's not an ebuild bug. An ebuild is allowed to use defaults if those defaults, as defined in the spec, work for it. There is nothing in the spec saying econf and the default src_* must only be used for autotools packages. And that's the point of EAPIs: we have a way of making these kinds of changes *without* breaking packages, and in such a way that developers don't need to make changes until they do the EAPI bump.

So this brings us back to the main point here: given that we know this kind of thing has caused problems previously, can you prove that this is a safe change?
Comment 40 Julian Ospald 2012-08-05 21:23:08 UTC
Well, I personally would rather fix those bugs introduced by the change as they come up, than filing endless verbosity-bugs, but for the sake of consensus I could agree on applying it retroactively on EAPI=4.

There we already parse --help, so that argument is out.

The other things will have to be fixed on eclass level or per-package. I will first check the former, so that I don't spam too many bugs which can still be solved above per-package level.
Comment 41 Ciaran McCreesh 2012-08-05 21:25:27 UTC
(In reply to comment #40)
> Well, I personally would rather fix those bugs introduced by the change as
> they come up, than filing endless verbosity-bugs

You mean, you'd rather we broke the tree...

> but for the sake of consensus I could agree on applying it retroactively on
> EAPI=4.

Since we all understand now why this has to be EAPI controlled, what's wrong with just doing it properly and sticking it in EAPI 5?
Comment 42 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-08-05 21:33:34 UTC
(In reply to comment #39)
> (In reply to comment #38)
> > > I don't follow what you mean by "expects autotools".
> > 
> > an autotools generated configure script where we expect certain options to
> > be existent anyway
> > 
> > The handcrafted configure scripts I know of in the tree call ./configure
> > instead of econf, because the unknown options already bork them.
> 
> The canonical example to the contrary used to be mplayer, which at one point
> worked with econf right up until we started being too clever with it...

And I believe you never had to set per-package EXTRA_ECONF just because someone smart used econf on non-autotools.
Comment 43 Ciaran McCreesh 2012-08-05 21:39:10 UTC
(In reply to comment #42)
> And I believe you never had to set per-package EXTRA_ECONF just because
> someone smart used econf on non-autotools.

What's that got to do with anything we're talking about?
Comment 44 Ryan Hill (RETIRED) gentoo-dev 2012-08-06 03:27:08 UTC
(In reply to comment #33)
> You fail to see how adding an argument to econf changes the behaviour of an
> ebuild? Well if it doesn't change the behaviour of an ebuild, why are we
> doing it?

Adding an argument that changes the verbosity of configure scripts if they support it to make the current behaviour explicit in the face of changing defaults that lie outside of the scope of EAPI is not a change in ebuild behaviour.


(In reply to comment #41)
> (In reply to comment #40)
> > Well, I personally would rather fix those bugs introduced by the change as
> > they come up, than filing endless verbosity-bugs
> 
> You mean, you'd rather we broke the tree...

Ciaran, that's a straw man and you know it.  Any package with a build system so fragile that a change like this would break it shouldn't be using econf in the first place.  This would be a great way to flush those out so we don't have to go through this again in the future.


> > but for the sake of consensus I could agree on applying it retroactively on
> > EAPI=4.
> 
> Since we all understand now why this has to be EAPI controlled, what's wrong
> with just doing it properly and sticking it in EAPI 5?

Because EAPI 5 will never get applied to half the tree, and even that will take years to propagate.  Meanwhile we get inconsistent behaviour across the tree for no good reason other than "the spec says so".  If portage is out of spec on this particular detail then let me be the first to say fuck the spec.
Comment 45 Ulrich Müller gentoo-dev 2012-08-06 06:45:11 UTC
(In reply to comment #41)
> > but for the sake of consensus I could agree on applying it retroactively on
> > EAPI=4.
> 
> Since we all understand now why this has to be EAPI controlled, what's wrong
> with just doing it properly and sticking it in EAPI 5?

Sure, introducing it with EAPI 5 would be the cleanest solution.

However, I wouldn't object to applying it retroactively to EAPI 4 where the output of configure --help is already parsed. It's hard to imagine how it could cause breakage there.
Comment 46 Ciaran McCreesh 2012-08-06 11:57:32 UTC
(In reply to comment #44)
> > You mean, you'd rather we broke the tree...
> 
> Ciaran, that's a straw man and you know it.  Any package with a build system
> so fragile that a change like this would break it shouldn't be using econf
> in the first place.

But plenty are, since econf is the default, and developers are told to use econf unless it doesn't work. And we know such packages exist, because every time we tinker with econf, we come across a few more of them...

> This would be a great way to flush those out so we don't have to go through
> this again in the future.

So you're back to saying we should break the tree, and that breaking the tree is a good thing?
Comment 47 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2012-08-06 12:11:03 UTC
(In reply to comment #46)
> (In reply to comment #44)
> > > You mean, you'd rather we broke the tree...
> > 
> > Ciaran, that's a straw man and you know it.  Any package with a build system
> > so fragile that a change like this would break it shouldn't be using econf
> > in the first place.
> 
> But plenty are, since econf is the default, and developers are told to use
> econf unless it doesn't work. And we know such packages exist, because every
> time we tinker with econf, we come across a few more of them...

Which developers? Who tells them?
Comment 48 Ciaran McCreesh 2012-08-06 12:18:16 UTC
(In reply to comment #47)
> Which developers? Who tells them?

Gentoo developers, who follow policy, and use econf where possible.
Comment 49 Zac Medico gentoo-dev 2012-08-07 21:11:26 UTC
I've reverted the patch to portage:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=9168d4c184a16e9b992e22bb5d6f3d88b25309b9

For those that want to apply this change retroactively to older EAPIs, I'd suggest to propose it to the Council.
Comment 51 Ulrich Müller gentoo-dev 2012-09-23 06:47:14 UTC
(In reply to comment #50)
> PMS patch:
> 
> http://git.overlays.gentoo.org/gitweb/?p=proj/pms.git;a=commit;
> h=b7750e67b4772c1064543defb7df6a556f09807b

Commit in master:
http://git.overlays.gentoo.org/gitweb/?p=proj/pms.git;a=commit;h=aac8b6b2dced29b498aa61a219111754e8b49a35