Many autotools packages now seem to default to using silent rules so that the actual compiler commands are not displayed during the build. While this may make things look a bit prettier, it makes it a lot hard to debug things when they go wrong. On a number of occasions recently, I've had failing ebuilds where I've had to go into the build tree, change AM_DEFAULT_VERBOSITY up to 1 and then rebuild to actually see what went wrong. Might it be worth fixing the econfigure rule to automatically add --disable-silent-rules? Fedora have also been discussing doing this; see http://lists.fedoraproject.org/pipermail/devel/2011-August/155411.html
We'd first have to check if configure supports the option, otherwise there would be a spurious warning. See bug 211529 comment 25.
I would reso WONTFIX, mostly because appending it without checking for its presence would cause _again_ the same issue we had with --disable-dependency-check. IMHO it's not much of a use to force it, it can be overridden on the make command line anyway: make V=1
Assuming you know about that option. You're also then building in a different environment to that of the ebuild.
it's not hard to check the configure script to see if it supports the option
(In reply to comment #4) > it's not hard to check the configure script to see if it supports the option ... especially since we already run configure --help: <http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=c2682228be270158966e99596b18d0f4eccca54a> It wouldn't be a problem to parse its output for the presence of both options.
indeed. conf_help=$(./configure --help) case ${conf_help} in *--disable-silent-rules*) set -- --disable-silent-rules "$@";; esac case ${conf_help} in *--disable-dependency-tracking*) set -- --disable-dependency-tracking "$@";; esac
(In reply to comment #6) > indeed. > > conf_help=$(./configure --help) > > case ${conf_help} in > *--disable-silent-rules*) set -- --disable-silent-rules "$@";; > esac > case ${conf_help} in > *--disable-dependency-tracking*) set -- --disable-dependency-tracking "$@";; > esac Any reason to go the "see if configure supports it and append it if so" versus diego's suggested env setting?
parsing the help output works all the time. adding random V=1 to MAKEOPTS is known to break builds which use V for something else. adding the option to EXTRA_ECONF means it only works for the few users who opt into that, and it gets added to configure scripts which don't actually support it.
I was thinking more thus AM_DEFAULT_VERBOSITY=1 then make V=1 (pardon, andrew's suggestion rather than diegos), but grok the point. Whatever the result, I could see mandating this in a specific EAPI, but for pkgcore I'd be looking at enabling it for all eapi's unless someone can think of a failure case...
adding AM_DEFAULT_VERBOSITY=0 to default make probably wouldn't have any random side effects like V=1. but i guess we would add that to the `emake` helper rather than default src_compile() ? in that case, i imagine we'd get the same result. i'd just prefer the econf approach as we're already parsing the output ...
(In reply to comment #6) > indeed. > > conf_help=$(./configure --help) > > case ${conf_help} in > *--disable-silent-rules*) set -- --disable-silent-rules "$@";; > esac > case ${conf_help} in > *--disable-dependency-tracking*) set -- --disable-dependency-tracking "$@";; > esac Can we apply that retroactively so that this issue will be solved above per package level?
(In reply to comment #11) > Can we apply that retroactively so that this issue will be solved above per > package level? Yes, it's now in Portage git: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=1cc39de72ac5311db748341ef9183586556719d9
(In reply to comment #12) > (In reply to comment #11) > > Can we apply that retroactively so that this issue will be solved above per > > package level? IMHO, such change of configure options need an EAPI bump. So this should be applied for EAPI 5 but not retroactively. > Yes, it's now in Portage git: > > http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit; > h=1cc39de72ac5311db748341ef9183586556719d9 Can you revert this please?
(In reply to comment #13) > IMHO, such change of configure options need an EAPI bump. So this should be > applied for EAPI 5 but not retroactively. Does it break stuff? If so, then that would be a good reason to tie it to EAPI. Otherwise, it should be safe to apply retroactively.
(In reply to comment #13) > (In reply to comment #12) > > (In reply to comment #11) > > > Can we apply that retroactively so that this issue will be solved above per > > > package level? > > IMHO, such change of configure options need an EAPI bump. So this should be > applied for EAPI 5 but not retroactively. > > > Yes, it's now in Portage git: > > > > http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit; > > h=1cc39de72ac5311db748341ef9183586556719d9 > > Can you revert this please? Doug requested to apply this retroactively, otherwise it is not above per package level and must be solved by eapi-bump for every single package.
Not every single package. Just those affected by it.
(In reply to comment #16) > Not every single package. Just those affected by it. erm... obviously. Judging from the gentoo developers feedback they don't want to fix it per-package, but on eclass or PM level. The bug bloat on the tracker was more than expected, so it's not just a few packages. Otherwise I would have to reopen most of them and start filing more...
The fix is to make packages that have build systems that do this use EAPI 5 when they're bumped.
(In reply to comment #18) > The fix is to make packages that have build systems that do this use EAPI 5 > when they're bumped. This is not a consistent fix, some packages may never get bumped again and some packages/maintainers use a lower EAPI on purpose. it ends up this way: - keep portage fix EAPI agnostic - start filing bugs against all affected packages again
(In reply to comment #19) > it ends up this way: > - keep portage fix EAPI agnostic > - start filing bugs against all affected packages again as in: either keep portage fix EAPI agnostic or start filing bugs against all affected packages again
(In reply to comment #19) > This is not a consistent fix, some packages may never get bumped again And those package that never get bumped are unlikely to get bumped to a newer auto* that enables silent rules by default. So we're down to the small number of packages that get bumped once, before we have EAPI 5, and then never again afterwards. > and some packages/maintainers use a lower EAPI on purpose. For the small number of legitimate cases, if any of them are affected by this, they can pass in the necessary argument themselves. > it ends up this way: > - keep portage fix EAPI agnostic > - start filing bugs against all affected packages again I think you're missing the point of having EAPIs here. Portage needs to have this change reverted, since it violates the spec.
(In reply to comment #21) > (In reply to comment #19) > > This is not a consistent fix, some packages may never get bumped again > > And those package that never get bumped are unlikely to get bumped to a > newer auto* that enables silent rules by default. So we're down to the small > number of packages that get bumped once, before we have EAPI 5, and then > never again afterwards. You assume that packages with lower EAPI are mostly outdated. That is not correct. > > > and some packages/maintainers use a lower EAPI on purpose. > > For the small number of legitimate cases, if any of them are affected by > this, they can pass in the necessary argument themselves. Using a lower EAPI is always legitimate. The numbers are not so small as you might think... I was already in the process of filing those bugs. > > > it ends up this way: > > - keep portage fix EAPI agnostic > > - start filing bugs against all affected packages again > > I think you're missing the point of having EAPIs here. Portage needs to have > this change reverted, since it violates the spec. Which one?
(In reply to comment #22) > You assume that packages with lower EAPI are mostly outdated. That is not > correct. No, I assume that the window between packages starting to use silent rules, and EAPI 5 being available, is small enough that most packages that were bumped inside that period will also be bumped afterwards. > Using a lower EAPI is always legitimate. ...for key packages that can't use a newer EAPI, yes. Otherwise, no. > > I think you're missing the point of having EAPIs here. Portage needs to have > > this change reverted, since it violates the spec. > > Which one? The arguments passed by econf are specified by PMS.
(In reply to comment #23) > (In reply to comment #22) > The arguments passed by econf are specified by PMS. Then PMS has to be fixed.
(In reply to comment #24) > (In reply to comment #23) > > (In reply to comment #22) > > The arguments passed by econf are specified by PMS. > > Then PMS has to be fixed. Again, you don't understand the point of having EAPIs.
(In reply to comment #25) > (In reply to comment #24) > > (In reply to comment #23) > > > (In reply to comment #22) > > > The arguments passed by econf are specified by PMS. > > > > Then PMS has to be fixed. > > Again, you don't understand the point of having EAPIs. And you did not bring up any technical reason.
The point of EAPIs is that they're *fixed* standards against which developers develop. Changes affect only new EAPIs, so that developers don't have to worry about whether the behaviour of their ebuild will change after it's written. But this isn't the place for this discussion.
If PMS prohibits package manager-specific extensions, then I believe PMS is broken.
Uhm, this isn't an extension. It's a behaviour change to a function visible to ebuilds. An example of an extension is allowing users to supply additional arguments to econf on a per-package basis. This isn't that.
(In reply to comment #29) > Uhm, this isn't an extension. It's a behaviour change to a function visible > to ebuilds. It is an extension.
(In reply to comment #30) > (In reply to comment #29) > > Uhm, this isn't an extension. It's a behaviour change to a function visible > > to ebuilds. > > It is an extension. No, it's really not.
I fail to see how this changes the behaviour of the ebuild. In fact, the change is being made to preserve the status quo as far as what the user experiences. When the behaviour of the tools outside of our control changes our from under us, we need to be able to compensate. We gain absolutely nothing by forcing this to go through the EAPI happydance.
You fail to see how adding an argument to econf changes the behaviour of an ebuild? Well if it doesn't change the behaviour of an ebuild, why are we doing it?
Also, doesn't the Portage patch turn on --help parsing for everyone, even though we know breaks some packages with slightly odd configure scripts?
(In reply to comment #34) > Also, doesn't the Portage patch turn on --help parsing for everyone No, only for econf which expects autotools.
(In reply to comment #34) > Also, doesn't the Portage patch turn on --help parsing for everyone, even > though we know breaks some packages with slightly odd configure scripts? Right, there are some hand-crafted configure scripts around that may do unexpected things. I know of at least one case where unknown options are ignored. It would be less problematic if the change was applied retroactively to EAPI 4 only (where configure --help is already called).
(In reply to comment #35) > (In reply to comment #34) > > Also, doesn't the Portage patch turn on --help parsing for everyone > > No, only for econf which expects autotools. I don't follow what you mean by "expects autotools". Please explain. As I understand it, before the patch, EAPI 0 packages which either call econf or use the defaults have their ./configure run once if it exists, whether or not they use autotools. After the patch, they'll have their ./configure run twice, which we know has caused problems with some packages previously.
(In reply to comment #37) > (In reply to comment #35) > > (In reply to comment #34) > > > Also, doesn't the Portage patch turn on --help parsing for everyone > > > > No, only for econf which expects autotools. > > I don't follow what you mean by "expects autotools". an autotools generated configure script where we expect certain options to be existent anyway The handcrafted configure scripts I know of in the tree call ./configure instead of econf, because the unknown options already bork them. How would the patch calling --help and redirecting error output change this? That is an ebuild bug. Econf is not meant for handcrafted configure scripts and it will break anyway if they bump it to a newer EAPI when I request it because of the verbose bug.
(In reply to comment #38) > > I don't follow what you mean by "expects autotools". > > an autotools generated configure script where we expect certain options to > be existent anyway > > The handcrafted configure scripts I know of in the tree call ./configure > instead of econf, because the unknown options already bork them. The canonical example to the contrary used to be mplayer, which at one point worked with econf right up until we started being too clever with it... > How would the patch calling --help and redirecting error output change this? We know we *used* to have packages which used "almost" autoconf-like configure scripts, which still used econf due to defaults. Are you certain there are no such packages? > That is an ebuild bug. Econf is not meant for handcrafted configure scripts > and it will break anyway if they bump it to a newer EAPI when I request it > because of the verbose bug. No, it's not an ebuild bug. An ebuild is allowed to use defaults if those defaults, as defined in the spec, work for it. There is nothing in the spec saying econf and the default src_* must only be used for autotools packages. And that's the point of EAPIs: we have a way of making these kinds of changes *without* breaking packages, and in such a way that developers don't need to make changes until they do the EAPI bump. So this brings us back to the main point here: given that we know this kind of thing has caused problems previously, can you prove that this is a safe change?
Well, I personally would rather fix those bugs introduced by the change as they come up, than filing endless verbosity-bugs, but for the sake of consensus I could agree on applying it retroactively on EAPI=4. There we already parse --help, so that argument is out. The other things will have to be fixed on eclass level or per-package. I will first check the former, so that I don't spam too many bugs which can still be solved above per-package level.
(In reply to comment #40) > Well, I personally would rather fix those bugs introduced by the change as > they come up, than filing endless verbosity-bugs You mean, you'd rather we broke the tree... > but for the sake of consensus I could agree on applying it retroactively on > EAPI=4. Since we all understand now why this has to be EAPI controlled, what's wrong with just doing it properly and sticking it in EAPI 5?
(In reply to comment #39) > (In reply to comment #38) > > > I don't follow what you mean by "expects autotools". > > > > an autotools generated configure script where we expect certain options to > > be existent anyway > > > > The handcrafted configure scripts I know of in the tree call ./configure > > instead of econf, because the unknown options already bork them. > > The canonical example to the contrary used to be mplayer, which at one point > worked with econf right up until we started being too clever with it... And I believe you never had to set per-package EXTRA_ECONF just because someone smart used econf on non-autotools.
(In reply to comment #42) > And I believe you never had to set per-package EXTRA_ECONF just because > someone smart used econf on non-autotools. What's that got to do with anything we're talking about?
(In reply to comment #33) > You fail to see how adding an argument to econf changes the behaviour of an > ebuild? Well if it doesn't change the behaviour of an ebuild, why are we > doing it? Adding an argument that changes the verbosity of configure scripts if they support it to make the current behaviour explicit in the face of changing defaults that lie outside of the scope of EAPI is not a change in ebuild behaviour. (In reply to comment #41) > (In reply to comment #40) > > Well, I personally would rather fix those bugs introduced by the change as > > they come up, than filing endless verbosity-bugs > > You mean, you'd rather we broke the tree... Ciaran, that's a straw man and you know it. Any package with a build system so fragile that a change like this would break it shouldn't be using econf in the first place. This would be a great way to flush those out so we don't have to go through this again in the future. > > but for the sake of consensus I could agree on applying it retroactively on > > EAPI=4. > > Since we all understand now why this has to be EAPI controlled, what's wrong > with just doing it properly and sticking it in EAPI 5? Because EAPI 5 will never get applied to half the tree, and even that will take years to propagate. Meanwhile we get inconsistent behaviour across the tree for no good reason other than "the spec says so". If portage is out of spec on this particular detail then let me be the first to say fuck the spec.
(In reply to comment #41) > > but for the sake of consensus I could agree on applying it retroactively on > > EAPI=4. > > Since we all understand now why this has to be EAPI controlled, what's wrong > with just doing it properly and sticking it in EAPI 5? Sure, introducing it with EAPI 5 would be the cleanest solution. However, I wouldn't object to applying it retroactively to EAPI 4 where the output of configure --help is already parsed. It's hard to imagine how it could cause breakage there.
(In reply to comment #44) > > You mean, you'd rather we broke the tree... > > Ciaran, that's a straw man and you know it. Any package with a build system > so fragile that a change like this would break it shouldn't be using econf > in the first place. But plenty are, since econf is the default, and developers are told to use econf unless it doesn't work. And we know such packages exist, because every time we tinker with econf, we come across a few more of them... > This would be a great way to flush those out so we don't have to go through > this again in the future. So you're back to saying we should break the tree, and that breaking the tree is a good thing?
(In reply to comment #46) > (In reply to comment #44) > > > You mean, you'd rather we broke the tree... > > > > Ciaran, that's a straw man and you know it. Any package with a build system > > so fragile that a change like this would break it shouldn't be using econf > > in the first place. > > But plenty are, since econf is the default, and developers are told to use > econf unless it doesn't work. And we know such packages exist, because every > time we tinker with econf, we come across a few more of them... Which developers? Who tells them?
(In reply to comment #47) > Which developers? Who tells them? Gentoo developers, who follow policy, and use econf where possible.
I've reverted the patch to portage: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=9168d4c184a16e9b992e22bb5d6f3d88b25309b9 For those that want to apply this change retroactively to older EAPIs, I'd suggest to propose it to the Council.
PMS patch: http://git.overlays.gentoo.org/gitweb/?p=proj/pms.git;a=commit;h=b7750e67b4772c1064543defb7df6a556f09807b Portage patch: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=7d7387b34fdfb6ac8e736ca4e73f1e3f8fa1a705
(In reply to comment #50) > PMS patch: > > http://git.overlays.gentoo.org/gitweb/?p=proj/pms.git;a=commit; > h=b7750e67b4772c1064543defb7df6a556f09807b Commit in master: http://git.overlays.gentoo.org/gitweb/?p=proj/pms.git;a=commit;h=aac8b6b2dced29b498aa61a219111754e8b49a35
