A missing file for /usr/share/doc/$P should not kill an emerge. Given how long it can take to compile a package, the fact that a README file no longer exists upstream should not prevent the merge. Unless a dev-oriented FEATURE flag is set (perhaps an existing one, else a new one). Even the main tree has ebuilds which die because the maintainers do not try with every combination of USE flags, which is often mathematically infeasible. It would be more useful to try to catch such issues with tools for the devs, than to rely on users complaining here.
There's already "nonfatal dodoc" for such purposes. Maybe the devmanual needs to document this better.
(In reply to Ulrich Müller from comment #1) > There's already "nonfatal dodoc" for such purposes. Maybe the devmanual > needs to document this better. Can this nonfatal version be the default, and the fatal one activated in dev profile(s)?
That would defeat the point of having strict helpers.
(In reply to Ciaran McCreesh from comment #3) > That would defeat the point of having strict helpers. Which is being put into dispute hereby - limited to dodoc/dohtml/doman/doinfo(?) I assume.
The point of strict helpers is that when you explicitly say "install blah", if blah doesn't exist then it's a strong indication that there's at either a mistake in the ebuild or a bigger problem. It's not somehow less of a problem if it happens for a user rather than a developer: on the contrary, it's a sign that something the developer didn't expect has happened, and it needs to be looked into.
I'd rather have all helpers behave consistently. What's the problem with prefixing them with "nonfatal" when you want the pre-EAPI-4 behaviour?
As a developer, I *want* fatal by default. I want to have a big, obvious failure in front of my face to show me that I wasn't paying attention when version-bumping an ebuild. If I was careless and didn't notice renamed documentation files, it probably means that I also missed some important changes in actual code.
> Can this nonfatal version be the default, and the fatal one activated in > dev profile(s)? That would be perfect. Fatal for the devs but just a warning for when one is just installing or upgrading a package. Perhaps, in addition to the possibility of making such issues fatal via a dev profile, perhaps something in an overlay’s profiles directory also could turn it on. That would allow one to see the problems when working on one’s own stuff without the inconvenience of having it on globally. In any case, fatal when one is working on ebuilds and non-fatal when one is just trying to *use* gentoo is the goal. (Which a number of the replies seem to have missed.)
(In reply to James Cloos from comment #8) > In any case, fatal when one is working on ebuilds and non-fatal when one is > just trying to *use* gentoo is the goal. Uh, let's think this through a bit. Suppose you're using Gentoo. The only time you'll see this message is when a developer has explicitly listed a file to be installed, AND the developer has committed the ebuild, AND that file isn't there for you. I hope we can assume that developers have carefully tested their code. So the only time you'll see this as a user is if something strange has happened. In this case, I highly doubt you'd want the install to proceed without a file that the package mangler has been explicitly told to install.
> Suppose you're using Gentoo. The only time you'll see this message is when a > developer has explicitly listed a file to be installed, AND the developer has > committed the ebuild, AND that file isn't there for you. And that exact scenario regularly happens to me. README files often get removed upstream, and devs often fail to notice because they do not test with every USE flag combination. Why else would I have posted this in the first place, eh?
(In reply to James Cloos from comment #10) > > Suppose you're using Gentoo. The only time you'll see this message is when a > > developer has explicitly listed a file to be installed, AND the developer has > > committed the ebuild, AND that file isn't there for you. > > And that exact scenario regularly happens to me. > > README files often get removed upstream, and devs often fail to notice > because they do not test with every USE flag combination. > > Why else would I have posted this in the first place, eh? Really? README files are regularly conditionally installed, and developers regularly fail to notice this when testing and when explicitly doing a dodoc README? Provide a list of ten examples of when this has happened.
If you can’t handle reality please stay away. Comment #11 is unethical, insulting, condescending horseshit.
(In reply to James Cloos from comment #10) > README files often get removed upstream, and devs often fail to notice > because they do not test with every USE flag combination. I wonder about this statement. dodoc is used for files that are not installed by upstream's build system. Typically static things like README or NEWS that are not influenced by USE flags. So I wonder how such a thing could get unnoticed by the developer committing the ebuild. And if it's unnoticed by the developer, then the bug should be caught while the ebuild is in testing. I would be surprised if there were cases that made it into the stable tree (but I'm ready to learn otherwise). (In reply to James Cloos from comment #12) > If you can’t handle reality please stay away. > > Comment #11 is unethical, insulting, condescending horseshit. Please, watch your language.
The most recent example, which triggered this report, is Ice: make: Leaving directory `/var/tmp/portage/dev-libs/Ice-3.5.0/work/Ice-3.5.0/rb' install: cannot stat 'rb/README': No such file or directory !!! dodoc: rb/README does not exist * ERROR: dev-libs/Ice-3.5.0 failed (install phase): * dodoc failed which I worked around by :; USE=-ruby emerge -v dev-libs/Ice The line: dodoc rb/CHANGES rb/README exists in each version in portage, but the rb/README does not exist in 3.5.0. That may have only wasted 8 minutes, but I’ve seen longer builds die in install due to missing dodoc files. The more packages one installs and the more USE flags one enables the more likely such problems are. Empirically speaking.
> So I wonder how such a thing could get unnoticed by the developer committing the ebuild. Simple. They do not test every USE flag combination every time. Just like the installs which fail for certain FEATURE flags (cf dev-lang/parrot vs the userpriv FEATURE flag; even with an open bug report they still haven’t added the necessary restrict to the ebuilds to skip userpriv if set, so as to allow the install phase to work; nor have they added any patches to work with userpriv). Clearly those maintaining the ebuilds do not test with all of the combinations used in the wild. And as I wrote initially, in some cases it is combinatorially infeasible even to try to test all of the possibilities. Something like dobin, dosbin, dolib or the ones which install into /etc are important and it is ok for them to die. But dohtml and dodoc are not *that* important and should not die by default.
So you've found one example of where the feature works as intended, and has allowed you to file a bug allowing the package to be fixed. Great!
There clearly is a problem, and hiding behind "the developer knows what they're doing with every USE flag in existence" is simply nonsense, especially coming from McCreesh. Nor do sardonic insinuations about the reporter's competence mean anything, except that he still hasn't grown-up. "A missing file for /usr/share/doc/$P should not kill an emerge" in the general case, unless the user is running with FEATURES=strict|er. Then you can have the "strict helpers for README files" that McCreesh professes to feel are so vital to the continued health of user machines. Note that *none* of this is about what happens in a developer profile, or machine, nor under repoman, so the "as a developer" part, while a valid concern, does not apply. "It would be more useful to try to catch such issues with tools for the devs, than to rely on users complaining here." Agreed. Less hassle and less work for everyone, so long as we get a QA warning about missing dodoc files when not strict. Failing a 15GB build because upstream removed a README file, is plain wrong. Complain loudly by all means, but do *not* mess the user around. As for "consistency across helpers" they have different names for different purposes. As such it is expected that there may be differences (such as automatic compression.) Otherwise you might as well just use 'install' and forget about all these variants, in the name of "consistency".
We're still discussing a hypothetical here. So far we've seen one case where the feature works as intended and allowed a bug to be fixed, and no demonstration that there's actually a problem.
An ebuild that is trying to install a file that doesn't exist is buggy and needs fixing. Furthermore, the desired behaviour is readily available with a) default_src_install() and b) "nonfatal dodoc". Therefore, I don't see what would be the issue with PMS here. Also I'm still waiting for a single example where an ebuild with a bad dodoc has made it into the stable tree: (In reply to Ulrich Müller from comment #13) > And if it's unnoticed by the developer, then the bug should be caught while > the ebuild is in testing. I would be surprised if there were cases that made > it into the stable tree (but I'm ready to learn otherwise).
You don't get to close this bug against portage like that.
It's not up to Portage. The spec mandates this behaviour.
Maybe to clarify this further, the following calls are equivalent: +--------------------+----------------+ | EAPI 3 and earlier | EAPI 4 and 5 | +--------------------+----------------+ | dodoc | nonfatal dodoc | +--------------------+----------------+ | dodoc || die | dodoc | +--------------------+----------------+ So both functionalities are available in all EAPIs, only their syntax has changed.
(In reply to comment #19 and comment #22) > An ebuild that is trying to install a file that doesn't exist is buggy and > needs fixing. Agreed, but please acknowledge that there is a *big* difference between plugin-foo/README and plugin-foo/foo.so > Furthermore, the desired behaviour is readily available with > a) default_src_install() and b) "nonfatal dodoc". And the opposite would equivalently be available if the default were switched, so that's irrelevant really, wrt what the default should be. > Maybe to clarify this further, the following calls are equivalent: > > +--------------------+----------------+ > | EAPI 3 and earlier | EAPI 4 and 5 | > +--------------------+----------------+ > | dodoc | nonfatal dodoc | > +--------------------+----------------+ > | dodoc || die | dodoc | > +--------------------+----------------+ > > So both functionalities are available in all EAPIs, only their syntax has > changed. So how plain 'dodoc' behaves changes according to EAPI. (Yes I'm aware of the argument that this is "consistent" with other handlers. That's the nux of the bug.) Did it occur to you that the prior default might actually have been desirable, and may have been the reason do* didn't die by default when first implemented? After all dodoc README NEWS INSTALL COPYING # and ignore the ones that aren't there, is a pretty reasonable thing to do, especially from a function. Sure you can put nonfatal in the new "convenient" manner, but why should you have to for a doc? || die is much clearer in intent to anyone reading it. > Also I'm still waiting for a single example where an ebuild with a bad dodoc > has made it into the stable tree: > (Ulrich Müller from comment #13) > > And if it's unnoticed by the developer, then the bug should be caught while > > the ebuild is in testing. I would be surprised if there were cases that made > > it into the stable tree (but I'm ready to learn otherwise). > Therefore, I don't see what would be the issue with PMS here. The issue is that there's no separation between what happens on a developer/strict machine and what happens on an end-user machine. Whether that's PMS or not is irrelevant as far as utility is concerned. In the latter case it's inappropriate for the ebuild to fail simply because upstream moved or removed a doc file for a plugin, for example, that the developer missed since they "cannot test every USE-flag combination". A user of an unstable ebuild doesn't want to see the build of a large package fail for that reason, any more than a user on pure-stable would. Sure, they might be prepared for certain failures: that doesn't mean it's the best we can do for them in this case. It's not. So let's do better for our users. Give out a loud QA warning instead, and work on automating QA reports if that's a concern. It is to me, and I consider it far more important than dying for *every* user because a doc file is missing, and the busy dev missed it because they couldn't test that USE-flag on their machine.
(In reply to Steve L from comment #23) > So how plain 'dodoc' behaves changes according to EAPI. (Yes I'm aware of > the argument that this is "consistent" with other handlers. That's the nux > of the bug.) No, the nux of the matter is if the problem is serious enough and occurs with such a frequency that it would justify inconsistent behaviour between handlers or ugly special casing depending on profiles. As for the frequency of the problem, EAPI 4 was approved in early 2011 and this bug is reported more than two years later. I've never had any bug reported because of a failing dodoc for any package that I maintain (O.K., so maybe I maintain only simple stuff). Also I can't remember encountering the problem for any package that I've emerged on one of my systems. Finally (and I am repeating this for the third time now), no evidence has been presented where a package in the stable tree was affected by the issue. Users accepting ~arch must beware of "stability issues, imperfect package handling, ... or broken packages" (quoting from the Handbook).
