Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 250077 - prepalldocs should be documented in PMS
Summary: prepalldocs should be documented in PMS
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Hosted Projects
Classification: Unclassified
Component: PMS/EAPI (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: PMS/EAPI
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-12-06 21:00 UTC by Ulrich Müller
Modified: 2009-03-08 14:45 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ulrich Müller gentoo-dev 2008-12-06 21:00:36 UTC
prepalldocs is used by several hundred ebuilds in the tree, Portage has the function since a long time, and it is mentioned in the old Ebuild HOWTO.

Clearly, the function should be documented in PMS.
Comment 1 Ciaran McCreesh 2008-12-06 21:05:08 UTC
It's documented as "reserved for package manager use and may not be used or relied upon by ebuilds". prep* are Portage internal functions that aren't supposed to be used by ebuilds, and that might get changed or removed at any time.
Comment 2 Ulrich Müller gentoo-dev 2008-12-06 21:07:45 UTC
(In reply to comment #1)
> It's documented as "reserved for package manager use and may not be used or
> relied upon by ebuilds". prep* are Portage internal functions that aren't
> supposed to be used by ebuilds, and that might get changed or removed at any
> time.

Then the documentation is wrong in this point and should be changed.
Comment 3 SpanKY gentoo-dev 2008-12-06 21:12:10 UTC
as it's always been and as ciaranm has always been told, yet he still ignores it

prepalldocs is part of EAPI-0 and will be part of all newer ones until an EAPI proposed includes a different solution
Comment 4 Ciaran McCreesh 2008-12-06 21:14:22 UTC
It never has been and a small number of developers keep on abusing it for horrible messy hacks rather than doing things properly. See, for example, bug 167029.
Comment 5 Ulrich Müller gentoo-dev 2008-12-06 21:19:00 UTC
(In reply to comment #3)
> prepalldocs is part of EAPI-0 and will be part of all newer ones until
> an EAPI proposed includes a different solution

Thank you for the clarification, Mike.
Comment 6 Samuli Suominen (RETIRED) gentoo-dev 2008-12-06 21:40:27 UTC
FYI, I've just used prepalldocs in x11-misc/wbarconf. I patched the upstream install.sh to install them to correct location, and it got accepted and applied. Now I don't have to use dodoc for them. Is that somehow wrong? I don't think so.
Comment 7 Ciaran McCreesh 2008-12-06 21:47:26 UTC
prep* don't have well defined or stable behaviour, and they can and do change arbitrarily between Portage releases. They've always been Portage internals that just happen to be in PATH because of how Portage lays things out, not externals for ebuild use.
Comment 8 SpanKY gentoo-dev 2008-12-06 22:21:33 UTC
wrong yet again.  this is the same "argument" you use every time and for some reason, you continue to ignore reality.  prepalldocs and prep*strip have always had rigid behavior.  the real world examples you cite are irrelevant as they apply to other prep* programs which everyone agrees should not be touched directly (prepall, prep{all,}man, prep{all,}info, preplib).

in fact, the example you cite where a developer hacked into prepall is due to limitations in the EAPI spec ... a future extension has already been proposed for that (mandatory execution of eclass hooks)
Comment 9 Ciaran McCreesh 2008-12-06 22:29:28 UTC
Reality is, you need to read bug 167029 again, especially the part that says "a package manager specific internal" and "the prepall can *vary* between those versions providing new functionality".
Comment 10 Ulrich Müller gentoo-dev 2008-12-06 22:39:15 UTC
(In reply to comment #9)
> "a package manager specific internal"

ebuild(5) says something else.
Comment 11 Ciaran McCreesh 2008-12-06 22:44:35 UTC
(In reply to comment #10)
> (In reply to comment #9)
> > "a package manager specific internal"
> 
> ebuild(5) says something else.

ebuild(5) says a lot of things that need updating to be in line with reality...
Comment 12 Ciaran McCreesh 2008-12-06 22:55:30 UTC
...and going even further back, Mike, you've already been told about this at length in bug 140697. Please stop posting the same misinformation over and over again.
Comment 13 SpanKY gentoo-dev 2008-12-06 23:19:47 UTC
wrong.  you keep pointing out functions *no one else is disputing*.  we dont care about prep*man or prep*docs or prepall.  this bug is about "prepalldocs" which has never changed behavior and has always been used by ebuilds and will continue to be used by ebuilds.
Comment 14 Samuli Suominen (RETIRED) gentoo-dev 2008-12-06 23:21:24 UTC
(In reply to comment #13)
> wrong.  you keep pointing out functions *no one else is disputing*.  we dont
> care about prep*man or prep*docs or prepall.  this bug is about "prepalldocs"
> which has never changed behavior and has always been used by ebuilds and will
> continue to be used by ebuilds.
> 

exactly, *I need* this function and because it's needed it should be documented.
Comment 15 Ciaran McCreesh 2008-12-06 23:22:45 UTC
And prepalldocs is not exported, in the same way that all the other prep* scripts are not exported. It is there for Portage internal use, not ebuild use.
Comment 16 SpanKY gentoo-dev 2008-12-06 23:27:51 UTC
sorry, my comment 13 had a typo ... "prep*man or prep*info or prepall"

it isnt in the PMS because every time someone posts a patch, it isnt merged.  the only person saying "it's for internal use only" is you.  your inability to see reality is irrelevant.  prep*docs is meant for ebuild use and is exported for ebuilds and should be documented in EAPI0+.
Comment 17 Ciaran McCreesh 2008-12-06 23:31:17 UTC
It isn't in PMS because every time we have this discussion you pop up and post the same nonsense rather than fix your hacky ebuilds. Your "it's there so I can use it" argument was thoroughly debunked a long time ago by a then Portage developer.
Comment 18 SpanKY gentoo-dev 2008-12-06 23:37:28 UTC
that logic makes no sense.  it isnt in PMS because i support the idea ?

you call these packages "hacks" yet provide no other solution that is compatible with reality.  and every comment you have against "prepalldocs" is wrong as it applies to unrelated commands.

reality: packages install into /usr/share/doc/$PF/ using upstream packaging methods or similar behavior.  files are not compressed unless they go through a command such as dodoc.  prepalldocs is there to catch this very common case and compress the remaining.

also, "your" is inappropriate.  do a count on the ebuilds in the portage tree actually using the command and tell me how many are "mine".  sure i use it, but i'm certainly not the largest consumer, and it'll continue to get used irregardless of what i do.
Comment 19 Ciaran McCreesh 2008-12-06 23:40:09 UTC
Files there are compressed or not based upon whether the package manager wants to go through and compress them after src_install is done, not based upon what src_install wants to do. If you want to catch the common case, do it automatically.
Comment 20 Samuli Suominen (RETIRED) gentoo-dev 2008-12-06 23:45:15 UTC
(In reply to comment #19)
> Files there are compressed or not based upon whether the package manager wants
> to go through and compress them after src_install is done, not based upon what
> src_install wants to do. If you want to catch the common case, do it
> automatically.
> 

No, this would break installations. Some packages use the documentations in their GUIs and so forth, from the /usr/share/doc/${PF}. Upstreams do. It's convinient to add prepalldocs to a certain point of src_install to compress the files that needs to be compressed, and then install the ones that can't be compressed.
Comment 21 Ciaran McCreesh 2008-12-06 23:47:31 UTC
(In reply to comment #20)
> No, this would break installations. Some packages use the documentations in
> their GUIs and so forth, from the /usr/share/doc/${PF}. Upstreams do. It's
> convinient to add prepalldocs to a certain point of src_install to compress the
> files that needs to be compressed, and then install the ones that can't be
> compressed.

Except that docdir doesn't use PF, so they don't end up there anyway.
Comment 22 SpanKY gentoo-dev 2008-12-06 23:53:13 UTC
except that forcing the policy of compression prevents packages from installing files into /usr/share/doc/$PF/ and not being compressed.  it is up to the package as to what should and should not be compressed and for the package manager to then do that compression based on user settings.
Comment 23 Samuli Suominen (RETIRED) gentoo-dev 2008-12-06 23:57:26 UTC
(In reply to comment #21)
> Except that docdir doesn't use PF, so they don't end up there anyway.

Except that most autotoolized packages have the option to specify docdir to a
wanted location, --docdir=/usr/share.. in econf, or in install for emake
docdir="/usr/share.." install. So saying "docdir doesn't use PF" is invalid,
it's a changing variable.
Comment 24 Ciaran McCreesh 2008-12-07 00:47:26 UTC
Then we might as well acknowledge that any attempt to apply the user's preferred colour of bikeshed to documentation files is futile anyway, and the only kind of consistency there can be across packages is to simply do nothing.
Comment 25 Brian Harring (RETIRED) gentoo-dev 2008-12-07 04:47:49 UTC
Few notes from the 'then' portage developer:

bug 140697: this was about ebuilds invoking things the manager did already, as such kindly avoid using it as an arguement for why prepalldocs shouldn't be in pms (the bug was about redundancy, not spec definition).

bug 167029: this isn't about prepallman; this is about the ruby class overiding a chain of prep* functionality to accomplish means available via other approaches.  Again, irrevelant to prepalldocs being in the spec.

So... with that noted, on w/ the fillerbusting.
Comment 26 Brian Harring (RETIRED) gentoo-dev 2008-12-07 04:49:07 UTC
(In reply to comment #25)
> bug 167029: this isn't about prepallman; this is about the ruby class overiding
s:prepallman:prepalldocc:, pardon, long day.

Either way, switch to discussing the merits of it being in the spec (or a better implentation for eapi3) please, the noise is deafening.
Comment 27 SpanKY gentoo-dev 2008-12-07 07:43:15 UTC
for someone who complains so loudly about breaking EAPI behavior, it's a bit ironic how the direction of your argument goes completely hand in hand with personal interest

the behavior has always been that the only files compressed in /usr/share/doc/$PF/ are ones the ebuild has specifically OK-ed.  i recall seeing a small handful of packages that actually read the files directly for display (some simple python packages), so blindly implementing a policy of compressing all the files will break things.  not to mention you arent going to do the actual work of verifying that every package in the tree wont be broken by this policy change you seem intent on forcing on others.

if you want to change future EAPIs, then go for it as then people who update to that EAPI will explicitly test for things.  but EAPI0 and all current EAPIs have an exact behavior: no files in /usr/share/doc/$PF/ are compressed unless explicitly told to do so via `dodoc` or `prepalldocs` or similar command.  files installed in there outside of a package manage command (i.e. manually with `cp` or the makefile) should not.  which brings us back to the original truth: prepalldocs is part of all current EAPIs.
Comment 28 Ciaran McCreesh 2008-12-07 14:49:36 UTC
I'm not the one with personal interest here...

The *best* description one could possibly specify for prepalldocs is "May or may not make arbitrary changes to /usr/share/doc/${PF}, and may or may not also make other arbitrary changes elsewhere in future Portage versions". This isn't something that should be in a spec.
Comment 29 Ulrich Müller gentoo-dev 2008-12-07 15:26:19 UTC
Fact is that there is not viable substitute for the function, and it is widely used throughout the tree. And its purpose is precisely defined, basically it's the same as dodoc for everything in doc/${PF}, but without the install part.

BTW, also the documentation of dodoc should be updated, since it doesn't mention compression.
Comment 30 Ciaran McCreesh 2008-12-07 15:34:17 UTC
So you're saying dodoc should also say "may make arbitrary changes to the file mentioned, and to anything in ${D}/usr/share/doc/${PF}"...

Which other utilities are EAPI-retroactively going to make arbitrary changes to the files you give them?
Comment 31 Samuli Suominen (RETIRED) gentoo-dev 2008-12-07 16:17:28 UTC
(In reply to comment #30)
> So you're saying dodoc should also say "may make arbitrary changes to the file
> mentioned, and to anything in ${D}/usr/share/doc/${PF}"...
> 
> Which other utilities are EAPI-retroactively going to make arbitrary changes to
> the files you give them?
> 

Stay on the topic and stop putting words to other peoples mouths, thanks.

We are talking about prepalldocs and dodoc, and how they are used to install and/or ecompress files installed into "${D}"/usr/share/doc/${PF}. Period.
Comment 32 Ciaran McCreesh 2008-12-07 16:22:47 UTC
Which is my point -- you're asking for PMS to change to say that they can't be relied upon to do anything in particular, which in turn means people will need a new dodoc whose definition won't be changed in the future. dodoc compression broke ebuilds.
Comment 33 SpanKY gentoo-dev 2008-12-07 18:22:37 UTC
the only reason retroactive changes are being made is because they were wrongly omitted in the first place.  the behavior of prepalldocs isnt going to change, nor has it changed.  EAPIs need to be updated to reflect reality.

dodoc has always been defined as "does compression", not "compresses with gzip".  same goes for prepalldocs.
Comment 34 Ciaran McCreesh 2008-12-07 18:27:40 UTC
That isn't what other Gentoo developers think, given their ebuilds...
Comment 35 SpanKY gentoo-dev 2008-12-07 19:50:49 UTC
the message output has already been addressed.  you still have no valid reason for blocking this inclusion.
Comment 36 Ciaran McCreesh 2008-12-07 19:55:44 UTC
So you're happy for PMS to say the following?

"May or may not do something arbitrary to /usr/share/doc/${PF}. Ebuilds must not rely upon any particular behaviour."

and for dodoc

"May or may not modify the file and its filename in some arbitrary way. Ebuilds must not rely upon any particular behaviour."
Comment 37 David Leverton 2008-12-07 20:13:30 UTC
(In reply to comment #36)
> "May or may not do something arbitrary to /usr/share/doc/${PF}. Ebuilds must
> not rely upon any particular behaviour."

That should be "May or may not do something arbitrary to /usr/share/doc/${PF}, excluding /usr/share/doc/${PF}/html."
Comment 38 SpanKY gentoo-dev 2008-12-07 20:21:06 UTC
that text is fine, but i would really put emphasis on the common behavior being arbitrary compression

the other key part is that package managers should not be doing arbitrary things to /usr/share/doc/${PF}/ unless dodoc was run on the file or prepalldocs was run
Comment 39 David Leverton 2008-12-07 20:30:54 UTC
(In reply to comment #37)
> (In reply to comment #36)
> > "May or may not do something arbitrary to /usr/share/doc/${PF}. Ebuilds must
> > not rely upon any particular behaviour."
> 
> That should be "May or may not do something arbitrary to /usr/share/doc/${PF},
> excluding /usr/share/doc/${PF}/html."
> 

*cough* "May or may not do something arbitrary to /usr/share/doc,
> excluding /usr/share/doc/${PF}/html."

(In reply to comment #20)
> It's convinient to add prepalldocs to a certain point of src_install to compress the
> files that needs to be compressed, and then install the ones that can't be
> compressed.
> 

Actually, after looking more closely, it seems this usage behaves differently between Portage versions.  With the following ebuild:

SLOT="0"
KEYWORDS="amd64"

src_install() {
	touch foo bar
	insinto /usr/share/doc/${PF}
	doins foo
	prepalldocs
	doins bar
}

Portage 2.1.4.5 gives

dir /usr
dir /usr/share
dir /usr/share/doc
dir /usr/share/doc/dummy-5
obj /usr/share/doc/dummy-5/foo.bz2 4059d198768f9f8dc9372dc1c54bc3c3 1228681604
obj /usr/share/doc/dummy-5/bar d41d8cd98f00b204e9800998ecf8427e 1228681604

in CONTENTS, whereas current SVN gives

dir /usr
dir /usr/share
dir /usr/share/doc
dir /usr/share/doc/dummy-5
obj /usr/share/doc/dummy-5/bar.bz2 4059d198768f9f8dc9372dc1c54bc3c3 1228681664
obj /usr/share/doc/dummy-5/foo.bz2 4059d198768f9f8dc9372dc1c54bc3c3 1228681664
Comment 40 Ulrich Müller gentoo-dev 2008-12-08 06:42:52 UTC
(In reply to comment #39)
> Actually, after looking more closely, it seems this usage behaves differently
> between Portage versions.

That's another reason why we need a specification how it *should* behave.
Comment 41 Ciaran McCreesh 2008-12-08 14:22:55 UTC
(In reply to comment #40)
> (In reply to comment #39)
> > Actually, after looking more closely, it seems this usage behaves differently
> > between Portage versions.
> 
> That's another reason why we need a specification how it *should* behave.

I'd say more that it's another reason it shouldn't be used... The specification is currently:

"May or may not do something arbitrary to /usr/share/doc/${PF},
excluding /usr/share/doc/${PF}/html, and may or may not make arbitrary changes to anything in /usr/share/doc/${PF} excluding /usr/share/doc/${PF}/html at an arbitrary later part of the build process."
Comment 42 David Leverton 2008-12-12 22:08:16 UTC
(In reply to comment #41)
> "May or may not do something arbitrary to /usr/share/doc/${PF},
> excluding /usr/share/doc/${PF}/html, and may or may not make arbitrary changes
> to anything in /usr/share/doc/${PF} excluding /usr/share/doc/${PF}/html at an
> arbitrary later part of the build process."

Turns out it doesn't even guaranteed to exclude the html subdirectory, at least in the "arbitrary later part of the build process" case: if the subdirectory doesn't exist when prepalldocs is called, but gets created and populated afterwards, then its contents /will/ be compressed.  Combining that with Mike's request in comment 38 and my amendment in comment 39 which Ciaran appears to have missed, and making it extra clear that we're talking about ${D} here, we get:

"May or may not do something arbitrary (often but not necessarily some form of compression) to ${D}/usr/share/doc, excluding ${D}/usr/share/doc/${PF}/html, and may or may not make arbitrary changes to anything in ${D}/usr/share/doc, maybe or maybe not excluding ${D}/usr/share/doc/${PF}/html, at an arbitrary later part of the build process."
Comment 43 Petteri Räty (RETIRED) gentoo-dev 2008-12-14 20:11:42 UTC
Looking at the heated discussion this seems like something that the council could resolve.
Comment 44 Steve L 2009-01-09 15:22:30 UTC
Missed notification on list. Can I just clarify that this is what is under discussion:
"For EAPI 0-2 no files in "/usr/share/doc/$PF/" are compressed unless the ebuild calls `dodoc` or `prepalldocs` or [?] when they shall be compressed according to the user configuration[1].

Files installed in there outside of a package manage command (e.g: manually
with `cp` or by the makefile) will not be, unless they are installed before a call to prepalldocs.

[1] Note that this is an arbitrary command and the ebuild MUST NOT rely on the name of the subsequently installed file."
..and ask what the resolution of this is?

(I appreciate that this brings user configuration into the issue, but it seems absurd that the Gentoo PMS doesn't mention it, since it affects so much of what happens.)
Comment 45 Ciaran McCreesh 2009-01-09 15:27:27 UTC
What's under discussion is whether dleverton's wording in comment #42 is suitable language for a standard, particularly given how Portage keeps changing the behaviour of the utility between releases.
Comment 46 Steve L 2009-01-14 06:04:37 UTC
(In reply to comment #45)
> What's under discussion is whether dleverton's wording in comment #42 is
> suitable language for a standard
You appear to be obfuscating deliberately, both here and in your above posts, leave alone that nonsensical suggestion.

I was simply asking whether the long-standing behaviour is or is not as outlined by Spanky, at least in intention if not always in fact (after all _every_ project has bugs) above over two or three posts, before things got 'heated', and summarised above (along with the warning about the command used to post-process docs being under user control and thus not something the ebuild could predict.) I'll take your response as a 'yes'.

Might be worth adding a bit:
..the ebuild MUST NOT rely on the name of the subsequently installed documentation file; it is entirely possible that it will not be installed at all.
Comment 47 David Leverton 2009-01-14 07:40:10 UTC
(In reply to comment #46)
> You appear to be obfuscating deliberately, both here and in your above posts,
> leave alone that nonsensical suggestion.

You appear to be desperate to attack and deliberately contradict the PMS team at all costs.  After all:

07:47 < igli> PMS needs a rationale, tho i can't see paludroids going for that; would be opening up to people who shouldn't be reading it or sth

07:48 < AnMaster> igli, and what is PMS?
07:48 < igli> the bane of our life ;-)

> I was simply asking whether the long-standing behaviour is or is not as
> outlined by Spanky, at least in intention if not always in fact (after all
> _every_ project has bugs) above over two or three posts, before things got
> 'heated', and summarised above (along with the warning about the command used
> to post-process docs being under user control and thus not something the ebuild
> could predict.)

The "long-standing behaviour" has been demonstrated not to be stable across portage versions, breaking any ebuilds using the technique described in comment 20 (and presumably there are some, or else it wouldn't have been mentioned).

> Might be worth adding a bit:
> ..the ebuild MUST NOT rely on the name of the subsequently installed
> documentation file; it is entirely possible that it will not be installed at
> all.

That's covered by "do something arbitrary", but if you really insist, I suppose it wouldn't hurt.
Comment 48 Ulrich Müller gentoo-dev 2009-01-14 08:21:01 UTC
A question for better understanding: Is the "prepalldocs" command implemented in Paludis? If yes, what does it do there?
Comment 49 Ciaran McCreesh 2009-01-14 15:16:31 UTC
Paludis has all the prep* commands. They issue a QA warning saying "Don't call prep*" and do nothing.
Comment 50 Steve L 2009-01-15 11:56:51 UTC
(In reply to comment #47)
> (In reply to comment #46)
<snip OT>
> The "long-standing behaviour" has been demonstrated not to be stable across
> portage versions,
See comment 40
> breaking any ebuilds using the technique described in comment
> 20 (and presumably there are some, or else it wouldn't have been mentioned).
This is why timing was specified: "unless they are installed before a call to prepalldocs."

> > Might be worth adding a bit:
> > ..the ebuild MUST NOT rely on the name of the subsequently installed
> > documentation file; it is entirely possible that it will not be installed at
> > all.
> 
> That's covered by "do something arbitrary", but if you really insist, I suppose
> it wouldn't hurt.
> 
My word, you appear to be under the impression that your contribution would be suitable for a specification.
Comment 51 Ciaran McCreesh 2009-01-15 15:41:09 UTC
(In reply to comment #50)
> My word, you appear to be under the impression that your contribution would be
> suitable for a specification.

David is providing useful contributions. You are going around trying to stir up trouble and derail the project. Kindly go away.
Comment 52 Donnie Berkholz (RETIRED) gentoo-dev 2009-01-22 20:00:19 UTC
Ulrich is requesting that the council get involved here. Mike and Ciaran, you seem like you're coming to some kind of agreement. Do you think you can get this worked out, or does the council need to make a decision?
Comment 53 Ciaran McCreesh 2009-01-22 20:04:00 UTC
Well... Does the Council really think that a utility whose behaviour keeps on changing in weird ways is suitable for formal specification? And can the Council make sure that the behaviour will remain as it is for stable Portage?

The alternative is to go through the tree and simply remove all prep* calls. Any ebuild that relies upon any of those calls doing anything in particular is already broken.
Comment 54 Ulrich Müller gentoo-dev 2009-01-22 20:11:52 UTC
(In reply to comment #53)
> The alternative is to go through the tree and simply remove all prep* calls.

It isn't an alternative because that would remove useful functionality.

> Any ebuild that relies upon any of those calls doing anything in particular is
> already broken.

Read comment #18 again.
Comment 55 Ciaran McCreesh 2009-01-22 20:19:13 UTC
(In reply to comment #54)
> (In reply to comment #53)
> > The alternative is to go through the tree and simply remove all prep* calls.
> 
> It isn't an alternative because that would remove useful functionality.

Not true. prep* has no useful functionality. At best it's bikeshedding.

> > Any ebuild that relies upon any of those calls doing anything in particular is
> > already broken.
> 
> Read comment #18 again.

Uh, you miss the point.

a) prep* being a no-op is totally fine and might happen, depending upon user configuration. If an ebuild relies upon it doing something specific, it's broken.

b) Read all of dleverton's analysis of what prepalldocs actually does in different Portage versions to see that it has no stable, well defined behaviour that can be relied upon anyway.
Comment 56 Ulrich Müller gentoo-dev 2009-01-23 01:06:05 UTC
(In reply to comment #19)
>> Files there are compressed or not based upon whether the package manager
>> wants to go through and compress them after src_install is done, not based
>> upon what src_install wants to do. If you want to catch the common case,
>> do it automatically.

That's not such a bad idea. At least it would make the behaviour consistent with that for man pages and GNU Info files.

However, the following is also a valid argument:

(In reply to comment #20)
> No, this would break installations. Some packages use the documentations in
> their GUIs and so forth, from the /usr/share/doc/${PF}. Upstreams do. It's
> convinient to add prepalldocs to a certain point of src_install to compress
> the files that needs to be compressed, and then install the ones that can't
> be compressed.

So maybe a function with the inverted meaning of prepalldocs should be introduced for these cases? I.e., it would mark its arguments (which would be files or directories) to be left alone by any compression mechanism of the package manager.

In fact, a similar exclusion mechanism has already been proposed in bug 164114. If something like that was implemented, then prepalldocs could be called by the package manager and there would be no more need for it to appear in ebuilds. (And since ebuilds could exclude all of /usr/share/doc, all cases should be covered. Or am I missing something?)
Comment 57 Steve L 2009-01-23 11:04:07 UTC
(In reply to comment #53)
> Well... Does the Council really think that a utility whose behaviour keeps on
> changing in weird ways is suitable for formal specification?
The purpose (as stated in comment 40) is to get a clear specification based on the intent of earlier code, and ebuild authors' and end-users' understanding of when PORTAGE_COMPRESS* (as documented in `man make.conf`) will be run on files in the doc hierarchy.

> And can the
> Council make sure that the behaviour will remain as it is for stable Portage?
>
How can the Council be responsible for bugs in projects which they do not run?
 
> The alternative is to go through the tree and simply remove all prep* calls.
Or get to a sane specification in line with user expectations.

> Any ebuild that relies upon any of those calls doing anything in particular is
> already broken.
> 
That's fine; it's users who want to rely on being able to control what happens to the documents installed on their systems.
Comment 58 Ciaran McCreesh 2009-01-25 21:08:28 UTC
(In reply to comment #57)
> The purpose (as stated in comment 40) is to get a clear specification based on
> the intent of earlier code, and ebuild authors' and end-users' understanding of
> when PORTAGE_COMPRESS* (as documented in `man make.conf`) will be run on files
> in the doc hierarchy.

But that's not what it does.

> > And can the
> > Council make sure that the behaviour will remain as it is for stable Portage?
> >
> How can the Council be responsible for bugs in projects which they do not run?

If you're going to carry on 'contributing' to PMS bugs, please at least read up on what the Council said about this. There are mechanisms in place for the Council to get Portage to comply with PMS where necessary.

> > The alternative is to go through the tree and simply remove all prep* calls.
> Or get to a sane specification in line with user expectations.

prep* doesn't do what the user expects.

> > Any ebuild that relies upon any of those calls doing anything in particular is
> > already broken.
> > 
> That's fine; it's users who want to rely on being able to control what happens
> to the documents installed on their systems.

Utterly pointless. Again, read the previous discussion with the Council.
Comment 59 Steve L 2009-01-29 02:26:04 UTC
(In reply to comment #58)
> (In reply to comment #57)
> > The purpose (as stated in comment 40) is to get a clear specification based on
> > the intent of earlier code, and ebuild authors' and end-users' understanding of
> > when PORTAGE_COMPRESS* (as documented in `man make.conf`) will be run on files
> > in the doc hierarchy.
> 
> But that's not what it does.
>
Ignoring the grammatically nonsensical reply, I refer you to comment 20:
> We are talking about prepalldocs and dodoc, and how they are used to install
> and/or ecompress files installed into "${D}"/usr/share/doc/${PF}. Period.
 
> > > And can the
> > > Council make sure that the behaviour will remain as it is for stable Portage?
> > >
> > How can the Council be responsible for bugs in projects which they do not run?
 
> If you're going to carry on 'contributing' to PMS bugs, please at least read up
> on what the Council said about this. There are mechanisms in place for the
> Council to get Portage to comply with PMS where necessary.
>
And there are other projects which are supposed to adhere to PMS; if you meant in portage, you should have specified that. There's also a helluva lot more chance of anyone adhering to it if it's actually specified (and not in nonsensical terms like you and dleverton spammed this bug with.)

> > > The alternative is to go through the tree and simply remove all prep* calls.
> > Or get to a sane specification in line with user expectations.
> 
> prep* doesn't do what the user expects.
>
comment 40, yet again. If you're going to keep filibustering, at least use a novel argument.

> > That's fine; it's users who want to rely on being able to control what happens
> > to the documents installed on their systems.
> 
> Utterly pointless. Again, read the previous discussion with the Council.
> 
Well that's big of you, as ever. You know full well there are no Council logs on the web after 10/23/08-- you've complained about it on the dev ml.

Seems to me that others have been calling for specification of the interaction with PORTAGE_COMPRESS*, which is what this bug is about. Since PMS is supposed first and foremost to document what Portage is intended to do, and Gentoo devs have repeatedly stated that they want this specified correctly so that they can move forward, and it's been in portage for such a long time, there is no reason not to specify it in line with their wishes. (Though I'm sure you'll drag one up.)

(In reply to comment #51)
> David is providing useful contributions.
If the language in comment 42 were a useful version of a specification, why did you query it in comment 45? Further, why did you not simply work on the specification in collaboration with Spanky et al after comment 38, when you got a grudging yes, so long as things could be better defined?

Removing my CC as I have no wish to run around any more circles with you; all you're doing is wasting everyone's time with repeated irrelevant argument, and jokes with your friend about who can make up the stupidest spec.

I'll check back in a while to see how far this has 'progressed'.
Comment 60 Ciaran McCreesh 2009-01-29 17:49:24 UTC
(In reply to comment #59)
> > But that's not what it does.
> >
> Ignoring the grammatically nonsensical reply, I refer you to comment 20:

The reply makes perfect sense. And yes, we are talking about prepalldocs, and no, what it does is not what you seem to want it to do.

> And there are other projects which are supposed to adhere to PMS; if you meant
> in portage, you should have specified that. 

Uh, yes, because other projects containing implementations of prepalldocs go around changing its behaviour all the time... Riiiiight.

> There's also a helluva lot more chance of anyone adhering to it if it's 
> actually specified (and not in nonsensical terms like you and dleverton 
> spammed this bug with.)

The specification we're working out *is* what it does, if you look across multiple Portage versions. The only nonsense here is Portage's behaviour.

> > prep* doesn't do what the user expects.
> >
> comment 40, yet again. If you're going to keep filibustering, at least use a
> novel argument.

We have yet to get a guarantee that specifying a particular behaviour will make any difference to what Portage does. If the Council's not prepared to demand Portage be reverted to stable behaviour and stay that way, there's absolutely no point.

> Seems to me that others have been calling for specification of the interaction
> with PORTAGE_COMPRESS*, which is what this bug is about.

The only person talking about PORTAGE_COMPRESS is you. The rest of us know that PORTAGE_COMPRESS is utterly beyond the scope of PMS and has nothing to do with any of this.

> Since PMS is supposed first and foremost to document what Portage is intended 
> to do, and Gentoo devs have repeatedly stated that they want this specified
> correctly so that they can move forward, and it's been in portage for such a 
>long time, there is no reason not to specify it in line with their wishes. 

PMS has to walk the fine line between specifying what Portage is intended to do and what it actually does. This is a major source of pain.

Also, other Gentoo devs (including more than one Council member) have agreed with my position that prep* should be nuked. And considering their wishes are a lot easier to go with, taking that route is a much better option.

> (In reply to comment #51)
> > David is providing useful contributions.
> If the language in comment 42 were a useful version of a specification, why did
> you query it in comment 45?

The query is about whether a utility that requires the degree of vagueness specified in comment 42 is suitable for standardisation.

> Further, why did you not simply work on the
> specification in collaboration with Spanky et al after comment 38, when you got
> a grudging yes, so long as things could be better defined?

Because it's been made clear that there are people more important than Mike who want to give serious consideration to getting prep* nuked rather than specifying dumb behaviour.

> Removing my CC as I have no wish to run around any more circles with you; all
> you're doing is wasting everyone's time with repeated irrelevant argument, and
> jokes with your friend about who can make up the stupidest spec.

Yes, by going along with what the Council are doing *I*'m the one wasting time. Again, please stop 'contributing' to PMS bugs.
Comment 61 Ulrich Müller gentoo-dev 2009-02-01 10:12:46 UTC
Steve, Ciaran, would you mind if we get back on topic? Removing userrel from CC, since I don't see what would be their business here.

(In reply to comment #60)
> The rest of us know that PORTAGE_COMPRESS is utterly beyond the scope of PMS

This may be true.

> and has nothing to do with any of this.

Fact is that Portage implements such a feature, i.e. it may compress some of the files an ebuild installs. On the other hand (see for example comment #27), some packages actually access their documentation files in /usr/share/doc at runtime, and will not function with compressed files.

So the ebuild needs some means to tell the package manager what files should be left unchanged, or - the logical inverse - what files it is allowed to change (especially to compress).

(In reply to comment #55)
> b) Read all of dleverton's analysis of what prepalldocs actually does in
> different Portage versions to see that it has no stable, well defined
> behaviour that can be relied upon anyway.

This is unfortunate and probably a bug in recent Portage. It used to behave as described in comment #20.
Comment 62 Ciaran McCreesh 2009-02-01 14:56:07 UTC
(In reply to comment #61)
> So the ebuild needs some means to tell the package manager what files should be
> left unchanged, or - the logical inverse - what files it is allowed to change
> (especially to compress).

No, it doesn't.

If an ebuild specifically requires some particular kind of compression, it can do it itself, in a way that it can control. Otherwise, compression is pointless.

> (In reply to comment #55)
> > b) Read all of dleverton's analysis of what prepalldocs actually does in
> > different Portage versions to see that it has no stable, well defined
> > behaviour that can be relied upon anyway.
> 
> This is unfortunate and probably a bug in recent Portage. It used to behave as
> described in comment #20.

No, it's an intentional change in behaviour, which shows that the Portage people don't consider it a stable utility with well defined behaviour.
Comment 63 Ulrich Müller gentoo-dev 2009-02-01 16:07:27 UTC
(In reply to comment #62)
> > So the ebuild needs some means to tell the package manager what files
> > should be left unchanged, or - the logical inverse - what files it is
> > allowed to change (especially to compress).
>
> No, it doesn't.
>
> If an ebuild specifically requires some particular kind of compression,
> it can do it itself, in a way that it can control.

You miss the point. The point is that Portage allows the user to specify compression and that some packages need documentation files that are _not_ compressed.

> Otherwise, compression is pointless.

What has this opinion of yours to do with this bug? Stay on topic, please.
Comment 64 Ciaran McCreesh 2009-02-01 16:12:34 UTC
(In reply to comment #63)
> You miss the point. The point is that Portage allows the user to specify
> compression and that some packages need documentation files that are _not_
> compressed.

That's ok, because the package manager doesn't compress things unless it's told to.

> > Otherwise, compression is pointless.
> 
> What has this opinion of yours to do with this bug? Stay on topic, please.

See the most recent Council meeting. This is not a matter of opinion.
Comment 65 Ulrich Müller gentoo-dev 2009-02-01 16:24:33 UTC
> That's ok, because the package manager doesn't compress things unless it's
> told to.

By prepalldocs. That's what this bug is about.
Comment 66 Ciaran McCreesh 2009-02-01 16:30:55 UTC
(In reply to comment #65)
> > That's ok, because the package manager doesn't compress things unless it's
> > told to.
> 
> By prepalldocs. That's what this bug is about.

As has already been stated several times:

prepalldocs is a) pointless and b) way too weirdly behaving.

If an ebuild specifically needs compression, it can't use prepalldocs because prepalldocs might not compress, or it might not compress the way the ebuild needs. If an ebuild doesn't specifically need compression, there's no point in letting the user choose what colour to paint the doorhandle on the bikeshed.

Please read the Council discussion on this before commenting further.
Comment 67 Ulrich Müller gentoo-dev 2009-02-01 16:51:51 UTC
> As has already been stated several times:
> 
> prepalldocs is a) pointless and b) way too weirdly behaving.

And there is currently no alternative for its functionality.

> If an ebuild doesn't specifically need compression, there's no point in
> letting the user choose

Earlier (comment #60) you said that "PORTAGE_COMPRESS is utterly beyond the scope of PMS". Fact is that Portage authors decided to let the user choose. And last time I checked, its default was to enable compression.

> what colour to paint the doorhandle on the bikeshed.

This discussion would really profit from a more factual diction.

> Please read the Council discussion on this before commenting further.

Read it yourself again and you may notice that I was present.
Comment 68 Ciaran McCreesh 2009-02-01 17:01:16 UTC
(In reply to comment #67)
> > As has already been stated several times:
> > 
> > prepalldocs is a) pointless and b) way too weirdly behaving.
> 
> And there is currently no alternative for its functionality.

And there is no need for its functionality.

> > If an ebuild doesn't specifically need compression, there's no point in
> > letting the user choose
> 
> Earlier (comment #60) you said that "PORTAGE_COMPRESS is utterly beyond the
> scope of PMS". Fact is that Portage authors decided to let the user choose. And
> last time I checked, its default was to enable compression.

You misunderstand. PORTAGE_COMPRESS is a user configuration variable that isn't meaningful for ebuilds. User configuration is beyond the scope of PMS.

> > what colour to paint the doorhandle on the bikeshed.
> 
> This discussion would really profit from a more factual diction.

The *fact* is, user selectable compression for docs is providing choice for the sake of doing so.

> > Please read the Council discussion on this before commenting further.
> 
> Read it yourself again and you may notice that I was present.

Then why do you keep repeating things that were already covered?
Comment 69 Alec Warner (RETIRED) archtester gentoo-dev Security 2009-02-01 21:46:42 UTC
(In reply to comment #68)
> (In reply to comment #67)
> > > As has already been stated several times:
> > > 
> > > prepalldocs is a) pointless and b) way too weirdly behaving.
> > 
> > And there is currently no alternative for its functionality.
> 
> And there is no need for its functionality.

Ciaran: Regardless of the *need* (which is debatable) people do use it.  I would be hard pressed to agree to an argument of it not being in EAPI-0.  Sure its portage specific but it is useful.  The fact that it may have changed between releases is unfortunate; but can easily be noted in PMS as well.  I could make an argument that the older versions were 'buggy' and the newer versions were bugfixes.

Ulrich:
I don't see a patch attached to this bug, so I have no idea what exactly you expect PMS to say about prepalldocs.  The discussion has tended toward discussing whether to include anything or not; which is a difficult topic.  Assume that something will be added; what should the text say.  Can you attach a patch?

-A

> 
> > > If an ebuild doesn't specifically need compression, there's no point in
> > > letting the user choose
> > 
> > Earlier (comment #60) you said that "PORTAGE_COMPRESS is utterly beyond the
> > scope of PMS". Fact is that Portage authors decided to let the user choose. And
> > last time I checked, its default was to enable compression.
> 
> You misunderstand. PORTAGE_COMPRESS is a user configuration variable that isn't
> meaningful for ebuilds. User configuration is beyond the scope of PMS.
> 
> > > what colour to paint the doorhandle on the bikeshed.
> > 
> > This discussion would really profit from a more factual diction.
> 
> The *fact* is, user selectable compression for docs is providing choice for the
> sake of doing so.
> 
> > > Please read the Council discussion on this before commenting further.
> > 
> > Read it yourself again and you may notice that I was present.
> 
> Then why do you keep repeating things that were already covered?
> 

Comment 70 Ciaran McCreesh 2009-02-01 21:49:56 UTC
(In reply to comment #69)
> Ciaran: Regardless of the *need* (which is debatable) people do use it. 

That's easily fixed by removing all calls to it and a simple repoman check.

> Sure its portage specific but it is useful.

It has no useful use. Nor is it Portage specific.

>  The fact that it may have changed between releases is unfortunate; but can
> easily be noted in PMS as well.  I could make an argument that the older 
> versions were 'buggy' and the newer versions were bugfixes.

There's no way the weird behaviour of newer versions can be classified as fixing anything.
Comment 71 Alec Warner (RETIRED) archtester gentoo-dev Security 2009-02-02 06:21:14 UTC
(In reply to comment #70)
> (In reply to comment #69)
> > Ciaran: Regardless of the *need* (which is debatable) people do use it. 
> 
> That's easily fixed by removing all calls to it and a simple repoman check.

Right but until then...we can document it.  I don't really expect you to head up some kind of eradication squad for prepalldocs.  All I expect from you is that you look at and possibly accept a patch from Ulrich.

> 
> > Sure its portage specific but it is useful.
> 
> It has no useful use. Nor is it Portage specific.

Ok sure, Paludis has an implementation.

Usefulness is hard to measure though.  If we look at invocations it is used by 1% of the tree; so possibly not too useful; but not useless either.  Usefulness is not defined by you.

> 
> >  The fact that it may have changed between releases is unfortunate; but can
> > easily be noted in PMS as well.  I could make an argument that the older 
> > versions were 'buggy' and the newer versions were bugfixes.
> 
> There's no way the weird behaviour of newer versions can be classified as
> fixing anything.
> 

So don't classify it as such.

Anyway as always I await a patch ;)

-Alec
Comment 72 Ciaran McCreesh 2009-02-02 14:15:10 UTC
(In reply to comment #71)
> Right but until then...we can document it.  I don't really expect you to head
> up some kind of eradication squad for prepalldocs.  All I expect from you is
> that you look at and possibly accept a patch from Ulrich.

Eradicating prepalldocs is a cleaner solution than documenting it.

> Usefulness is hard to measure though.  If we look at invocations it is used by
> 1% of the tree; so possibly not too useful; but not useless either.  Usefulness
> is not defined by you.

No, we look at what it does. And what it does is useless.
Comment 73 Ulrich Müller gentoo-dev 2009-02-02 15:13:05 UTC
(In reply to comment #71)
> Anyway as always I await a patch ;)

As long as there is no agreement that it should be documented, it's not very useful to prepare a patch ... Anyhow, here's my attempt to formulate the suggestion of comment #42 in a more neutral way.

"Allow the package manager to make implementation dependent modifications (usually some form of compression) to the files in ${D}/usr/share/doc, at an arbitratry later part of the build process."

This would be for EAPI<=2, for later EAPIs it may be changed (or removed) of course.
Comment 74 Thomas Anderson (tanderson) (RETIRED) gentoo-dev 2009-02-12 23:09:56 UTC
Council has banned PMS in current EAPIs(0,1,2). Closing.
Comment 75 Brian Harring (RETIRED) gentoo-dev 2009-02-12 23:16:46 UTC
I assume you meant "banned prepall" not PMS there...
Comment 76 Thomas Anderson (tanderson) (RETIRED) gentoo-dev 2009-02-12 23:17:53 UTC
er yeah, I fail.
Comment 77 Ulrich Müller gentoo-dev 2009-02-13 06:16:04 UTC
(In reply to comment #74)
> Council has banned PMS in current EAPIs(0,1,2).

And in their infinite wisdom, what do they propose as substitute for its functionality? Call ecompress/ecompressdir directly? :-(
Comment 78 Petteri Räty (RETIRED) gentoo-dev 2009-02-13 12:15:42 UTC
(In reply to comment #77)
> (In reply to comment #74)
> > Council has banned PMS in current EAPIs(0,1,2).
> 
> And in their infinite wisdom, what do they propose as substitute for its
> functionality? Call ecompress/ecompressdir directly? :-(
> 

Equally Portage internal. Use available public APIs or open a request for new stuff with clearly defined behaviour. This bug is getting quite long so opening a new one makes more sense IMHO.
Comment 79 Ulrich Müller gentoo-dev 2009-02-13 14:04:49 UTC
(In reply to comment #78)
> > And in their infinite wisdom, what do they propose as substitute for its
> > functionality? Call ecompress/ecompressdir directly? :-(
> 
> Equally Portage internal.

Internal, yes, but not "equally". Obviously prepalldocs is _not_ Portage internal, because it wasn't ever called internally, but only from ebuilds.
Comment 80 Ulrich Müller gentoo-dev 2009-02-24 11:59:30 UTC
(In reply to comment #78)
> Use available public APIs or open a request for new stuff with clearly
> defined behaviour. This bug is getting quite long so opening a new one
> makes more sense IMHO.

I've opened bug 260188 for my "docompress" proposal.
Comment 81 Ulrich Müller gentoo-dev 2009-03-08 14:45:11 UTC
(In reply to comment #80)
> I've opened bug 260188 for my "docompress" proposal.

Eh, that's bug 260118.