Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 120088 - dev-lang/php does not work on default USE
Summary: dev-lang/php does not work on default USE
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: High trivial (vote)
Assignee: PHP Bugs
URL:
Whiteboard:
Keywords:
: 122469 (view as bug list)
Depends on:
Blocks: 119737 123844
  Show dependency tree
 
Reported: 2006-01-23 11:54 UTC by Chris Gianelloni (RETIRED)
Modified: 2006-11-11 19:51 UTC (History)
12 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Chris Gianelloni (RETIRED) gentoo-dev 2006-01-23 11:54:31 UTC
Here's the problem... dev-lang/php does not work with the default USE from 2005.0, 2005.1, or 2006.0's profiles.  Forcing a user to edit package.use for something as simple as PHP is seriously uncalled for and should never be a requirement.

Take this as an example:

 * USE flag 'berkdb' needs these additional flag(s) set:
 *    dba

This is 100% incorrect.  If having USE="berkdb" requires that the functionality for USE="dba" be enabled, then USE="berkdb" should enable it when set.  The user should not have to be required to perform interactive steps for this to work.

Here is another example:

 * USE flag 'truetype' needs one of these additional flag(s) set:
 *    gd gd-external

Now, both berkdb and truetype are in default USE for the release, whereas dba and gd/gd-external are not.  This means dev-lang/php does not build.  This is against policy[1][2].  It also completely breaks any and all tinderbox and release builds using any package requiring PHP support.  This is *not* acceptable.

[1] http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=2&chap=1 (USE variables)
[2] http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=3&chap=1#doc_chap1
Comment 1 Jakub Moc (RETIRED) gentoo-dev 2006-01-23 12:02:14 UTC
Wonderful. So - will you now file blocker bugs about tens of other ebuilds that bail out when checking for kernel features, use flags or whatever? Fix portage, ktnxbye, dunno why are you assigning this bug to php. 

We can't automagically enable use flags since it breaks other ebuilds that check for them with built_with_use. 

Re-assigning to portage, enjoy. ;)
Comment 2 SpanKY gentoo-dev 2006-01-23 12:17:13 UTC
> Wonderful. So - will you now file blocker bugs about tens of other ebuilds 
> that bail out when checking for kernel features, use flags or whatever?

actually we did and in case you hadnt noticed, a lot less kernel packages abort now during emerge

this is a bug regardless of how you care to look at it
Comment 3 Chris Gianelloni (RETIRED) gentoo-dev 2006-01-23 13:59:48 UTC
Like I said, when you install from stage3, then immediately type "emerge php", I would expect it to work.  If it does not, then it is a failure in the ebuild and a bug.  I don't care if you want to blame it on portage or not, but it is your responsility to ensure that your package is not broken with a default installation.  If this is going to be an issue, then I'm simply going to ask that the older PHP stuff, which does work from a default install, remain in the tree until this bug is resolved.
Comment 4 Jakub Moc (RETIRED) gentoo-dev 2006-01-23 14:07:39 UTC
(In reply to comment #3)
> If this is going to be an issue, then I'm simply going to ask
> that the older PHP stuff, which does work from a default install, remain in the
> tree until this bug is resolved.

Sure, we plan that it will remain in the tree for a couple of months, package.masked. HTH. ;)
Comment 5 Chris Gianelloni (RETIRED) gentoo-dev 2006-01-23 14:21:12 UTC
So you are admitting that you are planning on intentionally masking the *only* versions that work from a default installation?

What I find so humorous is that rather than trying to work with anyone to actually resolve this, you've chosen to try to pass the buck and blame someone else.  Notice what my bug says... default USE.  This means if a working PHP were actually compiled with the default USE (read, profiles had proper USE as defaults) then this wouldn't be an issue.

Luckily, we won't have to visit this quite so soon since installations done using our sane release media won't be including this obviously broken version of PHP.  I truly fear for what lies in store for 2006.1, though.  You have ~6 months to figure out something.

I wish you all luck.
Comment 6 Jakub Moc (RETIRED) gentoo-dev 2006-01-23 15:08:23 UTC
(In reply to comment #5)
> So you are admitting that you are planning on intentionally masking the *only*
> versions that work from a default installation?

No, we are planning to mask broken, unmaintained and unsupported versions. Obviously there are much more important things and features, compared to asking users to put one or two use flags to /etc/make.conf or package.use. What you call "versions that work" fails to compile in numerous cases, has issues with collision-protect, etc. etc. 

> What I find so humorous is that rather than trying to work with anyone to
> actually resolve this, you've chosen to try to pass the buck and blame someone
> else.  

While talking about "trying to work with anyone", maybe you could visit us in #gentoo-apache or #gentoo-php or send email to php@gentoo.org next time (like many arch people did while keywording these ebuilds), instead of filing blocker bugs, ranting about breaking the policy, broken ebuilds and suggesting solutions that just won't work.

> Notice what my bug says... default USE.  This means if a working PHP
> were actually compiled with the default USE (read, profiles had proper USE as
> defaults) then this wouldn't be an issue.

Of course we can add the two flags to make.defaults (there's already lot of ebuild-specific crap anyway) and be done with it in five minutes. Could already be done probably, if you asked for it. 

> Luckily, we won't have to visit this quite so soon since installations done
> using our sane release media won't be including this obviously broken version
> of PHP. 

See above. What you call broken - we call major improvement. I've already explained why enabling features automagically won't fly (and this is a real, not theoretical problem with gd/gd-external e.g., there are ebuilds in the tree that do check for this). 

Frankly - you should really know better than insulting hard work done by lots of people, including the arches that have tested and keyworded the stuff. If you had any clue whatsoever about PHP stuff, you and your "sane release media" wouldn't force users to install unsupported, deprecated and broken stuff, that will make them just waste their time and cause unneeded migration issues. Or maybe you are volunteering to help those users with that (totally unneeded) migration? In that case you are welcome to provide them with support in #gentoo-php. We don't support those packages any more.
Comment 7 Chris Gianelloni (RETIRED) gentoo-dev 2006-01-23 15:38:43 UTC
(In reply to comment #6)
> No, we are planning to mask broken, unmaintained and unsupported versions.

That just happen to be the only versions that work from a default installation.  

Brilliant.

Have you ever heard of a regression?  This is a prime example.  When something used to work, and now it does not, it is a QA issue and should be treated as such.

> Obviously there are much more important things and features, compared to asking
> users to put one or two use flags to /etc/make.conf or package.use. What you
> call "versions that work" fails to compile in numerous cases, has issues with
> collision-protect, etc. etc.

No, what should have been done is that a workable solution should have been made before the packages ever went stable.  It is *your* responsibility to make sure that *your* packages work, not mine.  You are the maintainer.  This is why I am filing a bug and assigning it to *you* rather than fixing it myself.

> While talking about "trying to work with anyone", maybe you could visit us in
> #gentoo-apache or #gentoo-php or send email to php@gentoo.org next time (like
> many arch people did while keywording these ebuilds), instead of filing blocker
> bugs, ranting about breaking the policy, broken ebuilds and suggesting
> solutions that just won't work.

I was udner the impression that filing bugs was how we did things around here.  Excuse me for following a standard protocol for getting bugs resolved in Gentoo.  This could have easily been resolved long ago had proper care been taken proactively, rather than instead becoming defensive when someone points out a true and valid bug.

> Of course we can add the two flags to make.defaults (there's already lot of
> ebuild-specific crap anyway) and be done with it in five minutes. Could already
> be done probably, if you asked for it.

Perhaps you are missing the point of this bug, then.  See, I *am* asking for some action to be taken, you just seem to be missing it.  Why else would this bug have been filed?

> > Luckily, we won't have to visit this quite so soon since installations done
> > using our sane release media won't be including this obviously broken version
> > of PHP. 
> 
> See above. What you call broken - we call major improvement. I've already
> explained why enabling features automagically won't fly (and this is a real,
> not theoretical problem with gd/gd-external e.g., there are ebuilds in the tree
> that do check for this).

...and there are other solutions to work around this issue.  For example, if berkdb requires dba, then an ebuild that requires that dev-lang/php was built with dba could check that *either* flag were enabled.  This really is not that hard to accomplish.

> Frankly - you should really know better than insulting hard work done by lots
> of people, including the arches that have tested and keyworded the stuff. If
> you had any clue whatsoever about PHP stuff, you and your "sane release media"
> wouldn't force users to install unsupported, deprecated and broken stuff, that
> will make them just waste their time and cause unneeded migration issues. Or
> maybe you are volunteering to help those users with that (totally unneeded)
> migration? In that case you are welcome to provide them with support in
> #gentoo-php. We don't support those packages any more.

*sigh*

Since it is obvious that you're not going to listen to reason and instead are going to become defensive and say that I am "insulting" you by pointing out a verifiable *bug* in an ebuild, I'm not sure how much traction this is going to receive.  The reason why dev-php/php will be installed on many machines is because it is still the default on many architectures.  I'm not forcing anything on anyone, I simply expect the release media to actually work.  See, that is *my* job.  I am going to do what it takes to make sure our users have a pleasant experience installing Gentoo, otherwise *I* have failed at *my* job.  If that means ruffling a few feathers or bruising a few egos, so be it, but that doesn't excuse this as not being a bug, nor as not blocking proper releases.

I really don't understand the immediate sarcastic response to a proper bug report being filed.

The 2006.0 snapshot has been taken.  We currently have versions of dev-lang/php marked stable on a few architectures where it will not work out of the box.  This *will* need to be resolved before the release.  Now, what I am offering is suggestions on how this can be resolved.  Please take a step back and a deep breath and at least *attempt* to be responsive to these issues rather than immediately becoming defensive and accusing me of "insulting hard work".  I am simply trying to get a working release out the door.  I could give a damn about insulting you or your work.

There is a bug.  I am trying to get it fixed.

The simplest solution is to give me a list of USE flags, above what is enabled in the 2006.0 profiles, for each architecture that has taken dev-lang/php stable, that will at leats build a working version of dev-lang/php from the box.  I am willing to assist with this where I can, but I need you to be calm and helpful, rather than condescending and defensive.

When you are ready to work with us on this, please either file the requested information as patches (or even comments) or let us know what we can do to assist you.  Until then, I am leaving this alone as it is obvious that continuing to comment on this bug is adding nothig to getting to a resolution.

Good day.
Comment 8 Stuart Herbert (RETIRED) gentoo-dev 2006-01-23 15:45:38 UTC
This was discussed in Saturday's PHP herd meeting.  

Long and short of it is that we are waiting for newer versions of Portage to deliver the functionality we need, and until that happens, we're not aware of anything we can do.

--

The long answer:

We decided to maintain the current behaviour until we can find a solution which still allows other packages to use 'built_with_use' accurately.  Using your first example, when USE=berkdb is set, we can pass the correct configure options to configure to enable dba, but 'built_with_use dev-lang/php dba' will still return false, which is incorrect.

To use your second example, when USE=freetype is set, there are two conflicting choices available to the user.  They can choose the bundled gd library (USE=gd), or choose to link against the UPSTREAM gd library (USE=gd-external).  They can't choose both.

I took the decision in 2004 that the only way to handle the situation was to require the user to get their USE flags correct.  At the same time, Ciaran and I filed a bug w/ the Portage team for ebuilds to be able to report their USE flag requirements when the dep tree is created, so that users could be informed as early as possible of the problems with their USE flags.

The way the USE flags work atm is 100% correct with the layered nature of the PHP API.  They result in an installed PHP package that is exact and 100% correct to the user's wishes at all times.  Please visit http://www.php.net/manual/en/ for full documentation, including per-extension API dependencies and installation requirements.

Regarding the policy on USE flags ... I've been through the USE flags section several times, and I can't see anything saying that the behaviour of dev-lang/php wrt USE flags is incorrect.  The only possible part I could conceed is that we require a USE flag to be set for at least one of the supported SAPIs - which indeed is not optional.  However, after three years of living with separate ebuilds for separate SAPIs (which is how to comply with that policy),  all involved have agreed that the unified package is the right way to deliver PHP for Gentoo.  All other USE flags *are* there to enable optional functionality.  You don't need USE=dba unless you're switching on one of the PHP extensions that requires the dba extension.  In your own examples, you're talking about *optional* functionality that is being enabled outside our control in the various profiles.

Regarding the other policy you mention ... nowhere does it say that ebuilds cannot abort if there is a problem with the USE flags.  Is it possible you're mis-interpreting the following section of the policy document?

# Your package, when complete and unmasked, is supposed to "just work" for the end-user. Tweaking the installed product to get it to work should be optional; thus you need to install the package with reasonable default settings.

The key words there are "Tweaking the installed product".  As the ebuild aborts before anything is installed, requiring the user to get their act together and their USE flags in order is not against that section of the policy.  That section was originally added to ensure that packages installed sensible default config files - it was never intended to cover the emerge process itself.

Please quote the relevant sections of those two policy documents that you're referenced if I've missed them.

Regarding tinderbox ... file a bug with the tinderbox maintainers please.  We're not the maintainers of tinderbox, and are not responsible for any problems that it has compiling packages that Portage can compile (and Portage *can* compile our packages).

We're open to practical suggestions on how we can improve this, but we haven't found one yet.

The issue of working with default release profiles is a different story.  We'd love to get some sensible default USE flags for dev-lang/php into the profiles, and CHTEKK took an action from the meeting to look into that.  If that's not possible, then I'm not sure what we can do until Portage allows each ebuild to specify its own default USE flags.

We (the PHP team) wonder if there are more positive steps you could take to help the issue.  It might be worth breaking up the 'default' desktop-centric profile so that users are given the choice between a desktop profile and a profile more suited to server environments - with there being *no* default profile.  You could also offer to help us get our default USE flags into your profiles (something we need some advice on).  These are pro-active measures that'll get you a friendlier and more helpful response than taking a beligerent approach.

Downgrading bug from 'blocker' to 'trivial'.

Best regards,
Stu
--
PS: What packages are you including in 2006.1 that use PHP?  And exactly when were you going to let us know that PHP was being included on the 2006.1 release media? :)  Speaking personally, it'd nice for you to remember from time to time that we're all on the same team.
Comment 9 Jakub Moc (RETIRED) gentoo-dev 2006-01-23 15:50:23 UTC
(In reply to comment #7)
> (In reply to comment #6)
> > No, we are planning to mask broken, unmaintained and unsupported versions.
> 
> That just happen to be the only versions that work from a default installation. 

Frankly, I don't care much about your "default" installation. We have no control over it. When someone for whatever reason puts yet another crufty use flag there and "breaks" default install of whatever, we'll then stick another couple of use flags there to remedy it? Or what's your solution. There's apparent lack of per-package use flags, that could finally stop this never-ending profiles bloat. 


> Have you ever heard of a regression? 

Your bloated profiles are a regression. There's another regression with esd use flag added to 2006.0 profiles.

> No, what should have been done is that a workable solution should have been
> made before the packages ever went stable.  It is *your* responsibility to make
> sure that *your* packages work, not mine.  You are the maintainer.  

Noone ever mentioned you are going to put PHP on stages, great job, really. :P As is announcing a release 24 hours before it should happen.

> This could have easily been resolved long ago had proper care been
> taken proactively, rather than instead becoming defensive when someone points
> out a true and valid bug.

No, it couldn't. It could be worked around, it only could be solved when Bug 61732 is solved.

> ...and there are other solutions to work around this issue.  For example, if
> berkdb requires dba, then an ebuild that requires that dev-lang/php was built
> with dba could check that *either* flag were enabled.  This really is not that
> hard to accomplish.

No. Solution to this are USE-based dependencies, not reworking the ebuilds over and over again once a weird use flag changes somewhere. Also, the ebuilds don't find themselves, someone has to do the job.
Comment 10 SpanKY gentoo-dev 2006-01-23 15:53:31 UTC
think you guys could take your pissing fight off bugzilla and to e-mail ?  it's really not getting anything done
Comment 11 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2006-01-23 15:56:15 UTC
So to conclude, Chris requires a list of use flags to add to default USE on all arches that have new PHP stable in the 2006.0 snapshot, so that he can add them to the profiles so that PHP will compile properly.  This is ( as far as I am aware ) the only sane solution.

No need to get all pissy, or write a huge essay on why either side sucks at their job.  In the end it probably IS a portage issue, and workarounds need to be made.  Chris, PHP probably needs to know what arches have PHP marked stable in your snapshot.

Jakub, I don't appreciate the "it's a portage problem, so screw off" much either.  Obviously this is a problem for Chris, so lets work to get it resolved. Otherwise the portage team knows this is a problem and we are working on it;heck in this particular case, I am working on it ;)
Comment 12 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2006-01-23 15:57:47 UTC
er Severity changed on me ;)
Comment 13 Stuart Herbert (RETIRED) gentoo-dev 2006-01-23 16:03:20 UTC
(In reply to comment #5)
> So you are admitting that you are planning on intentionally masking the *only*
> versions that work from a default installation?

Chris - it's long been publically and prominently announced that the old-style dev-php/php packages are going to be dropped from the tree.  This should not be news to anyone.

The new dev-lang/php packages work.  You seem to have a personal objection to how they work, but I can't agree with your statement that they do not work.

> What I find so humorous is that rather than trying to work with anyone to
> actually resolve this, you've chosen to try to pass the buck and blame someone
> else.  

That's not true.  The meeting minutes from Saturday's PHP Herd meeting clearly state that CHTEKK is going to get some default USE flags added to the profiles.  That information had already been publically announced before you made this comment.

> Luckily, we won't have to visit this quite so soon since installations done
> using our sane release media won't be including this obviously broken version
> of PHP.  

That's very irresponsible on your part, as you are choosing to ship packages that you know full well are no longer supported, and which will be removed from Portage shortly.

If you're seriously going to ship dev-php/php on 2006.1, then I personally have to question whether your conduct is fit and proper for the lead of the releng team.  Instead of working with us to get our USE flags into the profile, you'd actively shaft our users?  That's not on.

We will work with you.  But we're not going to accept the way you think you can treat us or our contribution to Gentoo.  It is your attitude and continued poor manners and inter-personal skills which make it difficult for our team to work with you.

Best regards,
Stu
Comment 14 Stuart Herbert (RETIRED) gentoo-dev 2006-01-23 16:40:13 UTC
(In reply to comment #7)
> ...and there are other solutions to work around this issue.  For example, if
> berkdb requires dba, then an ebuild that requires that dev-lang/php was built
> with dba could check that *either* flag were enabled.  This really is not that
> hard to accomplish.

That is not a practical solution.  We are not going to push this level of knowledge down into web-based apps.  It's PHP's berkdb API that needs dba, not the web-based apps.

> Since it is obvious that you're not going to listen to reason and instead are
> going to become defensive and say that I am "insulting" you by pointing out a
> verifiable *bug* in an ebuild, I'm not sure how much traction this is going to
> receive.  

I've asked Jakub to calm it a bit.  I've been on the receiving end of his bug commenting in the past when he gets the bit between his teeth, and I didn't like it either :)

> The reason why dev-php/php will be installed on many machines is
> because it is still the default on many architectures.  

I guess you're refering to the snaphot.  But the snapshot's only used to build release media, right?  Once users have installed Gentoo, they'll just pick up the dev-lang/php packages (which are now default in the live tree for all except alpha).

> I am simply trying to get a working release out the door.  I could give a damn about
> insulting you or your work.
> 
> There is a bug.  I am trying to get it fixed.

If you did give a damn about how your comments are perceived, people might be more helpful in return.  There's nothing gets people's backs up like little footstampers who think they know it all - and that is how you come across to us, whether you mean to or not.

> The simplest solution is to give me a list of USE flags, above what is enabled
> in the 2006.0 profiles, for each architecture that has taken dev-lang/php
> stable, that will at leats build a working version of dev-lang/php from the
> box.  I am willing to assist with this where I can, but I need you to be calm
> and helpful, rather than condescending and defensive.

:)  We will get that done.  CHTEKK has an action from Saturday's meeting to do exactly that.

Best regards,
Stu
Comment 15 Brian Harring (RETIRED) gentoo-dev 2006-01-23 22:08:10 UTC
transeferring this; if you want it duped on other portage bugs regarding use deps/default use transfer it back >:)

The warring over this particular short term issue is y'alls mess to sort out; there's nothing the portage team can do in the time frame involved to help, plus we *are* working on long term solution to this.

Basically, solve the short term issue without spamming our bugzie addie, kthnx :)
Comment 16 Sebastian Bergmann (RETIRED) gentoo-dev 2006-01-23 22:23:49 UTC
The two given examples could be "easily" "fixed" (IMHO) as follows:

1. berkdb

When USE=berkdb and none of its required USE flags is set, display a warning, maybe wait a couple of seconds, and then proceed as if USE=-berkdb.

2. truetype

Either do the same as with 1. or default to the behaviour of USE="gd truetype" which uses the bundled GD library.
Comment 17 Luca Longinotti (RETIRED) gentoo-dev 2006-01-24 00:01:35 UTC
Hi all, as soon as I get home today I'll get in touch with releng/arch teams to fix the default USE issue, but anyway: those two cases can easily be fixed, just add "dba" and "gd-external" to the default USE, as PHP is the only package using them (plus an out-of-date thttpd version, so that doesn't really count, and when it will be updated, it will anyway have the same USE flags). Also the addition of "session", "pcre", "cli" and a couple of others to the profiles default USE would help us a lot, at least until Portage provides a definitive solution for this.
Best regards, CHTEKK.
Comment 18 Chris Gianelloni (RETIRED) gentoo-dev 2006-01-24 06:36:42 UTC
(In reply to comment #8)
> Long and short of it is that we are waiting for newer versions of Portage to
> deliver the functionality we need, and until that happens, we're not aware of
> anything we can do.

There is plenty that can be done to rectify this with the current stable portage.

> We decided to maintain the current behaviour until we can find a solution which
> still allows other packages to use 'built_with_use' accurately.  Using your

No offense, but why?  Why use built_with_use in this case at all?  Would it not be much simpler to keep your own metadata for this sort of thing, seeing as how built_with_use obviously does not work for you?  The way I see it, you guys are encouraging a fairly broken behavior simply to fit with what we already have in the tree, which isn't the best solution.

> first example, when USE=berkdb is set, we can pass the correct configure
> options to configure to enable dba, but 'built_with_use dev-lang/php dba' will
> still return false, which is incorrect.

Only if your checking does not take this into account.

if built_with_use dev-lang/php berkdb; then
  some_code_here that enables berkdb_and_dba
elif built_with_use dev-lang/php dba; then
  some_code_here that enables dba_only
fi

This really is not that hard, and allows for flags that *require* other flags be set to work without interaction.  The user has already chosen berkdb, which *requires* dba be enabled, so why even give them the option to *attempt* to run with USE="berkdb -dba" when it is an invalid combination?


> To use your second example, when USE=freetype is set, there are two conflicting
> choices available to the user.  They can choose the bundled gd library
> (USE=gd), or choose to link against the UPSTREAM gd library (USE=gd-external). 
> They can't choose both.

You're right, they cannot.  However, you can, and should, choose a sensible default.  I would say if USE="truetype" then you should default to USE="gd" *unless* the user *also* has USE="gd-external" in their flags.  Again, having a set of sensible defaults allows the package to build without requiring a portage feature that we don't have, making the ebuild interactive, bloating the profiles, or removing choice from the user.

> I took the decision in 2004 that the only way to handle the situation was to
> require the user to get their USE flags correct.  At the same time, Ciaran and
> I filed a bug w/ the Portage team for ebuilds to be able to report their USE
> flag requirements when the dep tree is created, so that users could be informed
> as early as possible of the problems with their USE flags.

I see no problem with this as a long-term solution.  However, why not provide a sensible set of defaults that works *now* instead?

> optional functionality.  You don't need USE=dba unless you're switching on one
> of the PHP extensions that requires the dba extension.  In your own examples,
> you're talking about *optional* functionality that is being enabled outside our
> control in the various profiles.

I'm sorry, but information in the profiles is very easy to get.  It is also the default.  Both berkdb and truetype have been defaults on x86 for at least 2 years now.  You can't feign ignorance on this issue.  Also, my point is not to disable the ability to disable extensions, but onyl to ensure that the *defaults* actually build.  This seems to be lost on everyone.

I did not file this bug because of *any* reason other than the simple *fact* that the ebuilds do not work out of the box.

You have been working on these ebuilds for months, correct?  Were you not aware that they would not build by default?

> Regarding the other policy you mention ... nowhere does it say that ebuilds
> cannot abort if there is a problem with the USE flags.  Is it possible you're
> mis-interpreting the following section of the policy document?

I never said that an ebuild cannot abort.  Nobody has said that.  Again, you are interjecting your own thoughts on what I am saying into the simple statements I am making.

> # Your package, when complete and unmasked, is supposed to "just work" for the
> end-user. Tweaking the installed product to get it to work should be optional;
> thus you need to install the package with reasonable default settings.

"thus you need to install the package with reasonable default settings."

I think not being able to build with the default USE on a package as critical to many people as PHP, is a failure in this policy.  If you continue to disagree, then we will most likely need to take this to an external entity (Council?) to decide on what the policy really means so one of us can change their expectations.  I don't mean this as a threat or anything like that.  I don't mean it as any sort of disciplinary action.  I simply mean that we are disagreeing on a policy from a technical level, and either need to come to a consensus, or need to have our decision-making body make a decision on this to resolve the conflict.

> The key words there are "Tweaking the installed product".  As the ebuild aborts
> before anything is installed, requiring the user to get their act together and
> their USE flags in order is not against that section of the policy.  That
> section was originally added to ensure that packages installed sensible default
> config files - it was never intended to cover the emerge process itself.

I disagree.  I care not about tweaking.  I care about installing the package with reasonable default settings, which is not being done.

> Regarding tinderbox ... file a bug with the tinderbox maintainers please. 
> We're not the maintainers of tinderbox, and are not responsible for any
> problems that it has compiling packages that Portage can compile (and Portage
> *can* compile our packages).

*sigh*

In this case, tinderbox is a catalyst target.  It uses portage 100% for its building.  It starts from a default stage3 and builds packages listed in the spec file using the default USE/CFLAGS/FEATURES.  If a package fails here, we consider it a QA problem and file bugs on it.  This is why I have filed this bug.

> We're open to practical suggestions on how we can improve this, but we haven't
> found one yet.

See above.

> The issue of working with default release profiles is a different story.  We'd
> love to get some sensible default USE flags for dev-lang/php into the profiles,
> and CHTEKK took an action from the meeting to look into that.  If that's not
> possible, then I'm not sure what we can do until Portage allows each ebuild to
> specify its own default USE flags.

Excellent.  I was unaware of this.  Perhaps had this been said from the beginning rather than the short and rude comments from Jakub, this would have been resolved much sooner and with much less friction.

> We (the PHP team) wonder if there are more positive steps you could take to
> help the issue.  It might be worth breaking up the 'default' desktop-centric
> profile so that users are given the choice between a desktop profile and a
> profile more suited to server environments - with there being *no* default

*sigh*

Apparently nobody pays any attention to emails sent to -dev anymore.  I had already mentioned something very similar to this on several occasions and even have a set of profiles that I have been playing ith in the tree for some time now.  There *will* always be a default profile, and that profile is the desktop profile.  However, I am also looking to have a server profile that works out of the box.  The concept is to have an easily selectable choice between a "default desktop" and "default server" setup.

> profile.  You could also offer to help us get our default USE flags into your
> profiles (something we need some advice on).  These are pro-active measures
> that'll get you a friendlier and more helpful response than taking a beligerent
> approach.

I took no such approach.

I filed a bug.  That is all.  Jakub immediately turned this into a pissing match, rather than working on *any* kind of actual solution.  I thank you for taking the time to look at this seriously and to respond with well thought out words, rather than rash off-the-cuff knee-jerk reactions.

> PS: What packages are you including in 2006.1 that use PHP?  And exactly when

None at this time.  I have a proposal for building separate "desktop" and "server" media, which would include PHP, but nothing is definite at this time.

> were you going to let us know that PHP was being included on the 2006.1 release
> media? :)  Speaking personally, it'd nice for you to remember from time to time
> that we're all on the same team.

In the past, PHP was included on the release media.  We removed it (along with other server products) when space became tight simply because we were unable to offer a comprehensive (or really usable) set of server packages.  There is a *proposal* out to change this in the future, that was sent out in public.  I cannot take it upon myself to contact every single herd or maintainer that might be affected by a proposal that hasn't been accepted and still get any actual work done.  I *did* notice an issue with dev-lang/php that should be resolved, which is the *exact* purpose of this email.  Essentially, I *am* contacting you on this issue.  Saying otherwise is simply false.
Comment 19 Chris Gianelloni (RETIRED) gentoo-dev 2006-01-24 06:50:47 UTC
(In reply to comment #9)
> Frankly, I don't care much about your "default" installation. We have no
> control over it. When someone for whatever reason puts yet another crufty use
> flag there and "breaks" default install of whatever, we'll then stick another
> couple of use flags there to remedy it? Or what's your solution. There's
> apparent lack of per-package use flags, that could finally stop this
> never-ending profiles bloat.

Given your position as a bug-wrangler, I find this attitude apalling.  For one, the USE flags in question have been in the default profiles for over 2 years on at least x86.  Second, you have control over what is in the profiles.  You can file bugs, JUST LIKE THIS ONE to get changes enacted that you require.  Using some feature that is not implemented as an excuse for not working with what you have now is not only poor taste, but rather pointless.

> > Have you ever heard of a regression? 
> 
> Your bloated profiles are a regression. There's another regression with esd use
> flag added to 2006.0 profiles.

Do you know what a "regression" actually means?  Adding additional functionality is not a regression.  Also, the esd USE flag has been in the tree for at least 2 years.  Again, get your facts stright before posting these rude and inflammatory comments.  You're only showing your lack of knowledge on the subject at hand.

> > No, what should have been done is that a workable solution should have been
> > made before the packages ever went stable.  It is *your* responsibility to make
> > sure that *your* packages work, not mine.  You are the maintainer.  
> 
> Noone ever mentioned you are going to put PHP on stages, great job, really. :P

Exactly.  *I* never mentioned that PHP would be in the stages *ever* in this post or anywhere else.  Do yourself a favor and *quit* putting words into my mouth.

> As is announcing a release 24 hours before it should happen.

Nobody announced a release 24 hours before it shoudl happen.  I announced a snapshot 24 hours before, though the general date of the snapshot was known since the posting of the last Release Engineering meeting notes were posted.

> No. Solution to this are USE-based dependencies, not reworking the ebuilds over
> and over again once a weird use flag changes somewhere. Also, the ebuilds don't
> find themselves, someone has to do the job.

That is a long-term solution.  However, we don't live in some fantasy land.  We live in the present.  This means you must adhere to the restrictions and limitations in our current system.

I have proposed  a possible solution and have even said that I would be willing to assist in making the changes.  What more could you ask for?
Comment 20 Chris Gianelloni (RETIRED) gentoo-dev 2006-01-24 06:56:59 UTC
(In reply to comment #13)
> Chris - it's long been publically and prominently announced that the old-style
> dev-php/php packages are going to be dropped from the tree.  This should not be
> news to anyone.

The packages spent very little time in the tree.  As soon as I noticed the issue with them, I filed this bug.

> The new dev-lang/php packages work.  You seem to have a personal objection to
> how they work, but I can't agree with your statement that they do not work.

We'll just have to agree to disagree here.  Yes, they do work.  However, they do not work by default, which is my issue here.

> That's not true.  The meeting minutes from Saturday's PHP Herd meeting clearly
> state that CHTEKK is going to get some default USE flags added to the profiles.
>  That information had already been publically announced before you made this
> comment.

I must admit that I, too, am guilty of not reading everything posted to dev or -core, so I had missed this piece of information.  As I have said before, this is a solution to the problem for releases, but I still don't feel it is a complete solution to the problem.  I apologize that I had not read your meeting summary, but I am still catching up on emails from my recent vacation along with doing work for building this release.

> > Luckily, we won't have to visit this quite so soon since installations done
> > using our sane release media won't be including this obviously broken version
> > of PHP.
> 
> That's very irresponsible on your part, as you are choosing to ship packages
> that you know full well are no longer supported, and which will be removed from
> Portage shortly.

No.  It isn't.

We don't include *any* version of PHP.

> If you're seriously going to ship dev-php/php on 2006.1, then I personally have
> to question whether your conduct is fit and proper for the lead of the releng
> team.  Instead of working with us to get our USE flags into the profile, you'd
> actively shaft our users?  That's not on.

No.  I wouldn't ship PHP, at all.  Apparently you were not paying attention when I stated that my job is to make sure our users don't get things broken.  That does not translate to "actively shafting" anyone.

> We will work with you.  But we're not going to accept the way you think you can
> treat us or our contribution to Gentoo.  It is your attitude and continued poor
> manners and inter-personal skills which make it difficult for our team to work
> with you.

Nobody is "treating" you any way.  I pointed out a *bug* in a package and was attacked for it.  I'm not sure how filign a well-worded and thought-out bug report on our bug tracker constitutes an attitude, but I digress.
Comment 21 Chris Gianelloni (RETIRED) gentoo-dev 2006-01-24 07:03:07 UTC
(In reply to comment #14)
> (In reply to comment #7)
> > ...and there are other solutions to work around this issue.  For example, if
> > berkdb requires dba, then an ebuild that requires that dev-lang/php was built
> > with dba could check that *either* flag were enabled.  This really is not that
> > hard to accomplish.
> 
> That is not a practical solution.  We are not going to push this level of
> knowledge down into web-based apps.  It's PHP's berkdb API that needs dba, not
> the web-based apps.

Why not?  That's what eclasses are for, to reduce the complexity and promote code reuse.

> I've asked Jakub to calm it a bit.  I've been on the receiving end of his bug
> commenting in the past when he gets the bit between his teeth, and I didn't
> like it either :)

Thank you.  This is very much appreciated and I hope that you and myself can work towards a solution to these issues without resorting to the rudeness seen earlier in this bug.

> > The reason why dev-php/php will be installed on many machines is
> > because it is still the default on many architectures.  
> 
> I guess you're refering to the snaphot.  But the snapshot's only used to build
> release media, right?  Once users have installed Gentoo, they'll just pick up
> the dev-lang/php packages (which are now default in the live tree for all
> except alpha).

Only if they perform an "emerge --sync" immediately afterwards.  As the Release Engineering lead, I have seen countless bug reports where a bug was fixed in the tree, but was not fixed in the release snapshot.  Unfortunately, the answer is invariably to resolve the bug as INVALID and to tell the user to "emerge --sync" to get the updated ebuilds, but wouldn't it be better if it just worked immediately?

> If you did give a damn about how your comments are perceived, people might be
> more helpful in return.  There's nothing gets people's backs up like little
> footstampers who think they know it all - and that is how you come across to
> us, whether you mean to or not.

I am not going to be apologetic for misinterpretation on your part.  I have been trying to post only technical information and all I have met with is resistance and rudeness (until your posts, which I appreciate).

> :)  We will get that done.  CHTEKK has an action from Saturday's meeting to do
> exactly that.

Thank you.  This will solve the block on bug #119737 and give us a working release.  I don't feel that it is a perfect long-term or intermediate solution, but it resolves the initial problem.
Comment 22 Chris Gianelloni (RETIRED) gentoo-dev 2006-01-24 07:04:35 UTC
(In reply to comment #17)
> Hi all, as soon as I get home today I'll get in touch with releng/arch teams to
> fix the default USE issue, but anyway: those two cases can easily be fixed,
> just add "dba" and "gd-external" to the default USE, as PHP is the only package
> using them (plus an out-of-date thttpd version, so that doesn't really count,
> and when it will be updated, it will anyway have the same USE flags). Also the
> addition of "session", "pcre", "cli" and a couple of others to the profiles
> default USE would help us a lot, at least until Portage provides a definitive
> solution for this.

The best solution here would be a simple list on this bug of what flags should be enabled by default to build a proper and working PHP and I can make sure that the proper flags get into the proper profiles.

Agsin, thank you for your response on the issue.
Comment 23 Luca Longinotti (RETIRED) gentoo-dev 2006-01-24 07:48:17 UTC
Ok, the list of USE flags we'd like to see enabled by default, needed to build a minimal, usable PHP are the following:

apache2 cli ctype expat fastbuild force-cgi-redirect ftp gd memlimit nls pcre posix session simplexml soap sockets spl ssl tokenizer truetype xml xsl zlib

These should be enabled possibly on all profiles, the majority of those are either already enabled or only used by PHP-related stuff, so I'm pretty sure most of those shouldn't pose a big problem.
I just tried to only emerge -pv dev-lang/php with no USE in package.use or make.conf (profiles only) and saw three USEs that could interrupt the emerge of dev-lang/php: berkdb, truetype and gdbm. berkdb and gdbm need dba set too, wich imo you can safely add too to all profiles default USE, as PHP is the only one using it. truetype needs either gd or gd-external set, now, we as PHP Herd would prefer if gd (as cited above in the list) would be enabled by default, as it's the bundled lib that has many enhancements... If this cannot be done, gd-external would be an acceptable alternative as "default USE" (only PHP uses it, so it's doable w/o problems), but gd is preferred. :)
So, that's the list etc., I hope we can now work on this and get it done, thanks!
Best regards, CHTEKK.
Comment 24 Chris Gianelloni (RETIRED) gentoo-dev 2006-01-24 09:01:06 UTC
I've added these to our release snapshot in a testing fashion (not in the best place, thrown into default-linux) but once I've verified that it doesn't break anything, we'll get them added into the proper locations for each individual profile.
Comment 25 Jakub Moc (RETIRED) gentoo-dev 2006-02-11 09:15:40 UTC
*** Bug 122469 has been marked as a duplicate of this bug. ***
Comment 26 solar (RETIRED) gentoo-dev 2006-03-13 20:04:25 UTC
(In reply to comment #24)
> I've added these to our release snapshot in a testing fashion (not in the best
> place, thrown into default-linux) but once I've verified that it doesn't break
> anything, we'll get them added into the proper locations for each individual
> profile.

This broke uclibc profiles. Thanks man. great QA skills you have.
http://sources.gentoo.org/viewcvs.py/gentoo-x86/profiles/default-linux/make.defaults?r1=1.12&r2=1.13
Comment 27 solar (RETIRED) gentoo-dev 2006-03-13 20:08:35 UTC
Being that is is marked trivial I'm reverting that change. 
Comment 28 Gustavo Zacarias (RETIRED) gentoo-dev 2006-03-14 09:02:05 UTC
Ok, PHP team, are all those USE flags REALLY needed for php to build?
Or are they just "expected basic functionality" for php?
If it's the later they should be hardcoded in the php ebuilds and be done with it. Otherwise you're enabling a ton of crud that affects other ebuilds too and bloats everything.
For example, i don't think xml+expat+simplexml are all required at the same time to build, that would be silly. Maybe you need one, but not the three of them.
I'm not about to accept all those USE as defaults (with respect to global ones) in the sparc profiles, and i think some other arch teams will agree on this.
Comment 29 Sebastian Bergmann (RETIRED) gentoo-dev 2006-03-14 23:12:09 UTC
PHP 5 has several extensions (DOM, SimpleXML, SOAP, WDDX, XML, XMLReader, XMLRPC, XMLWriter, XSL) that make up its XML support and that can be switched on/off independently. Following the Gentoo credo of choice, we added USE flags for each of these extensions.

The default USE flags that have been added to the profiles reflect the out-of-the-box configuration of the PHP distribution.

IMHO, the DOM, SimpleXML, XML, XMLReader, and XMLWriter extensions could all be controlled by a single USE=xml flag to reduce the amount of USE flags. This would, of course, restrict the user's choices.
Comment 30 Luca Longinotti (RETIRED) gentoo-dev 2006-03-15 01:12:37 UTC
As I already said, I don't believe removing the possibility of choiche from the PHP ebuilds (and consequently from the user) is the way to go, I won't do that. I also don't believe that adding them all to the profiles is the solution to this, the solution to "default PHP install" is adding support for per-package default USE flags to Portage, and that will afaik be done in the next months, so I'd defer solution of that aspect of the problem until then.
Wrt the problem of dieing on conflicting USE flags we're working with the Portage team to find a suitable hack to solve this (as it can't really be a "solution", but it will be a "hack" to workaround and try to solve this).
Best regards, CHTEKK.
Comment 31 Daniel Herzog 2006-03-15 23:21:26 UTC
Alot of discussion here, some thing else:
As long as the functionality isnt there in portage, is it possible to show all conflicts with useflags at once?
This would be quite helpful.

Oh - I dont want a discussion guys. Just a plain yes-or-no answer with appended "will do" or "wont do" could be enough as far as i am concerned :-) i do not have the energy to fight this out.
Comment 32 Daniel Herzog 2006-03-15 23:23:58 UTC
Or something like "more conflicts likely" or so you wont expect it to work now an start emerge -e world again. See what i mean?

(Sorry, this came to my mind right after i clicked "Commit")
Comment 33 Luca Longinotti (RETIRED) gentoo-dev 2006-03-15 23:48:43 UTC
We can change the PHP ebuild to show all conflicts at once, and we'll do that if we don't find any other suitable solution that's more complete. But it's not possible to do this at the emerge -e or u or whatever level, as Portage doesn't support this.
Best regards, CHTEKK.

PS: I'll also CC the Portage team as we're working with them to find a solution and eventually their input/ideas are very much appreciated! ;)
Comment 34 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2006-03-16 01:39:27 UTC
(In reply to comment #32)
> Or something like "more conflicts likely" or so you wont expect it to work now
> an start emerge -e world again. See what i mean?
> 
> (Sorry, this came to my mind right after i clicked "Commit")
> 

It's been discussed, most people prefer no conflicts at all; plus I haven't seen any patches for it hitting the portage-dev mailing list ;)

Comment 35 Simon Stelling (RETIRED) gentoo-dev 2006-03-16 04:17:43 UTC
(In reply to comment #34)
> It's been discussed, most people prefer no conflicts at all; plus I haven't
> seen any patches for it hitting the portage-dev mailing list ;)

This is not about PHP. It's about packages that require a certain setup to build properly. This may be a collision-free use flag setup, but it may as well be a certain kernel option that has to be enabled to build a kernel module. With something like pkg_depscan() mentioned in bug 75936 would not only make all those happy who simply don't like emerge to stop in the middle of a -e world but also make merging kernel module packages a lot more convenient. A lot of games' ebuilds would profit from a solution like this too (ACCEPT_KEYWORDS comes to mind). Beside that it can be used to avoid major breakage in cases like the one described in comment 4 of bug 75936.

If the user gives garbage input, it's not the ebuild's job to guess what the user might want. Warn the user, preferrably as soon as possible and all in one go so that he doesn't have to figure out a correct use flags combination by running emerge php 10 times in a row and make sure you get input that makes sense.

I'll try to put together a patch which fixes this issue. Meanwhile it would be very nice to see the php ebuild at least show all use flag collisions at the first go.
Comment 36 Marius Mauch (RETIRED) gentoo-dev 2006-03-16 04:33:42 UTC
(In reply to comment #33)
> PS: I'll also CC the Portage team as we're working with them to find a solution
> and eventually their input/ideas are very much appreciated! ;)

FYI: most portage devs are already on the QA alias and there are other bugs (as mentioned in comment #35) regarding a portage feature to avoid this, so removing dev-portage from CC.
Comment 37 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2006-03-16 10:51:23 UTC
So as a member of QA and portage here is me weighing in.
1.  The current situation is undesirable because php does not build in a default configuration.
2.  The current situation is undesirable because php has many USE conflicts that require resolution in order for it to build.

Both of these are technically fixable with the current portage and syntax.  However I do not feel it is pragmatic to do so.

First, there are a set of things that could be reduced, such as "xml" turning on all the xml features, and remove the xml-rpc, simplexml,soap,xsl,wddx
Another set is the 'dba' use flag, which could enable all database drivers, or none.  We could then remove 'berkdb' 'cdb' 'flatfile' 'qdbm' 'gdbm' 'inifile'

However the PHP team is against this simplification.

The alternative is to shift the onus of maintainership onto the PHP developers.
That is, if xmlrpc is set but xml is not set; ignore the xml-rpc flag.  This requires developers to use built-with-use xml && built-with-use xml-rpc, because xml-rpc by itself will not enable xml-rpc ( it requires xml ).  The documentation for this can be added to use.local.desc

However this also means that users can get confused when they enable xmlrpc but don't get a PHP that can do XMLRPC.

There is a use conflict between gd and gd-external, so you can ignore gd if gd-external is set...but then you have a more complicated built-with-use with that too ( built-with-use gd && !built_with_use gd-external )

In the end you can just end up ignoring user choices, remove user choices, reduce use choices, and in the case of the above examples, simply ignore their choices in order to gain this awesome "php installs out of the box".

However I don't feel this is worth it.  QA doesn't maintain PHP, QA is not volunteering to maintain PHP, QA should not push a solution such as this on the PHP maintainers who are trying to offer a very complicated and agile PHP package.  In the end you will get what you have here, a pissed off PHP team and a crappy solution to a problem that is really only a policy issue.  I will not support this unless actual harm to users is shown.  If you have proof that users are pissed off about the way PHP is written, then there is rationale for change.  For now I see none.
Comment 38 Chris Gianelloni (RETIRED) gentoo-dev 2006-03-16 13:39:39 UTC
(In reply to comment #37)
> So as a member of QA and portage here is me weighing in.

As the person who filed the bug, how about I throw my 0.02 USD in?

> 1.  The current situation is undesirable because php does not build in a
> default configuration.

This was "fixed" in a less-than-desired manner by adding USE flags to the default-linux/$arch/2006.0 profiles.  This does not fix anyone on the older profiles, however.

> 2.  The current situation is undesirable because php has many USE conflicts
> that require resolution in order for it to build.

Here's what I think many people are missing.  I think a USE conflict is OK, so long as it isn't present in the *default* USE.  You will notice that in my original request, I simply asked for sensible defaults to be assumed.  If we have USE="berkdb" and it *requires* USE="dba" being set, then just work as if it *was* set.  This is *very* simple to do and keeps things from being broken.

Yes, there are *tons* of *better* and *more correct* solutions, some of which require additional portage features, but my main concern was that dev-lang/php would build with the default USE.  I didn't ask for the choices to be reduced.  I didn't ask for anything requiring new features.  The PHP team has assisted me in making sure that dev-lang/php will build out-of-the-box for users on 2006.0, which is pretty much what I wanted.  Was adding a ton of additional USE flags to the profiles the best solution?  No.  Do I think it should be a permanent solution?  No.  However, it is a solution that works for us now.  It has less to do with QA and more to do with the out-of-the-box experience for our users.

> However the PHP team is against this simplification.

So am I.

> 
> The alternative is to shift the onus of maintainership onto the PHP developers.

This is probably the "best" solution.  It makes things a bit more difficult for the developers, which is not my intention, at all, but it makes things seamless for our users, which is why we *really* should be doing this.

> That is, if xmlrpc is set but xml is not set; ignore the xml-rpc flag.  This
> requires developers to use built-with-use xml && built-with-use xml-rpc,
> because xml-rpc by itself will not enable xml-rpc ( it requires xml ).  The
> documentation for this can be added to use.local.desc

I think that seems a bit backwards.  If xmlrpc is set, the user *obviously* wants XMLRPC support in PHP.  At that point, XML support is no longer "optional" as would be required by a USE flag, so it should be enabled.  A user can have XML support without XMLRPC, but not vice-versa.  The ebuild should reflect this.  Currently, it does it by using a die and asking the user to change his USE.  I think it should be seamless, which is why I filed the bug in the first place.

> However this also means that users can get confused when they enable xmlrpc but
> don't get a PHP that can do XMLRPC.

Right, which is why we don't do it that way but instead do it the way I mentioned.

> There is a use conflict between gd and gd-external, so you can ignore gd if
> gd-external is set...but then you have a more complicated built-with-use with
> that too ( built-with-use gd && !built_with_use gd-external )

More complicated ebuild syntax is unfortunately something we cannot avoid without adding more capabilities to portage to be "smarter" on these sort of things.

> In the end you can just end up ignoring user choices, remove user choices,
> reduce use choices, and in the case of the above examples, simply ignore their
> choices in order to gain this awesome "php installs out of the box".

Except that I have shown here how that is not necessarily the case.  Sure there are going to be some situations where a "die" is necessary.  The default set of USE should not be one of them unless there is absolutely no way around it.  As I see it, there *is* a way around it without removing choice, by simply ensuring that USE flags that *require* the functionality of another simply enable it.  Think of it the same as adding something to *DEPEND.  If it is needed to build/run, then it is a hard dependency.  If "xml" is required for "xmlrpc", then setting "xmlrpc" should be enough.  Since "xmlrpc" is not needed for "xml", though, we should not assume that the user wants "xmlrpc" just because they have "xml" in their USE.

Does this make sense?

This is exactly how we do things in games, where we have some very complex sets of interdependencies sometimes.  I don't see how it should be different here.

> However I don't feel this is worth it.  QA doesn't maintain PHP, QA is not
> volunteering to maintain PHP, QA should not push a solution such as this on the
> PHP maintainers who are trying to offer a very complicated and agile PHP
> package.  In the end you will get what you have here, a pissed off PHP team and
> a crappy solution to a problem that is really only a policy issue.  I will not
> support this unless actual harm to users is shown.  If you have proof that
> users are pissed off about the way PHP is written, then there is rationale for
> change.  For now I see none.

I'm a user.  I'm pissed off.  Does that work for you?

As I have said before, we initially started off on the wrong foot, but the PHP team and myself have been working to resolve this.  Even after they came up with a solution, even though it wasn't optimal, they continued to work with other teams to try to ensure that their final solution *is* what is best for our users.  I applaud them for this.

I've ranted on enough here.  Apologies to everyone receiving these emails.  ;]
Comment 39 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2006-03-16 14:01:07 UTC
(In reply to comment #38)
> (In reply to comment #37)
> > So as a member of QA and portage here is me weighing in.
> 
> As the person who filed the bug, how about I throw my 0.02 USD in?
> 
> > 1.  The current situation is undesirable because php does not build in a
> > default configuration.
> 
> This was "fixed" in a less-than-desired manner by adding USE flags to the
> default-linux/$arch/2006.0 profiles.  This does not fix anyone on the older
> profiles, however.
> 
> > 2.  The current situation is undesirable because php has many USE conflicts
> > that require resolution in order for it to build.
> 
> Here's what I think many people are missing.  I think a USE conflict is OK, so
> long as it isn't present in the *default* USE.  You will notice that in my
> original request, I simply asked for sensible defaults to be assumed.  If we
> have USE="berkdb" and it *requires* USE="dba" being set, then just work as if
> it *was* set.  This is *very* simple to do and keeps things from being broken.

I worked with CHTEKK to remove the dba flag...so that should simplify that a bit :)

> Yes, there are *tons* of *better* and *more correct* solutions, some of which
> require additional portage features, but my main concern was that dev-lang/php
> would build with the default USE.  I didn't ask for the choices to be reduced. 
> I didn't ask for anything requiring new features.  The PHP team has assisted me
> in making sure that dev-lang/php will build out-of-the-box for users on 2006.0,
> which is pretty much what I wanted.  Was adding a ton of additional USE flags
> to the profiles the best solution?  No.  Do I think it should be a permanent
> solution?  No.  However, it is a solution that works for us now.  It has less
> to do with QA and more to do with the out-of-the-box experience for our users.
> 
> > However the PHP team is against this simplification.
> 
> So am I.
> 
> > 
> > The alternative is to shift the onus of maintainership onto the PHP developers.
> 
> This is probably the "best" solution.  It makes things a bit more difficult for
> the developers, which is not my intention, at all, but it makes things seamless
> for our users, which is why we *really* should be doing this.
> 
> > That is, if xmlrpc is set but xml is not set; ignore the xml-rpc flag.  This
> > requires developers to use built-with-use xml && built-with-use xml-rpc,
> > because xml-rpc by itself will not enable xml-rpc ( it requires xml ).  The
> > documentation for this can be added to use.local.desc
> 
> I think that seems a bit backwards.  If xmlrpc is set, the user *obviously*
> wants XMLRPC support in PHP.  At that point, XML support is no longer
> "optional" as would be required by a USE flag, so it should be enabled.  A user
> can have XML support without XMLRPC, but not vice-versa.  The ebuild should
> reflect this.  Currently, it does it by using a die and asking the user to
> change his USE.  I think it should be seamless, which is why I filed the bug in
> the first place.
> 
> > However this also means that users can get confused when they enable xmlrpc but
> > don't get a PHP that can do XMLRPC.
> 
> Right, which is why we don't do it that way but instead do it the way I
> mentioned.
> 
> > There is a use conflict between gd and gd-external, so you can ignore gd if
> > gd-external is set...but then you have a more complicated built-with-use with
> > that too ( built-with-use gd && !built_with_use gd-external )
> 
> More complicated ebuild syntax is unfortunately something we cannot avoid
> without adding more capabilities to portage to be "smarter" on these sort of
> things.
> 
> > In the end you can just end up ignoring user choices, remove user choices,
> > reduce use choices, and in the case of the above examples, simply ignore their
> > choices in order to gain this awesome "php installs out of the box".
> 
> Except that I have shown here how that is not necessarily the case.  Sure there
> are going to be some situations where a "die" is necessary.  The default set of
> USE should not be one of them unless there is absolutely no way around it.  As
> I see it, there *is* a way around it without removing choice, by simply
> ensuring that USE flags that *require* the functionality of another simply
> enable it.  Think of it the same as adding something to *DEPEND.  If it is
> needed to build/run, then it is a hard dependency.  If "xml" is required for
> "xmlrpc", then setting "xmlrpc" should be enough.  Since "xmlrpc" is not needed
> for "xml", though, we should not assume that the user wants "xmlrpc" just
> because they have "xml" in their USE.
> 
> Does this make sense?
> 
Yes it makes sense, I offered that as a solution ( xmlrpc enables xml but built-with-use xml would fail ).  We got 6 or 7 of the collisions removed, the rest are quite difficult, if we take the above approach, we can eliminate more.

Was berkdb vs dba the only problem in the default profile USE?  Or were there others?
Comment 40 Luca Longinotti (RETIRED) gentoo-dev 2006-03-16 14:16:03 UTC
(In reply to comment #38)
> (In reply to comment #37) 
> > In the end you can just end up ignoring user choices, remove user choices,
> > reduce use choices, and in the case of the above examples, simply ignore their
> > choices in order to gain this awesome "php installs out of the box".
> 
> Except that I have shown here how that is not necessarily the case.  Sure there
> are going to be some situations where a "die" is necessary.  The default set of
> USE should not be one of them unless there is absolutely no way around it.  As
> I see it, there *is* a way around it without removing choice, by simply
> ensuring that USE flags that *require* the functionality of another simply
> enable it.  Think of it the same as adding something to *DEPEND.  If it is
> needed to build/run, then it is a hard dependency.  If "xml" is required for
> "xmlrpc", then setting "xmlrpc" should be enough.  Since "xmlrpc" is not needed
> for "xml", though, we should not assume that the user wants "xmlrpc" just
> because they have "xml" in their USE.
> 
> Does this make sense?

Yes and no. Problem with this is checking if PHP was built with something, we basically aren't really against doing magic in the ebuild to enable depending USE flags etc., but we have the big concern that PHP has a load of extensions and features (USE flags), and a webapp or any other package needs to be able to determine exactly what PHP was built with.
The XML case is very important in this case, especially for PHP5. So if we solve it like you said, we can have the case where the user just does USE="xmlrpc -xml", following your example he gets a PHP which has XMLRPC and XML extensions turned on, and that works, ok. Now he wants to install ZNF or some PEAR package which require XML support to work... How do we check this? Only way is to check using built_with_use on dev-lang/php, now if we do that for the "xml" USE, we'd get false, but the extension is actually enabled, compiled and loaded! That doesn't work... One solution here would be to simulate the same logic we use in the ebuild in the require_php_with_use function of depend.php eclass (which atm is just a wrapper for built_with_use for PHP), and then make it so all ebuilds in the tree use require_php_with_use to check for PHP USE flags... That, while adding extra maintainance and complexity, would probably be able to solve some of those cases, like the XML one and the ODBC one, and generally some other of the "USE depends on USE" cases, not all of them, as some really need the user to be aware if he really wants to emerge that.
But not all of our conflicts are fixable like this... There are some like gd VS gd-external that simply don't have a solution, we can't really force one or the other on the user, since both do the same thing but in totally different way and the user would really wonder if he wants gd-external, has gd enabled, and finds himself with gd in the end, and has to recompile whole PHP... I'd find this much more frustrating/problematic than an ebuild dieing and telling me exactly what went wrong, why, and how to fix it. Same goes for basically all our USE conflicts: oci8 VS oci8-instant-client, vm-switch VS vm-goto, libedit VS readline etc., those are only fixable if we actually force a choice on the user, and that's something I really don't like, as they too aren't conflicts where choosing the default is really simple, or at all desiderable imo...

> I'm a user.  I'm pissed off.  Does that work for you?

Heh ok. ;)

> As I have said before, we initially started off on the wrong foot, but the PHP
> team and myself have been working to resolve this.  Even after they came up
> with a solution, even though it wasn't optimal, they continued to work with
> other teams to try to ensure that their final solution *is* what is best for
> our users.  I applaud them for this.

Thanks. :)
Comment 41 Chris Gianelloni (RETIRED) gentoo-dev 2006-03-16 15:01:00 UTC
(In reply to comment #39)
> Was berkdb vs dba the only problem in the default profile USE?  Or were there
> others?

I know that berkdb and truetype were affected.  I didn't go much further than that, honestly.  You can check quite easily by setting your profile to default-linux/x86/2005.1 and commenting out your USE in make.conf and trying to emerge dev-lang/php.  Actually, I guess that would be a bit more complex than you would wish.  If you desire it, I can find all of the problems with the default USE.  I'd love to be able to drop most (all?) of the additional USE flags from the profiles, especially the global ones that affect more than just dev-lang/php.

(In reply to comment #40)
> But not all of our conflicts are fixable like this... There are some like gd VS
> gd-external that simply don't have a solution, we can't really force one or the
> other on the user, since both do the same thing but in totally different way
> and the user would really wonder if he wants gd-external, has gd enabled, and
> finds himself with gd in the end, and has to recompile whole PHP... I'd find
> this much more frustrating/problematic than an ebuild dieing and telling me

Unfortunately, I'm not sure that I understand what the problem is with this one, so I cannot comment on this conflict directly.  I will just say that in the list of stuff you gave me to put in the profiles, there was "gd" and not "gd-external".  Since USE="truetype" requires one or the other, and USE="truetype" is a default USE flag, I would think that USE="truetype" should enable the same support as USE="gd" *unless" USE="gd-external" is used.  This would resolve the conflict in the default USE, but as I said, I don't know the specifics of the implications of this on the package itself, so I definitely couldn't make that decision for you or even pretend to know what I'm talking about.  ;]

> exactly what went wrong, why, and how to fix it. Same goes for basically all
> our USE conflicts: oci8 VS oci8-instant-client, vm-switch VS vm-goto, libedit
> VS readline etc., those are only fixable if we actually force a choice on the
> user, and that's something I really don't like, as they too aren't conflicts
> where choosing the default is really simple, or at all desiderable imo...

Remember that I didn't ask for *all* of the conflicts to be removed.  I simply said it should work "by default".  Once a user starts changing their USE flags, it's up to them to make sure they work.  While we *do* (did?) have a policy that says that all USE combinations should work, we all know that it is not always possible.  I can accept that.  The point is to try to reduce the number of fatal conflicts, which is exactly what you guys are already doing, and I applaud you for it.
Comment 42 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2006-03-17 08:55:17 UTC
If anyone has qualms about the interactivity, please file a seperate bug.
Comment 43 Arthur Hagen 2006-03-23 15:15:13 UTC
(In reply to comment #38)
> 
> This was "fixed" in a less-than-desired manner by adding USE flags to the
> default-linux/$arch/2006.0 profiles.  This does not fix anyone on the older
> profiles, however.

Thank goodness.  (It was, for a brief moment, added to the default-linux/make.default, but thankfully someone stomped that out quickly)
I've just reverted three systems to 2005.1 because I do NOT run php, and do NOT want the USE flags that PHP slams on all users in the 2006.0/make.profile.
There's already too much bloat in the new make.profile as is, and this addition was enough to drop 2006.0

Sorry for bitching, but I'm one of the *users* that the devs want to make things easier for, and I ended up having to revert to a previous version after fiddling with minus flags in USE and wasting a day's work.  It's time that the devs realise that not everyone runs php, not everyone runs apache2, not everyone runs a desktop -- the strength of gentoo has been that you WOULDN'T compile in support for parts you don't need, or even /can't have/, due to corporate policies (ssl being a common one on hard-shell networks with internal monitoring policies).
Yes, php should compile on a default system.  Changing the default system to oblige php is, IMHO, not the way to accomplish that.
Especially not when this bug is marked "trivial" -- does that justify a global make.profile change?
Comment 44 Luca Longinotti (RETIRED) gentoo-dev 2006-03-23 16:21:56 UTC
I've just completed implementing and testing a solution for this bug, it will be added later today to Portage I think, you can already see it in our PHP Overlay [1].
Basically the eclasses were modified, and a new one called phpconfutils (to sobstitute confutils for a more specialized usage in PHP) added. With those, USE flags that depend on eachother are solved automatically, and the DEPENDs were modified accordingly, so that it works as expected, also require_php_with_use and require_php_with_any_use were updated/added to the depend.php eclass, enabling other ebuilds to know and require exactly what USE flags PHP was built with, which was our main concern.
This new dev-lang/php now dies only in a couple of cases, namely if NO SAPI was chosen, or if one of the direct conflicts between two USEs were triggered (those are 6-8, depending on the PHP version), as in that case it would make the whole thing much more complicated that needed to select a default for the user, a decision we don't want to make anyway for reasons of "choice goes to the user", in this case we believe... Anyway, this bug is about "does not work on default USE", and that was fully resolved, since none of those last remaining cases where it may die are ever triggered by a default USE flag set in the profiles, so I see this bug as resolved as soon as the new eclasses/ebuilds are added to Portage.
Doing this also resolves the need of having all those USE flags in the profiles, which, like the last comment stated, is not really liked by users (and I'd say devs/arch-teams too).
Basically I'd request that, if possible, all those USE sets for PHP are removed from the various profiles they were added, and that only a minimal, really minimal list be added to the base profile:

cli spl reflection session pcre

cli, spl and reflection are dev-lang/php-only USE flags, so they will cause no harm or problems to anybody, session is used by a really minor number of ebuilds in the tree, a good chunk of those PHP-related, and I don't think adding this will cause any bloat or particular problem for anyone, I'd say sessions are pretty important... pcre is up to debate, that would influence ~20 packages in the tree if I counted correctly, but otoh pcre is rather standard and I wouldn't see the bloat or problems enabling that one could cause to people.
So, after the new eclasses/ebuilds are in Portage, I'd really only request for those five to go into base and all the rest cleaned again, this should solve this bug and make all involved parties happy, and when Portage itself will have support for per-package USE-defaults, we may add new ones etc., but the really minimal needed ones are only those five mentioned above.
I hope this satisfies everyone, as we've worked rather hard to fix this, and thank all the ones that supported and helped us, especially antarus for assisting with Portage knowledge!
Best regards, CHTEKK.

[1] http://svn.gnqs.org/projects/gentoo-php-overlay/
Comment 45 Chris Gianelloni (RETIRED) gentoo-dev 2006-03-24 04:34:56 UTC
(In reply to comment #44)
> cli spl reflection session pcre

I would have no problem with reducing the flags to just these.  I'll check with the other arch teams over the next few days and see if they agree.
Comment 46 Luca Longinotti (RETIRED) gentoo-dev 2006-03-24 15:27:09 UTC
The updated PHP ebuilds/eclasses were committed into the tree, emerge --sync in a couple of hours to get them. :)
Thus I'd consider this bug closed on our part (PHP Herd), as dev-lang/php now correctly builds without any intervention from the user on default USE flags (provided at least one SAPI is specified out of cli, cgi, apache, apache2, but apache2 is default in the new profiles now and we asked for cli to be added to base profile anyway, so this will never trigger on default USE flags).
I'll leave the bug open for now, pending the change/addition of those five USE flags to the base profile, please close it as you see fit once those were added or not, or use it to discuss them.
Enjoy PHP on Gentoo. ;)
Best regards, CHTEKK.
Comment 47 Mark Loeser (RETIRED) gentoo-dev 2006-03-24 19:19:07 UTC
(In reply to comment #46)
> The updated PHP ebuilds/eclasses were committed into the tree, emerge --sync in
> a couple of hours to get them. :)

Before adding new eclasses to the tree, please post them to g-dev@.  We can't get rid of an eclass, so now we are stuck with what you added today (not saying it is bad, because I haven't looked at it, but people on g-dev@ may have found some problems or a better way to do it).
Comment 48 Stephen Bennett (RETIRED) gentoo-dev 2006-04-05 14:03:31 UTC
Do we really need USE=apache2 enabled by default? It gets more than slightly irritating when installing the subversion client on a fresh stage3 wants to bring in apache as a dep, and from the looks of things it was in the list of flags to be added as a result of this bug.
Comment 49 Luca Longinotti (RETIRED) gentoo-dev 2006-04-05 14:37:37 UTC
(In reply to comment #48)
> Do we really need USE=apache2 enabled by default? It gets more than slightly
> irritating when installing the subversion client on a fresh stage3 wants to
> bring in apache as a dep, and from the looks of things it was in the list of
> flags to be added as a result of this bug.
> 

It was on the list of suggested USE flags (that now was anyway trimmed down to just 5 flags and should be replaced by those 5, any update on this Chris?), but it was not added as a result of this bug, but already sooner with the agreement of the Apache Herd, see http://thread.gmane.org/gmane.linux.gentoo.devel/33454/focus=33454 for the discussion on -dev ML about it.
Best regards, CHTEKK.
Comment 50 Chris Gianelloni (RETIRED) gentoo-dev 2006-04-10 16:33:51 UTC
Sorry, but I've been terribly busy recently and haven't gotten to this.  I'll get on it immediately to get this resolved as soon as possible.  Is the eclass and everything in the tree now?

Arch teams, are there any objections to adding: cli spl reflection session pcre

If nobody says anything in say the next week, I'm going to remove the added dev-lang/php USE flags from all of the affected profiles and add these instead, which should really limit the cruft being added.
Comment 51 Luca Longinotti (RETIRED) gentoo-dev 2006-04-12 03:36:14 UTC
(In reply to comment #50)
> Sorry, but I've been terribly busy recently and haven't gotten to this.  I'll
> get on it immediately to get this resolved as soon as possible.

Np.

> Is the eclass and everything in the tree now?

Yes, since several weeks now, and we've had no problems with it so far, works as expected.

> If nobody says anything in say the next week, I'm going to remove the added
> dev-lang/php USE flags from all of the affected profiles and add these
> instead, which should really limit the cruft being added.

Perfect. :)
Best regards, CHTEKK.
Comment 52 Chris Gianelloni (RETIRED) gentoo-dev 2006-04-19 07:57:28 UTC
Profiles are updated.