Bugzilla Bug 25013

I believe previous versions of portage allowed DEPEND to be set
in an eclass like this:

DEPEND="$DEPEND foo bar"

Now, this is broken and newdepend is required to be used.

I noticed this when depclean wanted to remove gnuconfig.  I checked it out and
there are all kinds of things that *should* depend on it.  For example, readline
and wget both inherit the gnuconfig eclass and *should* DEPEND on gnuconfig.

However, with portage 2.0.48-r5, emerge -ep wget shows that it will not try to
install gnuconfig which shows that the DEPEND is being overwritten by the wget
ebuild.

No telling how many other problems this could cause.

Reproducible: Always
Steps to Reproduce:
1. emerge -ep wget | grep gnuconfig
2. cat wget-1.8.2-r2.ebuild | grep gnuconfig

Actual Results:  
See that emerge -ep wget won't get you gnuconfig, yet the ebuild needs it.

Expected Results:  
should have DEPENDed properly on gnuconfig.

Changing the DEPEND="$DEPEND...." in the gnuconfig eclass to newdepend fixes
the problem.

Portage 2.0.48-r5 (default-x86-1.4, gcc-3.2.3, glibc-2.3.2-r1)
=================================================================
System uname: 2.4.20-18.7 i686 AMD Athlon(tm) processor
GENTOO_MIRRORS="http://gentoo.oregonstate.edu
http://www.ibiblio.org/pub/Linux/distributions/gentoo"
CONFIG_PROTECT="/etc /var/qmail/control /usr/kde/2/share/config
/usr/kde/3/share/config /var/bind /usr/X11R6/lib/X11/xkb
/usr/kde/3.1/share/config /usr/share/config"
CONFIG_PROTECT_MASK="/etc/fonts /etc/bash_completion /usr/X11R6/lib/X11/xkb
/etc/X11/serverconfig /etc/X11/app-defaults /etc/X11/starthere /etc/ssmtp
/etc/sound/events /etc/X11/rstart /etc/X11/xdm /etc/pango /etc/gconf /etc/env.d"
PORTDIR="/usr/portage"
DISTDIR="/usr/portage/distfiles"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR_OVERLAY=""
USE="x86 oss 3dnow apm avi crypt cups encode foomaticdb gif jpeg libg++ libwww
mad mikmod mmx mpeg ncurses pdflib png quicktime spell truetype xml2 xmms xv
zlib gdbm berkdb slang readline arts java X sdl gpm tcpd pam ssl python esd
oggvorbis gnome gtk opengl cdr -svga -doc -motif -nls -imlib -emacs -kde -qt
perl mozilla dvd gtk2"
COMPILER="gcc3"
CHOST="i686-pc-linux-gnu"
CFLAGS="-mcpu=i686 -O3 -pipe"
CXXFLAGS="-mcpu=i686 -O3 -pipe"
ACCEPT_KEYWORDS="x86"
MAKEOPTS="-j2"
AUTOCLEAN="yes"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
FEATURES="sandbox ccache"

Looks like these eclasses need to be updated for sure:

aspell-dict.eclass
ccc.eclass
common-lisp.eclass
commonbox.eclass
crosscompile.eclass
cvs.eclass
ebook.eclass
elisp.eclass
enlightenment.eclass
eutils.eclass
freedict.eclass
games-q3mod.eclass
gcc.eclass
gnuconfig.eclass
ion.eclass
jakarta-commons.eclass
kernel.eclass
koffice-i18n.eclass
perl-module.eclass
php-ext.eclass
php.eclass
stardict.eclass
vim.eclass
xemacs-packages.eclass
zproduct.eclass

can everyone in the CC list please adjust your eclasses to use newdepend,
please?

perl-module is done and commited

I fixed freedict.eclass vim.eclass and aspell-dict.eclass.

mcummings - it's newdepend "foo bar", not newdepend="foo bar"

fixed gnuconfig.eclass since it seemed to be "group maintained"

bah, it's late, fixed :)

These seem like the owners to me:

absinthe        jakarta-commons.eclass
bass            ebook.eclass
coredumb        php-ext.eclass
danarmak        cvs.eclass
danarmak        koffice-i18n.eclass
kutsuya         zproduct.eclass
liquidx         stardict.eclass
lostlogic       kernel.eclass
mkennedy        common-lisp.eclass
mkennedy        elisp.eclass
robbat2         php.eclass
seemant         commonbox.eclass
taviso          ccc.eclass
vapier          enlightenment.eclass
vapier          eutils.eclass
vapier          games-q3mod.eclass
vapier          gcc.eclass
vapier          ion.eclass
zwelch          crosscompile.eclass

The rest have been fixed.

Just looking at the code of newdepend I have a problem with this.
newdepend 'dev-libs/foo'
adds 'dev-libs/foo' to BOTH RDEPEND and DEPEND.

In the PHP eclass, I explictily want DEPEND to contain more items than RDEPEND.

Eg, I have DEPEND="${DEPEND} !dev-libs/9libs" as 9libs replaces one of the glibc headers and this causes the compile of PHP to break in strange ways.
dev-libs/libiconv causes the same problems.
virtual/imap-c-client and dev-libs/libmcal get statically linked into PHP, so there is NO reason for them to need to exist at runtime.

Looking at the newdepend code in /usr/lib/portage/bin/ebuild.sh I see NO way to do this if I am required to use newdepend.

Could I instead suggest that code the packages are using that assumes RDEPEND = DEPEND if RDEPEND is unset in the ebuild is in error?

I have fixed common-lisp.eclass and elisp.eclass

George

ccc.eclass updated.

Ok...

We're pushing new *DEPEND handling.
There are new functions in portage-2.0.48-r6 and this
will be pushed stable very hard and fast. Test it please.

newdepend changed... Will post to -core and -dev about changes.

games-q3mod.eclass is updated 
 
i should have all ebuilds not using newdepend cleaned up tomorrow 

*** Bug 17031 has been marked as a duplicate of this bug. ***

*** Bug 26871 has been marked as a duplicate of this bug. ***

ok, this issue *needs* to be resolved ... stable portage doesnt work with these 
changes ... 

i guess the discussion happened on -core cause i cant find it on -dev on 
gmane.org and i've nuked my core/dev mailboxes since it happened ... 
basically, danarmak (maybe others, but he's really only one who spoke up since 
he wrote newdepend) is against using new{,r,c,p}depend for eclasses ... 

if we're not going to use that function than we should rename the functions (i 
think an altenate name was proposed) and we should (imo at least) remove 
newdepend from portage ... it can go into an eclass, but since it's always been
a 
little hazy to developers/users what it does, it should be put into an eclass
so 
danarmak still has access to it but it isnt officially part of core portage 

*** Bug 27352 has been marked as a duplicate of this bug. ***

its http://www.gentoo.org/doc/en/eclass-howto.xml updated to have these changes
?

the documentation wont be updated until this problem is resolved

danamark still has issues with the function naming and nick hasnt stepped
up to resolve it

seems distutils is affected too, see bug #30998

adding distutils maintainer for bug #30998

but distutils uses newdepend? or i'm confused .. 

oh and i just fixed stardict.eclass

What's the status of this?

the following eclasses aren't fixed yet:

apache-ant.eclass
aspell-dict.eclass
cannadic.eclass
commonbox.eclass
cvs.eclass
fixheadtails.eclass
gcc.eclass
gnustep.eclass
gtk-engines.eclass
jakarta-commons.eclass
kde.eclass
kernel.eclass
koffice-i18n.eclass
nxserver.eclass
php-2.eclass
php-ext-base.eclass
php-ext-source.eclass
php-ext.eclass
php-lib.eclass
php-pear.eclass
php.eclass
rpm.eclass
ruby-gnome2.eclass
tetex.eclass
vim-plugin.eclass
vim.eclass
webapp-apache.eclass
webapp.eclass

Is this bug still valid?  I thought that some changes made in the .49 series
alleviated this problem.  A quick use of the kde eclass based ebuilds seems to
work.

Not sure, My main interest in this is how ebuilds use newdepend, therefore
don't set RDEPEND, so our 2004.0 release will be completely broken in regards
to KDE.

I'm not sure wheter it is still relevant (DEPEND/RDEPEND seem to work)
but fixed cannadic.eclass, ruby-gnome2.eclass and tetex.eclass.

btw, rpm.eclass doesn't have the problem.

I think this may be invalid.  Here's why:

ebuilds now automatically inherit dependencies from inherited eclasses.  This means in an ebuild you no longer have to do a DEPEND="$DEPEND...".  I believe this was a commit fixed in portage by drobbins in July.

This means that eclasses can also get away with DEPEND="" or using newdepend if desired.  But, the kicker is if you create a convenience function that calls a newdepend.  If you ebuild calls this function, it will overwrite any DEPEND="" you may have unless you put this call AFTER the DEPEND="" in your ebuild.  This is what we're trying to avoid having to do...strategically placing eclass functions within an ebuild.

Can someone still come up with a test case where this fails?  Or do we need to rescend this "fix"?

IMHO the best fix to this bug is to get rid of newdepend and friends
completely.  Use DEPEND="${DEPEND} mystuff" ... it really isn't hard, people! 
What is the real point of these "convenience" functions anyway?  They just
obsfucate the code because it's not clear how they affect DEPEND.

Why don't we just get rid of them, fix up the eclasses to set DEPEND directly,
and get this bug closed?

so we've all moved to newdepend's, and now we find out that it wasn't needed? 

Aron and Caleb are right though, DEPEND="${DEPEND} some-dep" are exported properly to the ebuilds because of the plumbing in ebuild.sh's inherit() function. this really isn't a blocker or a bug anymore. 

deprecating newdepend should probably be discussed on gentoo-dev

any progress on this?

I think Aron is right.  I think that we should get rid of newdepend completely
as the normal DEPEND= stuff is clearer and works correctly with current 
versions of portage.

ive talked to nick sometime ago and verified ...

so this is how it works:
in eclasses, put DEPEND="your eclass depend stuff"
in ebuilds, put DEPEND="your ebuild depend stuff"

do _not_ try and get smart and do DEPEND="${DEPEND} blah blah" in ebuilds, let portage handle bringing the DEPEND's together between eclasses and ebuilds (and it does work)

as regards to 'new?depend', pretend it doesnt exist ... trim it from your eclasses and ebuilds and never mention it again

Iow I'll do a grep on "newdepend" through the documentation and remove all
references to it. How's that for pretending :)

What about things like "need-kde 3", which sets up dependencies, but doesn't
inherit them.

if you do this:

need-kde 3
DEPEND="app-foo/bar"

Then you lose the kde deps.

(Note: I didn't design "need-kde" so I have no reservations about trying to get
rid of it)

Created an attachment (id=27952) [details]
list of eclasses which use newdepend

There doesn't seem to have been any progress on cleaning up the eclasses. 
Please
find your eclass in the list and clean it up please.

gnome2.eclass done

Can someone once and for all clarify whether we should be moving to newdepend
or not?  From the comments attached to this bug, it doesn't seem to be a done
deal.

Stuart, the conclusion of this bug is that we are (gradually) getting rid of
newdepend and friends completely.  There is no reason for them.  Portage
handles setting of DEPEND/RDEPEND in eclasses correctly.

Created an attachment (id=28429) [details]
patch to fix all the eclasses.

Here's a patch that replaces all the instances of newdepend and newrdepend in
the
eclasses.

pelr-modules.eclass re-fixed. Dropped newdepend, added DEPEND="dev-lang/perl"

Created an attachment (id=29000) [details]
better patch to ebuilds.

Portage handles setting DEPEND="cat/package" in eclasses just fine now so I've
updated the patch to not do DEPEND="${DEPEND} cat/package"

Please find your eclass and update.

Fixed elisp.eclass, iiimf.eclass, latex-package.eclass,
ruby-gnome2.eclass, ruby.eclass, sgml-catalog.eclass and tetex.eclass.

I went ahead and fixed tla.eclass since it's not used.

mr_bones_ : you may as well just apply the fixes -- we'll be waiting a year and
a day otherwise.

Created an attachment (id=29064) [details]
list of ebuilds that use new.?depend

Done.

Now to address the 183 ebuilds that use these functions.

Nothing in web-apps is affected by this bug ... removing us from the CC list.

Stu

Portage does handle the setting of DEPEND= fine now during an inherit, but it
doesn't work during a function call to an eclass.  Thus newdepend still has
some merit from within an eclass if you're inside of a function call that is
initiated from outside the eclass.

An example to my above comment:


inherit kde
need-kde 3.1

...

DEPEND="foo/bar"

the need-kde 3.1 call causes any DEPEND settings to be lost when later on DEPEND="" is set by the ebuild.

No!  Use DEPEND="${DEPEND} whatever" and RDEPEND="${RDEPEND} whatever"

There's no reason so continue to use the new*depend functions.

That will only work at the ebuild level, however.  That means every KDE related
ebuild will have to be updated, vs. changing one function call in the eclass.

Plus, isn't it somewhat messy within the ebuild to have to do the:

DEPEND="$DEPEND
        next/dep..

Which is what was attempted to be fixed in the first place.

Caleb, in response to your comments, Mr_Bones_ is right.  The new*depend
functions are going away, for reasons already discussed in this bug.  There are
two cases to consider (and only two):

1. In the context of an ebuild, you should use DEPEND="blah" for the first
setting of DEPEND, and DEPEND="${DEPEND} blah" for following settings.  Portage
will properly inherit the DEPEND from eclasses.

2. In the context of an eclass, you should use DEPEND="blah" for the first
setting of DEPEND, and DEPEND="${DEPEND} blah" for following settings.  Portage
will properly maintain the DEPEND setting from the ebuild.

The case you've mentioned, where the ebuild is calling a function from the
eclass, is really just a special case of #1.  You should use DEPEND="${DEPEND}
blah" in the function.

Regarding messiness, that is a matter of opinion.  In my humble opinion, the
new*depend functions were problematic because it was never clear what variables
they affected and how.  Chaining variable settings (var="${var} stuff") is
pretty standard, readable bash, and it's the right way to handle this
situation.

I hope this clears up your questions.  Thanks!

So, in a nutshell, we should move "need-kde" to the bottom of the ebuild?

actually, my above comments are completely invalid as it now seems to be
working as I hoped it would.  I suppose I just had something cached and didn't
realize it.

/me looks back at your comment #51 to comprehend

Yes, in that case, you should either move need-kde to the bottom of the ebuild, or use DEPEND="${DEPEND} foo/bar".  Sorry if that means a lot of work for you!  Hopefully we won't have to make this kind of change again.

Caleb, feel free to tag me on #-dev if you want to talk more about this, or
point me at an ebuild and I can tell you what I think needs to be done...

Looks like the solution here is to make sure that need-kde comes after all
DEPEND= and RDEPEND= within the ebuild.

If that's the final solution (moving need-kde after DEPEND and RDEPEND),
note that for those ebuilds that set DEPEND but not RDEPEND you should
manually add RDEPEND=${DEPEND} before need-kde.
Alternatively, you should define RDEPEND in kde-functions.eclass only if it
is not empty, something like
[ -z "${RDEPEND}" ] || RDEPEND="${RDEPEND} >=kde-base/kdelibs-${selected_version}"

*** Bug 48650 has been marked as a duplicate of this bug. ***

A pretty dumb question: Wouldn't it be simpler to have a NEED_KDE variable and
a optional special function in eclasses, which is called by portage when the
ebuild variables are known? This would be a more general approach, than this
position dependent eclass function call.

for Caleb and kde team:
I have a large set of trivial patches that move kde-need down in each kde
related ebuild, and substitute new?depend with ?DEPEND (they nearly empty
the list).

are they useful? should I post them somewhere?

wanna start a new bug and attach there please?  assign to me personally.

so is this finally fixed?

sure