Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 67179 - --depclean removes packages that --update --deep requires
Summary: --depclean removes packages that --update --deep requires
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All All
: High minor (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
: 78408 97392 133269 145166 (view as bug list)
Depends on:
Blocks: 155723 136244
  Show dependency tree
 
Reported: 2004-10-11 17:31 UTC by Colin Macdonald
Modified: 2006-11-19 15:26 UTC (History)
12 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge -pd depclean output (emergepddepclean.bz2,13.00 KB, application/octet-stream)
2006-01-15 20:30 UTC, Sean Kennedy
Details
The output of emerge -pd --depclean (depclean.log,200.16 KB, text/plain)
2006-01-18 12:04 UTC, Robert Forsman
Details
Compressed emerge --debug --pretend --depclean output (depclean-debug.out.gz,51.76 KB, application/octet-stream)
2006-08-12 00:54 UTC, Richard Fish
Details
Alternate depclean implementation (alt-depclean.patch,6.52 KB, patch)
2006-08-15 08:46 UTC, Jason Stubbs (RETIRED)
Details | Diff
Alternate depclean implementation (w/ portdb) (alt-depclean.patch,6.64 KB, patch)
2006-08-15 19:50 UTC, Jason Stubbs (RETIRED)
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Colin Macdonald 2004-10-11 17:31:07 UTC
When I run "emerge --depclean", glibmm is removed but revdep-rebuild then wants to rebuild all of the following:

[ebuild   R   ] app-office/openoffice-bin-1.1.3
[ebuild   R   ] dev-cpp/gtkmm-2.4.5
[ebuild   R   ] dev-cpp/gconfmm-2.6.1
[ebuild   R   ] dev-cpp/libglademm-2.4.0
[ebuild   R   ] dev-cpp/libgnomecanvasmm-2.6.1
[ebuild   R   ] dev-cpp/libgnomemm-2.6.0

Emerging gtkmm will fix the problem:

aconite gtkmm # emerge -pv gtkmm

These are the packages that I would merge, in order:

Calculating dependencies ...done!
[ebuild  N    ] dev-cpp/glibmm-2.4.4  -debug 788 kB
[ebuild   R   ] dev-cpp/gtkmm-2.4.5  -debug 0 kB

But then emerge --depclean will remove it again.  I checked the gtkmm ebuild and it lists glibmm as a dependency.  I'm at a loss for what else to check.

Emerge info:
Portage 2.0.51_rc9 (default-x86-2004.2, gcc-3.3.4, glibc-2.3.3.20040420-r2, 2.4.27 i686)
=================================================================
System uname: 2.4.27 i686 AMD Athlon(tm) XP 1800+
Gentoo Base System version 1.5.3
distcc 2.17 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
Autoconf: sys-devel/autoconf-2.59-r5
Automake: sys-devel/automake-1.8.5-r1
Binutils: sys-devel/binutils-2.14.90.0.8-r1
Headers:  sys-kernel/linux-headers-2.4.22
Libtools: sys-devel/libtool-1.5.2-r5
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CFLAGS="-march=athlon-xp -O3 -pipe"
CHOST="i686-pc-linux-gnu"
COMPILER=""
CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3.3/share/config:/usr/kde/3.3/env:/usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/mozilla/defaults/pref /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-march=athlon-xp -O3 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs ccache collision-protection distlocks sandbox"
GENTOO_MIRRORS="ftp://206.75.217.180/ http://gentoo.osuosl.org/ ftp://gentoo.ccccom.com ftp://ftp.ussg.iu.edu/pub/linux/gentoo http://gentoo.ccccom.com"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X Xaw3d aalib alsa apm arts berkdb bitmap-fonts cdr cjk crypt cups dga directfb dvd eds emacs encode esd f77 faad fbcon fftw flac gcj gdbm gif gimpprint ginac gnome gphoto2 gpm gstreamer gtk gtk2 guile imlib jack java jpeg ldap leim libg++ libwww lirc live mad matroska mikmod mmx mng motif mozilla mpeg mysql nas ncurses nls objc offensive oggvorbis opengl oss pam pdflib perl plotutils png ppds python qhull qt quicktime radeon readline rtc scanner sdk sdl slang speex spell sse ssl svg tcltk tcpd tetex theora tiff truetype usb v4l wxwindows x86 xinerama xml xml2 xmms xosd xprint xv xvid zlib video_cards_radeon video_cards_mach64"
Comment 1 foser (RETIRED) gentoo-dev 2004-10-12 09:34:20 UTC
depclean -afaik- as it is isn't very safe for general use.
Comment 2 Kenyon Ralph 2005-01-11 17:57:07 UTC
Indeed I'm seeing this same activity.  Seems like this would be a problem in one of the ebuilds since depclean and revdep-rebuild et al work fine with everything else.  I'm going to take a look at them...

An emerge --depclean shows this:
Calculating depclean dependencies ... done!

>>> These are the packages that I would unmerge:

 dev-cpp/glibmm
    selected: 2.4.4
   protected: none
     omitted: none

After unmerging that, doing an emerge -Duavt world wants to install glibmm again because gtkmm-2.4.8 apparently depends on it.

# emerge --info
Portage 2.0.51-r8 (default-linux/x86/2004.3, gcc-3.4.3, glibc-2.3.4.20041102-r0, 2.6.10-cko3 i686)
=================================================================
System uname: 2.6.10-cko3 i686 Intel(R) Pentium(R) 4 CPU 2.80GHz
Gentoo Base System version 1.6.8
Python:              dev-lang/python-2.3.4 [2.3.4 (#1, Jan  6 2005, 15:58:34)]
ccache version 2.3 [enabled]
dev-lang/python:     2.3.4
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.4_p6, 1.8.5-r2, 1.6.3, 1.9.4, 1.5, 1.7.9
sys-devel/binutils:  2.15.92.0.2-r2
sys-devel/libtool:   1.5.10-r2
virtual/os-headers:  2.6.8.1-r2
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CFLAGS="-O3 -march=pentium4 -msse2 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O3 -march=pentium4 -msse2 -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperms"
GENTOO_MIRRORS="http://gentoo.ccccom.com http://gentoo.mirrors.pair.com/ http://chod.cwru.edu/gentoo http://mirror.datapipe.net/gentoo http://gentoo.osuosl.org http://www.ibiblio.org/pub/Linux/distributions/gentoo"
LDFLAGS=""
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"
USE="x86 X aac aalib acl alsa apm audiofile avi berkdb bitmap-fonts caps cdparanoia cdr crypt cups dvd encode fam flac font-server foomaticdb fortran gdbm gif gnome gphoto2 gpm gtk gtk2 guile imagemagick imlib ipv6 java jikes jpeg kde lcms libwww mad mikmod mmx mng mozilla mpeg ncurses nls nptl nvidia offensive oggvorbis opengl pam pdflib perl pic png ppds python qt quicktime readline real samba sdl slang sndfile spell sse ssl svg svga tcpd theora tiff truetype unicode usb wmf xine xml xml2 xmms xprint xv xvid xvmc zlib"
Comment 3 Bob 2005-01-12 12:44:38 UTC
I have run into a similar problem with gtkmm and depclean/rev-rebuild.  

revdep-rebuild chokes when trying to emerge gtkmm, yielding the following fatal error:

checking for getc_unlocked... yes 
checking for pkg-config... /usr/bin/pkg-config 
checking for glibmm-2.4 >= 2.4.0 atk >= 1.6.0... Package glibmm-2.4 was not found in the pkg-config search path. 
Perhaps you should add the directory containing `glibmm-2.4.pc' 
to the PKG_CONFIG_PATH environment variable 
No package 'glibmm-2.4' found 

configure: error: Library requirements (glibmm-2.4 >= 2.4.0 atk >= 1.6.0) not met; consider adjusting the PKG_CONFIG_PATH environment variable if your libraries are in a nonstandard prefix so pkg-config can find them. 

!!! ERROR: dev-cpp/gtkmm-2.4.8 failed. 
!!! Function econf, Line 447, Exitcode 1 
!!! econf failed 
!!! If you need support, post the topmost build error, NOT this status message.

here is a thread i have started at Portage and Programming:
http://forums.gentoo.org/viewtopic.php?t=278631

emerge info will follow as an attachment
Comment 4 Bob 2005-01-12 12:47:59 UTC
Portage 2.0.51-r8 (default-linux/x86/2004.3, gcc-3.4.3, glibc-2.3.4.20041102-r0, 2.6.10-gentoo-r1 i686) 
================================================================= 
System uname: 2.6.10-gentoo-r4 i686 Pentium III (Coppermine) 
Gentoo Base System version 1.6.8 
Python:              dev-lang/python-2.3.4 [2.3.4 (#1, Dec 29 2004, 22:58:57)] 
dev-lang/python:     2.3.4 
sys-devel/autoconf:  2.59-r6, 2.13 
sys-devel/automake:  1.8.5-r2, 1.5, 1.4_p6, 1.6.3, 1.7.9, 1.9.4 
sys-devel/binutils:  2.15.92.0.2-r2 
sys-devel/libtool:   1.5.10-r2 
virtual/os-headers:  2.6.8.1-r2 
ACCEPT_KEYWORDS="x86 ~x86" 
AUTOCLEAN="yes" 
CFLAGS="-O3 -march=pentium3 -mtune=pentium3 -fforce-addr -fomit-frame-pointer -frename-registers -fweb -ftracer -pipe" 
CHOST="i686-pc-linux-gnu" 
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control" 
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" 
CXXFLAGS="-O3 -march=pentium3 -mtune=pentium3 -fforce-addr -fomit-frame-pointer -frename-registers -fweb -ftracer -pipe -fvisibility-inlines-hidden" 
DISTDIR="/usr/portage/distfiles" 
FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperms" 
GENTOO_MIRRORS="http://gentoo.netnitco.net http://gentoo.osuosl.org http://www.ibiblio.org/pub/Linux/distributions/gentoo" 
LDFLAGS="" 
MAKEOPTS="-j2" 
PKGDIR="/usr/portage/packages" 
PORTAGE_TMPDIR="/var/tmp" 
PORTDIR="/usr/portage" 
PORTDIR_OVERLAY="" 
SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage" 
USE="x86 X acl acpi alsa apm arts avi berkdb bitmap-fonts cdinstall cdparanoia cdr cdrom cgi chroot client crypt cups dvd dvdr dvdread encode esd fam flac foomaticdb fortran freetype gdbm gif gimp gimpprint gnome gnomedb gnuplot gpm gtk gtk2 gtkhtml hal ide imagemagick imlib ipv6 ithreads java javacomm javadocjavamail javascript jpeg jpeg2k kde ldap libwww mad mikmod motif mpeg mysql ncurses nls nptl oggvorbis opengl oss pam pdf pdflib perl png posix print pthreads python qt quicktime readline samba sdl shaper slang snmp spell ssl svga tcpd tiff truetype userlocales xine xinerama xml xml2 xmms xscreensaver xv zlib"
Comment 5 Wilson M. Michaels 2005-01-30 20:56:21 UTC
There is some sort of versioning problem with dev-cpp/gtkmm-2.2.12 in the unmasked tree. When I run under 2.6.10:

emerge -ua --deep world

it will compile gtkmm-2-2-11 like this:

[ebuild     UD] dev-cpp/gtkmm-2.2.11 [2.2.12]

When I rerun the same emerge command again it will compile gtkmm-2.2.12

[ebuild     U ] dev-cpp/gtkmm-2.2.12 [2.2.11]

It just toggles back and forth.
Comment 6 Zac Medico gentoo-dev 2005-08-05 13:54:01 UTC
Today I had dev-cpp/gtkmm-2.6.3 in my "emerge -a depclean" output despite it's
providing a dependency to inkscape-0.4.2 for DEPEND=">=dev-cpp/gtkmm-2.4". 
Apparently it was because after "emerge sync", inkscape-0.4.2 had been hard
masked in package.mask.  After I unmasked inkscape-0.4.2, "emerge -a depclean"
reported nothing to clean.

Bug 77669 and Bug 78408 may be related.
Comment 7 Alec Warner archtester Gentoo Infrastructure gentoo-dev Security 2005-08-12 21:02:50 UTC
*** Bug 97392 has been marked as a duplicate of this bug. ***
Comment 8 Jason Stubbs (RETIRED) gentoo-dev 2005-10-16 01:16:47 UTC
*** Bug 78408 has been marked as a duplicate of this bug. ***
Comment 9 Jason Stubbs (RETIRED) gentoo-dev 2005-11-08 04:09:09 UTC
Can anybody still experiencing this attach the output of `emerge -pd depclean` 
please? 
Comment 10 Colin Macdonald 2005-11-08 22:10:13 UTC
I haven't used emerge --depclean in a a long time but I currently cannot
reproduce this.  I will play with it for a few days and reopen if I can find any
strange behavior from depclean/revdep-rebuild
Comment 11 Sean Kennedy 2006-01-15 20:30:18 UTC
Created attachment 77225 [details]
emerge -pd depclean output
Comment 12 Robert Forsman 2006-01-18 12:04:39 UTC
Created attachment 77451 [details]
The output of emerge -pd --depclean

depclean tells me to remove some packages that would break installed stuff.  I have attached the output of emerge -pd --depclean, and here are some equery results that show --depclean crazy:

thoth@alexandria ~ $ equery depends oaf

*** You are not in the portage group. You may experience cache problems
*** due to permissions preventing the creation of the on-disk cache.
*** Please add this user to the portage group if you wish to use portage.

[ Searching for packages depending on oaf... ]
gnome-base/gnome-vfs-1.0.5-r4
gnome-base/gconf-1.0.9
thoth@alexandria ~ $ epm -q gconf
gconf-2.10.1-r1
gconf-1.0.9

My first instinct after scanning the log would be that --depclean is not considering packages required by gconf-1.0.9, only gconf-2.10.1-r1 .
Comment 13 Jakub Moc (RETIRED) gentoo-dev 2006-05-14 01:21:30 UTC
*** Bug 133269 has been marked as a duplicate of this bug. ***
Comment 14 Richard Fish 2006-08-12 00:54:41 UTC
Created attachment 94037 [details]
Compressed emerge --debug --pretend --depclean output

I'm seeing this now with docbook-xml-dtd-4.4-r1:

carcharias rjf # emerge --depclean --pretend world
<snip>
 app-text/docbook-xml-dtd
    selected: 4.4-r1
   protected: none
     omitted: 4.1.2-r6 4.3-r1

After running without --pretend:
carcharias rjf # emerge -Dvpt world

These are the packages that would be merged, in reverse order:

Calculating world dependencies... done!
[nomerge      ] media-video/mplayer-1.0_pre20060810  USE="X aac alsa arts dga doc dts dv dvd encode gif gtk jpeg mad mmx mmxext opengl png rtc samba sdl sse sse2 theora truetype unicode v4l v4l2 vorbis win32codecs xv xvid xvmc -3dfx -3dnow -3dnowext -aalib (-altivec) -amr -bidi -bindist -bl -cdparanoia -cpudetection -custom-cflags -debug -directfb -dvb -dvdread -enca -esd -fbcon -ggi -iconv -ipv6 -jack -joystick -libcaca -lirc -live -livecd -lzo -matrox -musepack -nas -openal -oss -real -speex -svga -tga -x264 -xanim -xinerama -xmms"
[ebuild  NS   ]  app-text/docbook-xml-dtd-4.4-r1  0 kB

emerge --info:
carcharias rjf # emerge --info
Portage 2.1.1_pre5 (default-linux/x86/2006.0, gcc-4.1.1/vanilla, glibc-2.4-r3, 2.6.18-rc2 i686)
=================================================================
System uname: 2.6.18-rc2 i686 Genuine Intel(R) CPU           T2500  @ 2.00GHz
Gentoo Base System version 1.12.4
Last Sync: Sat, 12 Aug 2006 03:20:01 +0000
app-admin/eselect-compiler: 2.0.0_rc2-r1
dev-lang/python:     2.4.3-r1
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     [Not Present]
dev-util/confcache:  0.4.2-r1
sys-apps/sandbox:    1.2.18.1
sys-devel/autoconf:  2.13, 2.60
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.17
sys-devel/gcc-config: 2.0.0_rc1
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.17
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=pentium4 -Os -fomit-frame-pointer -pipe -fweb"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/eselect/compiler /etc/gconf /etc/init.d /etc/java-config/vms/ /etc/revdep-rebuild /etc/splash /etc/terminfo"
CXXFLAGS="-march=pentium4 -Os -fomit-frame-pointer -pipe -fweb"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig buildpkg confcache distlocks fixpackages metadata-transfer sandbox sfperms splitdebug strict"
GENTOO_MIRRORS="http://mirror.datapipe.net/gentoo/"
LINGUAS=""
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://tacklebox.fishnet/gentoo-portage"
USE="x86 X alsa arts bzip2 crypt cups elibc_glibc encode flac gtk hal input_devices_evdev input_devices_keyboard input_devices_mouse input_devices_synaptics jpeg kde kdeenablefinal kdehiddenvisibility kernel_linux mmx mp3 pam pic png python qt qt3 qt4 readline samba sse ssl tiff truetype unicode userland_GNU video_cards_fbdev video_cards_nv video_cards_nvidia video_cards_vesa vorbis zlib"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, PORTAGE_RSYNC_EXTRA_OPTS

Compressed output of emerge --debug --pretend --depclean world is attached.
Comment 15 Simon Stelling (RETIRED) gentoo-dev 2006-08-12 04:16:26 UTC
reopening on behalf of Richard as he lacks priviledges to do so
Comment 16 Carsten Lohrke (RETIRED) gentoo-dev 2006-08-12 05:04:00 UTC
I don't know about the original example, but the one Richard is mentioning, is a side-effect of Portage's broken behaviour wrt to slotted packages. It wants always  to update to the latest ebuild of a package, even if the installed packages need only specific (older) slotted versions. So this is basically a dupe of bug 93469.
Comment 17 Zac Medico gentoo-dev 2006-08-12 11:17:14 UTC
n Richard's case, it seems that mplayer-1.0_pre20060810 pulled in the update via it's unspecific dep on app-text/docbook-xml-dtd.  If the mplayer ebuild had a more specific atom, then he wouldn't experience this problem.  Slot deps would certainly be useful here, but the ~ or * operators could also be of use.
Comment 18 Jason Stubbs (RETIRED) gentoo-dev 2006-08-14 17:30:21 UTC
Portage shouldn't try to update a package to the latest slot unless it's in the world file. I haven't read every new comment on this bug but the following situation might bring it about:

world-pkg-a-1:DEPEND="slot-pkg"
world-pkg-b-1:DEPEND="<slot-pkg-2"
slot-pkg-1
slot-pkg-2

`emerge world-pkg-b world-pkg-a` here would only bring in slot-pkg-1. A following `emerge --update --deep` would bring in slot-pkg-2 as well. A --depclean will (likely) remove slot-pkg-2 if world-pkg-b is processed before world-pkg-a.
Comment 19 Carsten Lohrke (RETIRED) gentoo-dev 2006-08-14 18:26:20 UTC
(In reply to comment #18)
> Portage shouldn't try to update a package to the latest slot unless it's in the
> world file.

Interesting idea to look up the world file for this. This would work without having real slot dependencies, assuming the slotted dependencies are not referenced >=foo-x.y, which will work most of the time. I'm all for having this implemented asap. :)


> I haven't read every new comment on this bug but the following
> situation might bring it about:
> 
> world-pkg-a-1:DEPEND="slot-pkg"
> world-pkg-b-1:DEPEND="<slot-pkg-2"
> slot-pkg-1
> slot-pkg-2

The problem is that there are cases, it doesn't work. Think about packages A and B listing the dependencies >=foo-3.x respective >=foo-3.x+1 in their ebuilds, where =foo-3* has the same slot, but 3.x+1 provides extended functionality, needed by B. This can be needed, because <foo-4 doesn't work, when <foo-3 is incompatible and =foo-3* of course doesn't work either in this scenario, so >= dependencies, restricted to a specific slot are still needed.
Comment 20 Jason Stubbs (RETIRED) gentoo-dev 2006-08-15 08:46:59 UTC
Created attachment 94328 [details, diff]
Alternate depclean implementation

(In reply to comment #19)
> (In reply to comment #18)
> > Portage shouldn't try to update a package to the latest slot unless it's in the
> > world file.
> 
> Interesting idea to look up the world file for this. This would work without
> having real slot dependencies, assuming the slotted dependencies are not
> referenced >=foo-x.y, which will work most of the time. I'm all for having this
> implemented asap. :)

Actually, I started out with the premise that slots are _only_ upgraded when in the world file and then proceeded to prove myself wrong due to rushing too early in the morning. ;)

> > I haven't read every new comment on this bug but the following
> > situation might bring it about:
> > 
> > world-pkg-a-1:DEPEND="slot-pkg"
> > world-pkg-b-1:DEPEND="<slot-pkg-2"
> > slot-pkg-1
> > slot-pkg-2
> 
> The problem is that there are cases, it doesn't work. Think about packages A
> and B listing the dependencies >=foo-3.x respective >=foo-3.x+1 in their
> ebuilds, where =foo-3* has the same slot, but 3.x+1 provides extended
> functionality, needed by B. This can be needed, because <foo-4 doesn't work,
> when <foo-3 is incompatible and =foo-3* of course doesn't work either in this
> scenario, so >= dependencies, restricted to a specific slot are still needed.

This patch is a complete rewrite of --depclean and should cover the above case as well as the case I mentioned "safely". I've had it sitting outside of emerge for over a year now and the only change of made here is to add all found vdb packages rather than the best(). I'm also not dying on cases where blocked packages are installed - they will just be left as is.

Positives? Uses USE flags from vardb so -uDN doesn't need to be run before doing a depclean. Doesn't touch ebuilds or binary packages or bother creating a graph which makes it a bit faster.

Negatives? Slots that aren't needed will be left behind. Possible bugs. It's also missing nice error messages and other niceties like ignoring world packages that aren't installed.

The basic algorithm is:

1) Create a list of unresolved atoms from world and system lists
2) Pull an atom from the list
3) Find all installed packages that match the atom
4) For each of those packages that haven't already been processed,
  a) Grab the dependencies taking into account compile-time USE flags
  c) Add those dependencies to the unresolved atoms list
5) If there are still unresolved atoms, go back to 2.

With the horrible hack in dep_zapdeps, || () deps are taken care of just as they would have been with building a tree normally... Anything else I'm missing?

Patch is against emerge from portage-2.1.1_pre4-r1
Comment 21 Jason Stubbs (RETIRED) gentoo-dev 2006-08-15 09:02:04 UTC
As it doesn't seem obvious after re-reading, I'll explaing why I quoted "safely". Essentially, the only time that this version depclean should be able to break installed packages is when those packages have linked against packages outside of those specified in their dependencies. With the added conservative removal of packages (aka keeping more packages than actually necessary) I feel that it is now "overly cautious" rather than "safe".

Either way, the goal in being overly cautious was to ensure that --uD would not require anything that --depclean was happy to remove rather than to ensure safety...
Comment 22 Zac Medico gentoo-dev 2006-08-15 16:55:16 UTC
The alternate depclean implementation looks pretty good.  The only problem that I forsee is inconsistency between /var/db/pkg/*DEPEND and the current corresponding values in the portage tree.  Experience as show, at least when running ~arch, that the *DEPEND values in the tree tend to diverge from those of the installed packages.  Due to bug 48195, this divergence will cause the depclean algorithm to view the dependency tree differently than -uDN world algorithm.  That opens up the potential for depclean removing stuff that -uDN world wants to install, once again.  However, we could alter your patch to pull *DEPEND from the portage tree so that they'll be consistent.
Comment 23 Jason Stubbs (RETIRED) gentoo-dev 2006-08-15 19:50:28 UTC
Created attachment 94367 [details, diff]
Alternate depclean implementation (w/ portdb)

While it hurts my sensibilities that identically versioned ebuilds can be wildly different, I know that you're right. Only a two line change, but I'm reuploading anyway. Giving it a sping locally gave me different results which I guess is enough proof of the need. :/

Still using USE flags from vardb because if they've changed --newuse should pick it up. Any ebuilds unavailable from portdb are of course taken from vardb.
Comment 24 Zac Medico gentoo-dev 2006-08-15 22:37:31 UTC
Nice work.  The patch is in svn r4267.
Comment 25 Zac Medico gentoo-dev 2006-08-16 15:53:47 UTC
(In reply to comment #23)
> While it hurts my sensibilities that identically versioned ebuilds can be
> wildly different, I know that you're right.

We also need to fix bug 122089 in order to account for package moves.  As soon as 48195 and 122089 have been fixed, we can eliminate the portdb usage.
Comment 26 Zac Medico gentoo-dev 2006-08-17 00:15:06 UTC
This has been released in 2.1.1_pre5-r2.
Comment 27 Carsten Lohrke (RETIRED) gentoo-dev 2006-08-17 13:24:05 UTC
We need to revert this one, as it is problematic, annoying and maybe broken. On the other hand it's quite nice, as it shed light on some problems.


emerge --depclean --pretend results on my box in:


Calculating dependencies... done!
The following are required but not installed:
* dev-perl/Date-Calc
* >=dev-lang/fpc-2.0.0
* =dev-util/scons-0.96.1
* virtual/glibc
* ~kde-base/kdelibs-3.5.3
* ~kde-base/kdelibs-3.5.2
* >=media-plugins/libvisual-plugins-0.2
* app-misc/colordiff
* =virtual/libstdc++-3.3*
* dev-perl/Data-Dumper
* =x11-libs/qt-3.3.4-r8
* kde-base/smoke
* dev-perl/PostScript-Simple
* =net-www/apache-1*


The problems are quite different, do I'm explaining the entries:


- scons, libvisual-plugins and libstdc++ 

In terms of Portage's dep calcuation correct, no need to discuss them.

- apache

Made me aware that I still had the slotted mod_perl ebuild for Apache 1 installed.

- perl packages

Having taskjuggler via ACCEPT_KEYWORDS="~x86" emerge  installed and emerge --depclean afterwards explains that dev-perl/PostScript-Simple never showed up on x86 emerge -u world. Probably the same for the others. Since I use this only for quick & dirty build testing, it's a nonissue. I assume the other perl packages fall into this category as well.

- smoke

see perl

- fpc

This one is interesting. While for the same reason as the perl packages and smoke it never showed up before, the real problem is quite different: dev-lang/fpc has been moved to dev-lang/fpc-bin. Later a new package dev-lang/fpc has been createad. While I did remove the move entry a while ago, the local db aparently has been changed to the moved package, while another one (dev-lang/fpc-ide) still depends on dev-lang/fpc: I'm not sure, if this is a special hickup or if dependencies on a renamed/moved package don't get properly fixed in the package db. Nevertheless Portage should check, if the destination package it wants to move a package to, exists already and deny, warn, bail out maybe. I might add, that moving packages (as well as versioning changes within a package) can e.g. break GLSA's. We need to rethink this and come to a better solution.

- virtual/glibc

There is no package for this virtual

- app-misc/colordiff

I've no f*cking clue about this one. Neither is it referenced in /var/db/pkg, nor is there a package in the tree that depends on it. The patch looks rather sane to me, though.

-  kdelibs (and I suppose qt as well)

We update dependencies as needed in the tree, not in /var/db/pkg, so we don't have to issue new ebuilds for nothing and our userbase is not forced to have to (re-)build packages for nothing. Workig on the installed package information is at the moment not possible. I might add that this is not specific to the KDE team, but small dependency fixes happen on the live tree all the time. The current way we work is a showstopper for a getting bug 48195 fixed.


Last but not least I want to add that - while it revealed some problems - it's quite annoying to be confronted with a list of packges Portage is missing, while I'm solely interested to clean out unneeded ones. Moreso the output "Apache 1 is missing" is semantically not correcrt and some unexperiencd user will only say wtf.
Comment 28 Zac Medico gentoo-dev 2006-08-17 16:03:38 UTC
(In reply to comment #27)
> We update dependencies as needed in the tree, not in /var/db/pkg, so we don't
> have to issue new ebuilds for nothing and our userbase is not forced to have to
> (re-)build packages for nothing. Workig on the installed package information is
> at the moment not possible. I might add that this is not specific to the KDE
> team, but small dependency fixes happen on the live tree all the time. The
> current way we work is a showstopper for a getting bug 48195 fixed.

I brought up that issue in comment #22 and the latest patch (already released in pre5-r2) already accounts for that. This new depclean implementation is superior to the old one by far.  Maybe some adjustments are necessary, but to go back to the old one would be insane imo.
Comment 29 Carsten Lohrke (RETIRED) gentoo-dev 2006-08-17 16:39:45 UTC
(In reply to comment #28)
> I brought up that issue in comment #22 and the latest patch (already released
> in pre5-r2) already accounts for that. This new depclean implementation is
> superior to the old one by far.  Maybe some adjustments are necessary, but to
> go back to the old one would be insane imo.

You haven't read what I wrote, have you? It's completely broken, unless Portage on every --sync diffs the ebuilds (or is the stored env used?) in /var/db/pkg with the ones in /usr/portage (+ overlays) and patches those in /var/db/pkg when they differ (trivial header changes aside), so they match the current tree.
Comment 30 Zac Medico gentoo-dev 2006-08-17 17:27:07 UTC
(In reply to comment #29)
> You haven't read what I wrote, have you?

Yes, I have. Portage will disregard /var/db/pkg/*/*/*DEPEND as long as there is a matching ebuild it the tree to pull the dependencies from.
Comment 31 David Li 2006-08-17 19:30:55 UTC
Hmm, well the new version seems to work very well for me.
There used to be 3 packages that depclean tried to remove but update world tried to reinstall.

Looks like good work.
Comment 32 Zac Medico gentoo-dev 2006-08-18 13:14:55 UTC
(In reply to comment #31)
> There used to be 3 packages that depclean tried to remove but update world
> tried to reinstall.

As far as I can tell, the new depclean algorithm complements the update algorithm perfectly.  This bug seems to be fixed.
Comment 33 Richard Fish 2006-08-18 13:26:04 UTC
(In reply to comment #32)
> As far as I can tell, the new depclean algorithm complements the update
> algorithm perfectly.  This bug seems to be fixed.

Worked for me.  Even makes updating with binary packages built on another system smoother.
Comment 34 Michiel de Bruijne 2006-08-19 16:47:40 UTC
I have the same problem as Carsten, with this enabled I can't run emerge --depclean anymore. Can you explain why it's better to stop emerge --depclean if Portage thinks that some packages are required but not installed. Why is it not possible to remove the packages that are not required but installed? Also if these packages are required shouldn't a emerge world -DNuv install those? At the moment it appears emerge --depclean has a different depgraph than emerge world -DNuv. From my point of view the current behaviour doesn't seem logical.

emerge --depclean -vp

*** WARNING ***  Depclean may break link level dependencies.  Thus, it is
*** WARNING ***  recommended to use a tool such as `revdep-rebuild` (from
*** WARNING ***  app-portage/gentoolkit) in order to detect such breakage.
*** WARNING ***
*** WARNING ***  Also study the list of packages to be cleaned for any obvious
*** WARNING ***  mistakes. Packages that are part of the world set will always
*** WARNING ***  be kept.  They can be manually added to this set with
*** WARNING ***  `emerge --noreplace <atom>`.

Calculating dependencies... done!
The following are required but not installed:
* virtual/glibc
Comment 35 Zac Medico gentoo-dev 2006-08-19 17:00:10 UTC
(In reply to comment #34)
> Can you explain why it's better to stop emerge --depclean
> if Portage thinks that some packages are required but not installed. Why is it
> not possible to remove the packages that are not required but installed?

It's possible, but not very safe.  When the dependency graph is incomplete, it's  unknown exactly how badly incomplete it is.  In that case, depclean has a high probability of removing packages that the user would probably want to keep.

> if these packages are required shouldn't a emerge world -DNuv install those? 

Yes, it should.

> Calculating dependencies... done!
> The following are required but not installed:
> * virtual/glibc

Hmm, I wonder what pulled that in.  Please file a new bug and post the output of `find /var/db/pkg -name "*DEPEND" | xargs grep -l virtual/glibc`.
Comment 36 Michiel de Bruijne 2006-08-19 17:14:14 UTC
I have just checked the current state on the PC of a friend of mine. His situation is worse;

notice a few strange things;
- emerge --depclean states that games-fps/quake3 and virtual/glibc are required but not installed (he even doesn't want quake3), but emerge -DNuv doesn't mention both
- emerge --depclean states that x11-misc/xdg-utils and net-libs/gnutls are required but not installed. The good thing is that emerge world -DNuv also mention both packages. The bad thing is that now it's mandatory to install these packages before I can remove unnecessary packages. IMHO users/administrators shouldn't be obligated to upgrade the complete system with new versions/packages just to remove unneeded packages of the current installation. For me the new behaviour isn't exactly an improvement, but I understand the previous behaviour (removing packages with --depclean that reappear with -DNuv) isn't very nice either. I hope we can come with some solution in the middle that everybody can work with.


emerge --depclean -vp

*** WARNING ***  Depclean may break link level dependencies.  Thus, it is
*** WARNING ***  recommended to use a tool such as `revdep-rebuild` (from
*** WARNING ***  app-portage/gentoolkit) in order to detect such breakage.
*** WARNING ***
*** WARNING ***  Also study the list of packages to be cleaned for any obvious
*** WARNING ***  mistakes. Packages that are part of the world set will always
*** WARNING ***  be kept.  They can be manually added to this set with
*** WARNING ***  `emerge --noreplace <atom>`.

Calculating dependencies... done!
The following are required but not installed:
* x11-misc/xdg-utils
* games-fps/quake3
* net-libs/gnutls
* virtual/glibc

emerge world -DNuvp

These are the packages that would be merged, in order:

Calculating world dependencies |
... done!
[ebuild     U ] app-admin/perl-cleaner-1.04.3 [1.04.1] 5 kB
[ebuild  N    ] x11-misc/xdg-utils-1.0_beta3  0 kB
[ebuild     U ] app-text/libpaper-1.1.19 [1.1.14.8] 0 kB
[ebuild  N    ] app-crypt/opencdk-0.5.7  USE="-doc" 0 kB
[ebuild  N    ] dev-libs/lzo-2.02-r1  USE="-examples" 0 kB
[ebuild  N    ] net-libs/gnutls-1.2.11  USE="crypt zlib -doc" 0 kB
[ebuild   R   ] net-print/cups-1.2.2  USE="X% dbus jpeg nls pam png ppds ssl tiff -samba -slp" 0 kB
[ebuild     U ] media-gfx/exiv2-0.10-r1 [0.10] USE="unicode -doc" 0 kB
Comment 37 Zac Medico gentoo-dev 2006-08-19 17:24:18 UTC
(In reply to comment #36)
> I have just checked the current state on the PC of a friend of mine. His
> situation is worse;

Again, please file a new bug.
Comment 38 Jason Stubbs (RETIRED) gentoo-dev 2006-08-19 21:32:29 UTC
Bug #144486 for behavioural issues with the new depclean.
Comment 39 Jakub Moc (RETIRED) gentoo-dev 2006-08-26 05:10:10 UTC
*** Bug 145166 has been marked as a duplicate of this bug. ***
Comment 40 Alon Bar-Lev (RETIRED) gentoo-dev 2006-09-06 10:14:27 UTC
Hello,
When will it be stable?

I am having trouble with lzo and with cups.

lzo has side-by-side problem, depclean tries to remove the newest slot.
This is solved by sys-apps/portage-2.1.1_rc1-r4.

cups has or dependencies when installed:
net-print/cups-pdf-2.4.1
net-print/foomatic-db-ppds
net-print/foomatic-filters-ppds

When cups-pdf is installed, depclean tries to remove foomatic-db-ppds and foomatic-filters-ppds.
This is NOT solved by sys-apps/portage-2.1.1_rc1-r4.
Maybe the dependency of cups is incurrent... I think the or is inclusive... But if not... Then I will open a separate bug for cups so it will or and and all combinations of dependencies.
Comment 41 Zac Medico gentoo-dev 2006-09-06 10:25:38 UTC
(In reply to comment #40)
> Hello,
> When will it be stable?

2.1.1 final is planned to be released on Sept. 8 (2 days from now).

> When cups-pdf is installed, depclean tries to remove foomatic-db-ppds and
> foomatic-filters-ppds.
> This is NOT solved by sys-apps/portage-2.1.1_rc1-r4.

Please file a new bug with the ouput of `emerge -ped world`.