Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 531770

Summary: emerge -aev1 @ world gives different results than emerge -av1 @installed.
Product: Portage Development Reporter: Bob Raitz <pappy_mcfae>
Component: Core - Interface (emerge)Assignee: Portage team <dev-portage>
Status: RESOLVED INVALID    
Severity: normal CC: esigra, pacho
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=350230
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 155723    
Attachments: debug.log.xz
Results of requested operation used on initial system with differing install.
differences between -aev world and @installed --nodeps on proj_x

Description Bob Raitz 2014-12-06 00:13:21 UTC
When updating system gcc, we are advised to use emerge -aev1 @ world (or some equivalent) to update all packages. While doing this, I noted that the @installed set gave differing results as to number of packages installed as a result of each emerge function.

Reproducible: Always

Steps to Reproduce:
1. use emerge -aev1 @world.
2. use emerge -av1 @installed.
3. compare number of packages marked for emerge.



I use the --nodeps with the @installed set to avoid possible conflicts or other issues that remain with the @installed set.
Comment 1 Bob Raitz 2014-12-06 00:21:18 UTC
In order to be consistent, I will use the oldest setup for this.

Results of emerge -av1 @installed --nodeps:

Total: 1429 packages (9 upgrades, 1420 reinstalls), Size of downloads: 0 KiB
Fetch Restriction: 2 packages

Would you like to merge these packages? [Yes/No] n

Results of emerge -aev1 @world:

Total: 1241 packages (1 upgrade, 5 new, 1235 reinstalls, 11 uninstalls), Size of downloads: 395 KiB
Fetch Restriction: 1 package
Conflict: 11 blocks

Would you like to merge these packages? [Yes/No] n
Comment 2 Bob Raitz 2014-12-06 00:23:03 UTC
emerge --info for the system being discussed:

core-too ~ # emerge --info
Portage 2.2.14 (python 2.7.8-final-0, default/linux/amd64/13.0, gcc-4.8.3, glibc-2.20, 3.17.4 x86_64)
=================================================================
System uname: Linux-3.17.4-x86_64-Intel-R-_Core-TM-2_Duo_CPU_E6550_@_2.33GHz-with-gentoo-2.2
KiB Mem:     2563980 total,    979480 free
KiB Swap:    1951892 total,   1951892 free
Timestamp of tree: Wed, 03 Dec 2014 08:00:01 +0000
ld GNU ld (Gentoo 2.24 p1.4) 2.24
distcc 3.2rc1 x86_64-pc-linux-gnu [enabled]
app-shells/bash:          4.3_p30-r1
dev-java/java-config:     2.2.0
dev-lang/perl:            5.20.1-r3
dev-lang/python:          2.7.8, 3.2.5-r6, 3.3.5-r1, 3.4.2
dev-util/cmake:           3.0.2
dev-util/pkgconfig:       0.28-r2
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.13.6
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.11.6-r1, 1.14.1
sys-devel/binutils:       2.24-r3
sys-devel/gcc:            4.8.3::Local-Portage-Overlay
sys-devel/gcc-config:     1.8
sys-devel/libtool:        2.4.3-r2
sys-devel/make:           4.1-r1
sys-kernel/linux-headers: 3.17-r1 (virtual/os-headers)
sys-libs/glibc:           2.20
Repositories: gentoo Local-Portage-Overlay
Installed sets: @system
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA dlj-1.1 googleearth AdobeFlash-11.x Oracle-BCLA-JavaSE"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=core2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.5/ext-active/ /etc/php/apache2-php5.6/ext-active/ /etc/php/cgi-php5.5/ext-active/ /etc/php/cgi-php5.6/ext-active/ /etc/php/cli-php5.5/ext-active/ /etc/php/cli-php5.6/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -march=core2 -pipe"
DISTDIR="/mnt/seeds/64-too/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--autounmask=n --quiet-build=n"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distcc distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j4"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://192.168.0.105/gentoo-portage"
USE="X a52 aac acpi ads alsa amd64 apache2 apng audiofile avi bash-completion berkdb bitmap-fonts bzip2 cairo cdr clang cli consolekit cracklib crypt css ctype cups cxx dbus dlloader dri drm dvd dvdr dvdread emboss encode fam ffmpeg fftw filter flac fontconfig fortran gdbm gif glib gnutls gpm gsl gtk gudev hwdb iconv icu jpeg kde kdenlive kdgraphics ladspa lame ldap libkms libsamplerate libwww live lm_sensors mad mikmod mmx mng modules mp3 mp4 mpeg multilib mysql nas ncurses nepomuk nls nptl nptlonly objc ogg opengl openmp openvg pam pcre pdcre pdf pdflib perl php plasma png ppds python qt3support qt4 quicktime readline reflection samba script sdl semantic-desktop session slang slp smp sndfile spell spl sql sqlite sse sse2 ssl ssse3 startup-notification svg swat tcpd theora thunar tiff truetype udev udisks unicode upcall v4l v4l2 vhosts vorbis winbind wxwindows x264 xml xmp xorg xscreensaver xv xvid zlib" ABI_X86="64" ALSA_CARDS="intel-hda" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status thunar unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby20" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
Comment 3 Zac Medico gentoo-dev 2014-12-06 06:38:31 UTC
Please post debug.log.xz created as follows:

   emerge -pv --depclean --debug > debug.log 2>&1
   xz debug.log
Comment 4 Zac Medico gentoo-dev 2014-12-06 06:53:04 UTC
Also, it's useful to know the content of /etc/portage/profile/package.provided if it exists,
Comment 5 Bob Raitz 2014-12-07 09:34:38 UTC
I don't have any packages installed outside portage on any of my systems, so the file you seek does not exist. The file you asked me to create was on the order of 12.2 megs. At 12.2 megs, pastebin is having none of it. 

Because of that, I am going to have to source it locally. Please, I ask that those concerned download the resulting file, and let me know when you have it so I can take it down. I don't want to tie up the network here anymore than absolutely necessary. 

http://142.129.188.109/  #Ain't apache all that?

Thanks in advance.
Comment 6 Zac Medico gentoo-dev 2014-12-07 10:07:28 UTC
Created attachment 391114 [details]
debug.log.xz

(In reply to Bob Raitz from comment #5)
Got it thanks. Attached is the same file compressed with xz.

The summary at the bottom shows this:

Packages installed:   1429
Packages in world:    454
Packages in system:   43
Required packages:    1222
Number to remove:     207

It shows that you only have 1222 required packages, which explains why your 'emerge -aev1 @world' from comment #1 only pulled in 1241 packages. The numbers are not identical because the 'emerge -aev1 @world' command is pulling in updates, while the --depclean command is not.

When your system is in a state like this, it's normal for @installed to pull in more packages than 'emerge -e @world'. In order to make them pull in an identical number of packages, you have to either let --depclean remove those 207 packages, or else add those 207 packages to your world file so that 'emerge -e @world' can pull them in.
Comment 7 Bob Raitz 2014-12-08 09:10:44 UTC
(In reply to Zac Medico from comment #6)
> Created attachment 391114 [details]
> debug.log.xz
> 
> (In reply to Bob Raitz from comment #5)
> Got it thanks. Attached is the same file compressed with xz.
> 
> The summary at the bottom shows this:
> 
> Packages installed:   1429
> Packages in world:    454
> Packages in system:   43
> Required packages:    1222
> Number to remove:     207
> 
> It shows that you only have 1222 required packages, which explains why your
> 'emerge -aev1 @world' from comment #1 only pulled in 1241 packages. The
> numbers are not identical because the 'emerge -aev1 @world' command is
> pulling in updates, while the --depclean command is not.
> 
> When your system is in a state like this, it's normal for @installed to pull
> in more packages than 'emerge -e @world'. In order to make them pull in an
> identical number of packages, you have to either let --depclean remove those
> 207 packages, or else add those 207 packages to your world file so that
> 'emerge -e @world' can pull them in.

Funny thing about those packages. Many of them were deleted in order to get @installed to work. Some reinstalled themselves during the initial -aev world I performed in an attempt to stabilize portage and get rid of lots of dead packages that kept killing the @installed set. That is a bit puzzling.

That also doesn't take into account the fact that this happens with all my installs, including the most recent Funtoo install which awaits a machine on which it can fly solo. While I can believe this machine has built up a lot of cruft over the years, I can't believe that a setup that has been updated only a few times would have accumulated as much cruft as one that's been operating for at least five years continuously.

I will check on my other systems and report any other issues. I want to be sure that everything is as it should be.

I will try your commands on all my setups to see if this is an actual fix, or if there's still something not quite right.
Comment 8 Zac Medico gentoo-dev 2014-12-08 15:55:45 UTC
(In reply to Bob Raitz from comment #7)
> That also doesn't take into account the fact that this happens with all my
> installs, including the most recent Funtoo install which awaits a machine on
> which it can fly solo. While I can believe this machine has built up a lot
> of cruft over the years, I can't believe that a setup that has been updated
> only a few times would have accumulated as much cruft as one that's been
> operating for at least five years continuously.

I see --depclean remove packages after practically every update, so it doesn't surprise me that you have stale packages if you're not using --depclean to remove stale packages after *every* update.
Comment 9 Bob Raitz 2014-12-08 22:23:14 UTC
Created attachment 391246 [details]
Results of requested operation used on initial system with differing install.

This shows a drastic example of the way that --depclean treats a relatively new system installation.
Comment 10 Bob Raitz 2014-12-08 22:24:14 UTC
(In reply to Zac Medico from comment #8)
> (In reply to Bob Raitz from comment #7)
> > That also doesn't take into account the fact that this happens with all my
> > installs, including the most recent Funtoo install which awaits a machine on
> > which it can fly solo. While I can believe this machine has built up a lot
> > of cruft over the years, I can't believe that a setup that has been updated
> > only a few times would have accumulated as much cruft as one that's been
> > operating for at least five years continuously.
> 
> I see --depclean remove packages after practically every update, so it
> doesn't surprise me that you have stale packages if you're not using
> --depclean to remove stale packages after *every* update.

You want to know why I don't use --depclean? Read the attachment. This installation is a fresh install from a Funtoo tarball onto a spare hard drive to be developed for a planned new laptop come the new year. According to the extensive list of files to be nuked by depclean, I'll be lucky to have anything resembling a working system with fully functional X, xfce4, and kde.

That, in a nutshell is why I don't use --depclean. When the time comes that I can invoke it on a freshly setup system and have it want to destroy all the work I've done to get it where it is, there's something wrong somewhere. Simply dismissing this issue as me not tending to my machinery seems wrong, very wrong.

Just to be sure, yes, this install is on its own drive. Yes, it has booted a system, and brought forth X functionality. That tells me it's setup properly. So, why on Earth does --depclean want to nuke everything?

If this is another bug, then I'll keep writing them until either I get serious carpal tunnel stress, or the bottom of this issue is found.
Comment 11 Bob Raitz 2014-12-08 22:28:34 UTC
emerge --info for this installation:

(proj_x) core-too / # emerge --info
Portage 2.3.6-r9 (funtoo/1.0/linux-gnu/arch/x86-64bit, gcc-4.8.3, glibc-2.19, 3.17.4 x86_64)
=================================================================
System uname: Linux-3.17.4-x86_64-Intel-R-_Core-TM-2_Duo_CPU_E6550_@_2.33GHz-with-gentoo-2.2.0
KiB Mem:     2563980 total,   1361308 free
KiB Swap:    1951892 total,   1951892 free
Timestamp of tree: Unknown
ld GNU ld (Gentoo 2.24 p1.4) 2.24
app-shells/bash:          4.3_p30-r1
dev-java/java-config:     2.2.0-r1000
dev-lang/perl:            5.20.1-r3
dev-lang/python:          2.7.8-r1000, 3.3.6-r1000
dev-util/cmake:           3.0.2
sys-apps/baselayout:      2.2.0-r6
sys-apps/openrc:          0.12.3-r3
sys-apps/sandbox:         2.6-r2
sys-devel/autoconf:       2.69
sys-devel/automake:       1.11.6, 1.13.4
sys-devel/binutils:       2.24-r3
sys-devel/gcc:            4.8.3
sys-devel/gcc-config:     1.8
sys-devel/libtool:        2.4.3-r2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.17-r1 (virtual/os-headers)
sys-libs/glibc:           2.19
Repositories: gentoo Local-Portage-Overlay
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA AdobeFlash-11.x"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=core2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -march=core2 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--autounmask=n --quiet-build=n"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch xattr"
FFLAGS=""
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--sort-common -Wl,--as-needed"
LINGUAS="en"
MAKEOPTS="-j4"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="git://github.com/funtoo/ports-2012.git"
SYNC_USER="root"
USE="# #initial X \ acl acpi alsa amd64 apng avi bash-completion berkdb bitmap-fonts bzip2 cairo cli cracklib crypt cups cxx dbus declarative ffmpeg fftw fortran gdbm gif glib gpm gtk gudev iconv icu jpeg kde minizip mmx mng modules mp3 mp4 mpeg mudflap multilib mysql ncurses nls nptl ogg opengl openmp pam pcre png policykit python qt3support qt4 readline resolvconf script set spell sql sse sse2 ssl ssse3 startup-notification svg tcpd thunar tiff truetype udev udisks unicode vorbis webkit xattr xfce4 xml xmp xorg xscreensaver xv xvid zlib" ABI_X86="64" ALSA_CARDS="intel-hda" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias authn_core authz_core socache_shmcb unixd" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_ABIS="2.7 3.3" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" QEMU_SOFTMMU_TARGETS="i386 x86_64" QEMU_USER_TARGETS="i386 x86_64" RUBY_TARGETS="ruby19 ruby20 ruby21" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, SYNC_UMASK
Comment 12 Bob Raitz 2014-12-08 22:31:35 UTC
Created attachment 391248 [details]
differences between -aev world and @installed --nodeps on proj_x

Results of -aev world and @installed on proj_x.
Comment 13 Arfrever Frehtes Taifersar Arahesis 2014-12-08 23:42:31 UTC
(In reply to Bob Raitz from comment #10)
> That, in a nutshell is why I don't use --depclean. When the time comes that
> I can invoke it on a freshly setup system and have it want to destroy all
> the work I've done to get it where it is, there's something wrong somewhere.

That wrong something is in procedure of your work. You should have added packages needed by you to world file:

Example:
emerge --noreplace www-client/firefox-bin x11-base/xorg-x11
Comment 14 Alexander Berntsen (RETIRED) gentoo-dev 2014-12-09 08:47:25 UTC
Zac, I do not think it is pertinent to waste your time with this. I suggest we close as INVALID.

Bob, this is bugs.gentoo.org not bugs.funtoo.org. And I don't know what you are doing to screw things up this bad. Arfrever is right — the packages should be in your world. Feel free to stop by #gentoo-portage on irc.freenode.net and maybe we can investigate further what you are doing wrong. But this is not a bug.
Comment 15 Zac Medico gentoo-dev 2014-12-10 09:03:25 UTC
Yeah, all the evidence I've seen here shows emerge acting as designed.

A good counter-example would be bug 350230.
Comment 16 Bob Raitz 2014-12-17 07:36:02 UTC
(In reply to Alexander Berntsen from comment #14)
> Zac, I do not think it is pertinent to waste your time with this. I suggest
> we close as INVALID.
> 
> Bob, this is bugs.gentoo.org not bugs.funtoo.org. And I don't know what you
> are doing to screw things up this bad. Arfrever is right — the packages
> should be in your world. Feel free to stop by #gentoo-portage on
> irc.freenode.net and maybe we can investigate further what you are doing
> wrong. But this is not a bug.

So, the fact that other systems were able to update world the world file without --norepace has no bearing on what's wrong here?

Frankly, at this point, I'm glad I took the time away from this. 

As long as the @installed set stays, I'm fine with it all, as I said before, many, many times.

As for whether it's Funtoo or Gentoo, you're writing emerge for both systems, unless Funtoo has decided to ditch y'all. Of all the people on the planet, I figured you folks would know that. :shrug:

Anyway, do as you will. I consider the fact that I got rid of years of crap to be a blessing. The minutia is of no concern at this point. 

Thanks for your efforts.