Today once again I did an emerge -Dup world which told me koffice libs, data and kivio where to be updated but not the rest of koffice which has been upgraded to 1.4.1. Reproducible: Always Steps to Reproduce: 1. emerge -Dup world 2. 3. Actual Results: not all updates are emerged. Expected Results: All updates are emerged Koffice was emerged from kde-meta and the bug does seem to be related to meta packages as if emerge only looks at the world file and not at what is installed and requires updating. I have flagged this as a major as it could have security implications. emerge info Portage 2.0.51.22-r2 (default-linux/x86/2005.0, gcc-3.3.5, glibc-2.3.5-r0, 2.6.12-gentoo-r6 i686) ================================================================= System uname: 2.6.12-gentoo-r6 i686 AMD Athlon(tm) processor Gentoo Base System version 1.6.13 ccache version 2.3 [enabled] dev-lang/python: 2.3.5 sys-apps/sandbox: 1.2.11 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.5 sys-devel/binutils: 2.15.92.0.2-r1, 2.15.92.0.2-r10 sys-devel/libtool: 1.5.18-r1 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-mcpu=athlon-xp -march=athlon-xp -O3 -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/splash /etc/terminfo /etc/env.d" CXXFLAGS="-mcpu=athlon-xp -march=athlon-xp -O3 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache distcc distlocks sandbox sfperms strict userpriv" GENTOO_MIRRORS="http://gentoo.blueyonder.co.uk ftp://mirrors.blueyonder.co.uk/mirrors/gentoo http://www.mirrorservice.org/sites/www.ibiblio.org/gentoo/ ftp://ftp.mirrorservice.org/sites/www.ibiblio.org/gentoo/" LINGUAS="en_GB" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="x86 3dnow 3dnowext X X509 a52 aac acpi acpi4linux aim alsa aotuv apm arts atm audiofile avantgo avi bash-completion berkdb bitmap-fonts bluetooth browserplugin cddb cdparanoia cdr chroot crypt cups curl dar32 dbus dga directfb divx4linux djbfft dpms dv dvb dvd dvdr dvdread dxr3 edl eds emboss encode esd exif extensions faad fam fame fb fbcon ffmpeg fftw flac font-server foomaticdb fortran fpx ftp gdbm gif gimp gimpprint gnokii gnome gphoto2 gpm gstreamer gtk gtk2 hal icq idea imagemagick imlib ipv6 irmc ithreads jabber java javascript joystick jpeg jpeg2k kde kdepim kdexdeltas lcms ldap libg++ libwww lirc lm_sensors lzo lzw-tiff mad mikmod mime mjpeg mmx mmxext mng motif mp3 mpeg mpi mplayer msn multislot mysql nas ncurses network nls no-old-linux nptl nptlonly offensive ogg oggvorbis opengl opie oscar oss pam pam_chroot pcre pda pdflib perl pic png ppds private python qt quicktime rar rdesktop readline real samba sblive scanner sdl silc slang smime sms sndfile sockets sox speex spell sqlite sse ssl svg svga symlink sysfs tcpd tga theora threads tiff truetype truetype-fonts type1-fonts unicode usb usepackagedmakefiles v4l v4l2 vcd vidix visualization vorbis wifi win32codecs wmf xanim xface xine xml2 xmms xpm xscreensaver xv xvid xvmc yahoo yv12 zeroconf zlib zvbi linguas_en_GB userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS
Also I have just noticed that not everything I emerge is being put int the world file. I did explicitly emerge koffice-meta and it is not in my world file as neither is kde-meta which I also explicitly emerged. The last thing in my world file is net-analyzer/tcpdump which was emerged way back in March of this year and nothing has been added since.
I have added the contents of my package.keywords to the world file and now an emerge -Dup world gives:- These are the packages that I would merge, in order: Calculating world dependencies ...done! [ebuild U ] app-arch/arj-3.10g [3.10.21] [ebuild U ] kde-base/kmail-3.4.1-r1 [3.4.1] [ebuild U ] kde-base/kweather-3.4.1-r1 [3.4.1] [ebuild U ] media-gfx/gphoto2-2.1.6 [2.1.5] [ebuild U ] kde-base/ksysguard-3.4.1-r1 [3.4.1] [ebuild U ] dev-util/valgrind-2.4.0 [2.2.0-r2] [ebuild U ] dev-util/callgrind-0.9.11-r1 [0.9.10-r1] [ebuild U ] app-office/koshell-1.4.1 [1.4.0] [ebuild U ] media-sound/ogg2mp3-0.5 [0.3] [ebuild U ] net-wireless/wireless-tools-27-r1 [27] [ebuild U ] app-pda/gtkpod-0.94.0 [0.88.2] [ebuild U ] app-office/kformula-1.4.1 [1.4.0] [ebuild U ] x11-misc/xscreensaver-4.22-r2 [4.20] [ebuild U ] app-office/kchart-1.4.1 [1.4.0] [ebuild U ] app-office/kspread-1.4.1 [1.4.0] [ebuild U ] app-office/kugar-1.4.1 [1.4.0] [ebuild U ] kde-base/kig-3.4.1-r10 [3.4.1] [ebuild U ] net-firewall/iptables-1.3.2 [1.3.1-r4] [ebuild U ] app-office/kpresenter-1.4.1 [1.4.0] [ebuild U ] media-plugins/gst-plugins-ffmpeg-0.8.5 [0.8.4-r1] [ebuild U ] app-office/kword-1.4.1 [1.4.0] [ebuild U ] app-office/kexi-1.4.1 [1.4.0] [ebuild U ] app-office/krita-1.4.1 [1.4.0] [ebuild U ] app-office/koffice-meta-1.4.1 [1.4.0] [ebuild U ] media-plugins/gst-plugins-mpeg2dec-0.8.10 [0.8.9] [ebuild U ] media-libs/libdts-0.0.2-r2 [0.0.2-r1] This shows that emerge is only looking at the world file and not what is installed as I would expect it to. If portage was adding correctly to the world file it would still not update dependencies in this case, which could have serious security implications for a server which is on the net. I wonder how long this has been like this? At least since March of this year and I have seen one other bug logged against this which was closed and should have been looked at further.
Probably can close as a dupe of bug 97616 "This shows that emerge is only looking at the world file and not what is installed as I would expect it to." FYI, Portage only looks at the world file to calculate world dependencies.
I will close if you can explain why any new package I explicitly emerge does not get put into the world file otherwise I would like this left open until both bugs are resolved.
The other thing is what about files that get emerged as dependencies that never get added to the world file and pulled in by meta files. They always used to get updated so has emerged changed in some way?
Ok so you used the kde-meta ebuild to get your kde installed. Do you have a copy of your world file prior to adding your package.keywords stuff to it? If not could you post your current world file please.
Is there any chance that your system has run out of space at some point when you've been running emerge? My world file has been deleted/zeroed numerous times because of that, and I've had to refill it manually by looking at the output of emerge depclean -p|less.
Created attachment 64561 [details] world file before edits My world file before it was edited and I have not run out of disk space ever:- df -h Filesystem Size Used Avail Use% Mounted on /dev/hda3 56G 8.0G 48G 15% / udev 442M 2.7M 440M 1% /dev /dev/sda1 76G 14G 59G 19% /mnt/disk1 hda3 is my root partition and has never gone above the 15% to 20% usage I am aware of what running out of space can do. The system was rebuilt at the end of Feb beginning of March this year due to a failed disk.
Can you attach a compressed ( bzip2, gzip ) copy of your emerge.log? It would be helpful to see what emerge commands were run to duplicate this problem.
Created attachment 64620 [details] emerge log Emerge log as requested.
here is an example of emergeing a new package:- emerge sdparm snip >>> Recording sys-apps/sdparm in "world" favorites file... >>> clean: No packages selected for removal. >>> Auto-cleaning packages ... >>> No outdated packages were found on your system. * GNU info directory index is up-to-date. localhost ~ # tail /var/lib/portage/world app-portage/gentoolkit media-video/lve dev-perl/XML-Simple media-video/em8300-libraries kde-base/qtsharp dev-perl/Tie-IxHash kde-base/kdebase-applnk kde-base/kenolaba net-analyzer/tcpdump dev-perl/MIME-Lite It has not recorded it to the world file although it says it has.
Use grep rather than tail as the world file is not added to by appending. It is recreated each time with no specific order.
g1gsw@localhost ~ $ grep -i sdparm /var/lib/portage/world g1gsw@localhost ~ $
Very strange. Do you have any aliases for emerge? Although permissions should cause visible problems from emerge, what are the permissions on /, /var, /var/lib, /var/lib/portage and /var/lib/portage/world? What options is/are the filesystem(s) mounted with? Obviously you have things in your world file so it would appear to be an intermittent thing. Can you reliably reproduce it? Can you reliably not reproduce it? Any other possible information?
ls -la /var total 2 drwxr-xr-x 12 root root 336 Jul 25 18:16 . drwxr-xr-x 20 root root 456 Jul 29 21:25 .. -rw-r--r-- 1 root root 0 Jul 20 17:21 .keep drwxr-xr-x 6 root root 144 Jun 4 14:28 cache drwxr-xr-x 3 root root 72 Oct 27 2004 db drwxr-xr-x 2 root root 72 May 20 22:02 empty drwxr-xr-x 24 root root 672 Jul 20 17:11 lib drwxrwxr-x 3 root uucp 96 Jul 29 21:24 lock drwxr-xr-x 8 root root 784 Jul 29 21:28 log lrwxrwxrwx 1 root root 15 May 31 23:48 mail -> /var/spool/mail drwxr-xr-x 12 root root 744 Jul 29 21:28 run drwxr-xr-x 7 root root 192 Jun 4 14:28 spool drwxr-xr-x 2 root root 72 Oct 27 2004 state drwxrwxrwt 8 root root 240 Jul 30 11:26 tmp ls -la /var/lib total 5 drwxr-xr-x 24 root root 672 Jul 20 17:11 . drwxr-xr-x 12 root root 336 Jul 25 18:16 .. drwxr-xr-x 3 root root 88 Jun 26 18:33 bluetooth drwxr-xr-x 3 root root 80 Mar 4 01:38 cache drwxr-xr-x 2 root root 88 Jul 14 18:52 dbus drwxr-xr-x 6 root root 256 Jul 29 21:28 init.d drwxr-xr-x 2 root root 72 Jul 26 21:13 ip6tables drwxr-xr-x 2 root root 104 Jul 26 21:13 iptables drwxr-xr-x 2 root root 72 Jul 29 21:28 kdm -rw-r--r-- 1 root root 257 May 15 03:10 logrotate.status drwxr-xr-x 2 root root 72 Oct 27 2004 misc drwxr-xr-x 2 root root 72 Jul 6 22:37 module-rebuild drwxr-x--- 4 mysql mysql 96 Jul 25 20:00 mysql drwxr-xr-x 3 root root 128 Jul 20 17:11 net-scripts drwxr-xr-x 2 ntp ntp 80 Jun 4 10:51 ntp drwx------ 2 ldap ldap 72 Jul 13 18:12 openldap-data drwx------ 2 ldap ldap 72 Jul 13 18:12 openldap-ldbm drwx------ 2 ldap ldap 72 Jul 13 18:12 openldap-slurp drwxrws--- 2 root portage 96 Jul 30 11:33 portage drwxr-xr-x 6 root root 144 Jun 4 14:28 samba drwxr-xr-x 2 root root 72 May 21 18:15 sasl2 drwxr-xr-x 36 root root 904 Jul 7 22:05 scrollkeeper drwxr-x--- 2 root slocate 104 Jul 30 03:16 slocate drwx------ 3 root root 104 Jul 7 19:36 xdm drwxr-xr-x 2 root root 72 Jul 29 21:28 xkb ls -la /var/lib/portage/ total 25 drwxrws--- 2 root portage 96 Jul 30 11:33 . drwxr-xr-x 24 root root 672 Jul 20 17:11 .. -rw-rw---- 1 root portage 6400 Jul 30 11:33 config -rw-rw---- 1 root portage 14336 Jul 30 11:28 world /dev/hda3 is root cat /etc/fstab /dev/hda3 / reiserfs sync 0 1 It would appear that since I re installed in early March it stopped adding things to world about 2 to 3 weeks later and now whenever I install anything with emrge packagename it is not added to the world file. I can reproduce it all the time. I cannot give any more info as I am at a loss as to what can cause it. I have been using Gentoo for about 18 months and Linux for about 10 years so I am not a novice and can resolve most things. I always run emerge as the root user.
No idea on the cause, but it seems to becoming a more and more common bug. Perhaps an strace might yield something?
Created attachment 64734 [details, diff] Process versioned atoms correctly Try this out. I really don't think it'll help with the behaviour you are describing but it might. In cases like `emerge =sys-apps/portage-2.0.51.22-r2` nothing is being added to world. This patch fixes that.
Sorry for the late reply but as you summised the patch does not resolve the problem.
Try sorting the world file, emerge something (and confirm it wasn't added) and check to see if the world file is still sorted. If it isn't, it means that it's not being written to at all. If it is, it means that emerge didn't add the package to the to-be-added-to-world list.
Well I don't know what has changed but I just emerged 2 packages and they are both are in world. I suggest we close the bug and see what happens. Thanks for all your help and patience.
I've been tracking this bug for about a week now, as I have the same problem. One interesting thing I should note is that I tried app-portage/porthole-0.4.1, and it seems to find all the updates that emerge misses. eg. My current emerge -pDu world shows no updates: --- tweak bin # emerge -pDut world These are the packages that I would merge, in reverse order: Calculating world dependencies ...done! tweak bin # echo $? 0 --- However porthole lists 18, including: gnome-base/gnome gnome-base/control-center dev-perl/perl-ldap www-client/epiphany ... and a slew of others, none of which are in my 'world' file. One further thing to note, when I ask emerge to update dev-perl/perl-ldap, it recognizes that there is an update to be made: --- tweak portage # emerge -pD perl-ldap These are the packages that I would merge, in order: Calculating dependencies ...done! [ebuild U ] dev-perl/perl-ldap-0.33 [0.31] --- I have never ran out of disk space, and I can't think of any other incident that would cause my 'world' file to corrupt or disappear. My current world file only has 86 lines in it. From /var/db/pkg I count that I should have at least hundreds more. Would it be a good/bad idea to make a list of packages installed using /var/db/pkg, and cat that all onto world? Carl
I have noticed this too and did use it to find all packages missing from my world file. For some reason portage has just started to work again, so whether one of the files that got updated by porthole fixed portage I will never know.
(In reply to comment #22) > I have noticed this too and did use it to find all packages missing from my > world file. For some reason portage has just started to work again, so whether > one of the files that got updated by porthole fixed portage I will never know. I looked at porthole's source, it will iterate through all packages running emerge -u <pkg> and then check the output. This will catch all packages, as opposed to just packages in world.
(In reply to comment #3) > Probably can close as a dupe of bug 97616 > > "This shows that emerge is only looking at the world file and not what is > installed as I would expect it to." > > FYI, Portage only looks at the world file to calculate world dependencies. Yes, this looks like Bug 97616. (In reply to comment #23) > I looked at porthole's source, it will iterate through all packages running > emerge -u <pkg> and then check the output. This will catch all packages, as > opposed to just packages in world. > That explains the difference. I suggest that people with this problem run "emerge -a depclean" and check the output for packages they want to keep. Append those packages to /var/lib/portage/world.
*** Bug 97616 has been marked as a duplicate of this bug. ***
*** Bug 100527 has been marked as a duplicate of this bug. ***
Duped the others on this as this one has a patch for at least one issue. Note for everybody interested in this bug, `emerge -uD world` is not meant to update all packages. It is only meant to update what is in world and their dependencies - that is, only packages which have been specified on the command line without --oneshot. I've posted one patch already that fixes a bug where `emerge >=some/package-ver` wasn't adding some/package to the world file. If anybody can reproducably emerge in some way that should add packages to world but doesn't, please post steps on how to reproduce.
*** Bug 102107 has been marked as a duplicate of this bug. ***
I would recommend that --oneshot and other modified things get recorded in a file like /etc/portage/portage.noautoupdate. All other files should be updated if possible to prevent security issues. My recommendation is to have something like cd /var/db/pkg ; find -mindepth 2 -maxdepth 2 -type d | sed s,'-r[0-9]*$',,g | sed s,'\(./\)\([^/]*\)/\(.*\)-\([^-]*\)','\2/\3',g | xargs emerge -pu to emerge all installed packages plus a filter that filters out everything in /etc/portage/portage.noautoupdate. Thus, one always has an up-to-date system with all required packages. The world-file can not do this, especially if USE-flags change, since then dependencies change and packages that are installed and may be exploited do not get updated. My complete packagelist and emerge infos are in Bug 102107, sorry that I didn't find this one.
This should be 100% fixed by 2.0.53_rc3. If not, please reopen and provide details. Remember, if it's not in world it doesn't count.
