Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 215116 - preserve-libs: backtrace on injecting a directory
Summary: preserve-libs: backtrace on injecting a directory
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All All
: High normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-03-27 21:57 UTC by Fabian Groffen
Modified: 2008-04-12 09:00 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
fixing the wrong NEEDED entries by reading scanelf's output correctly (portage-scanelf.patch,664 bytes, patch)
2008-03-28 21:18 UTC, Fabian Groffen
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Fabian Groffen gentoo-dev 2008-03-27 21:57:11 UTC
The error:

injecting /Library/Gentoo/var/lib into /Library/Gentoo/var/tmp/portage/app-editors/emacs-22.2/image/
Traceback (most recent call last):
  File "/Library/Gentoo/usr/bin/emerge", line 20, in <module>
    retval = _emerge.emerge_main()
  File "/Library/Gentoo/usr/lib/portage/pym/_emerge/__init__.py", line 7783, in emerge_main
    myopts, myaction, myfiles, spinner)
  File "/Library/Gentoo/usr/lib/portage/pym/_emerge/__init__.py", line 7148, in action_build
    retval = mergetask.merge(pkglist, favorites, mtimedb)
  File "/Library/Gentoo/usr/lib/portage/pym/_emerge/__init__.py", line 4435, in merge
    return self._merge(mylist, favorites, mtimedb)
  File "/Library/Gentoo/usr/lib/portage/pym/_emerge/__init__.py", line 4721, in _merge
    prev_mtimes=ldpath_mtimes)
  File "/Library/Gentoo/usr/lib/portage/pym/portage/__init__.py", line 4981, in doebuild
    vartree=vartree, prev_mtimes=prev_mtimes)
  File "/Library/Gentoo/usr/lib/portage/pym/portage/__init__.py", line 5176, in merge
    mydbapi=mydbapi, prev_mtimes=prev_mtimes)
  File "/Library/Gentoo/usr/lib/portage/pym/portage/dbapi/vartree.py", line 2542, in merge
    mydbapi=mydbapi, prev_mtimes=prev_mtimes)
  File "/Library/Gentoo/usr/lib/portage/pym/portage/dbapi/vartree.py", line 2550, in _merge
    cleanup=cleanup, mydbapi=mydbapi, prev_mtimes=prev_mtimes)
  File "/Library/Gentoo/usr/lib/portage/pym/portage/dbapi/vartree.py", line 1968, in treewalk
    self._preserve_libs(srcroot, destroot, myfilelist+mylinklist, counter)
  File "/Library/Gentoo/usr/lib/portage/pym/portage/dbapi/vartree.py", line 1675, in _preserve_libs
    os.path.join(srcroot, x.lstrip(os.sep)))
  File "/Library/Gentoo/usr/lib/python2.5/shutil.py", line 91, in copy2
    copyfile(src, dst)
  File "/Library/Gentoo/usr/lib/python2.5/shutil.py", line 46, in copyfile
    fsrc = open(src, 'rb')
IOError: [Errno 21] Is a directory


After some digging, it appears that in _preserve_libs(self, srcroot, destroot, mycontents, counter) in pym/portage/dbapi/vartree.py preserve_libs eventually contains an entry 'lib', this is mapped on the directory /var/lib, because emacs appears to install /var/lib/games/emacs/.keep.

There are a number of weird things going on here:
- why does libary_consumers have a "lib" entry?
  lib /Library/Gentoo/usr/lib/postgresql/autoinc.so,/Library/Gentoo/usr/lib/...
  but:
  % scanelf -n /Library/Gentoo/usr/lib/postgresql/autoinc.so
    TYPE   NEEDED FILE 
  XET_DYN  /Library/Gentoo/usr/lib/postgresql/autoinc.so 
- where does "lib" come from?  to end up in preserved_libs, it needs to be in 
  old_libs:
  old_contents = self._installed_instance.getcontents().keys()
  old_libs = set([os.path.basename(x) for x in old_contents]).intersection(libmap)
  but, there is no "lib" file in emacs:
  % qlist emacs | grep lib
  /Library/Gentoo/usr/share/emacs/22.1/etc/celibacy.1
  /Library/Gentoo/usr/share/emacs/22.1/lisp/calendar/diary-lib.el
  /Library/Gentoo/usr/share/emacs/22.1/lisp/calendar/diary-lib.elc
  /Library/Gentoo/usr/libexec/emacs/22.1/i386-pc-solaris2.10/sorted-doc
  /Library/Gentoo/usr/libexec/emacs/22.1/i386-pc-solaris2.10/digest-doc
  /Library/Gentoo/usr/libexec/emacs/22.1/i386-pc-solaris2.10/hexl
  /Library/Gentoo/usr/libexec/emacs/22.1/i386-pc-solaris2.10/cvtmail
  /Library/Gentoo/usr/libexec/emacs/22.1/i386-pc-solaris2.10/vcdiff
  /Library/Gentoo/usr/libexec/emacs/22.1/i386-pc-solaris2.10/update-game-score
  /Library/Gentoo/usr/libexec/emacs/22.1/i386-pc-solaris2.10/fakemail
  /Library/Gentoo/usr/libexec/emacs/22.1/i386-pc-solaris2.10/movemail
  /Library/Gentoo/usr/libexec/emacs/22.1/i386-pc-solaris2.10/rcs2log
  /Library/Gentoo/usr/libexec/emacs/22.1/i386-pc-solaris2.10/profile
  /Library/Gentoo/var/lib/games/emacs/.keep_app-editors_emacs-22
- because "lib" has external consumers (weirdness in the library_consumers
  file) the dir is not removed, eventually shutil dies.  Should the code be
  more defensive, or is the library_consumers file supposed to be 100% correct?
  Note that I had to patch a bit there, because it only works for ELF/Linux.
  See also commits 9497-9499:
  http://sources.gentoo.org/viewcvs.py/portage/main/branches/prefix/pym/portage/dbapi/vartree.py?rev=9499&view=log

(I'm filing a bug, because this is genone's stuff, who is hard to catch in IRC, obviously, I'm using Prefix Portage.)
Comment 1 Fabian Groffen gentoo-dev 2008-03-28 21:18:25 UTC
Created attachment 147575 [details, diff]
fixing the wrong NEEDED entries by reading scanelf's output correctly

Ok, the bug is caused by a wrong NEEDED file:

% grep earthdistance.so $EPREFIX/var/db/pkg/dev-db/postgresql-8.2.7/NEEDED
/Library/Gentoo/usr/lib/postgresql/earthdistance.so /Library/Gentoo/usr/lib/postgresql/earthdistance.so:/Library/Gentoo/usr/i386-pc-solaris2.10/lib:/Library/Gentoo/usr/i386-pc-solaris2.10/lib/gcc:/Library/Gentoo/usr/lib:/Library/Gentoo/lib

This is caused by bin/misc-functions.sh around line 160.

The scanelf command returns:
% scanelf -qyRF '%p:%r %n' /Library/Gentoo/usr/lib/postgresql/earthdistance.so
/Library/Gentoo/usr/lib/postgresql/earthdistance.so:/Library/Gentoo/usr/i386-pc-solaris2.10/lib:/Library/Gentoo/usr/i386-pc-solaris2.10/lib/gcc:/Library/Gentoo/usr/lib:/Library/Gentoo/lib 

That is: the filename, the runpath, a space, and nothing (there are no deps)

The code in bin/misc-functions.sh fails to parse it, since it uses default IFS, so the trailing space is removed by bash, making sure that the loop that handles it doesn't see the space, and hence considers the runpath as needed lib, and hence "lib" shows up as required lib.

I fixed this in the attached patch, but this is SpanKY's territory, IMO.

While I was at it, I also noted that rpath is wrongly extracted from ${l}, if you have more than one path stored in it, rpath only contains the last one.  If you are unfortunately enough to have spaces in your paths, you even will end up with an incomplete rpath.

I applied the patch to prefix in r9582 as it fixes the problem for me.
Comment 2 Thomas Kjosmoen 2008-04-08 10:42:56 UTC
I had the same problem on OS X 10.5.2, using portage-2.2.00.9583, when emerging Python:
dev-lang/python-2.5.1-r5  USE="doc ncurses readline ssl tk -berkdb -bootstrap -build -examples -gdbm -ipv6 -nothreads -sqlite -ucs2"

Using FEATURES="-preserve-libs" solved the problem.
Comment 3 Fabian Groffen gentoo-dev 2008-04-12 09:00:09 UTC
fixed in prefix.