Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 347565 - dev-lang/python site.py crashes on python startup due to defective .pth file
Summary: dev-lang/python site.py crashes on python startup due to defective .pth file
Status: RESOLVED UPSTREAM
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords:
: 347573 (view as bug list)
Depends on:
Blocks:
 
Reported: 2010-12-02 15:13 UTC by Donald Wallace Rouse II
Modified: 2010-12-27 16:53 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Donald Wallace Rouse II 2010-12-02 15:13:23 UTC
starting python 2.7 yields the following error message:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site.py", line 554, in <module>
    main()
  File "/usr/lib64/python2.7/site.py", line 537, in main
    known_paths = addsitepackages(known_paths)
  File "/usr/lib64/python2.7/site.py", line 316, in addsitepackages
    addsitedir(sitedir, known_paths)
  File "/usr/lib64/python2.7/site.py", line 192, in addsitedir
    addpackage(sitedir, name, known_paths)
  File "/usr/lib64/python2.7/site.py", line 162, in addpackage
    exec line
  File "<string>", line 1, in <module>
KeyError: 'zope'

This is due to a bug in the file /usr/lib64/python2.7/site-packages/zope.app.schema-3.5.0-py2.7-nspkg.pth, which I am reporting as a separate bug unrelated to python 2.7 itself.
However, this bug causes things like portage to fail completely, which is a Very Bad Thing.

In the file /usr/lib64/python2.7/site.py, at about line 162, you have an unprotected "exec" line:
    if line.startswith(("import ", "import\t")):
        exec line

If the execution of the line fails, python generates an uncaught exception. This places python at the mercy of bugs in third-party software.

The "exec" line should be bracketed by "try/except" to catch such errors:
    if line.startswith(("import ", "import\t")):
        try:
            exec line
        except:
            print "Failed to exec line " + line + "in file " + fullname
        continue

(Or maybe replace the "print" line with "pass", depending on how you want to handle it.)

Reproducible: Always

Steps to Reproduce:
1. Install zope under python 2.6 (before they fix their bug).
2. Install python 2.7.
3. Run python-updater.

After "net-zope/zope-app-schema-3.5.0" is updated, everything after will fail to be updated, portage will fail completely, and running anything that uses python 2.7 will fail.
Actual Results:  
I get the traceback message shown above when I try to run any software (like the "emerge" portage command) that uses python. Note that to someone who doesn't know how to fix site.py, this is a showstopper.

Expected Results:  
I expected programs using python not to crash.

Here is the output from emerge --info (after I made the temporary fix shown above):

Failed to exec line import sys,types,os; p = os.path.join(sys._getframe(1).f_locals['sitedir'], *('zope', 'app')); ie = os.path.exists(os.path.join(p,'__init__.py')); m = not ie and sys.modules.setdefault('zope.app',types.ModuleType('zope.app')); mp = (m or []) and m.__dict__.setdefault('__path__',[]); (p not in mp) and mp.append(p); m and setattr(sys.modules['zope'], 'app', m)
in file /usr/lib64/python2.7/site-packages/zope.app.schema-3.5.0-py2.7-nspkg.pth
Portage 2.1.9.25 (default/linux/amd64/10.0, gcc-4.4.5, glibc-2.12.1-r3, 2.6.36-gentoo-r3 x86_64)
=================================================================
System uname: Linux-2.6.36-gentoo-r3-x86_64-AMD_Athlon-tm-_64_X2_Dual_Core_Processor_4200+-with-gentoo-2.0.1
Timestamp of tree: Wed, 01 Dec 2010 04:45:01 +0000
app-shells/bash:     4.1_p9
dev-java/java-config: 2.1.11-r2
dev-lang/python:     2.6.6-r1, 2.7.1, 3.1.3
dev-util/cmake:      2.8.1-r2
sys-apps/baselayout: 2.0.1-r1
sys-apps/openrc:     0.6.6
sys-apps/sandbox:    2.4
sys-devel/autoconf:  2.13, 2.68
sys-devel/automake:  1.4_p6-r1, 1.7.9-r2, 1.8.5-r4, 1.9.6-r3, 1.10.3, 1.11.1
sys-devel/binutils:  2.20.1-r1
sys-devel/gcc:       4.4.5, 4.5.1-r1
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.4-r1
sys-devel/make:      3.82
virtual/os-headers:  2.6.36.1 (sys-kernel/linux-headers)
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA DOOM3 ETQW IBM-J1.6 PUEL dlj-1.1 sun-bcla-java-vm"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -O2 -pipe -g -ggdb"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config /var/bind /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/eselect/postgresql /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=athlon64 -O2 -pipe -g -ggdb"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs distlocks fixlafiles fixpackages installsources news nostrip parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
GENTOO_MIRRORS="http://www.gtlib.gatech.edu/pub/gentoo                 http://distro.ibiblio.org/pub/linux/distributions/gentoo/                http://mirrors.rit.edu/gentoo/"
LANG="en_US.UTF-8"
LC_ALL="POSIX"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en_US en"
MAKEOPTS="-j1"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/var/lib/layman/sunrise /var/lib/layman/voyageur /var/lib/layman/kde-sunset /var/lib/layman/roslin /var/lib/layman/haskell"
SYNC="rsync://rsync5.us.gentoo.org/gentoo-portage"
USE="3dnow X a52 aac acl acpi aim alsa amd64 ao apache2 apm atm audiofile avahi bash-completion bcmath bidi bindist blas branding bsf bzip2 cairo calendar caps cdda cddb cdparanoia cdr cgi chasen cjk clamav cli cracklib crypt cscope css ctype cups curl curlwrappers cvs cxx dbi dbus dbx dga directfb djvu doc dri dts dv dvb dvd dvdr encode enscript esd evo examples exif expat fam fastcgi fbcon ffmpeg fftw flac fltk fontconfig foomaticdb fortran freewnn ftp gcj gd gdbm geoip ggi gif gimp ginac glut gmp gnome gnome-keyring gnuplot gnustep gnutls gphoto2 gpm graphviz gsl gsm gstreamer gtk gtkhtml guile handbook hardened hddtemp iconv icu idn imagemagick imap imlib inifile innodb iodbc ipv6 jack java java6 javascript jbig jpeg jpeg2k kde kerberos kontact krb4 ladspa lame lapack lash latex lcms ldap libcaca libedit libgda libnotify libsamplerate libwww lm_sensors lua lzo m17n-lib mad maildir mailwrapper matroska mbox memlimit mhash migemo mikmod mime mmap mmx mng modplug modules mono motif mp3 mp4 mpeg mpi mplayer mtp mudflap multilib musepack musicbrainz mysql mysqli nas ncurses neXt netcdf networkmanager nis nls nntp nocd nptl nptlonly nsplugin ocaml ocamlopt odbc offensive ogg openal openexr opengl openmp osc pam pcntl pcre pdf plasma plotutils png policykit portaudio posix postgres ppds pppd prelude profile pulseaudio python qt3 qt4 quicktime raw readline recode rss ruby samba sasl scanner sdl seamonkey semantic-desktop session sharedmem shorten simplexml skey smartcard smp sndfile snmp soap sockets socks5 source sox speex spell sqlite sqlite3 sse sse2 ssl startup-notification static-libs subversion suid svg symlink sysfs syslog sysvipc szip taglib tcl tcpd theora threads tidy tiff timidity tk truetype udev unicode usb vcd vhosts videos vim-syntax vorbis wavpack wddx wmf wxwidgets x264 xattr xcb xcomposite xface xft xine xinetd xml xmlrpc xmp xorg xosd xpm xscreensaver xsl xulrunner xv xvid zeroconf zlib zsh-completion" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" CAMERAS="canon directory" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev keyboard mouse wacom" KERNEL="linux" LINGUAS="en_US en" NETBEANS_MODULES="apisupport cnd dlight enterprise ergonomics groovy gsf harness ide identity j2ee java mobility nb profiler ruby webcommon websvccommon" PHP_TARGETS="php5-2" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="fbdev nv nvidia vesa virtualbox" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 1 Donald Wallace Rouse II 2010-12-02 15:58:27 UTC
I have reported the actual bug that revealed this problem at http://bugs.gentoo.org/show_bug.cgi?id=347573
Comment 2 Donald Wallace Rouse II 2010-12-02 17:06:48 UTC
OK, this is embarrassing.
The workaround that I provided above was sufficient to allow emerge --info to work, but when I tried to actually emerge something, it failed.
Replacing the line with "print >>sys.stderr, ..." seems to work but causes lots of output. I guess the solution is to use "pass", but of course that means that bugs related to problems with .pth files will be more difficult to diagnose.
Comment 3 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2010-12-05 20:03:30 UTC
1. This behavior exists in all versions of Python and Setuptools/Distribute.
2. You can file improvement request in upstream bug tracker:
   http://bugs.python.org/
3. The direct cause of your problem is that you manually created /usr/lib64/python2.7/site-packages/zope/__init__.py file. Please remove it:
   rm -f /usr/lib64/python2.7/site-packages/zope/__init__.py*

(There are some plans to change handling of something related to this bug.)
Comment 4 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2010-12-05 20:03:49 UTC
*** Bug 347573 has been marked as a duplicate of this bug. ***
Comment 5 Donald Wallace Rouse II 2010-12-07 02:44:46 UTC
RE: Comment #3: I most certainly did _not_ create /usr/lib64/python2.7/site-packages/zope/__init__.py manually; I was not even aware of that file until you mentioned it in your comment. An install process must have created it.

Here is the output from "equery belongs /usr/lib64/python2.7/site-packages/zope/__init__.py":
   * Searching for /usr/lib64/python2.7/site-packages/zope/__init__.py ...
  net-zope/zope-interface-3.6.1 (/usr/lib64/python2.7/site-packages/zope/__init__.py)

So there's still a bug in there somewhere.
Comment 6 Donald Wallace Rouse II 2010-12-07 02:46:14 UTC
Sorry. forgot to check "Reopen bug".
Comment 7 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2010-12-07 07:38:07 UTC
Attach full build log of net-zope/zope-interface.
Comment 8 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2010-12-27 16:53:03 UTC
Not ignoring of errors at startup is intentional:
http://svn.python.org/view?view=rev&revision=78826
http://svn.python.org/view?view=rev&revision=78872

site.py will tolerate broken .pth files in Python 2.7.2, 3.1.4 and 3.2.
(>=2.7.2_pre20110102:2.7, >=3.1.4_pre20110102:3.1 and >=3.2_pre20110102:3.2 from python overlay will also contain this change.)
http://svn.python.org/view?view=rev&revision=87497
http://svn.python.org/view?view=rev&revision=87499
http://svn.python.org/view?view=rev&revision=87500