Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 234607

Summary: sandbox access violations regarding portage .pyo files
Product: Portage Development Reporter: Wolfram Schlich (RETIRED) <wschlich>
Component: SandboxAssignee: Portage team <dev-portage>
Status: RESOLVED NEEDINFO    
Severity: major CC: python
Priority: High    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---

Description Wolfram Schlich (RETIRED) gentoo-dev 2008-08-13 10:21:28 UTC
this happens on almost every emerge:
--8<--
[...]
>>> Source compiled.
--------------------------- ACCESS VIOLATION SUMMARY ---------------------------
LOG FILE = "/var/log/sandbox/sandbox-8585.log"

unlink:    /usr/lib/portage/pym/portage_exception.pyo
unlink:    /usr/lib/portage/pym/portage.pyo
unlink:    /usr/lib/portage/pym/cache/__init__.pyo
unlink:    /usr/lib/portage/pym/cache/cache_errors.pyo
unlink:    /usr/lib/portage/pym/cvstree.pyo
unlink:    /usr/lib/portage/pym/xpak.pyo
unlink:    /usr/lib/portage/pym/getbinpkg.pyo
unlink:    /usr/lib/portage/pym/output.pyo
unlink:    /usr/lib/portage/pym/portage_const.pyo
unlink:    /usr/lib/portage/pym/portage_util.pyo
unlink:    /usr/lib/portage/pym/portage_dep.pyo
unlink:    /usr/lib/portage/pym/portage_versions.pyo
unlink:    /usr/lib/portage/pym/portage_data.pyo
unlink:    /usr/lib/portage/pym/portage_manifest.pyo
unlink:    /usr/lib/portage/pym/portage_checksum.pyo
unlink:    /usr/lib/portage/pym/portage_exec.pyo
unlink:    /usr/lib/portage/pym/portage_gpg.pyo
unlink:    /usr/lib/portage/pym/portage_locks.pyo
unlink:    /usr/lib/portage/pym/portage_localization.pyo
unlink:    /usr/lib/portage/pym/eclass_cache.pyo
unlink:    /usr/lib/portage/pym/portage_update.pyo
unlink:    /usr/lib/portage/pym/portage_selinux.pyo
--------------------------------------------------------------------------------
--8<--

env -i emerge --info:
--8<--
Portage 2.1.5.6 (default/linux/x86/2008.0/server, gcc-4.1.2, glibc-2.6.1-r0, 2.6.24-gentoo-r5-1 i686)
=================================================================
System uname: 2.6.24-gentoo-r5-1 i686 AMD Athlon(tm) MP
Timestamp of tree: Mon, 11 Aug 2008 00:45:01 +0000
distcc[14321] (dcc_get_top_dir) Warning: HOME is not set; can't find distcc directory [enabled]
ccache version 2.4 [disabled]
app-shells/bash:     3.2_p17-r1
dev-java/java-config: 1.3.7, 2.0.33-r1
dev-lang/python:     2.4.4-r13, 2.5.2-r6
dev-python/pycrypto: 2.0.1-r6
dev-util/ccache:     2.4-r7
sys-apps/baselayout: 1.12.12
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r1
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1
sys-devel/binutils:  2.18-r3
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool:   1.5.26
virtual/os-headers:  2.6.23-r3
ACCEPT_KEYWORDS="x86"
ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci"
ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol"
APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config log_forensic logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_ftp proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias"
APACHE2_MPMS="prefork"
ARCH="x86"
AUTOCLEAN="yes"
BUILD_PREFIX="/var/tmp/portage"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=athlon-mp -pipe"
CHOST="i686-pc-linux-gnu"
CLASSPATH="."
CLEAN_DELAY="5"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/udev/rules.d"
CVS_RSH="ssh"
CXXFLAGS="-O2 -march=athlon-mp -pipe"
DCCC_PATH="/usr/lib/distcc/bin"
DISTCC_LOG="/var/log/distcc/distcc.log"
DISTCC_VERBOSE="1"
DISTDIR="/local/portage/distfiles"
ELIBC="glibc"
EMERGE_DEFAULT_OPTS="-v"
EMERGE_WARNING_DELAY="10"
FEATURES="buildpkg collision-protect distcc distlocks parallel-fetch sandbox sfperms strict unmerge-orphans userfetch userpriv usersandbox"
FETCHCOMMAND="/usr/bin/wget -t 5 -T 60 --passive-ftp -O "${DISTDIR}/${FILE}" "${URI}""
GCC_SPECS=""
GENERATION="2"
GENTOO_MIRRORS="http://distfiles.gentoo.org     http://sunsite.cnlab-switch.ch/ftp/mirror/gentoo        http://ftp.snt.utwente.nl/pub/os/linux/gentoo   http://gentoo.inode.at        http://ftp.ussg.iu.edu/linux/gentoo     http://gentoo.oregonstate.edu   http://distro.ibiblio.org/pub/linux/distributions/gentoo"
INFOPATH="/usr/share/info:/usr/share/binutils-data/i686-pc-linux-gnu/2.18/info:/usr/share/gcc-data/i686-pc-linux-gnu/4.1.2/info"
INPUT_DEVICES="keyboard mouse evdev"
INSTALL_MASK=" /etc/apache2/modules.d/*default*vhost* /etc/apache2/vhosts.d/*default*vhost* /etc/postfix/saslpass /etc/cron.monthly/update-pciids /etc/stunnel/stunnel.crt /etc/stunnel/stunnel.csr /etc/stunnel/stunnel.key /etc/stunnel/stunnel.pem /etc/openldap/ssl/ldap.crt /etc/openldap/ssl/ldap.csr /etc/openldap/ssl/ldap.key /etc/openldap/ssl/ldap.pem /etc/ssl/dovecot/server.crt /etc/ssl/dovecot/server.csr /etc/ssl/dovecot/server.key /etc/ssl/dovecot/server.pem /etc/xinetd.d/* /etc/ppp/ip-* /etc/ppp/ipv6-* /etc/dspam/mysql.data /var/bind/pri/127.zone /etc/nagios/objects/*.cfg"
JAVAC="/opt/sun-jdk-1.4.2.15/bin/javac"
JAVA_HOME="/opt/sun-jdk-1.4.2.15"
JDK_HOME="/opt/sun-jdk-1.4.2.15"
KERNEL="linux"
LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text"
LDFLAGS="-Wl,-O1"
LESS="-R -M --shift 5"
LESSOPEN="|lesspipe.sh %s"
LINGUAS="en de"
MAKEOPTS="-j6"
MANPATH="/usr/local/share/man:/usr/share/man:/usr/share/binutils-data/i686-pc-linux-gnu/2.18/man:/usr/share/gcc-data/i686-pc-linux-gnu/4.1.2/man:/opt/sun-jdk-1.4.2.15/man:/et
c/java-config/system-vm/man/:/usr/lib/php5/man/:/usr/lib/erlang/man"
PAGER="/usr/bin/less"
PATH="/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/4.1.2:/opt/sun-jdk-1.4.2.15/bin:/opt/sun-jdk-1.4.2.15/jre/bin:/opt/sun-jdk-1.4.2.15/jre/javaws:/usr/games/bin"
PKGDIR="/local/portage/packages"
PKG_TMPDIR="/var/tmp/portage-pkg"
PORTAGE_ARCHLIST="ppc s390 amd64 x86 ppc64 x86-fbsd m68k arm sparc sh mips ia64 alpha hppa sparc-fbsd"
PORTAGE_BINHOST_CHUNKSIZE="3000"
PORTAGE_BIN_PATH="/usr/lib/portage/bin"
PORTAGE_COMPRESS="bzip2"
PORTAGE_COMPRESS_FLAGS="-9"
PORTAGE_CONFIGROOT="/"
PORTAGE_DEBUG="0"
PORTAGE_DEPCACHEDIR="/var/cache/edb/dep"
PORTAGE_ELOG_CLASSES="info warn error log"
PORTAGE_ELOG_MAILFROM="portage@prometheus.bla.fasel.org"
PORTAGE_ELOG_MAILSUBJECT="Portage: ${PACKAGE} merged on ${HOST} with notice"
PORTAGE_ELOG_MAILURI="hostmaster@bla.fasel.org mail.bla.fasel.org"
PORTAGE_ELOG_SYSTEM="save:* mail_summary:warn,error"
PORTAGE_FETCH_CHECKSUM_TRY_MIRRORS="5"
PORTAGE_FETCH_RESUME_MIN_SIZE="350K"
PORTAGE_GID="250"
PORTAGE_INST_GID="0"
PORTAGE_INST_UID="0"
PORTAGE_NICENESS="19"
PORTAGE_PYM_PATH="/usr/lib/portage/pym"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --ex
clude=/packages"
PORTAGE_RSYNC_RETRIES="3"
PORTAGE_TMPDIR="/var/tmp"
PORTAGE_WORKDIR_MODE="0700"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/local/portage/layman/wschlich /local/portage/layman/gentoo-de /local/portage/layman/wschlich-testing /local/portage/layman/sunrise /local/portage/layman/cen
terim /local/portage/layman/dertobi123 /usr/local/portage"
PORT_LOGDIR="/var/log/portage"
PYTHONPATH="/usr/lib/portage/pym"
RESUMECOMMAND="/usr/bin/wget -c -t 5 -T 60 --passive-ftp -O "${DISTDIR}/${FILE}" "${URI}""
ROOT="/"
ROOTPATH="/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/4.1.2:/opt/sun-jdk-1.4.2.15/bin:/opt/sun-jdk-1.4.2.15/jre/bin:/opt/sun-jdk-1.4.2.15/jre/javaws"
RPMDIR="/usr/portage/rpm"
SYNC="rsync://rsync.de.gentoo.org/gentoo-portage"
USE="X509 aalib abook accounting acl adns aim ansi apache2 authfile automount bacula-console bash-completion bcmath berkdb bidi bind-mysql bzip2 calendar caps ccache cgi chip
card chroot command-args courier cpudetection cpulimit cracklib crypt cscope css csv ctype cups curl curlwrappers customlog cvsgraph cyrus dba dbi dbm dbx devmap dhcp dlopen
dnsdb ecc exif expat fam fastcgi finger flac flash flatfile foomaticdb ftp gd gdbm gif gssapi gzip hpn http iconv icq idea idled idn imagemagick imap imlib imlib2 inifile inn
odb ipalias ipv6 jabber jpeg ldap libwww lmtp logrotate mailbox maildir maildrop mbox mcal memlimit mgetty mhash mime mimencode ming mmap mpi mppe-mppc mysql mysqli nagios-dn
s nagios-game nagios-ntp nagios-ping nagios-ssh ncurses netboot network nfs nis nls nntp nptl nptlonly oav odbc ogg oggvorbis openssh openssl pam pam_chroot pam_timestamp par
se-clocks pcntl pcre pdf pdfkit pdflib perl php pic png posix postfix postgres ppds procmail pwdb python qdbm quotas radius readline recode reiserfs rrdtool rss ruby samba sa
mple sasl screen sendfile sensord session sftp sftplogging shaper shared sharedext sharedmem simplexml skey slang slp smartcard smime smp sms smtp snmp soap sockets socks5 sp
amassassin sql sqlite sqlite3 sse ssl stream subversion suhosin sysfs syslog sysvipc tcpd theora threads tiff tokenizer transparent-proxy truetype unicode usb userlocales utf
8 vim vim-pager virtual-users virus-scan withsamplescripts x86 xattr xfs xinetd xml xml2 xmlrpc xsl xslt yahoo zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0
106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm
alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODU
LES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user au
toindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config log_forensic logio mem_ca
che mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_ftp proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias"
APACHE2_MPMS="prefork" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" L
INGUAS="en de" USERLAND="GNU" VIDEO_CARDS="fbdev glint i810 mach64 mga neomagic nv r128 radeon savage sis tdfx trident vesa vga via vmware voodoo"
USERLAND="GNU"
USE_EXPAND="ALSA_CARDS ALSA_PCM_PLUGINS APACHE2_MODULES APACHE2_MPMS CAMERAS CROSSCOMPILE_OPTS DVB_CARDS ELIBC FCDSL_CARDS FOO2ZJS_DEVICES FRITZCAPI_CARDS INPUT_DEVICES KERNE
L LCD_DEVICES LINGUAS LIRC_DEVICES MISDN_CARDS USERLAND VIDEO_CARDS"
USE_EXPAND_HIDDEN="CROSSCOMPILE_OPTS ELIBC KERNEL USERLAND"
USE_ORDER="env:pkg:conf:defaults:pkginternal:env.d"
VIDEO_CARDS="fbdev glint i810 mach64 mga neomagic nv r128 radeon savage sis tdfx trident vesa vga via vmware voodoo"
VMHANDLE="sun-jdk-1.4"
--8<--

I remerged python-2.4, portage and sandbox several times.
Then I upgraded to python-2.5 and remerged all dev-python
packages.
No luck.
Comment 1 Zac Medico gentoo-dev 2008-08-13 10:32:22 UTC
There's nothing sandbox can do about it. There's got to be some reason that python wants to replace those *.pyo files with new ones.
Comment 2 Jesus Rivero (RETIRED) gentoo-dev 2008-08-14 04:59:24 UTC
Hmm, Ive seen this before..

  When Python runs with the -O option it will re-generate .pyo files for .py files that have changed, replacing old pyo files with new ones. Now, this may have happened because portage got upgraded and the removal of the old version didn't take care of old pyo files (Same thing happened with scons in bug #107013). 

  One thing I would suggest so you get rid of this is to merge something trivial with FEATURES='-sandbox' so .pyo files get generated and everything should be back to normal afterwards. 

  Although it would be real nice if you could provide us with more info on when exactly you started getting this message (after upgrading portage perhaps? or python?)

Kind regards, 
Comment 3 Wolfram Schlich (RETIRED) gentoo-dev 2008-08-14 08:20:15 UTC
(In reply to comment #2)
> Hmm, Ive seen this before..
> 
>   When Python runs with the -O option it will re-generate .pyo files for .py
> files that have changed, replacing old pyo files with new ones. Now, this may
> have happened because portage got upgraded and the removal of the old version
> didn't take care of old pyo files (Same thing happened with scons in bug
> #107013). 
> 
>   One thing I would suggest so you get rid of this is to merge something
> trivial with FEATURES='-sandbox' so .pyo files get generated and everything
> should be back to normal afterwards. 

I cannot confirm this -- I've been merging quite some packages with
FEATURES="-sandbox -usersandbox" for some time, and it didn't fix it.

>   Although it would be real nice if you could provide us with more info on when
> exactly you started getting this message (after upgrading portage perhaps? or
> python?)

It's been too long now to reconstruct that, sorry :(
It happens on 3 machines for me.
Comment 4 Zac Medico gentoo-dev 2008-08-14 09:00:55 UTC
If nothing you do ever solves it then it seems like maybe the *.pyo files are constantly being updated over and over again by python.
Comment 5 Jesus Rivero (RETIRED) gentoo-dev 2008-08-14 14:26:08 UTC
Im going to shoot at the dark here, as I can't reproduce this so I can't test properly. Did you try to remove all pyc, pyo files from /usr/lib/portage/pym and then emerge with -sandbox?

Another thing, Have you tried to update portage to see if that helps with this issue? 

Have you checked that those pyo files actually change in time when using portage? And could you check that running one of those files with python -O directly produces the same result pyo file that the one being generated by emerge?

Thanks, 
Comment 6 Wolfram Schlich (RETIRED) gentoo-dev 2009-02-13 18:34:38 UTC
meanwhile I've upgraded all those machines to portage-2.2_rc*,
no problems anymore :)
Comment 7 Zac Medico gentoo-dev 2009-02-13 18:49:47 UTC
Please reopen if it happens again.