Bug 492288 – =sys-apps/portage-2.2.7 segfaults in visit_decref at Modules/gcmodule.c:361 - 'str' object has no attribute 'cpv_split' in portage/versions.py, line 336 called from portage/versions.py, line 382 with both mydata and cpv set to 'sys-devel/automake-1.12'

• Status: RESOLVED OBSOLETE
• Product: Portage Development

[Description Justin Lecher (RETIRED) 2013-11-22 11:32:25 UTC]

I am calling 
emerge --with-bdeps y -uvaDNt1 system world
and see segf.

Please find the complete report attached


$ einfo portage
Portage 2.2.7 (default/linux/amd64/13.0, gcc-4.8.2-asneeded, glibc-2.17, 3.12.1-lh x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-3.12.1-lh-x86_64-Intel-R-_Core-TM-_i7-2600_CPU_@_3.40GHz-with-gentoo-2.2
KiB Mem:    16468040 total,   5414016 free
KiB Swap:    4194300 total,   4194300 free
Timestamp of tree: Fri, 22 Nov 2013 10:00:01 +0000
ld GNU gold (GNU Binutils 2.23.2) 1.11
ccache version 3.1.9 [enabled]
app-shells/bash:          4.2_p45
dev-java/java-config:     2.2.0
dev-lang/python:          2.7.5-r4, 3.2.5-r3, 3.3.3
dev-util/ccache:          3.1.9-r3
dev-util/cmake:           2.8.12.1-r1
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.4
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.10.3, 1.11.6, 1.12.6, 1.13.4, 1.14
sys-devel/binutils:       2.23.2
sys-devel/gcc:            4.1.2, 4.7.3-r1, 4.8.2
sys-devel/gcc-config:     1.8
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.11 (virtual/os-headers)
sys-libs/glibc:           2.17
Repositories: gentoo sunrise bicatali betagarden dummy Neurogeek science last-hope
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -ftracer   -march=native -frecord-gcc-switches -g -Wimplicit-function-declaration"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/eselect/ /usr/share/gnupg/qualified.txt /usr/share/nano/ /var/lib/hsqldb /var/spool/munin-async/.ssh /var/spool/torque"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -ftracer   -march=native -frecord-gcc-switches -g -Wenum-compare"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS=" -vt --keep-going --autounmask-write --autounmask --quiet-build=n"
FCFLAGS="-O2 -pipe -ftracer   -march=native -frecord-gcc-switches -g"
FEATURES="assume-digests binpkg-logs buildsyspkg ccache cgroup collision-protect compressdebug distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news noinfo parallel-fetch parallel-install preserve-libs protect-owned sandbox sfperms sign split-log splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -ftracer   -march=native -frecord-gcc-switches -g"
GENTOO_MIRRORS=" /mnt/tmpfs/ http://gentoo.j-schmitz.net/mirror/ ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo ftp://ftp.tu-clausthal.de/pub/linux/gentoo/"
LANG="en_GB.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -Wl,-O1 -Wl,--hash-style=gnu -Wl,--sort-common"
MAKEOPTS="-j4 -l8"
PKGDIR="/usr/portage/packages"
PORTAGE_COMPRESS="xz"
PORTAGE_COMPRESS_FLAGS="-z -e -9 -v"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="--omit-dir-times"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/local/overlays/sunrise-reviewed /local/overlays/bicatali /local/overlays/betagarden /local/overlays/dummy /local/overlays/neurogeek /local/overlays/sci /local/overlays/lh-overlay"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X acl alsa amd64 avx bash-completion berkdb branding bzip2 cairo caps cli cracklib crypt cups cxx dbus dri fortran gdbm gif gmp gstreamer gtk iconv icu ipv6 jpeg jpeg2k ldap logrotate mmx mmxext modules mudflap multilib ncurses network-cron nls nptl nsplugin numa opengl openmp pam pcre pgo png pulseaudio qt3support readline session smp sse sse2 sse2_4way sse2check sse3 sse4 sse41 sse4_1 sse4a sse5 ssl ssse3 startup-notification tcpd threads tiff truetype unicode vaapi vdpau xattr xinerama zlib" ABI_X86="64" ALSA_CARDS="hda-intel" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="plymouth" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" GRUB_PLATFORMS="efi-32 efi-64" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer pdfimport scripting-javascript wiki-publisher nlpsolver" LINGUAS="en" NGINX_MODULES_HTTP="access auth_basic autoindex browser charset empty_gif fastcgi geo gzip limit_conn limit_req map memcached proxy referer rewrite scgi split_clients ssi upstream_ip_hash userid uwsgi stub_status" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-4" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" QEMU_SOFTMMU_TARGETS="i386 x86_64 arm armeb" QEMU_USER_TARGETS="i386 x86_64 arm armeb" RUBY_TARGETS="ruby19" USERLAND="GNU" VIDEO_CARDS="nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
USE_PYTHON="2.7 3.3"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND

=================================================================
                        Package Settings
=================================================================

sys-apps/portage-2.2.7 was built with the following:
USE="(ipc) (multilib) python3 xattr -build -doc -epydoc (-pypy2_0) -python2 (-selinux)" ABI_X86="64" LINGUAS="-ru" PYTHON_TARGETS="python2_7 python3_3 (-pypy2_0) -python2_6 -python3_2 -python3_4"

[Comment 1 Justin Lecher (RETIRED) 2013-11-22 11:32:59 UTC]

Created attachment 363794 [details]
abrt.log

Full trace log

[Comment 2 Sebastian Luther (few) 2013-11-22 11:55:13 UTC]

Is this reproducible?

What segfaults here is python. Hardware problems could easily cause this.

[Comment 3 Justin Lecher (RETIRED) 2013-11-22 12:09:46 UTC]

It did so 4 times in a row. But now it isn't doing it anymore.

[Comment 4 Sebastian Luther (few) 2013-11-25 11:38:20 UTC]

Any news from your side? Otherwise I don't see anything we can do here.

[Comment 5 Justin Lecher (RETIRED) 2013-11-25 11:47:33 UTC]

(In reply to Sebastian Luther (few) from comment #4)
> Any news from your side? Otherwise I don't see anything we can do here.

Probably this is related to the sandbox problem with linux-3.12*

[Comment 6 Tom Wijsman (TomWij) (RETIRED) 2013-11-25 22:40:04 UTC]

The output is mangled with the backtrace, noteworthy is the summary ($$$):

> :#0  visit_decref (op=<unknown at remote 0x80>, data=0x0) at /var/tmp/portage/dev-lang/python-3.3.3/work/Python-3.3.3/Modules/gcmodule.c:361
> $$$ :Python Exception <type 'exceptions.TypeError'> object of type 'FakeRepr' has no len(): $$$
> :#1  0x00002b10febd42f8 in dict_traverse (op=, visit=0x2b10fecbcf40 <visit_decref>, arg=0x0) at /var/tmp/portage/dev-lang/python-3.3.3/work/Python-3.3.3/Objects/dictobject.c:2402

If we look further we can see that Python gives us the values in the parameters, which learns us more:

> 'str' object has no attribute 'cpv_split'

So, eh, which 'str' object?

> Frame 0x13bd220, for file /usr/lib64/portage/pym/portage/versions.py, line 336, in catpkgsplit (mydata='sys-devel/automake-1.12', silent=1, eapi=None)

Ah!

> def catpkgsplit(mydata, silent=1, eapi=None):
> ...
>     return mydata.cpv_split

Bummer. Hmm, so, 'mydata' is a string; huh, what calls it then?

> Frame 0x13bae60, for file /usr/lib64/portage/pym/portage/versions.py, line 382, in __init__ (self=<_pkg_str at remote 0x2b1104a353d0>, cpv='sys-devel/automake-1.12', metadata=None, settings=None, eapi=None, repo=None, slot=None)

Note that the 'cpv' parameter appears to be fine.

If we look a bit up, we get to see the cpv parameter to the catpkgsplit call getting initialized this way:

> if not isinstance(cpv, _unicode):
>     # Avoid TypeError from _unicode.__init__ with PyPy.
>     cpv = _unicode_decode(cpv)
>     _unicode.__init__(cpv)

So, appears we have something related to Unicode failing. Seems that something in this _unicode functionality might change the object into a string.

I'm not convinced that this is caused by the kernel, but regardless; we'll follow the bug to follow along what's going on.

I suggest checking whether this is reproducible on a different version of Python as well as trying a non-3.12 kernel. Alternatively you can try to apply the commit that patches it (http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=ede4cebce16f5643c61aedd6d88d9070a1d23a68) so you don't have to build the whole kernel again.

(In reply to Justin Lecher from comment #3)
> It did so 4 times in a row. But now it isn't doing it anymore.

Interesting, sounds like an odd random bug in either Python or the kernel...

Hmm, if you can't consistently reproduce it, that might complicate things. :/

[Comment 7 Tom Wijsman (TomWij) (RETIRED) 2013-11-25 22:48:47 UTC]

http://bugs.python.org/issue17951 and http://bugs.python.org/msg188888 appear to tell me that the summary I first caught between the output is a false positive which is a result of gdb debug, I'll update the summary with the Python errors.

[Comment 8 Tom Wijsman (TomWij) (RETIRED) 2013-11-25 22:59:27 UTC]

Oh look, something very similar reported upstream by Arfrever. \o/

Not sure if this is the exact same, but it is very likely; can you try 3.2?

[Comment 9 Tom Wijsman (TomWij) (RETIRED) 2013-11-26 01:33:04 UTC]

> #gentoo-python [Arfrever] | CPython bug 16089 is fixed in CPython >=3.3.1.

[Comment 10 Justin Lecher (RETIRED) 2013-11-26 07:18:22 UTC]

(In reply to Tom Wijsman (TomWij) from comment #8)
> Not sure if this is the exact same, but it is very likely; can you try 3.2?

It is not reproducible at the moment. And I never saw this with py3.2.