Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 487196 - sys-apps/portage-2.2.7 fails while building dev-python/pygobject-3.8.3
Summary: sys-apps/portage-2.2.7 fails while building dev-python/pygobject-3.8.3
Status: UNCONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 1 vote (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-10-07 09:23 UTC by Dieter Verfaillie
Modified: 2014-06-12 17:28 UTC (History)
7 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
pygobject-3.8.3 build.log (pygobject-3.8.3_build.log,276.34 KB, text/plain)
2013-10-07 09:25 UTC, Dieter Verfaillie
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Dieter Verfaillie 2013-10-07 09:23:26 UTC
Disabled MAKEOPTS and set
USE_PYTHON='-2.6 2.7 3.2 3.3'
PYTHON_TARGETS="python2_7 python3_2 python3_3"

snipped output (will attach build.log in a moment):

libtool: install: warning: remember to run `libtool --finish /usr/lib64/python2.7/site-packages/gi'
 /bin/mkdir -p '/var/tmp/portage/dev-python/pygobject-3.8.3/image//usr/lib64/python2.7/site-packages/gi'
 /var/tmp/portage/._portage_reinstall_.lgibpp/bin/ebuild-helpers/xattr/install -c -m 644 __init__.py types.py module.py importer.py pygtkcompat.py '/var/tmp/portage/dev-python/pygobject-3.8.3/image//usr/lib64/python2.7/site-packages/gi'
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site.py", line 62, in <module>
    import os
  File "/usr/lib64/python2.7/os.py", line 49, in <module>
    import posixpath as path
  File "/usr/lib64/python2.7/posixpath.py", line 17, in <module>
    import warnings
  File "/usr/lib64/python2.7/warnings.py", line 8, in <module>
    import types
  File "types.py", line 27, in <module>
    from . import _gobject
ValueError: Attempted relative import in non-package
make[3]: *** [install-pygiPYTHON] Error 1
make[3]: Leaving directory `/var/tmp/portage/dev-python/pygobject-3.8.3/work/pygobject-3.8.3-python2_7/gi'
make[2]: *** [install-am] Error 2
make[2]: Leaving directory `/var/tmp/portage/dev-python/pygobject-3.8.3/work/pygobject-3.8.3-python2_7/gi'
make[1]: *** [install-recursive] Error 1
make[1]: Leaving directory `/var/tmp/portage/dev-python/pygobject-3.8.3/work/pygobject-3.8.3-python2_7/gi'
make: *** [install-recursive] Error 1
 * ERROR: dev-python/pygobject-3.8.3::gentoo failed (install phase):


Looks to me like "/var/tmp/portage/._portage_reinstall_.lgibpp/bin/ebuild-helpers/xattr/install"
might be a python script that doesn't like the types.py copy in cwd
(ie "/var/tmp/portage/dev-python/pygobject-3.8.3/work/pygobject-3.8.3-python2_7/gi"). Similar
output can be seen earlier (when installing for Python 3.X), but those don't seem to halt the
build...


Reproducible: Always
Comment 1 Dieter Verfaillie 2013-10-07 09:25:32 UTC
Created attachment 360298 [details]
pygobject-3.8.3 build.log
Comment 2 Dieter Verfaillie 2013-10-07 12:08:58 UTC
Hmmm, looking a bit closer reveals that $__PORTAGE_HELPER_CWD
is set to ${PWD} or in other words "/var/tmp/portage/dev-python/pygobject-3.8.3/work/pygobject-3.8.3-python2_7/gi".
in "/usr/lib64/portage/bin/ebuild-helpers/xattr/install", which
is supposed to avoid unsafe imports as noted in bug #469338.

So, when Python evaluates "import os", eventually we get an
"import types" [1] which results in
"/var/tmp/portage/dev-python/pygobject-3.8.3/work/pygobject-3.8.3-python2_7/gi/types.py"
being loaded which is clearly not what we want...

Honestly, I fail to see what's safe about this as just about any package
using the "install" command might come with a file shadowing a highly
used Python module name. Such package doesn't even need to actually install
such a file, it just needs to exist.

Anyway, setting $__PORTAGE_HELPER_CWD to whatever else works around this
specific issue but then later the pygobject build fails with:
    /usr/bin/install: cannot stat ‘AUTHORS’: No such file or directory
    !!! dodoc: AUTHORS does not exist
    /usr/bin/install: cannot stat ‘ChangeLog’: No such file or directory
    !!! dodoc: ChangeLog does not exist
    /usr/bin/install: cannot stat ‘NEWS’: No such file or directory
    !!! dodoc: NEWS does not exist
    /usr/bin/install: cannot stat ‘README’: No such file or directory
    !!! dodoc: README does not exist
     * ERROR: dev-python/pygobject-3.8.3::gentoo failed (install phase):

Looks like I'm stuck then :(


[1] ~ $ python -v -S -c 'import os'
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
import encodings # directory /usr/lib64/python2.7/encodings
# /usr/lib64/python2.7/encodings/__init__.pyc matches /usr/lib64/python2.7/encodings/__init__.py
import encodings # precompiled from /usr/lib64/python2.7/encodings/__init__.pyc
# /usr/lib64/python2.7/codecs.pyc matches /usr/lib64/python2.7/codecs.py
import codecs # precompiled from /usr/lib64/python2.7/codecs.pyc
import _codecs # builtin
# /usr/lib64/python2.7/encodings/aliases.pyc matches /usr/lib64/python2.7/encodings/aliases.py
import encodings.aliases # precompiled from /usr/lib64/python2.7/encodings/aliases.pyc
# /usr/lib64/python2.7/encodings/utf_8.pyc matches /usr/lib64/python2.7/encodings/utf_8.py
import encodings.utf_8 # precompiled from /usr/lib64/python2.7/encodings/utf_8.pyc
Python 2.7.5 (default, Aug 20 2013, 11:00:01) 
[GCC 4.7.3] on linux2
# /usr/lib64/python2.7/os.pyc matches /usr/lib64/python2.7/os.py
import os # precompiled from /usr/lib64/python2.7/os.pyc
import errno # builtin
import posix # builtin
# /usr/lib64/python2.7/posixpath.pyc matches /usr/lib64/python2.7/posixpath.py
import posixpath # precompiled from /usr/lib64/python2.7/posixpath.pyc
# /usr/lib64/python2.7/stat.pyc matches /usr/lib64/python2.7/stat.py
import stat # precompiled from /usr/lib64/python2.7/stat.pyc
# /usr/lib64/python2.7/genericpath.pyc matches /usr/lib64/python2.7/genericpath.py
import genericpath # precompiled from /usr/lib64/python2.7/genericpath.pyc
# /usr/lib64/python2.7/warnings.pyc matches /usr/lib64/python2.7/warnings.py
import warnings # precompiled from /usr/lib64/python2.7/warnings.pyc
# /usr/lib64/python2.7/linecache.pyc matches /usr/lib64/python2.7/linecache.py
import linecache # precompiled from /usr/lib64/python2.7/linecache.pyc
# /usr/lib64/python2.7/types.pyc matches /usr/lib64/python2.7/types.py
import types # precompiled from /usr/lib64/python2.7/types.pyc
    <snipped rest of output>
Comment 3 Arfrever Frehtes Taifersar Arahesis 2013-10-08 00:33:38 UTC
Maybe reinstall sys-apps/portage.
Comment 4 Dieter Verfaillie 2013-10-08 11:12:50 UTC
Did some more debugging and both
"/usr/lib64/portage/bin/ebuild-helpers/xattr/install" and
"/usr/lib64/portage/bin/install.py" are working just fine.

Looking into "/var/tmp/portage/dev-python/pygobject-3.8.3/work/pygobject-3.8.3-python2_7/gi/Makefile"
we see that "am__py_compile" is called right after "INSTALL_DATA":

install-pygiPYTHON: $(pygi_PYTHON)
        @$(NORMAL_INSTALL)
        @list='$(pygi_PYTHON)'; dlist=; list2=; test -n "$(pygidir)" || list=; \
        if test -n "$$list"; then \
          echo " $(MKDIR_P) '$(DESTDIR)$(pygidir)'"; \
          $(MKDIR_P) "$(DESTDIR)$(pygidir)" || exit 1; \
        fi; \
        for p in $$list; do \
          if test -f "$$p"; then b=; else b="$(srcdir)/"; fi; \
          if test -f $$b$$p; then \
            $(am__strip_dir) \
            dlist="$$dlist $$f"; \
            list2="$$list2 $$b$$p"; \
          else :; fi; \
        done; \
        for file in $$list2; do echo $$file; done | $(am__base_list) | \
        while read files; do \
          echo " $(INSTALL_DATA) $$files '$(DESTDIR)$(pygidir)'"; \
          $(INSTALL_DATA) $$files "$(DESTDIR)$(pygidir)" || exit $$?; \
        done || exit $$?; \
        if test -n "$$dlist"; then \
          $(am__py_compile) --destdir "$(DESTDIR)" \
                            --basedir "$(pygidir)" $$dlist; \
        else :; fi

And:
am__py_compile = PYTHON=$(PYTHON) $(SHELL) $(py_compile)
PYTHON = /usr/bin/python2.7
SHELL = /bin/sh
py_compile = $(top_srcdir)/py-compile
top_srcdir = ..
and just in case:
abs_top_srcdir = /var/tmp/portage/dev-python/pygobject-3.8.3/work/pygobject-3.8.3-python2_7

This means that after "__init__.py types.py module.py
importer.py and pygtkcompat.py" are copied to
"/var/tmp/portage/dev-python/pygobject-3.8.3/image//usr/lib64/python2.7/site-packages/gi"
the command "PYTHON=/usr/bin/python2.7 SHELL=/bin/sh /var/tmp/portage/dev-python/pygobject-3.8.3/work/pygobject-3.8.3-python2_7/py-compile"
is executed.

That's a shell script created by automake which executes
a "/usr/bin/python2.7 -c 'a whole bunch of code including import os'",
which is what's causing the build to fail due to the current working
directory having a types.py file shadowing python's stdlib types module.

Conclusion: this is neither a pygobject, pygobject-3.8.3.ebuild nor portage
but an automake bug (in py-compile). We can work around it though by getting
rid of py-compile with this simple (but evil) patch:

--- /usr/portage/dev-python/pygobject/pygobject-3.8.3.ebuild	2013-10-03 07:01:15.000000000 +0200
+++ pygobject-3.8.3.ebuild	2013-10-08 13:07:20.629800637 +0200
@@ -52,6 +52,10 @@ RDEPEND="${COMMON_DEPEND}
 	!<dev-python/pygtk-2.13
 	!<dev-python/pygobject-2.28.6-r50:2[introspection]"
 
+evil_py_compile_patcher() {
+        echo "#!/bin/bash\n/bin/true" > py-compile
+}
+
 src_prepare() {
 	DOCS="AUTHORS ChangeLog* NEWS README"
 
@@ -76,6 +80,7 @@ src_configure() {
 }
 
 src_compile() {
+	python_foreach_impl run_in_build_dir evil_py_compile_patcher
 	python_foreach_impl run_in_build_dir gnome2_src_compile
 }
Comment 5 Pacho Ramos gentoo-dev 2013-10-08 18:54:00 UTC
Please provide "emerge --info", I think latest automake versions fix a bug with py-compile
Comment 6 Dieter Verfaillie 2013-10-09 07:00:02 UTC
(In reply to Pacho Ramos from comment #5)
> Please provide "emerge --info", I think latest automake versions fix a bug
> with py-compile

Sadly a 3-way diff suggests there's very little differences regarding
py-compile between automake 1.11.6, 1.12.6 and 1.14 and no single
difference related to "python -c" usage without guarding against
shadowed imports...

Portage 2.2.7 (default/linux/amd64/13.0/desktop/gnome, gcc-4.8.1, glibc-2.17, 3.11.4-gentoo x86_64)
=================================================================
System uname: Linux-3.11.4-gentoo-x86_64-Intel-R-_Core-TM-_i7-2600_CPU_@_3.40GHz-with-gentoo-2.2
KiB Mem:    12241712 total,  10368848 free
KiB Swap:    1749996 total,   1749996 free
Timestamp of tree: Wed, 09 Oct 2013 00:45:01 +0000
ld GNU ld (GNU Binutils) 2.23.2
app-shells/bash:          4.2_p45
dev-java/java-config:     2.2.0
dev-lang/python:          2.6.8-r3, 2.7.5-r2, 3.2.5-r2, 3.3.2-r2
dev-util/cmake:           2.8.11.2
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.1
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.11.6, 1.12.6, 1.14
sys-devel/binutils:       2.23.2
sys-devel/gcc:            4.7.3-r1, 4.8.1-r1
sys-devel/gcc-config:     1.8
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.11 (virtual/os-headers)
sys-libs/glibc:           2.17
Repositories: gentoo bugfix_overlay
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA AdobeFlash-10.3 dlj-1.1 Oracle-BCLA-JavaSE PUEL"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=native -pipe -O1 -fno-inline -fno-omit-frame-pointer -g3 -Wall"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=native -pipe -O1 -fno-inline -fno-omit-frame-pointer -g3 -Wall"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--complete-graph y --with-bdeps y"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news nostrip parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.utf8"
LC_ALL="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/home/dieterv/dev/gentoo.org/bugfix-overlay"
USE="X a52 aac acl acpi alsa amd64 audit autoipd avahi berkdb bluetooth bzip2 cairo caps cdda cdr cjk cli colord cracklib crypt cups curl cxx dbus dconf dga djvu doctool dri dts dvd dvdr eds emboss encode epydoc equilizer esd evo examples exif fam ffmpeg firefox flac fontconfig fortran gallium gcj gd gdbm gdm gedit gif gmp gnome gnome-keyring gnome-online-accounts gpm graphviz gstreamer gtk gtk2 gtk3 guile highlight http iconv icu idn inotify introspection iodbc ipv6 ithreads jadetex java jpeg jpeg2k lapack lcms ldap libffi libkms libnotify libsamplerate libsecret lzma lzo mad mmx mng modules mp3 mp4 mpeg mssql mtp mudflap multilib natspec nautilus ncurses networking networkmanager nls nptl nsplugin odbc ogg opengl openmp pam pango pcre pdf perl png policykit ppds pulseaudio python qt3support rdp readline samba sdl secure-delete session slp socialweb spell sqlite sse sse2 sse3 ssl startup-notification svg systemd tcpd telepathy theora threads tiff tk truetype udev udisks unicode upcall upower usb uuid vala vorbis winbind wxwidgets x264 xattr xcb xml xv xvfb xvid zlib" ABI_X86="64" ALSA_CARDS="hda-intel" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="gensplash" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en nl fr" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_6 python2_7 python3_2 python3_3" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="nouveau" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
USE_PYTHON="2.6 2.7 3.2 3.3"
Unset:  CPPFLAGS, CTARGET, INSTALL_MASK, MAKEOPTS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 7 Pacho Ramos gentoo-dev 2013-10-09 16:54:44 UTC
I was talking about:
https://bugs.gentoo.org/show_bug.cgi?id=448896#c9

But this looks like a different problem, not sure what would be the best way to disable py-compile then :/
Comment 8 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-10-09 17:27:22 UTC
Pacho, can you reproduce this? Because it builds fine for me with the same flags, portage, Python versions...
Comment 9 Pacho Ramos gentoo-dev 2013-10-09 17:40:19 UTC
Me not, I always run testing portage, but on a mostly stable system (apart of gnome/systemd...) :/
Comment 10 David Mohr 2013-11-19 16:29:57 UTC
I have the same problem with pygobject-3.2.2-r1 (and portage-2.2.7).

On another machine pygobject-3.2.2-r1 installed just fine on the 7th (at that point portage-2.2.7 was already installed), but now it fails to recompile as well.

The error is exactly the same:
---
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site.py", line 62, in <module>
    libtool: install: warning: relinking `_gi.la'
import os
  File "/usr/lib64/python2.7/os.py", line 49, in <module>
    import posixpath as path
  File "/usr/lib64/python2.7/posixpath.py", line 17, in <module>
    import warnings
  File "/usr/lib64/python2.7/warnings.py", line 8, in <module>
    import types
  File "types.py", line 26, in <module>
    from . import _gobject
ValueError: Attempted relative import in non-package
make[3]: *** [install-pygiPYTHON] Error 1
---
Comment 11 David Mohr 2013-11-19 17:52:39 UTC
I can't figure out what changed from the last time it was compiled: I checked the dependencies, automake, autoconf and everything is older than pygobject. I even reverted the eclass that changed since then, but compilation still fails...
Comment 12 Greg Trigg 2014-01-07 12:27:16 UTC
The evil patch worked for me.  Any way we can get it added to an "-r1" ebuild?
Comment 13 Guenther Brunthaler 2014-02-05 07:19:13 UTC
Same problem here (amd64, too).

And the "evil patch" also solved the problem for me. Thanks, man!
Comment 14 Frederico 2014-03-06 14:44:38 UTC
The same here with pygobject-3.2.2-r1 and portage-2.2.7.
Comment 15 David Mohr 2014-06-12 17:28:06 UTC
Still fails for me with pygobject-3.10.2 with the following deps:

app-shells/bash:          4.2_p45
dev-java/java-config:     2.2.0
dev-lang/python:          2.7.6, 3.3.3
dev-util/ccache:          3.1.9-r3
dev-util/cmake:           2.8.12.2-r1
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.4
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.13.4
sys-devel/binutils:       2.23.2
sys-devel/gcc:            4.7.3-r1
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.13 (virtual/os-headers)
sys-libs/glibc:           2.17

Updated evil patch:
# diff -u /usr/portage/dev-python/pygobject/pygobject-3.10.2.ebuild pygobject-3.10.2-r1.ebuild
--- /usr/portage/dev-python/pygobject/pygobject-3.10.2.ebuild   2014-04-21 04:31:14.000000000 -0600
+++ pygobject-3.10.2-r1.ebuild  2014-06-12 11:10:19.447440456 -0600
@@ -54,12 +54,17 @@
        !<dev-python/pygtk-2.13
        !<dev-python/pygobject-2.28.6-r50:2[introspection]"
 
+evil_py_compile_patcher() {
+        echo -e "#!/bin/bash\n/bin/true" > py-compile
+}
+
 src_prepare() {
        gnome2_src_prepare
        python_copy_sources
 }
 
 src_configure() {
+    python_foreach_impl run_in_build_dir evil_py_compile_patcher
        # Hard-enable libffi support since both gobject-introspection and
        # glib-2.29.x rdepend on it anyway
        # docs disabled by upstream default since they are very out of date