Bug 533970 – dev-python/cython-0.21 without dev-python/pip - The package setup script has attempted to modify files on your system that are not within the EasyInstall build area, and has been aborted.

• Status: RESOLVED INVALID
• Product: Gentoo Linux

[Description Chris Folsom 2014-12-29 20:49:00 UTC]

Installing dev-python/cython fails when python use flag is set. Pip is not pulled in as a dependency by portage, but when the build runs, easy_install attempts to fetch pip and install it outside the sandbox causing the build to fail.

Reproducible: Always

Steps to Reproduce:
1. euse -E python
2. emerge -C dev-python/pip
3. emerge dev-python/cython
Actual Results:  
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Compiler/__init__.py to __init__.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Compiler/Main.py to Main.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Compiler/UtilityCode.py to UtilityCode.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Compiler/CmdLine.py to CmdLine.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Compiler/Builtin.py to Builtin.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Compiler/ParseTreeTransforms.py to ParseTreeTransforms.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Compiler/AnalysedTreeTransforms.py to AnalysedTreeTransforms.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Compiler/Naming.py to Naming.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Compiler/PyrexTypes.py to PyrexTypes.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Compiler/Future.py to Future.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Compiler/UtilNodes.py to UtilNodes.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Compiler/TreePath.py to TreePath.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Compiler/Lexicon.py to Lexicon.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Compiler/AutoDocTransforms.py to AutoDocTransforms.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/TestUtils.py to TestUtils.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Build/Dependencies.py to Dependencies.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Build/Inline.py to Inline.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Build/Cythonize.py to Cythonize.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Build/Tests/TestIpythonMagic.py to TestIpythonMagic.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Build/Tests/__init__.py to __init__.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Build/Tests/TestInline.py to TestInline.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Build/Tests/TestStripLiterals.py to TestStripLiterals.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Build/__init__.py to __init__.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Build/IpythonMagic.py to IpythonMagic.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Build/BuildExecutable.py to BuildExecutable.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/__init__.py to __init__.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/StringIOTree.py to StringIOTree.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Debugger/libcython.py to libcython.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Debugger/libpython.py to libpython.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Debugger/Tests/TestLibCython.py to TestLibCython.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Debugger/Tests/test_libpython_in_gdb.py to test_libpython_in_gdb.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Debugger/Tests/test_libcython_in_gdb.py to test_libcython_in_gdb.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Debugger/Tests/__init__.py to __init__.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Debugger/__init__.py to __init__.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Debugger/DebugWriter.py to DebugWriter.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Debugger/Cygdb.py to Cygdb.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Tempita/_looper.py to _looper.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Tempita/_tempita.py to _tempita.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Tempita/__init__.py to __init__.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Tempita/compat3.py to compat3.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/CodeWriter.py to CodeWriter.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Utility/__init__.py to __init__.pyc
byte-compiling /var/tmp/portage/dev-python/cython-0.21/image//_python2.7/usr/lib64/python2.7/site-packages/Cython/Utils.py to Utils.pyc
writing byte-compilation script '/var/tmp/portage/dev-python/cython-0.21/temp/tmpH_zyed.py'
/usr/bin/python2.7 -OO /var/tmp/portage/dev-python/cython-0.21/temp/tmpH_zyed.py
removing /var/tmp/portage/dev-python/cython-0.21/temp/tmpH_zyed.py
running install_egg_info
running egg_info
creating /var/tmp/portage/dev-python/cython-0.21/work/Cython-0.21-python2_7/Cython.egg-info
Searching for pip
Reading https://pypi.python.org/simple/pip/
Best match: pip 6.0.3
Downloading https://pypi.python.org/packages/source/p/pip/pip-6.0.3.tar.gz#md5=1ca6788e57a176abbdf6d99d69f54ae0
Processing pip-6.0.3.tar.gz
Writing /var/tmp/portage/dev-python/cython-0.21/temp/easy_install-QJ2Wn3/pip-6.0.3/setup.cfg
Running pip-6.0.3/setup.py -q bdist_egg --dist-dir /var/tmp/portage/dev-python/cython-0.21/temp/easy_install-QJ2Wn3/pip-6.0.3/egg-dist-tmp-eHIgTa
error: Setup script exited with error: SandboxViolation: mkdir('/var/tmp/portage/dev-python/cython-0.21/work/Cython-0.21-python2_7/pip.egg-info', 511) {}

The package setup script has attempted to modify files on your system
that are not within the EasyInstall build area, and has been aborted.

This package cannot be safely installed by EasyInstall, and may not
support alternate installation locations even if you run its setup
script by hand.  Please inform the package's author and the EasyInstall
maintainers to find out if a fix or workaround is available.
 [31;01m*[0m ERROR: dev-python/cython-0.21::gentoo failed (install phase):
 [31;01m*[0m   (no error message)
 [31;01m*[0m 
 [31;01m*[0m Call stack:
 [31;01m*[0m     ebuild.sh, line   93:  Called src_install
 [31;01m*[0m   environment, line 4092:  Called distutils-r1_src_install
 [31;01m*[0m   environment, line 1323:  Called _distutils-r1_run_foreach_impl 'distutils-r1_python_install'
 [31;01m*[0m   environment, line  288:  Called python_foreach_impl 'distutils-r1_run_phase' 'distutils-r1_python_install'
 [31;01m*[0m   environment, line 3582:  Called multibuild_foreach_variant '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'distutils-r1_python_install'
 [31;01m*[0m   environment, line 2699:  Called _multibuild_run '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'distutils-r1_python_install'
 [31;01m*[0m   environment, line 2697:  Called _python_multibuild_wrapper 'distutils-r1_run_phase' 'distutils-r1_python_install'
 [31;01m*[0m   environment, line  791:  Called distutils-r1_run_phase 'distutils-r1_python_install'
 [31;01m*[0m   environment, line 1292:  Called distutils-r1_python_install
 [31;01m*[0m   environment, line 1219:  Called esetup.py 'install' '--root=/var/tmp/portage/dev-python/cython-0.21/image//_python2.7'
 [31;01m*[0m   environment, line 1773:  Called die
 [31;01m*[0m The specific snippet of code:
 [31;01m*[0m       "${@}" || die
 [31;01m*[0m 
 [31;01m*[0m If you need support, post the output of `emerge --info '=dev-python/cython-0.21::gentoo'`,
 [31;01m*[0m the complete build log and the output of `emerge -pqv '=dev-python/cython-0.21::gentoo'`.
 [31;01m*[0m The complete build log is located at '/var/tmp/portage/dev-python/cython-0.21/temp/build.log'.
 [31;01m*[0m The ebuild environment file is located at '/var/tmp/portage/dev-python/cython-0.21/temp/environment'.
 [31;01m*[0m Working directory: '/var/tmp/portage/dev-python/cython-0.21/work/Cython-0.21'
 [31;01m*[0m S: '/var/tmp/portage/dev-python/cython-0.21/work/Cython-0.21'
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
job-working-directory: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory
chdir: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory

Expected Results:  
PIP is pulled in as part of the dependency tree when python use flag is set or installation proceeds using setuptools without installing pip.

Work around: Manually install pip.

[Comment 1 Mike Gilbert 2014-12-29 23:51:11 UTC]

(In reply to Chris Folsom from comment #0)
> Installing dev-python/cython fails when python use flag is set.

The 'python' use flag has no effect on the cython ebuild.

> Pip is not pulled in as a dependency by portage, but when the build runs, easy_install attempts to fetch pip and install it outside the sandbox causing the build to fail.

I cannot reproduce this.

[Comment 2 Ian Delaney (RETIRED) 2014-12-30 00:22:17 UTC]

This looks a red herring.  pip ought have no part in this anywhere, however, the reality is that most upstream write setup.py with pip set to install anything not found in the system as par for the course.  Having pip itself not installed is even more bizarre.  floppym's comment and his 'cannot replicate' is more than enough to indicate this is no bug

[Comment 3 Chris Folsom 2014-12-30 02:53:21 UTC]

    @Mike

    > The 'python' use flag has no effect on the cython ebuild.

    That was my fault. Try setting python use flag for sys-apps/kmod to replicate. This should cause cython to get rebuilt and reproduce the error.

    >> the reality is that most upstream write setup.py with pip set to install anything not found in the system as par for the course.  

    That seems to be a fairly recent trend. My understanding was that in the past most people used setuptools / easy_install. 

    > Having pip itself not installed is even more bizarre.

    Why? It's perfectly legitimate to have python installed without pip or virtualenv. I've been running Gentoo for over ten years and never once has any machine had pip installed with any version of python.

    emerge -s dev-python/pip
    *  dev-python/pip
          Latest version available: 1.5.6
          Latest version installed: [ Not Installed ]

    >> This looks a red herring. 

    Mmm...If I were going to choose two words to describe it, they'd be "global disaster." Python versionitis and linking issues are epidemic. This isn't just an issue for this package. It's a problem anywhere a library gets rebuilt on Gentoo for python. See: bug #533972, bug #533968
      

You have the following choices:

[Comment 4 Mike Gilbert 2014-12-30 02:58:41 UTC]

We really need more precise information to get to the bottom of this. As I said, I cannot reproduce the problem you are having.

It could be related to the versions of various packages you have installed, or that you have installed something python-related outside of portage.

[Comment 5 Mike Gilbert 2014-12-30 03:03:45 UTC]

(In reply to Ian Delaney from comment #2)

I don't mean to be rude, but your comment is just confusing.

It is perfectly normal for a Gentoo system to not have pip installed; I'm not sure where that idea came from.

[Comment 6 Ian Delaney (RETIRED) 2014-12-30 03:17:24 UTC]

I thought the main part of my comment was to back up yours and now you make out mine to be confusing.
<It is perfectly normal for a Gentoo system to not have pip installed>
is the very point I make or emphasise. I'm not sure it's worth pursuing this aside any further though, so I won't.
Carry on

[Comment 7 Chris Folsom 2014-12-30 03:23:54 UTC]

> As I said, I cannot reproduce the problem you are having.

Did you try any of the other packages listed with the other defects? All I do is enable python USE flag and emerge -DNu world to rebuild and multiple packages start breaking. Everything works fine if I manually install pip or disable the python USE flag.

> It could be related to the versions of various packages you have installed,
> or that you have installed something python-related outside of portage.

It might be the former. I dont think it's the later since haven't installed any libraries (python or otherwise) on this system without using portage. 

How can I help? I'm not really super-familiar with setup.py, but I'm trying to figure it out.

[Comment 8 Mike Gilbert 2014-12-30 03:34:15 UTC]

(In reply to Chris Folsom from comment #7)
> > As I said, I cannot reproduce the problem you are having.
> 
> Did you try any of the other packages listed with the other defects?

I cannot reproduce the issue with cracklib or protobuf.

> All I do is enable python USE flag and emerge -DNu world to rebuild and multiple packages start breaking. Everything works fine if I manually install pip or disable the python USE flag.

This basically tells me that you have very few packages with python support enabled. It is possible that something has been broken with your python installation for a while and you never noticed.

Just making some guesses, I would try re-installing the following:

dev-lang/python:2.7
dev-lang/python:3.3
dev-python/setuptools

> > It could be related to the versions of various packages you have installed,
> > or that you have installed something python-related outside of portage.
> 
> It might be the former. I dont think it's the later since haven't installed
> any libraries (python or otherwise) on this system without using portage. 
> 
> How can I help? I'm not really super-familiar with setup.py, but I'm trying
> to figure it out.

It's a bet of a chicken-and-egg problem: I don't understand the problem well enough to know what to ask you for.

[Comment 9 Mike Gilbert 2014-12-30 03:42:14 UTC]

(In reply to Mike Gilbert from comment #8)
> Just making some guesses, I would try re-installing the following:
> 
> dev-lang/python:2.7
> dev-lang/python:3.3
> dev-python/setuptools

Also, reinstall dev-lang/python-exec; sometimes /usr/lib/python-exec/python-exec2 gets overwritten, which causes strange behavior.

[Comment 10 Chris Folsom 2014-12-30 03:57:13 UTC]

(In reply to Mike Gilbert from comment #9)

Tried the following:

emerge --oneshot dev-lang/python:2.7
emerge --oneshot dev-lang/python:3.3
emerge --oneshot dev-lang/python:3.4
emerge --oneshot dev-lang/python-exec
emerge --oneshot setuptools


Setup tools fails for the same reason. It tries to download and install pip outside sandbox.

[Comment 11 Mike Gilbert 2014-12-30 04:03:25 UTC]

(In reply to Chris Folsom from comment #10)

As an experiment, could you try renaming /usr/lib64/python2.7/site-packages, and then try installing dev-python/setuptools?

That would essentially be starting from a blank slate.

This may break the emerge command if python2.7 is your main python interpreter; in that case, just invoke it by running python3.3 /usr/bin/emerge ...

[Comment 12 Mike Gilbert 2014-12-30 04:08:30 UTC]

Another idea: enable FEATURES="network-sandbox" and we might get a better stack trace when it attempts to download pip.

[Comment 13 Chris Folsom 2014-12-30 04:13:31 UTC]

(In reply to Mike Gilbert from comment #11)

> As an experiment, could you try renaming /usr/lib64/python2.7/site-packages,
> and then try installing dev-python/setuptools?

Awesome! Looks like it worked. What's the easiest way to get all of my python2.7 libs back? setuptools installed, but emerge on 2.7 is still broken.

[Comment 14 Mike Gilbert 2014-12-30 04:18:25 UTC]

(In reply to Chris Folsom from comment #13)
> Awesome! Looks like it worked. What's the easiest way to get all of my
> python2.7 libs back? setuptools installed, but emerge on 2.7 is still broken.

emerge --oneshot /usr/lib64/python2.7/site-packages will reinstall anything that had previously installed files there.

However, if you want to get to the bottom of this problem, move your old site-packages directory back into place, and run the following:

find /usr/lib64/python2.7/site-packages -exec qfile -o '{}' +

This will give you a list of "orphaned" files -- stuff that is not owned by any package that portage installed.

The answer to this mystery likely lies in one of these orphans.

[Comment 15 Chris Folsom 2014-12-30 05:08:42 UTC]

(In reply to Mike Gilbert from comment #14)

Fixed. It looks like virtualenv and stevedore got into the python2.7 installation somehow.