Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 436274 - www-client/firefox-15.0.1: virtual embedded python fails to handle libdirs properly (and bundles simplejson)
Summary: www-client/firefox-15.0.1: virtual embedded python fails to handle libdirs pr...
Status: RESOLVED OBSOLETE
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Mozilla Gentoo Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: x32 427048
  Show dependency tree
 
Reported: 2012-09-26 05:22 UTC by SpanKY
Modified: 2017-08-26 17:57 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
correctly support multiarch (python_virtualenv_multiarch_fixup.patch,4.85 KB, patch)
2013-01-17 15:37 UTC, Jory A. Pratt
Details | Diff
build.log (firefox-build.log,17.47 KB, text/plain)
2016-02-26 13:53 UTC, Dainius Masiliūnas
Details
01-proper-multilib-support.patch (01-proper-multilib-support.patch,10.39 KB, patch)
2016-02-26 13:58 UTC, Dainius Masiliūnas
Details | Diff
02-update-base64.patch (02-update-base64.patch,22.75 KB, patch)
2016-02-26 14:02 UTC, Dainius Masiliūnas
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description SpanKY gentoo-dev 2012-09-26 05:22:58 UTC
i have no idea what this new "virtualenv embedded" business is about, but it's doing things really horribly and really wrong

for some reason, it sets up:
./mozilla-release/obj-x86_64-unknown-linux-gnu/_virtualenv/

inside of there, it copies python from the host system (mostly) before executing it locally (for some reason).  it inserts its own copy of site.py instead of using the python site.py we specifically installed.  because this site.py lacks our fixes related to multilib, it ends up failing to locate libraries properly and the build fails with errors like:
...
" /var/tmp/portage/www...ols-0.6c11-py2.7.egg:
  Traceback (most recent call last):
  File "<string>", line 67, in <module>
ImportError: No module named md5
...
make[3]: Entering directory `/var/tmp/portage/www-client/firefox-15.0.1/work/mozilla-release/obj-x86_64-unknown-linux-gnu/build/virtualenv'
cd /var/tmp/portage/www-client/firefox-15.0.1/work/mozilla-release/other-licenses/simplejson-2.1.1/; /var/tmp/portage/www-client/firefox-15.0.1/work/mozilla-release/obj-x86_64-unknown-linux-gnu/_virtualenv/bin/python setup.py develop
Traceback (most recent call last):
  File "setup.py", line 7, in <module>
    from ez_setup import use_setuptools
  File "/var/tmp/portage/www-client/firefox-15.0.1/work/mozilla-release/other-licenses/simplejson-2.1.1/ez_setup.py", line 67, in <module>
    except ImportError: from md5 import md5
ImportError: No module named md5
make[3]: *** [default] Error 1
make[3]: Leaving directory `/var/tmp/portage/www-client/firefox-15.0.1/work/mozilla-release/obj-x86_64-unknown-linux-gnu/build/virtualenv'
...
Comment 1 Marien Zwart (RETIRED) gentoo-dev 2012-09-27 19:39:24 UTC
You'll likely find gentoo's virtualenv package has the same problem. If you install dev-python/virtualenv and run "virtualenv myenv" you'll find the same site.py that's causing you pain installed in there. It might make sense to fix up dev-python/virtualenv and then apply the same fix to firefox, but before I can try to do that I'd probably have to build myself an x32 chroot so I can test the result actually works.

The "virtualenv embedded business" is a directory with files that get embedded into virtualenv.py (the python module) "in order to support single-file use of virtualenv.py without installing it" (see the contributing section of the virtualenv website). The copying of the host python is what virtualenv is supposed to do: it attempts to build an isolated python environment by copying over the host python and instructing it to only look in its own stdlib directory and the new environment's site-packages, not the system site-packages directory.

virtualenv's site.py has diverged from CPython's pretty significantly, but the difference between CPython upstream's and Gentoo's is quite small (the only difference we actually need here is two lib -> lib64 changes).
Comment 2 SpanKY gentoo-dev 2012-09-28 21:58:55 UTC
so firefox is also duplicating the system package dev-python/virtualenv instead of depending on it ?  nice.

x32 stage3 tarballs are already readily available on the mirrors.  but really the only fix you need is to use the stuff we have in the python tarball to rewrite lib64 and friends to $(get_libdir).  nowhere should any code ever check for "lib64" -- it should check for what the build system has told it to use.
Comment 3 Jory A. Pratt gentoo-dev 2013-01-17 15:37:47 UTC
Created attachment 335918 [details, diff]
correctly support multiarch

Please test and let us know, thank you
Comment 4 SpanKY gentoo-dev 2013-01-18 02:45:41 UTC
(In reply to comment #3)

doesn't apply to 18.0.  which ver was i to test ?
Comment 5 Dainius Masiliūnas 2016-02-26 13:53:05 UTC
Created attachment 426636 [details]
build.log

This is still an issue for firefox 38 and 44. I thought it's because, as mentioned, virtualenv does not have support for multilib as it should. I thus applied the patch in https://github.com/pypa/virtualenv/pull/516 which should properly fix that, but no luck either. I get an error "ImportError: No module named __future__" with or without the patch applied.

Here's the build.log file of firefox-44.
Comment 6 Dainius Masiliūnas 2016-02-26 13:58:48 UTC
Created attachment 426638 [details, diff]
01-proper-multilib-support.patch

For reference, this is the patch I used. It's specifically merged against the version of virtualenv that firefox-44 uses (12.0.5).
Comment 7 Dainius Masiliūnas 2016-02-26 14:02:26 UTC
Created attachment 426640 [details, diff]
02-update-base64.patch

And this is the patch that updates the base64-encoded blob to the result of the previous patch. It is really inconvenient that it has to be done, and it needs to be updated for every single change in site.py, so it will no longer apply even after a minor update to the build system, without redoing it all.
Comment 8 Jory A. Pratt gentoo-dev 2017-08-26 17:57:19 UTC
If you feel I have closed your bug and it is still a current issue, please reopen and update it completely. We will not work bugs that have no ebuild in tree any longer or can not be reproduced with a current system.

Thank You for your support and understanding
The Mozilla Team