I've been working to get portage ready for python-2.3 (long overdue now), but I've run into a problem with portage.py which used to depend on an incorrect behaviour of os.environ.copy() that has been fixed now in python2.3. Basically, in .copy() would not actually copy the contents by just point to the same structure. Hence, any additions in settings via __setitem__ would directly affect os.environ. This presents problems when portage.py runs getstatusoutput("/usr/sbin/ebuild.sh depend") in doebuild(), because it expects EBUILD and other environment variables to be set. Attached is an example fix that emulates the old behaviour in class config. However there are other solutions such as replacing getstatusoutput() with an exec* function that can grab the environment from settings.environ(). Also attached is a new portage ebuild that dynamically checks for the operating python2 version and another patch that replaces the occurances of "/usr/bin/env python2.2" with "/usr/bin/env python2" so portage works for both 2.2 and 2.3. Reproducible: Always Steps to Reproduce:
Created attachment 17417 [details] portage-2.0.49-r50.ebuild this is an example portage ebuild that detects the version of python installed and installed *.py accordingly.
Created attachment 17418 [details, diff] portage-2.0.49-python2.3.patch changes all headers to #!/usr/bin/env python2
Created attachment 17419 [details, diff] portage-2.0.49-r3-portage.py2.3.patch example fix to portage.py's behaviour regarding the os.environ and getstatusoutput()
*** Bug 28327 has been marked as a duplicate of this bug. ***
http://bugs.gentoo.org/attachment.cgi?id=17391&action=view this patch is probably much cleaner than mine as it preserves both the old python2.2 behaviour in python2.3. can someone please that a look at both the patches to invoking the python2.2 intepreter, the portage.py patch and the revamped portage ebuild? this is blocking python2.3 from being introduced into portage. (on related news, python-2.3.1 is going to be coming out next week ..)
I've been running python 2.3 for the last week or so, doing several emerges of all my installed software. Portage is actually working quite alright with the patches applied, but I see a few different problems that needs solving. 1. The upgrade path isn't exactly smooth. There are two different upgrade paths as of now. 2a. Upgrade portage to a python-2.* version before upgrade python to 2.3. This breaks portage after upgrading python as portage will look for it's modules in /usr/lib/python-2.3/site-packages which is empty. 2b. upgrade to python-2.3, then upgrade portage. This should actually work as the 'old' portage would still be using python-2.2.x. This however requires that python-2.3 is slotted and that python-2.2 isn't removed. 2. Several ebuilds depends on python-2.2*. If you have upgraded to python-2.3, upgraded portage, then unmerged python-2.2 this will break portage, as emerging python-2.2 changes the /usr/bin/python symlink. This presents two different problems: 2a. We need to have a closer look at these packages and decide wether the dependencies should allow python-2.3. 2b. emerging python changes the /usr/bin/python link thereby changing the site-packages location. We need to handle this situation better. 3. We need to figure out how to handle changing python versions without breaking random python modules.
Created attachment 18138 [details] ebuilds that refer to python2.2
Comment on attachment 18138 [details] ebuilds that refer to python2.2 List of ebuilds somehow mentioning python2.2. We need to take a closer look at those.
yes, i agree with you. i have a python-rebuilding script that basically goes thru your installed packages and re-merges all the pacakges that have things in /usr/lib/python2.2/site-packages that is probably best discussed in bug #25591 rather than this bug. thanks for the list for packages with the 2.2 requirement. this bug is only to do with getting portage to not peg on python2.2, so we can move stages to python 2.3 only.
Created attachment 18635 [details, diff] portage-2.0.49-py2.3_headers.patch this is patch changes all #!/usr/bin/env python2.2 to #!/usr/bin/env python (against -r7)
Created attachment 18636 [details, diff] portage-2.0.49-py2.3_os.environ.patch this is the same patch by Carlos Castillo in bug #28327
Created attachment 18637 [details, diff] portage-2.0.49-py2.3_sandbox.patch adds new /usr/lib/portage/pym path to sandbox exclusions to protect against .pyc/.pyo violating sandbox.
Created attachment 18638 [details] portage-2.0.49-r50.ebuild a new revision of the portage ebuild that changes the following: 1. locks on to -r7 and applies the 3 attached patches (remove this when it goes in portage) 2. applies the above patches ins src_unpack (again remove this later) 3. reformatted RDEPEND to be more readable 4. added >=dev-lang/python-2.3.1 as a dependency because only that version has a /usr/lib/portage/pym in its default search path 5. installed all *.py into /usr/lib/portage/pym 6. moved emergehelp.py out of /usr/lib/portage/bin and into /usr/lib/portage/pym 7. put .pyo .pyc compilation command into a for loop 8. added a python_version function and clean-up function in pkg_postinst() to make sure no remaining portage.pyc/pyo will override the ones in /usr/lib/portage/pym this version requires the python-2.3.2_rc1.ebuild that is in bug #25591 that uses patch to installed /usr/lib/portage/pym. i'd suggest the patches be applied first to portage source tree first so at least portage works on python-2.3 regardless. and then work on putting in the ebuild that moves portage python modules to /usr/lib/portage/pym. i'll backport the changes to add /usr/lib/portage/pym to new revision to python-2.2.3-r2 so we don't have to depend on a python-2.3 version.
Look at portage-2.0.49-r10 It should have most everything taken care of... Sandbox stuff might be an exception, but it shouldn't be an issue anyway.
carpaski, you should probably change the newer portages to dep on dev-lang/python-2.2.3-r2, because that is the only python-2.2 that has the /usr/lib/portage/pym support
Created attachment 18950 [details, diff] portage-2.0.49-r12-pym.diff carpaski, there's a little mistake in the -r12 ebuild. it's installing the python modules in /etc instead of /usr/lib/portage/pym here's the patch to fix it. PS. sys-apps/portage doesn't have a metadata.xml ;)
another thing, i noticed that emergehelp.py is in /usr/lib/portage/bin and not in pym. i have a feeling it should be moved because /usr/lib/portage/bin isn't in the search path
sys.path += ["/usr/lib/portage/pym"] and the patch you submitted is pretty close to what I lost in an 'emerge sync' earlier today. I have emergehelp.py installing into pym now.
yep. looks good to go then. works fine on my python-2.3.2 install
after some testing, everything looks fine. there's some issues with python and sandbox, but that is being solved using the python.eclass' so it is time for closing.
*** Bug 31047 has been marked as a duplicate of this bug. ***
portage-2.0.49-r15 sill needs python2.3 fixes. replacing python2.2 with python2 in all py source file headers (# /usr/bin/python2) helped here.
