Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 575840 - sys-apps/portage: Add PyPy3 support
Summary: sys-apps/portage: Add PyPy3 support
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All All
: Normal enhancement (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-02-27 18:15 UTC by Petros
Modified: 2020-01-04 08:24 UTC (History)
9 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Petros 2016-02-27 18:15:55 UTC
[ebuild   R    ] sys-apps/portage-2.2.27::gentoo  USE="doc (ipc) -build -epydoc (-selinux) -xattr" LINGUAS="-ru" PYTHON_TARGETS="pypy python2_7 python3_3 python3_4 python3_5" 0 KiB


Could a python_targets_pypy3 use flag be added? Would portage work with PyPy3 ( dev-python/pypy3)?
Comment 1 Brian Dolbec (RETIRED) gentoo-dev 2016-03-02 14:32:48 UTC
As I don't have pypy3 installed, I don't know.

Also I don't know if we can add a pypy3 target to Travis testing.

You can just make a copy of the ebuild in a local overlay and add that target, re-install, keeping at least one of the other supported mainstream pythons as backup and test.  

But, you may have to add a pypy3 target to the testsuite script to do manual test suite runs on a local git checkout of the source.

If you load the runtests script in an editor you can add the pypyp3 target to:

# The rest are just "nice to have".
PYTHON_NICE_VERSIONS = [
	'pypy',
	'3.5',
]


and run it.

hmm time to move 3.5 up to PYTHON_SUPPORTED_VERSIONS.

If you do run the testsuite with pypy3 please report your results good or bad please.
Comment 2 Zac Medico gentoo-dev 2016-03-02 16:54:51 UTC
(In reply to Brian Dolbec from comment #1)
> But, you may have to add a pypy3 target to the testsuite script to do manual
> test suite runs on a local git checkout of the source.

Added minimal support:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=6193b030737d23148e5b131ee771ddbb52a895cb

Try it like this:

./runtests --python-versions pypy3

> If you do run the testsuite with pypy3 please report your results good or
> bad please.

I ran the tests with dev-python/pypy3-2.4.0-r1 and there were lots of failures. There's no point in adding the pypy3 target to the ebuild yet.
Comment 3 Zac Medico gentoo-dev 2017-01-28 22:38:02 UTC
It passes test locally with dev-python/pypy3-bin-5.5.0_alpha.

I've sent this pull request to enable pypy3.3-5.2-alpha1 in travis:

https://github.com/gentoo/portage/pull/104

However, in travis it currently fails to build lxml 3.7.2 as follows:

    building 'lxml.etree' extension
    creating build/temp.linux-x86_64-3.3
    creating build/temp.linux-x86_64-3.3/src
    creating build/temp.linux-x86_64-3.3/src/lxml
    cc -O2 -fPIC -Wimplicit -I/usr/include/libxml2 -Isrc/lxml/includes -I/home/travis/virtualenv/pypy3.3-5.2-alpha1/include -c src/lxml/lxml.etree.c -o build/temp.linux-x86_64-3.3/src/lxml/lxml.etree.o -w
    src/lxml/lxml.etree.c: In function ‘__Pyx_PyGen_FetchStopIterationValue’:
    src/lxml/lxml.etree.c:235851:23: error: ‘PyStopIterationObject’ undeclared (first use in this function)
    src/lxml/lxml.etree.c:235851:23: note: each undeclared identifier is reported only once for each function it appears in
    src/lxml/lxml.etree.c:235851:46: error: expected expression before ‘)’ token
    src/lxml/lxml.etree.c:235891:38: error: expected expression before ‘)’ token
    Compile failed: command 'cc' failed with exit status 1
    cc -O2 -fPIC -Wimplicit -I/usr/include/libxml2 -I/usr/include/libxml2 -c /tmp/xmlXPathInitdi_0my.c -o tmp/xmlXPathInitdi_0my.o
    /tmp/xmlXPathInitdi_0my.c:2:1: warning: return type defaults to ‘int’ [-Wimplicit-int]
    cc tmp/xmlXPathInitdi_0my.o -L/usr/lib/x86_64-linux-gnu -lxml2 -o a.out
    error: command 'cc' failed with exit status 1
    
    ----------------------------------------
Command "/home/travis/virtualenv/pypy3.3-5.2-alpha1/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-u9k6_x/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-srxg0v-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/travis/virtualenv/pypy3.3-5.2-alpha1/include/site/python3.3/lxml" failed with error code 1 in /tmp/pip-build-u9k6_x/lxml/


The command "pip install lxml" failed and exited with 1 during .
Comment 4 Zac Medico gentoo-dev 2017-01-28 22:51:59 UTC
This lxml issue for pypy3 has been open for more than 2 years:

https://bugs.launchpad.net/lxml/+bug/1377501

This more recently opened issue has been marked as "won't fix" because it's up to PyPy to come up with a working pypy3 release:

https://bugs.launchpad.net/lxml/+bug/1442427

We only use lxml in repoman's PkgMetadata module, since we migrated from the python's xml.etree module in this commit:

https://gitweb.gentoo.org/proj/portage.git/commit/?id=633cabfb9215633c554fd967b9875310be3718bd
Comment 5 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-01-29 08:06:01 UTC
I think I've removed pypy3 from Portage because of some missing dep that was missing PYTHON_USEDEP previously.
Comment 6 Zac Medico gentoo-dev 2017-01-29 18:31:17 UTC
dev-python/pyxattr is missing pypy3 in PYTHON_COMPAT. CPython 3.4 added xattr functions in the os module, but pypy3.3-5.2-alpha1 doesn't have have them because it's based on CPython 3.3.5.
Comment 7 Arfrever Frehtes Taifersar Arahesis 2017-06-09 09:03:06 UTC
(In reply to Zac Medico from comment #6)
> CPython 3.4 added xattr functions in the os module

Actually these functions in CPython are since 3.3.

(In reply to Zac Medico from comment #6)
> but pypy3.3-5.2-alpha1 doesn't have have them because
> it's based on CPython 3.3.5.

Newer versions of PyPy3 have getxattr(), setxattr(), removexattr() and listxattr() functions which do nothing and accept no arguments:
https://bitbucket.org/pypy/pypy/commits/664a8d8d8eb4328036d5a26caea80b6d305e0f76
https://bitbucket.org/pypy/pypy/src/a37ecfe5f142bc971a86d17305cc5d1d70abec64/pypy/module/posix/interp_posix.py?at=release-pypy3.5-v5.8.0&fileviewer=file-view-default#interp_posix.py-2272
Comment 8 Manuel Rüger (RETIRED) gentoo-dev 2017-07-04 22:19:02 UTC
I've added pypy3 to travis in this PR allowing it to fail. 

https://github.com/gentoo/portage/pull/177
Comment 9 Petross404(Petros S) 2018-01-14 14:53:03 UTC
Any news on this?
Comment 10 Zac Medico gentoo-dev 2018-01-15 03:22:10 UTC
All of portage's unit tests now pass with dev-python/pypy3-bin-5.10.0, and os.setxattr appears to be working.

So, it looks like the only remaining blocker is lxml, as discussed in comment #4.
Comment 11 Arfrever Frehtes Taifersar Arahesis 2018-01-15 07:19:25 UTC
(In reply to Arfrever Frehtes Taifersar Arahesis from comment #7)

Real implementations of xattr functions are available in PyPy3 >=5.10.0.
https://bitbucket.org/pypy/pypy/pull-requests/586
Comment 12 Reva Denis 2020-01-04 07:22:00 UTC
What about now?
Comment 13 Larry the Git Cow gentoo-dev 2020-01-04 08:16:22 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=15e94fc75e4dc5251d38a8abd6c64b2e046a8766

commit 15e94fc75e4dc5251d38a8abd6c64b2e046a8766
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2020-01-04 07:44:05 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2020-01-04 08:14:58 +0000

    travis.yml: enable pypy3
    
    Bug: https://bugs.gentoo.org/575840
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 .travis.yml | 1 +
 1 file changed, 1 insertion(+)
Comment 14 Larry the Git Cow gentoo-dev 2020-01-04 08:22:39 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=6dee3b6f12cb8bbc93dc93f24facdeaa0eb41268

commit 6dee3b6f12cb8bbc93dc93f24facdeaa0eb41268
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2020-01-04 08:19:18 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2020-01-04 08:22:32 +0000

    sys-apps/portage: PYTHON_COMPAT+=(pypy3)
    
    Closes: https://bugs.gentoo.org/575840
    Package-Manager: Portage-2.3.84, Repoman-2.3.20
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 sys-apps/portage/portage-2.3.84-r1.ebuild | 4 ++--
 sys-apps/portage/portage-9999.ebuild      | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
Comment 15 Zac Medico gentoo-dev 2020-01-04 08:24:40 UTC
(In reply to Reva Denis from comment #12)
> What about now?

Yeah that tests pass now:

https://travis-ci.org/gentoo/portage/builds/632575638

Thanks for the reminder!