Bug 941111 – dev-python/pypy3_10-exe: add USE=pgo

• Status: CONFIRMED
• Product: Gentoo Linux

[Description Sam James 2024-10-07 10:53:42 UTC]

pypy3, like cpython, supports PGO. See https://pythonfiles.wordpress.com/2017/05/12/enabling-profile-guided-optimizations-for-pypy/.

I'm testing this trivial change to see if it works:
```
--- a/dev-python/pypy3_10-exe/pypy3_10-exe-7.3.17.ebuild
+++ b/dev-python/pypy3_10-exe/pypy3_10-exe-7.3.17.ebuild
@@ -100,6 +100,7 @@ src_configure() {

        local args=(
                --no-shared
+               --profopt
                $(usex jit -Ojit -O2)

                ${jit_backend}
```

[Comment 1 Sam James 2024-10-07 10:56:26 UTC]

https://doc.pypy.org/en/latest/config/translation.profopt.html

It defaults to using the testsuite: https://github.com/pypy/pypy/blob/ad651f6f674e5df08eabb7289fec4df5b4259224/pypy/goal/targetpypystandalone.py#L275

Note that it doesn't have --pgo-extended (py2's regrtest.py lacks it) which we use in dev-lang/python.

[Comment 2 Sam James 2024-10-07 11:05:24 UTC]

My first attempt didn't seem to do anything, I don't see -fprofile* appear in the build log at all.

[Comment 3 Sam James 2024-10-07 11:18:31 UTC]

OK, if I change src_compile to run the 'profopt' target, it seems to "work", but it doesn't actually generate much data because:

/var/tmp/portage/dev-python/pypy3_10-exe-7.3.17/temp/usession-master-0/testing_1/pypy3.10-c "/var/tmp/portage/dev-python/pypy3_10-exe-7.3.17/work/pypy3.10-v7.3.17-src/rpython"/../lib-pyt
hon/2.7/test/regrtest.py --pgo -x test_asyncore test_gdb test_multiprocessing test_subprocess || true
/var/tmp/portage/dev-python/pypy3_10-exe-7.3.17/temp/usession-master-0/testing_1/pypy3.10-c: can't open file /var/tmp/portage/dev-python/pypy3_10-exe-7.3.17/work/pypy3.10-v7.3.17-src/lib
-python/2.7/test/regrtest.py: [Errno 2] No such file or directory

(It ignores this and carries on, but it means we get no benefit.)

[Comment 4 Sam James 2024-10-07 11:21:30 UTC]

I guess we want to use /var/tmp/portage/dev-python/pypy3_10-exe-7.3.17/work/pypy3.10-v7.3.17-src/lib-python/3/test/regrtest.py instead.

Presumably we can use --pgo-extended then too.

[Comment 5 Michał Górny 2024-10-07 14:44:05 UTC]

I wonder if that's actually supported at all these days.  Remember that PyPy runs the test suite via pytest, and that we restrict it because they never really got it to pass (or skip the tons of failing tests).

[Comment 6 Sam James 2024-10-07 15:14:50 UTC]

Ah, I didn't realise that :(

FWIW, anything would work as long as it's representative or representative enough.