Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 619648 - dev-python/pypy3-5.7.1-r2[libressl] and dev-python/pypy3-5.8.0[libressl] expected identifier or ‘(’ before numeric constant
Summary: dev-python/pypy3-5.7.1-r2[libressl] and dev-python/pypy3-5.8.0[libressl] expe...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Python Gentoo Team
URL: https://bitbucket.org/pypy/pypy/issue...
Whiteboard:
Keywords: NeedPatch, UPSTREAM
Depends on:
Blocks: libressl-support
  Show dependency tree
 
Reported: 2017-05-25 06:54 UTC by Alessandro Barbieri
Modified: 2017-10-22 11:13 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
info (info,14.22 KB, text/plain)
2017-05-25 11:36 UTC, Alessandro Barbieri
Details
dev-python:pypy3-5.7.1-r2:20170525-215219.log.xz (dev-python:pypy3-5.7.1-r2:20170525-215219.log.xz,36.81 KB, application/x-xz)
2017-05-26 02:51 UTC, Alessandro Barbieri
Details
pretend (pretend,17.91 KB, text/plain)
2017-05-26 02:55 UTC, Alessandro Barbieri
Details
info (info,29.85 KB, text/plain)
2017-05-26 02:55 UTC, Alessandro Barbieri
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alessandro Barbieri 2017-05-25 06:54:16 UTC
Running 

sudo rosdep init

led to

ERROR: cannot download default sources list from:
https://raw.githubusercontent.com/ros/rosdistro/master/rosdep/sources.list.d/20-default.list
Website may be down.

but I can access https://raw.githubusercontent.com/ros/rosdistro/master/rosdep/sources.list.d/20-default.list via browser ...
Comment 1 Jonas Stein gentoo-dev 2017-05-25 10:45:51 UTC
works here. Can you retry please?
Comment 2 Alessandro Barbieri 2017-05-25 11:26:25 UTC
Did you run "rosdep init" or just checked the URI?
Because the URI works for me too but rosdep can't access it for some reason.
Comment 3 Alessandro Barbieri 2017-05-25 11:36:53 UTC
Created attachment 474250 [details]
info

info
Comment 4 Alessandro Barbieri 2017-05-25 11:37:59 UTC
sudo emerge -pv dev-util/rosdep

These are the packages that would be merged, in order:

Calculating dependencies                   ... done!
[ebuild   R   ~] dev-util/rosdep-0.11.5::gentoo  USE="{-test}" PYTHON_TARGETS="pypy pypy3 python2_7 python3_4 python3_5 -python3_6" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB
Comment 5 Jonas Stein gentoo-dev 2017-05-25 11:42:02 UTC
Looks fine here, or did I miss the point?

$ sudo rosdep init
Wrote /etc/ros/rosdep/sources.list.d/20-default.list
Recommended: please run

	rosdep update

$ rosdep update
reading in sources list data from /etc/ros/rosdep/sources.list.d
Hit https://raw.githubusercontent.com/ros/rosdistro/master/rosdep/osx-homebrew.yaml
Hit https://raw.githubusercontent.com/ros/rosdistro/master/rosdep/base.yaml
Hit https://raw.githubusercontent.com/ros/rosdistro/master/rosdep/python.yaml
Hit https://raw.githubusercontent.com/ros/rosdistro/master/rosdep/ruby.yaml
Hit https://raw.githubusercontent.com/ros/rosdistro/master/releases/fuerte.yaml
Query rosdistro index https://raw.githubusercontent.com/ros/rosdistro/master/index.yaml
Add distro "groovy"
Add distro "hydro"
Add distro "indigo"
Add distro "jade"
Add distro "kinetic"
Add distro "lunar"
updated cache in /home/stein/.ros/rosdep/sources.cache
Comment 6 Alessandro Barbieri 2017-05-25 12:16:24 UTC
Re-tested, same problem. I'm out of ideas, your have the same version and is from ::gentoo?
Comment 7 Jonas Stein gentoo-dev 2017-05-25 13:25:20 UTC
tested with rosdep-0.11.5, perhaps ask on IRC or the forum / mailinglist so that we can find a pattern why it fails or not.
Comment 8 Alessandro Barbieri 2017-05-25 14:28:05 UTC
Ok now I found the problem: works with python3, python2 and pypy but with pypy3 (my preferred python implementation and first automatically tried) throw this error
Comment 9 Alexis Ballier gentoo-dev 2017-05-25 15:04:34 UTC
does:
$ pypy3 -c "from urllib.request import urlopen; urlopen('https://raw.githubusercontent.com/ros/rosdistro/master/rosdep/sources.list.d/20-default.list')"

works for you ?
Comment 10 Alexis Ballier gentoo-dev 2017-05-25 16:54:16 UTC
same here after upgrading to pypy3-5.7.1-r1:

[ebuild   R    ] dev-python/pypy3-5.7.1-r1:0/57::gentoo  USE="bzip2 gdbm jit ncurses tk -libressl -low-memory (-sandbox) -sqlite" CPU_FLAGS_X86="(-sse2)" 0 KiB


$  pypy3 -c "from urllib.request import urlopen; urlopen('https://raw.githubusercontent.com/ros/rosdistro/master/rosdep/sources.list.d/20-default.list')"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/pypy3/lib-python/3/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib64/pypy3/lib-python/3/urllib/request.py", line 466, in open
    response = self._open(req, data)
  File "/usr/lib64/pypy3/lib-python/3/urllib/request.py", line 489, in _open
    'unknown_open', req)
  File "/usr/lib64/pypy3/lib-python/3/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib64/pypy3/lib-python/3/urllib/request.py", line 1324, in unknown_open
    raise URLError('unknown url type: %s' % type)
urllib.error.URLError: <urlopen error unknown url type: https>



pypy3 doesnt link to openssl at all
Comment 11 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-05-25 17:46:01 UTC
Could you try with pypy3-bin? That's my build, and it works for me.

(In reply to Alexis Ballier from comment #10)
> pypy3 doesnt link to openssl at all

pypy3 uses OpenSSL via CFFI.
Comment 12 Alexis Ballier gentoo-dev 2017-05-25 18:11:03 UTC
simpler case:

$ pypy3 -c 'import ssl'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/pypy3/lib-python/3/ssl.py", line 99, in <module>
    import _ssl             # if we can't import it, let the error propagate
  File "/usr/lib64/pypy3/lib_pypy/_ssl/__init__.py", line 1, in <module>
    from _cffi_ssl._stdssl import (_PROTOCOL_NAMES, _OPENSSL_API_VERSION,
  File "/usr/lib64/pypy3/lib_pypy/_cffi_ssl/_stdssl/__init__.py", line 6, in <module>
    from _pypy_openssl import ffi
ImportError: No module named '_pypy_openssl'


(I didnt have build log storage enabled, rebuilding it to be able to grep)
Comment 13 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-05-25 18:37:40 UTC
I see the bug now. The pypy3 ebuild misses running CFFI gen for ssl module (pypy3-bin is fine). Fix in a minute.
Comment 14 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-05-25 18:42:41 UTC
commit 6c9d094f59282be1e894c7311cdc1af3f5a302ef
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: Thu May 25 20:38:47 2017
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: Thu May 25 20:40:50 2017

    dev-python/pypy3: Fix missing ssl CFFI module in 5.7.1, #619648
Comment 15 Alessandro Barbieri 2017-05-26 02:47:57 UTC
Emerge fail with:

>>> Install pypy3-5.7.1-r2 into /var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/ category dev-python
 * Installing PyPy ...
 * PT_PAX marking -m /var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/pypy3-c with paxctl
 * PT_PAX marking -m /var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/pypy3-c with paxctl-ng
 * PT_PAX marking -m /var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/libpypy3-c.so with paxctl
 * PT_PAX marking -m /var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/libpypy3-c.so with paxctl-ng
 * XATTR_PAX marking -m /var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/pypy3-c with paxctl-ng
 * XATTR_PAX marking -me /var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/pypy3-c with setfattr
 * Failed to set XATTR_PAX markings -me /var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/pypy3-c.
 * XATTR_PAX marking -me /var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/libpypy3-c.so with paxctl-ng
 * XATTR_PAX marking -me /var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/libpypy3-c.so with setfattr
 * Failed to set XATTR_PAX markings -me /var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/libpypy3-c.so.
 * Generating caches and byte-compiling ...
_lzma_cffi.c: In function ‘_cffi_checkfld_typedef_lzma_stream’:
_lzma_cffi.c:828:29: warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
   { lzma_allocator * *tmp = &p->allocator; (void)tmp; }


In file included from /usr/include/openssl/x509.h:595:0,
                 from /usr/include/openssl/engine.h:96,
                 from _pypy_openssl.c:518:
_pypy_openssl.c:2749:19: error: expected identifier or ‘(’ before numeric constant
 static const long X509_V_ERR_HOSTNAME_MISMATCH = 0;
                   ^
_pypy_openssl.c:2750:19: error: expected identifier or ‘(’ before numeric constant
 static const long X509_V_ERR_EMAIL_MISMATCH = 0;
                   ^
_pypy_openssl.c:2751:19: error: expected identifier or ‘(’ before numeric constant
 static const long X509_V_ERR_IP_ADDRESS_MISMATCH = 0;
                   ^
Traceback (most recent call last):
  File "/var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/lib-python/3/distutils/unixccompiler.py", line 143, in _compile
    extra_postargs)
  File "/var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/lib-python/3/distutils/ccompiler.py", line 909, in spawn
    spawn(cmd, dry_run=self.dry_run)
  File "/var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/lib-python/3/distutils/spawn.py", line 36, in spawn
    _spawn_posix(cmd, search_path, dry_run=dry_run)
  File "/var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/lib-python/3/distutils/spawn.py", line 159, in _spawn_posix
    % (cmd, exit_status))
distutils.errors.DistutilsExecError: command 'gcc' failed with exit status 1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "_ssl_build.py", line 53, in <module>
    ffi.compile()
  File "/var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/lib_pypy/cffi/api.py", line 684, in compile
    compiler_verbose=verbose, debug=debug, **kwds)
  File "/var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/lib_pypy/cffi/recompiler.py", line 1484, in recompile
    compiler_verbose, debug)
  File "/var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/lib_pypy/cffi/ffiplatform.py", line 20, in compile
    outputfilename = _build(tmpdir, ext, compiler_verbose, debug)
  File "/var/tmp/portage/dev-python/pypy3-5.7.1-r2/image/usr/lib64/pypy3/lib_pypy/cffi/ffiplatform.py", line 56, in _build
    raise VerificationError('%s: %s' % (e.__class__.__name__, e))
cffi.error.VerificationError: CompileError: command 'gcc' failed with exit status 1
 * ERROR: dev-python/pypy3-5.7.1-r2::gentoo failed (install phase):
 *   Failed to build CFFI bindings for ssl
 *
 * Call stack:
 *     ebuild.sh, line 115:  Called src_install
 *   environment, line 4241:  Called die
 * The specific snippet of code:
 *           "${PYTHON}" "_${t}_build.py" || die "Failed to build CFFI bindings for ${t}";
Comment 16 Alessandro Barbieri 2017-05-26 02:51:36 UTC
Created attachment 474342 [details]
dev-python:pypy3-5.7.1-r2:20170525-215219.log.xz

build log
Comment 17 Alessandro Barbieri 2017-05-26 02:55:28 UTC
Created attachment 474344 [details]
pretend

pretend
Comment 18 Alessandro Barbieri 2017-05-26 02:55:51 UTC
Created attachment 474346 [details]
info

info
Comment 19 Alexis Ballier gentoo-dev 2017-05-26 08:45:42 UTC
(In reply to Michał Górny from comment #11)
> (In reply to Alexis Ballier from comment #10)
> > pypy3 doesnt link to openssl at all
> 
> pypy3 uses OpenSSL via CFFI.

FTR: checking what it links to was still the proper way.

/usr/lib64/pypy3/lib_pypy/_pypy_openssl.pypy3-57-x86_64-linux-gnu.so
links to openssl here
Comment 20 Alessandro Barbieri 2017-05-26 13:06:37 UTC
I'll upgrade libressl to latest and retry.
Comment 21 Alessandro Barbieri 2017-05-26 15:17:47 UTC
Same error
Comment 22 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-05-26 15:20:24 UTC
Could you or the libressl team take this upstream? I don't have libressl installed here, and I don't have the resources to chase this.
Comment 23 Alessandro Barbieri 2017-05-28 09:41:56 UTC
I was asking for this right now :)
Ok, so I'll look for upstream support
Thanks
Comment 24 Alessandro Barbieri 2017-05-28 16:03:36 UTC
Upstream closed my bug as invalid saying that libressl is not officially supported and the bug came not from their build-system.
Comment 25 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-05-28 17:34:58 UTC
Well, I guess they would accept a patch if someone made one.
Comment 26 Marien Zwart 2017-06-19 13:45:26 UTC
It's pretty likely that backporting https://github.com/pyca/cryptography/commit/8d85b9564284a66aa28a0cbb759090b777698e43 and https://github.com/pyca/cryptography/commit/8bb9cc629b33e80a8544d7a3a2a55f96549a0259 or updating libressl's bundled cryptography would help.

The bug is in version checking in code PyPy bundles from cryptography (it's assuming that some #defines provided by newer openssl are never provided by libressl, but they actually are). That version checking code was recently reworked, and judging from the commit description it addresses this issue.
Comment 27 Alessandro Barbieri 2017-10-21 16:43:52 UTC
same with pypy3-5.9.0
Comment 28 Alessandro Barbieri 2017-10-21 17:11:59 UTC
(In reply to Alessandro Barbieri from comment #27)
> same with pypy3-5.9.0

pypy3-5.9.0 throws other errors (translation error, maybe not libressl related) while pypy3-5.8.0 have the same issue
Comment 29 Marien Zwart 2017-10-22 01:22:14 UTC
pypy-5.9.0 and pypy3-5.9.0 built fine here against libressl-2.6.2.

If your translation error with 5.9.0 does not reference openssl or libressl, it's probably a different bug (I recommend filing one with a build log attached).
Comment 30 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-10-22 07:42:09 UTC
(In reply to Marien Zwart from comment #29)
> pypy-5.9.0 and pypy3-5.9.0 built fine here against libressl-2.6.2.
> 
> If your translation error with 5.9.0 does not reference openssl or libressl,
> it's probably a different bug (I recommend filing one with a build log
> attached).

Does 'built fine' imply that it works, or merely doesn't fail during the build?
Comment 31 Marien Zwart 2017-10-22 08:07:42 UTC
As far as I can tell it works, but I don't know how best to exercise PyPy's SSL support.

urllib's https support works (this spits out a bunch of html):

$ pypy -c 'from urllib2 import urlopen; print urlopen("https://www.gentoo.org/").read()'
$ pypy3 -c 'from urllib.request import urlopen; print(urlopen("https://www.gentoo.org/").read())'

This seems to confirm it's really using LibreSSL:

$ pypy -c 'import ssl; print ssl.OPENSSL_VERSION'
LibreSSL 2.6.2                                                                                                         

$ pypy3 -c 'import ssl; print(ssl.OPENSSL_VERSION)'
LibreSSL 2.6.2                                                                                                         

That's with:

$ qlist -Iv dev-python/pypy
dev-python/pypy3-5.9.0                                                                                                 
dev-python/pypy-5.9.0

Anything else you'd like me to try?
Comment 32 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2017-10-22 11:13:44 UTC
Good enough for me.

Feel free to reopen if you notice any problems.