| Summary: | dev-lang/python: blake2 dies with illegal instruction | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Jorge Manuel B. S. Vicetto (RETIRED) <jmbsvicetto> |
| Component: | Current packages | Assignee: | Python Gentoo Team <python> |
| Status: | RESOLVED INVALID | ||
| Severity: | normal | CC: | mgorny, slyfox |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
emerge --info
emerge --info from 2nd host gdb output with python 3.6 and 3.7 gcc output emerge --info for VM 3 |
||
|
Description
Jorge Manuel B. S. Vicetto (RETIRED)
2019-09-24 00:53:31 UTC
Created attachment 590848 [details]
emerge --info from 2nd host
I've hit this error on 2 new VMs I installed this weekend. The 1st emerge --info is from the 2nd host. The 2nd emerge --info is from the first host - the one I used to get the errors and to run gdb (the output will be added on a later attachment). Created attachment 590850 [details]
gdb output with python 3.6 and 3.7
Created attachment 590852 [details]
gcc output
Could you run cpuid2cpuflags on one of those VMs? Can you also run 'disassemble' in gdb session to catch exact invalid instruction being executed? It likely comes directly from dev-python/pyblake2 package (it has build-time instruction set selection based on -march=), not dev-lang/python itself. Meanwhile building and checking final binary locally:
$ CFLAGS="-march=core2 -O2 -pipe" ebuild pyblake2-1.1.2.ebuild clean install
$ elfx86exts image/usr/lib64/python3.6/site-packages/pyblake2.cpython-36m-x86_64-linux-gnu.so
MODE64 (call)
SSE2 (movdqu)
SSE1 (movhps)
SSSE3 (pshufb)
CPU Generation: Unknown
pshufb looks like to only suspicious instruction here. -march=core2 does imply ssse3 in gcc:
$ diff -U0 <(gcc -Q --help=target) <(gcc -Q --help=target -march=core2)
...
- -mssse3 [disabled]
+ -mssse3 [enabled]
...
You emerge --info reports:
> System uname: Linux-5.3.0-gentoo-x86_64-QEMU_Virtual_CPU_version_2.5+-with-gentoo-2.6
which sounds like generic QEMU CPU. How do you run qemu to get core2 features passed through? Specifically the '-cpu ' parameter.
I suspect you get generic x86_64 by default and you want something like '-cpu host' or '-cpu core2duo'.
Or you need to drop -march=core2 from CFLAGS.
(In reply to Michał Górny from comment #5) > Could you run cpuid2cpuflags on one of those VMs? I forgot to mention that I had used it to set the CPU flags for the VMs. $ cpuid2cpuflags CPU_FLAGS_X86: mmx mmxext sse sse2 sse3 (In reply to Sergei Trofimovich from comment #7) <snip> > You emerge --info reports: > > > System uname: Linux-5.3.0-gentoo-x86_64-QEMU_Virtual_CPU_version_2.5+-with-gentoo-2.6 > > which sounds like generic QEMU CPU. How do you run qemu to get core2 > features passed through? Specifically the '-cpu ' parameter. > > I suspect you get generic x86_64 by default and you want something like > '-cpu host' or '-cpu core2duo'. > > Or you need to drop -march=core2 from CFLAGS. I understand your point and I can certainly try picking the correct arch, but for reference, I have other VMs, running on the same hosts with the same CFLAGS setting - I'm attaching emerge --info for a 3rd VM on those hosts. The most significant difference in the libvirt xml files for these VMs (1 and 3) is: --- /etc/libvirt/qemu/<vm3>.xml 2019-07-04 03:38:02.982414238 +0000 +++ /etc/libvirt/qemu/<vm1>.xml 2019-09-23 17:43:46.530415128 +0000 - <type arch='x86_64' machine='pc-i440fx-2.5'>hvm</type> + <type arch='x86_64' machine='pc-q35-4.0'>hvm</type> No other VM is experiencing this issue. Created attachment 590862 [details]
emerge --info for VM 3
Rebuilding python with --sse3 disabled as suggested by Arfrever "fixes" the issue. I'll try to enable core2duo in qemu for these vms and will try again. $ sudo PORTAGE_CHECKSUM_FILTER="* -blake2b" CFLAGS="-march=core2 -O2 -pipe -mno-sse3 -mno-ssse3" emerge -av1 --quiet-build=y dev-lang/python These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R ] dev-lang/python-3.7.4-r1:3.7/3.7m::gentoo USE="gdbm hardened ipv6 ncurses readline ssl xml -bluetooth -build -examples -libressl -sqlite -test -tk -wininst" 0 KiB Total: 1 package (1 reinstall), Size of downloads: 0 KiB Would you like to merge these packages? [Yes/No] >>> Verifying ebuild manifests >>> Emerging (1 of 1) dev-lang/python-3.7.4-r1::gentoo >>> Installing (1 of 1) dev-lang/python-3.7.4-r1::gentoo >>> Jobs: 1 of 1 complete Load avg: 1.08, 0.89, 0.43 >>> Auto-cleaning packages... >>> No outdated packages were found on your system. * GNU info directory index is up-to-date. ~ $ python3.7 -c 'import hashlib; print(hashlib.blake2b(b"a").digest())' b'3?\xcbN\xe1\xaa|\x11SU\xecf\xce\xac\x91|\x8b\xfd\x81[\xf7X}2Z\xec\x18d\xed\xd2N4\xd5\xab\xe2\xc6\xb1\xb5\xee?\xac\xe6/\xedx\xdb\xef\x80/*\x85\xcb\x91\xd4U\xa8\xf5$\x9d3\x08S\xcb<' ~ $ sudo emerge -av1 --quiet-build=y less These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R ] sys-apps/less-551::gentoo USE="pcre unicode" 0 KiB Total: 1 package (1 reinstall), Size of downloads: 0 KiB Would you like to merge these packages? [Yes/No] >>> Verifying ebuild manifests >>> Emerging (1 of 1) sys-apps/less-551::gentoo >>> Installing (1 of 1) sys-apps/less-551::gentoo >>> Jobs: 1 of 1 complete Load avg: 0.80, 0.20, 0.07 >>> Auto-cleaning packages... >>> No outdated packages were found on your system. * GNU info directory index is up-to-date. In the meantime, I've noticed that the 3rd VM is now throwing the same illegal instruction error and I haven't updated this system since May 20th, so this is likely related to an update in qemu or the change from a host with an Intel XEON processor to an AMD Opteron system (Dell R710 -> Dell R815) In any case, this isn't a python issue, so I'm closing this as invalid (In reply to Jorge Manuel B. S. Vicetto from comment #12) > In the meantime, I've noticed that the 3rd VM is now throwing the same > illegal instruction error and I haven't updated this system since May 20th, > so this is likely related to an update in qemu or the change from a host > with an Intel XEON processor to an AMD Opteron system (Dell R710 -> Dell > R815) > > In any case, this isn't a python issue, so I'm closing this as invalid Oh, hardware change makes sense. Internets say Dell R815 is an AMD Opteron 6100 series CPU. Those support "SSE2, SSE3, SSE4a", but not "SSSE3". https://en.wikipedia.org/wiki/List_of_AMD_Opteron_microprocessors#Opteron_6100-series_%22Magny-Cours%22_(45_nm) https://en.wikipedia.org/wiki/SSSE3#CPUs_with_SSSE3 $ diff -U0 <(gcc -Q --help=target -march=core2) <(gcc -Q --help=target -march=opteron-sse3) ... @@ -159 +159 @@ - -mssse3 [enabled] + -mssse3 [disabled] qemu can't emulate those instructions in software. |
