Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 636400 - dev-lang/python-3.5.4: test failures on sparc (test_ctypes test_datetime test_hash test_plistlib test_strptime test_time)
Summary: dev-lang/python-3.5.4: test failures on sparc (test_ctypes test_datetime test...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: Sparc64 Linux
: Normal normal (vote)
Assignee: Python Gentoo Team
URL: https://github.com/python/cpython/pul...
Whiteboard:
Keywords: PATCH, TESTFAILURE
Depends on:
Blocks:
 
Reported: 2017-11-03 16:34 UTC by Rolf Eike Beer
Modified: 2018-05-13 12:08 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build.log (build.log,257.91 KB, text/plain)
2017-11-03 16:35 UTC, Rolf Eike Beer
Details
build.log of python 3.4.5 (python-3.4.5-build.log,213.69 KB, text/x-log)
2017-11-10 17:50 UTC, Rolf Eike Beer
Details
build.log of python 3.4.5 (second run) (python-3.4.5-build.log,142.21 KB, text/x-log)
2017-11-10 17:51 UTC, Rolf Eike Beer
Details
Patch for Python 3.6.3-r1 (hash-unaligned.patch,1.26 KB, patch)
2017-11-29 18:49 UTC, Rolf Eike Beer
Details | Diff
hashlib patch for 3.5.4-r1 (hash-unaligned-3.5.patch,1.42 KB, patch)
2017-12-17 21:12 UTC, Rolf Eike Beer
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Rolf Eike Beer 2017-11-03 16:34:31 UTC
373 tests OK.
6 tests failed:
    test_ctypes test_datetime test_hash test_plistlib test_strptime
    test_time
18 tests skipped:
    test_curses test_devpoll test_kqueue test_msilib test_ossaudiodev
    test_smtpnet test_socketserver test_startfile test_timeout
    test_tix test_tk test_ttk_guionly test_urllib2net test_urllibnet
    test_winreg test_winsound test_xmlrpc_net test_zipfile64
Re-running failed tests in verbose mode
Re-running test 'test_plistlib' in verbose mode
test_appleformatting (test.test_plistlib.TestPlistlib) ... Fatal Python error: Bus error

Portage 2.3.8 (python 2.7.14-final-0, default/linux/sparc/13.0, gcc-5.4.0, glibc-2.25-r8, 4.13.8 sparc64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-4.13.8-sparc64-sun4v-with-gentoo-2.3
KiB Mem:    33133648 total,  13577792 free
KiB Swap:          0 total,         0 free
Timestamp of repository gentoo: Fri, 03 Nov 2017 13:15:01 +0000
Head commit of repository gentoo: 2ce521bb27399e0d205a10b3f6e7e85745db6509
sh bash 4.3_p48-r1
ld GNU ld (Gentoo 2.28.1 p1.0) 2.28.1
app-shells/bash:          4.3_p48-r1::gentoo
dev-lang/perl:            5.24.3::gentoo
dev-lang/python:          2.7.14::gentoo, 3.4.5::gentoo, 3.5.4::gentoo
dev-util/cmake:           3.8.2::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.3::gentoo
sys-apps/openrc:          0.32.1::gentoo
sys-apps/sandbox:         2.10-r4::gentoo
sys-devel/autoconf:       2.69::gentoo
sys-devel/automake:       1.11.6-r1::gentoo, 1.14.1::gentoo, 1.15-r2::gentoo
sys-devel/binutils:       2.28.1::gentoo
sys-devel/gcc:            5.4.0-r3::gentoo
sys-devel/gcc-config:     1.8-r1::gentoo
sys-devel/libtool:        2.4.6-r3::gentoo
sys-devel/make:           4.2.1::gentoo
sys-kernel/linux-headers: 4.4::gentoo (virtual/os-headers)
sys-libs/glibc:           2.25-r8::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000

ACCEPT_KEYWORDS="sparc"
ACCEPT_LICENSE="* -@EULA"
CBUILD="sparc-unknown-linux-gnu"
CFLAGS="-O2 -mcpu=niagara2 -pipe"
CHOST="sparc-unknown-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/spool/torque"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.6/ext-active/ /etc/php/apache2-php7.0/ext-active/ /etc/php/cgi-php5.6/ext-active/ /etc/php/cgi-php7.0/ext-active/ /etc/php/cli-php5.6/ext-active/ /etc/php/cli-php7.0/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -mcpu=niagara2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS=""
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS=""
GENTOO_MIRRORS="ftp://ftp.wh2.tu-dresden.de/pub/mirrors/gentoo ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j20"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="acl berkdb big-endian bzip2 cli cracklib crypt cxx dri fortran gdbm iconv ipv6 modules ncurses nls nptl openmp pam pcre readline session sparc ssl tcpd unicode xattr zlib" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" POSTGRES_TARGETS="postgres9_5" PYTHON_SINGLE_TARGET="python3_4" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby21 ruby22" USERLAND="GNU" VIDEO_CARDS="fbdev glint mach64 mga r128 radeon sunbw2 suncg14 suncg3 suncg6 sunffb sunleo tdfx dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

=================================================================
                        Package Settings
=================================================================

dev-lang/python-3.5.4::gentoo was built with the following:
USE="build examples gdbm ipv6 ncurses readline sqlite ssl (threads) tk wininst xml (-hardened) (-libressl)"
CFLAGS="-O2 -mcpu=niagara2 -pipe -fwrapv"
CXXFLAGS="-O2 -mcpu=niagara2 -pipe -fwrapv"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -L."


Reproducible: Always
Comment 1 Rolf Eike Beer 2017-11-03 16:35:02 UTC
Created attachment 501990 [details]
build.log
Comment 2 Rolf Eike Beer 2017-11-03 16:35:49 UTC
If you need more files just tell me, I leave the build tree around.
Comment 3 Mike Gilbert gentoo-dev 2017-11-03 21:17:49 UTC
test_time is failing because you haven't set a timezone properly. Please make sure /etc/localtime is a valid zoneinfo file.

The other test failures warrant further investigation by someone with access to sparc hardware. They are almost certainly hardware-specific.
Comment 4 Mike Gilbert gentoo-dev 2017-11-03 21:18:17 UTC
Copying the sparc alias.
Comment 5 Rolf Eike Beer 2017-11-10 17:49:44 UTC
No regression compared to 3.4.5, which also fails (logs coming).
Comment 6 Rolf Eike Beer 2017-11-10 17:50:58 UTC
Created attachment 503500 [details]
build.log of python 3.4.5
Comment 7 Rolf Eike Beer 2017-11-10 17:51:46 UTC
Created attachment 503502 [details]
build.log of python 3.4.5 (second run)
Comment 8 Rolf Eike Beer 2017-11-13 06:30:42 UTC
Just a sum up of some further testing compared to 3.4.5:

-test_ctypes does not fail in 3.4.5, but this is probably because it was simply not tested (the testsuite for 3.5.4 has much more subtests here)

-hash fails in both versions

Python 3.5.4 has been marked stable on sparc nevertheless because it's probably no worse than before.
Comment 9 Rolf Eike Beer 2017-11-29 18:49:42 UTC
Created attachment 507212 [details, diff]
Patch for Python 3.6.3-r1

The same problem exists with 3.6.3-r1. 3 of the 5 failures go away with the attached patch. Upstream has a special memcpy() macro in this file so they may want to tweak it, but it should be straight forward.
Comment 10 Rolf Eike Beer 2017-11-29 18:53:56 UTC
The ctypes issue looks like one once fixed for Windows which I can't find in the python bugtracker anymore, the reason was that the argument is copied by reference and no intermediate copy was created by libffi.
Comment 11 Rolf Eike Beer 2017-12-17 21:12:03 UTC
Created attachment 510538 [details, diff]
hashlib patch for 3.5.4-r1
Comment 12 Mike Gilbert gentoo-dev 2017-12-17 21:32:15 UTC
Thanks for the patches. Would you mind sending them upstream?
Comment 13 Larry the Git Cow gentoo-dev 2018-04-19 19:44:16 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0d123a4bd5046559e97746f6598c7f6030dccdda

commit 0d123a4bd5046559e97746f6598c7f6030dccdda
Author:     Mike Gilbert <floppym@gentoo.org>
AuthorDate: 2018-04-19 19:43:52 +0000
Commit:     Mike Gilbert <floppym@gentoo.org>
CommitDate: 2018-04-19 19:44:13 +0000

    dev-lang/python: fix unaligned memory access
    
    Thanks to Rolf Eike Beer for the patches.
    
    Closes: https://bugs.gentoo.org/636400
    Package-Manager: Portage-2.3.24, Repoman-2.3.6_p81

 .../python/files/python-3.5.5-hash-unaligned.patch | 43 ++++++++++++++++++++++
 .../python/files/python-3.6.5-hash-unaligned.patch | 42 +++++++++++++++++++++
 dev-lang/python/python-3.4.8.ebuild                |  1 +
 dev-lang/python/python-3.5.5.ebuild                |  1 +
 dev-lang/python/python-3.6.5.ebuild                |  1 +
 5 files changed, 88 insertions(+)
Comment 14 Rolf Eike Beer 2018-05-13 12:08:34 UTC
Just for the record: the patch has been merged upstream for 3.6+