Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 636400

Summary: dev-lang/python-3.5.4: test failures on sparc (test_ctypes test_datetime test_hash test_plistlib test_strptime test_time)
Product: Gentoo Linux Reporter: Rolf Eike Beer <eike>
Component: Current packagesAssignee: Python Gentoo Team <python>
Status: RESOLVED FIXED    
Severity: normal CC: sparc
Priority: Normal Keywords: PATCH, TESTFAILURE
Version: unspecified   
Hardware: Sparc64   
OS: Linux   
URL: https://github.com/python/cpython/pull/6123
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: build.log
build.log of python 3.4.5
build.log of python 3.4.5 (second run)
Patch for Python 3.6.3-r1
hashlib patch for 3.5.4-r1

Description Rolf Eike Beer archtester 2017-11-03 16:34:31 UTC
373 tests OK.
6 tests failed:
    test_ctypes test_datetime test_hash test_plistlib test_strptime
    test_time
18 tests skipped:
    test_curses test_devpoll test_kqueue test_msilib test_ossaudiodev
    test_smtpnet test_socketserver test_startfile test_timeout
    test_tix test_tk test_ttk_guionly test_urllib2net test_urllibnet
    test_winreg test_winsound test_xmlrpc_net test_zipfile64
Re-running failed tests in verbose mode
Re-running test 'test_plistlib' in verbose mode
test_appleformatting (test.test_plistlib.TestPlistlib) ... Fatal Python error: Bus error

Portage 2.3.8 (python 2.7.14-final-0, default/linux/sparc/13.0, gcc-5.4.0, glibc-2.25-r8, 4.13.8 sparc64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-4.13.8-sparc64-sun4v-with-gentoo-2.3
KiB Mem:    33133648 total,  13577792 free
KiB Swap:          0 total,         0 free
Timestamp of repository gentoo: Fri, 03 Nov 2017 13:15:01 +0000
Head commit of repository gentoo: 2ce521bb27399e0d205a10b3f6e7e85745db6509
sh bash 4.3_p48-r1
ld GNU ld (Gentoo 2.28.1 p1.0) 2.28.1
app-shells/bash:          4.3_p48-r1::gentoo
dev-lang/perl:            5.24.3::gentoo
dev-lang/python:          2.7.14::gentoo, 3.4.5::gentoo, 3.5.4::gentoo
dev-util/cmake:           3.8.2::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.3::gentoo
sys-apps/openrc:          0.32.1::gentoo
sys-apps/sandbox:         2.10-r4::gentoo
sys-devel/autoconf:       2.69::gentoo
sys-devel/automake:       1.11.6-r1::gentoo, 1.14.1::gentoo, 1.15-r2::gentoo
sys-devel/binutils:       2.28.1::gentoo
sys-devel/gcc:            5.4.0-r3::gentoo
sys-devel/gcc-config:     1.8-r1::gentoo
sys-devel/libtool:        2.4.6-r3::gentoo
sys-devel/make:           4.2.1::gentoo
sys-kernel/linux-headers: 4.4::gentoo (virtual/os-headers)
sys-libs/glibc:           2.25-r8::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000

ACCEPT_KEYWORDS="sparc"
ACCEPT_LICENSE="* -@EULA"
CBUILD="sparc-unknown-linux-gnu"
CFLAGS="-O2 -mcpu=niagara2 -pipe"
CHOST="sparc-unknown-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/spool/torque"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.6/ext-active/ /etc/php/apache2-php7.0/ext-active/ /etc/php/cgi-php5.6/ext-active/ /etc/php/cgi-php7.0/ext-active/ /etc/php/cli-php5.6/ext-active/ /etc/php/cli-php7.0/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -mcpu=niagara2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS=""
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS=""
GENTOO_MIRRORS="ftp://ftp.wh2.tu-dresden.de/pub/mirrors/gentoo ftp://ftp.join.uni-muenster.de/pub/linux/distributions/gentoo"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j20"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="acl berkdb big-endian bzip2 cli cracklib crypt cxx dri fortran gdbm iconv ipv6 modules ncurses nls nptl openmp pam pcre readline session sparc ssl tcpd unicode xattr zlib" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" POSTGRES_TARGETS="postgres9_5" PYTHON_SINGLE_TARGET="python3_4" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby21 ruby22" USERLAND="GNU" VIDEO_CARDS="fbdev glint mach64 mga r128 radeon sunbw2 suncg14 suncg3 suncg6 sunffb sunleo tdfx dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

=================================================================
                        Package Settings
=================================================================

dev-lang/python-3.5.4::gentoo was built with the following:
USE="build examples gdbm ipv6 ncurses readline sqlite ssl (threads) tk wininst xml (-hardened) (-libressl)"
CFLAGS="-O2 -mcpu=niagara2 -pipe -fwrapv"
CXXFLAGS="-O2 -mcpu=niagara2 -pipe -fwrapv"
LDFLAGS="-Wl,-O1 -Wl,--as-needed -L."


Reproducible: Always
Comment 1 Rolf Eike Beer archtester 2017-11-03 16:35:02 UTC
Created attachment 501990 [details]
build.log
Comment 2 Rolf Eike Beer archtester 2017-11-03 16:35:49 UTC
If you need more files just tell me, I leave the build tree around.
Comment 3 Mike Gilbert gentoo-dev 2017-11-03 21:17:49 UTC
test_time is failing because you haven't set a timezone properly. Please make sure /etc/localtime is a valid zoneinfo file.

The other test failures warrant further investigation by someone with access to sparc hardware. They are almost certainly hardware-specific.
Comment 4 Mike Gilbert gentoo-dev 2017-11-03 21:18:17 UTC
Copying the sparc alias.
Comment 5 Rolf Eike Beer archtester 2017-11-10 17:49:44 UTC
No regression compared to 3.4.5, which also fails (logs coming).
Comment 6 Rolf Eike Beer archtester 2017-11-10 17:50:58 UTC
Created attachment 503500 [details]
build.log of python 3.4.5
Comment 7 Rolf Eike Beer archtester 2017-11-10 17:51:46 UTC
Created attachment 503502 [details]
build.log of python 3.4.5 (second run)
Comment 8 Rolf Eike Beer archtester 2017-11-13 06:30:42 UTC
Just a sum up of some further testing compared to 3.4.5:

-test_ctypes does not fail in 3.4.5, but this is probably because it was simply not tested (the testsuite for 3.5.4 has much more subtests here)

-hash fails in both versions

Python 3.5.4 has been marked stable on sparc nevertheless because it's probably no worse than before.
Comment 9 Rolf Eike Beer archtester 2017-11-29 18:49:42 UTC
Created attachment 507212 [details, diff]
Patch for Python 3.6.3-r1

The same problem exists with 3.6.3-r1. 3 of the 5 failures go away with the attached patch. Upstream has a special memcpy() macro in this file so they may want to tweak it, but it should be straight forward.
Comment 10 Rolf Eike Beer archtester 2017-11-29 18:53:56 UTC
The ctypes issue looks like one once fixed for Windows which I can't find in the python bugtracker anymore, the reason was that the argument is copied by reference and no intermediate copy was created by libffi.
Comment 11 Rolf Eike Beer archtester 2017-12-17 21:12:03 UTC
Created attachment 510538 [details, diff]
hashlib patch for 3.5.4-r1
Comment 12 Mike Gilbert gentoo-dev 2017-12-17 21:32:15 UTC
Thanks for the patches. Would you mind sending them upstream?
Comment 13 Larry the Git Cow gentoo-dev 2018-04-19 19:44:16 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0d123a4bd5046559e97746f6598c7f6030dccdda

commit 0d123a4bd5046559e97746f6598c7f6030dccdda
Author:     Mike Gilbert <floppym@gentoo.org>
AuthorDate: 2018-04-19 19:43:52 +0000
Commit:     Mike Gilbert <floppym@gentoo.org>
CommitDate: 2018-04-19 19:44:13 +0000

    dev-lang/python: fix unaligned memory access
    
    Thanks to Rolf Eike Beer for the patches.
    
    Closes: https://bugs.gentoo.org/636400
    Package-Manager: Portage-2.3.24, Repoman-2.3.6_p81

 .../python/files/python-3.5.5-hash-unaligned.patch | 43 ++++++++++++++++++++++
 .../python/files/python-3.6.5-hash-unaligned.patch | 42 +++++++++++++++++++++
 dev-lang/python/python-3.4.8.ebuild                |  1 +
 dev-lang/python/python-3.5.5.ebuild                |  1 +
 dev-lang/python/python-3.6.5.ebuild                |  1 +
 5 files changed, 88 insertions(+)
Comment 14 Rolf Eike Beer archtester 2018-05-13 12:08:34 UTC
Just for the record: the patch has been merged upstream for 3.6+