Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 768048 - <dev-lang/lua-5.3.0: incorrect string.format output on hppa, riscv, sparc32, ...
Summary: <dev-lang/lua-5.3.0: incorrect string.format output on hppa, riscv, sparc32, ...
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal minor (vote)
Assignee: William Hubbs
Depends on:
Blocks: 783405
  Show dependency tree
Reported: 2021-01-31 10:49 UTC by Rolf Eike Beer
Modified: 2021-08-17 14:05 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---

build.log (build.log,15.30 KB, text/plain)
2021-01-31 10:49 UTC, Rolf Eike Beer

Note You need to log in before you can comment on or make changes to this bug.
Description Rolf Eike Beer archtester 2021-01-31 10:49:34 UTC
Created attachment 685236 [details]

/var/tmp/portage/dev-lang/lua-5.2.4-r3/work/lua-5.2.4-.hppa/src/lua5.2: strings.lua:211: assertion failed!
stack traceback:
        [C]: in function 'assert'
        strings.lua:211: in main chunk
        [C]: in function 'olddofile'
        all.lua:157: in main chunk
        [C]: in ?
..>>> closing state <<<

Portage 3.0.13 (python 3.8.7-final-0, default/linux/hppa/17.0, gcc-9.3.0, glibc-2.32-r6, 5.10.10-gentoo-parisc64 parisc64)
System uname: Linux-5.10.10-gentoo-parisc64-parisc64-PA8800_-Mako-with-glibc2.2
KiB Mem:     8228856 total,   4657304 free
KiB Swap:    4194300 total,   4190940 free
Timestamp of repository gentoo: Sun, 31 Jan 2021 01:15:01 +0000
Head commit of repository gentoo: d8cbaa62dbf4abc31cc74d2cc12e3d0c7ac9b70a
sh bash 5.0_p11-r1
ld GNU ld (Gentoo 2.34 p6) 2.34.0
app-shells/bash:          5.0_p11-r1::gentoo
dev-lang/perl:            5.30.3::gentoo
dev-lang/python:          2.7.18-r6::gentoo, 3.6.12-r2::gentoo, 3.7.9-r2::gentoo, 3.8.7-r1::gentoo, 3.9.1-r1::gentoo
dev-util/cmake:           3.18.5::gentoo
sys-apps/baselayout:      2.7::gentoo
sys-apps/openrc:          0.42.1::gentoo
sys-apps/sandbox:         2.20::gentoo
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r5::gentoo
sys-devel/automake:       1.16.2-r1::gentoo
sys-devel/binutils:       2.34-r2::gentoo, 2.35.1-r1::gentoo
sys-devel/gcc:            9.3.0-r1::gentoo
sys-devel/gcc-config:     2.3.2::gentoo
sys-devel/libtool:        2.4.6-r6::gentoo
sys-devel/make:           4.3::gentoo
sys-kernel/linux-headers: 5.4-r1::gentoo (virtual/os-headers)
sys-libs/glibc:           2.32-r6::gentoo

    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://
    priority: -1000
    sync-rsync-verify-max-age: 24
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-metamanifest: no

CFLAGS="-O2 -pipe -march=2.0"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -pipe -march=2.0"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
USE="acl berkdb big-endian bzip2 cli crypt cups firefox foomaticdb fortran gdbm gpm hppa iconv imlib ipv6 libglvnd libwww ncurses nls nptl openmp pam pcre readline seccomp spell split-usr ssl tcpd unicode xattr zlib" ADA_TARGET="gnat_2018" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LUA_SINGLE_TARGET="lua5-1" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3 php7-4" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_8" PYTHON_TARGETS="python3_8" RUBY_TARGETS="ruby26" USERLAND="GNU" VIDEO_CARDS="fbdev dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Comment 1 Vadim A. Misbakh-Soloviov (mva) gentoo-dev 2021-01-31 13:12:08 UTC
1) looks like this only affects tests.
If you just need a package to be installed, you can disable USE=test and build will pass (although, 5.2 is deprecated already).

2) The line it mentions contain
> assert(not pcall(string.format, "%d", 2^63))

i.e. it asserts that 2^63 would be out of range for (decimal) integer, so `string.format()` would throw an error, and so `pcall()` would return `false` (so, after `not()` it will became `true`.

So, as for me, it smells like endianness (big vs little) issue.
Comment 2 Rolf Eike Beer archtester 2021-01-31 14:04:04 UTC
I hit this during testing for bug 766528, otherwise I don't care for it. Maybe it does not know that hppa is big endian? Other big endian arches were successfully stabilized already, so it should be fine I think.
Comment 3 Conrad Kostecki gentoo-dev 2021-04-17 12:26:44 UTC
Any news on stabilizing? I guess, this should be, despite of this failing test, fine, as all other lua versions are stable.
Comment 4 Conrad Kostecki gentoo-dev 2021-05-25 20:44:57 UTC
Comment 5 Conrad Kostecki gentoo-dev 2021-07-06 19:49:14 UTC
@marecki,@williamh: Could we still stabilize lua5.2, as rest of all slotted versions are stabilized.
Comment 6 Rolf Eike Beer archtester 2021-07-06 20:35:27 UTC
Sorry, this fell of my list. This is the only test failure I have seen with this version, so if you think this is worth being stabilized then just do it.
Comment 7 Marek Szuba archtester gentoo-dev 2021-07-08 21:11:48 UTC
Interestingly enough, this is exactly the same test failure I saw while trying to keyword lua-5.2 on riscv. Not an endianness problem then, I guess... Interesting that it's only these two, rather different, arches that fail this test.
Comment 8 Marek Szuba archtester gentoo-dev 2021-07-08 22:32:06 UTC
OK, having tried running 'print(string.format, "%d", 2^63)' under different Lua versions things look rather interesting. For reference, 2 to the 63rd power is 9223372036854775808.

On amd64:
 - 5.1 - prints -9223372036854775808, i.e. right absolute value but wrong sign
 - 5.2, 5.3, 5.4 - "bad argument" error
On riscv:
 - 5.1, 5.2 - prints 9223372036854775807, i.e. is off by one
 - 5.3, 5.4 - "bad argument" error

Furthermore, it seems that in spite of the above the whole "integers out of range" section of strings.lua tests appears to be gone from the 5.3 test suite (haven't checked 5.4).

Finally, seeing as string.format in Lua calls sprintf() under the bonnet I've also looked at the output of the C function call 'printf("%ld", 1L << 63);'. On both arches the result is -9223372036854775808.
Comment 9 Rolf Eike Beer archtester 2021-07-09 06:06:16 UTC
(In reply to Marek Szuba from comment #8)
> OK, having tried running 'print(string.format, "%d", 2^63)' under different
> Lua versions things look rather interesting. For reference, 2 to the 63rd
> power is 9223372036854775808.

But only for unsigned. You are doing signed math, so you actually have only 63 bits, i.e. sign is negative and all other bits are 0 -> INT64_MIN, not UINT64_MAX.

So the versions that print a negative value are right.
Comment 10 Marek Szuba archtester gentoo-dev 2021-07-09 09:09:35 UTC
(In reply to Rolf Eike Beer from comment #9)

> You are doing signed math, so you actually have only
> 63 bits, i.e. sign is negative and all other bits are 0 -> INT64_MIN, not

Oh right, I forgot about the two's complement thing. Still, the output on riscv is clearly wrong... At least it seems the problem is just with printing, as having just converted 2^63 into LE binary using Lua 5.1 I get 0000000000000000000000000000000000000000000000000000000000000001 on both amd64 and riscv.
Comment 11 Rolf Eike Beer archtester 2021-07-09 10:00:29 UTC
Can you send me a test script or command so I can test on hppa?
Comment 12 Marek Szuba archtester gentoo-dev 2021-07-09 10:03:55 UTC
For printing the numbers I copied the line from the tests and replaced assert() with print() i.e. 'print(pcall(string.format, "%d", 2^63))' (the line in Comment #8 has been pasted wrong, sorry). For conversion to binary, I've used the function from .
Comment 13 Rolf Eike Beer archtester 2021-07-09 10:48:22 UTC

echo 'print(pcall(string.format, "%d", 2^63))' | lua5.1
true    2147483647
echo 'print(pcall(string.format, "%d", 2^63))' | lua5.2
true    9223372036854775807
echo 'print(pcall(string.format, "%d", 2^63))' | lua5.3
false   bad argument #2 to 'string.format' (number has no integer representation)

Sparc 32bit:

echo 'print(pcall(string.format, "%d", 2^63))' | lua5.1
true    2147483647
echo 'print(pcall(string.format, "%d", 2^63))' | lua5.2                                                                                                                                                                                                      
true    9223372036854775807
echo 'print(pcall(string.format, "%d", 2^63))' | lua5.3
false   bad argument #2 to 'string.format' (number has no integer representation)
Comment 14 Marek Szuba archtester gentoo-dev 2021-07-09 10:55:51 UTC I wonder how we've ever got 5.2 keyworded on all these arches in the first place! At least we've got some consistency here (the least said about 5.1 the better, especially given both its age and its abysmal test coverage).

Okay, so what do you guys reckon we should do? 5.2 will hopefully no longer be a problem soon but the fact we've got
Comment 15 Marek Szuba archtester gentoo-dev 2021-07-09 11:00:40 UTC
Argh, the browser has somehow sent the page too early.

Anyway, as I was saying - 5.2 will hopefully no longer be a problem soon but we've still got 5.1 to think of... I reckon that at this point we might just assume either that this only affects string.format or that it is a fact of life regarding  this version of Lua), shrug and carry on as usual (and perhaps even keyword it on riscv after all, with lua5-1 still being the default target and all) - but I can't say I personally would be entirely happy with this.
Comment 16 Marek Szuba archtester gentoo-dev 2021-08-17 14:05:21 UTC
I think this is in the end same problem as the one I stumbled upon while attempting to keyword dev-python/xarray for riscv. In short:
 * numbers in Lua are all 64-bit floats;
 * ergo, 2^63 is - as described in the code - out of range;
 * the relevant string.format line casts its input to an integer. However, the result of casting a float64 NaN to an int is undefined in C;
 * what we see here is how different platforms handle such an undefined cast.

Anyway, IMHO this issue is safe to close now. lua-5.2 is gone now, 5.3+ handle this operation differently under the bonnet, and I dare say we know what is going on with 5.1.