Compilation failure on dev-lang/python-3.7.8-r2::gentoo Fails only on 32bit installation, not on 64bit! Message: Traceback (most recent call last): File "./setup.py", line 4, in <module> import sys, os, importlib.machinery, re, optparse File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/optparse.py", line 422, in <module> _builtin_cvt = { "int" : (_parse_int, _("integer")), File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 625, in gettext return dgettext(_current_domain, message) File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 588, in dgettext codeset=_localecodesets.get(domain)) File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 533, in translation t = _translations.setdefault(key, class_(fp)) File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 260, in __init__ self._parse(fp) File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 344, in _parse from struct import unpack File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/struct.py", line 13, in <module> from _struct import * ModuleNotFoundError: No module named '_struct' make: *** [Makefile:628: sharedmods] Error 1 * ERROR: dev-lang/python-3.7.8-r2::gentoo failed (compile phase): * emake failed emerge --info '=dev-lang/python-3.7.8-r2::gentoo' Portage 2.3.89 (python 3.6.11-final-0, default/linux/x86/17.0/desktop, gcc-9.3.0, glibc-2.29-r7, 5.4.28-gentoo i686) ================================================================= System Settings ================================================================= System uname: Linux-5.4.28-gentoo-i686-Intel-R-_Core-TM-_i5-3570_CPU_@_3.40GHz-with-gentoo-2.6 KiB Mem: 3631804 total, 3422776 free KiB Swap: 1047548 total, 1047548 free Timestamp of repository gentoo: Sun, 02 Aug 2020 00:45:01 +0000 Head commit of repository gentoo: 3f389ecfd471a3003664c5e83003d147958a6ec1 sh bash 4.4_p23-r1 ld GNU ld (Gentoo 2.33.1 p2) 2.33.1 distcc 3.3.3 i686-pc-linux-gnu [disabled] ccache version 3.7.7 [disabled] app-shells/bash: 4.4_p23-r1::gentoo dev-java/java-config: 2.2.0-r4::gentoo dev-lang/perl: 5.30.1::gentoo dev-lang/python: 2.7.18-r1::gentoo, 3.6.11-r2::gentoo dev-util/ccache: 3.7.7-r1::gentoo dev-util/cmake: 3.16.5::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.6-r1::gentoo sys-apps/openrc: 0.42.1::gentoo sys-apps/sandbox: 2.13::gentoo sys-devel/autoconf: 2.13-r1::gentoo, 2.69-r4::gentoo sys-devel/automake: 1.13.4-r2::gentoo, 1.16.1-r1::gentoo sys-devel/binutils: 2.33.1-r1::gentoo sys-devel/gcc: 9.3.0::gentoo sys-devel/gcc-config: 2.2.1::gentoo sys-devel/libtool: 2.4.6-r6::gentoo sys-devel/make: 4.2.1-r4::gentoo sys-kernel/linux-headers: 5.4::gentoo (virtual/os-headers) sys-libs/glibc: 2.29-r7::gentoo Repositories: gentoo location: /var/portage/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-rsync-verify-metamanifest: yes sync-rsync-verify-max-age: 24 sync-rsync-extra-opts: --compress-level=9 sync-rsync-verify-jobs: 1 detlev location: /var/portage/overlay masters: gentoo priority: 0 ACCEPT_KEYWORDS="x86" ACCEPT_LICENSE="@FREE" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=pentium4 -mmmx -msse -msse2 -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -march=pentium4 -mmmx -msse -msse2 -pipe" DISTDIR="/var/distfiles" EMERGE_DEFAULT_OPTS="--buildpkg --with-bdeps=y --keep-going=y" ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -march=i686 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -march=i686 -pipe" GENTOO_MIRRORS="/var/distfiles /var/distfiles-Asterix ftp://mirror.netcologne.de/gentoo ftp://ftp.fau.de/gentoo ftp://mirror.switch.ch/mirror/gentoo" LANG="de_CH.utf8" LC_ALL="de_CH.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="de" MAKEOPTS="-j4" PKGDIR="/var/portage/binpkgs" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_EXTRA_OPTS="--compress-level=9" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/tmp" USE="X a52 aac acl acpi alsa berkdb branding bzip2 cairo cdda cdr cjk cli crypt cups dbus dri dts dvd dvdr elogind emboss encode exif ffmpeg flac fontconfig gd gdbm gif gimp gnome-keyring gnutls gphoto2 gpm gstreamer gtk gtk3 iconv icu java jpeg lcms libinput libnotify libtirpc lm_sensors lzma mad mng mp3 mp4 mpeg mysql ncurses nls nptl nsplugin offlinehelp ogg opengl openmp pam pango pch pcre pdf pm-utils png policykit ppds qt5 quicktime raw readline sbcl sdl seccomp sound spell split-usr sqlite ssl startup-notification subversion svg tcpd threads tiff timidity tk truetype udev udisks unicode upower usb vcd video vorbis wxwidgets x264 x86 xcb xfce xml xscreensaver xv xvid zlib" ABI_X86="32" ADA_TARGET="gnat_2018" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" CAMERAS="directory" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="garmin" INPUT_DEVICES="libinput" KERNEL="linux" L10N="de" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer nlpsolver" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2 php7-3" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_7" PYTHON_TARGETS="python2_7 python3_7" RUBY_TARGETS="ruby25" SANE_BACKENDS="hp" USERLAND="GNU" VIDEO_CARDS="vesa vboxvideo" XFCE_PLUGINS="clock power trash" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS emerge -pqv '=dev-lang/python-3.7.8-r2::gentoo' [ebuild NS ] dev-lang/python-3.7.8-r2 [2.7.18-r1, 3.6.11-r2] USE="gdbm ncurses readline sqlite ssl tk xml -bluetooth -build -examples -hardened -ipv6 -libressl -test -wininst (-threads%*)" [ebuild U ] sys-libs/glibc-2.30-r8 [2.29-r7] USE="(crypt%*) gd multiarch ssp (static-libs%*) -audit -caps (-cet) -compile-locales -custom-cflags% -doc -headers-only (-multilib) -nscd -profile (-selinux) -suid -systemtap -test (-vanilla)" !!! The following installed packages are masked: - dev-python/backports-lzma-0.0.13::gentoo (masked by: package.mask) /var/portage/portage/profiles/package.mask: # Michał Górny <mgorny@gentoo.org> (2020-08-01) # Python 2 backport, not used by anything anymore. # Removal in 30 days. For more information, see the MASKED PACKAGES section in the emerge man page or refer to the Gentoo Handbook.
Full build.log please.
Created attachment 655616 [details] Full build log Full build log of python 3.7.8 emerge run
Did previous versions of Python build fine for you, or is this the first version you're building?
Until python 3.4 everything was fine. Problems started with python 3.5, same error. Somebody from German Gentoo forum provided a patch for "Lib/gettext.py", where i should insert "import struct" in line 53 With this patch it compiled without error. (Forum link is "https://forums.gentoo.org/viewtopic-t-1108800.html", but it's all in German...) Same worked for python 3.6 From python 3.7 this did not work anymore. I also tried re-emerge of python packages without success. Funny, that it works on my amd64 box, but not on the x86 box (which i have as a backup machine)
Do I understand correctly that in the linked Forum thread the problem seems specific to x86 as well?
Could you try setting LC_ALL=C and let me know if that changes anything? Do you have the same locale on amd64 machine?
Yes, forum thread is about x86 I will try the locale setting C later and will post the resul
ok, i run <LC_ALL=C emerge -u python>, no change, error still there. Changing locale with eselect locale set 1 (C) -> same result
I suppose you could also try -j1 for completeness but I doubt it's relevant. I *suspect* it may have to do with some .mo file on your system that our systems don't have. Can you reproduce the error by running setup.py directly from the work directory, the same way Makefile does? i.e.: cd /tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8 LD_LIBRARY_PATH=/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8 CC='i686-pc-linux-gnu-gcc -pthread' LDSHARED='i686-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed -L. ' CFLAGS='-Wno-unused-result -Wsign-compare -DNDEBUG -O2 -march=pentium4 -mmmx -msse -msse2 -pipe -fwrapv ' _TCLTK_INCLUDES='' _TCLTK_LIBS='' ./python -E ./setup.py build
ok, -j1 makes no difference What i did now: emerged again until it crashes, then i used the remaining temp portage directory to copy+execute your command. The error lines are displayed immediately: root@Idefix /tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8 # LD_LIBRARY_PATH=/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8 CC='i686-pc-linux-gnu-gcc -pthread' LDSHARED='i686-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed -L. ' CFLAGS='-Wno-unused-result -Wsign-compare -DNDEBUG -O2 -march=pentium4 -mmmx -msse -msse2 -pipe -fwrapv ' _TCLTK_INCLUDES='' _TCLTK_LIBS='' ./python -E ./setup.py build Traceback (most recent call last): File "./setup.py", line 4, in <module> import sys, os, importlib.machinery, re, optparse File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/optparse.py", line 422, in <module> _builtin_cvt = { "int" : (_parse_int, _("integer")), File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 625, in gettext return dgettext(_current_domain, message) File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 588, in dgettext codeset=_localecodesets.get(domain)) File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 533, in translation t = _translations.setdefault(key, class_(fp)) File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 260, in __init__ self._parse(fp) File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 344, in _parse from struct import unpack File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/struct.py", line 13, in <module> from _struct import * ModuleNotFoundError: No module named '_struct'
Ok, now I'm going to ask you to do some debugging. For your convenience, you may want to put the whole directory under VCS control (git init; git add -A; git commit -m 1) or make backups of files you modify. For a start, open Lib/gettext.py and before line 532 (that is: with open(mofile, 'rb') as fp: add: print(mofile) So that we can see which files it tries to open. For extra kudos, you could run qfile (from portage-utils) to check which packages the printed files belong to.
ok, i'm not familiar with git, so i changed this: gettext.py: if t is None: print(">>>>> debug") print(mofile) with open(mofile, 'rb') as fp: t = _translations.setdefault(key, class_(fp)) executing the build command: root@Idefix /tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8 # LD_LIBRARY_PATH=/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8 CC='i686-pc-linux-gnu-gcc -pthread' LDSHARED='i686-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed -L. ' CFLAGS='-Wno-unused-result -Wsign-compare -DNDEBUG -O2 -march=pentium4 -mmmx -msse -msse2 -pipe -fwrapv ' _TCLTK_INCLUDES='' _TCLTK_LIBS='' ./python -E ./setup.py build >>>>> debug /usr/share/locale/de/LC_MESSAGES/messages.mo Traceback (most recent call last): File "./setup.py", line 4, in <module> import sys, os, importlib.machinery, re, optparse File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/optparse.py", line 422, in <module> _builtin_cvt = { "int" : (_parse_int, _("integer")), File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 627, in gettext return dgettext(_current_domain, message) File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 590, in dgettext codeset=_localecodesets.get(domain)) File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 535, in translation t = _translations.setdefault(key, class_(fp)) File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 260, in __init__ self._parse(fp) File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/gettext.py", line 344, in _parse from struct import unpack File "/tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8/Lib/struct.py", line 13, in <module> from _struct import * ModuleNotFoundError: No module named '_struct' and finally: equery b /usr/share/locale/de/LC_MESSAGES/messages.mo * Searching for /usr/share/locale/de/LC_MESSAGES/messages.mo ... root@Idefix /tmp/portage/dev-lang/python-3.7.8-r2/work/Python-3.7.8 # Did not find any package for this mo file. File itself is from year 2009, huu... So i renamed it from "messages.mo" to "messages.zzz" Then i did a full fresh emerge -u python ... What should i say: IT COMPILES :-D :-D :-D Great job, great support Many thanks!
Let's close it as system problem. If you notice the file again or figure out where it came from, please tell us.
(In reply to woct002 from comment #12) messages.po is default output file of xgettext tool: https://www.gnu.org/software/gettext/manual/html_node/xgettext-Invocation.html msgfmt tool compiles *.po files into binary *.mo files: https://www.gnu.org/software/gettext/manual/html_node/msgfmt-Invocation.html msgunfmt tool can mostly reverse above operation: https://www.gnu.org/software/gettext/manual/html_node/msgunfmt-Invocation.html If you want to find hints about origin of this messages.mo file, use msgunfmt to create messages.po from this messages.mo file and check content of messages.po.
I had a quick look into the (binary) messages.mo before i deleted it, must be part of the "xaos" fractal generator. But i think it was changed somehow (or the file date was tampered), so it was not properly removed by unmerging the "xaos" package. No problem for 11 years until python 3.x came along, bad luck ;-)
