This seems to be because python is solely linking to ncurses[w]. I believe it needs also to link to tinfo. See also: https://bugs.gentoo.org/690222 I'm not sure what the proper fix is because I don't fully understand python's build system, but I'll attach a proof-of-concept patch that at least got my system to compile python successfully. Reproducible: Always
Created attachment 586782 [details, diff] proof-of-concept patch
Please attach the entire build log to this bug report. Please post your `emerge --info` output in a comment.
Comment on attachment 586782 [details, diff] proof-of-concept patch setup.py should check whether terminal info specific functions are provided by libncurses or by libtinfo, instead of hard-coding it as in your patch.
I apologize, I completely failed to mention this is related to uclibc-ng, and blueness is in on it. Here's emerge --info, build log to follow once the system can produce it emerge --info Portage 2.3.69 (python 3.6.5-final-0, default/linux/arm/17.0/uclibc/armv6j, gcc-8.3.0, uclibc-ng-1.0.31, 4.19.66-gentoo armv6l) ================================================================= System uname: Linux-4.19.66-gentoo-armv6l-Intel-R-_Core-TM-_i7-4770_CPU_@_3.40GHz-with-gentoo-2.6 KiB Mem: 16336000 total, 6210652 free KiB Swap: 0 total, 0 free Timestamp of repository gentoo: Thu, 15 Aug 2019 10:15:01 +0000 Head commit of repository gentoo: 9a1a265385cfcf00088a15ba81d54a791a504ce5 sh bash 4.4_p23-r1 ld GNU ld (Gentoo 2.32 p2) 2.32.0 app-shells/bash: 4.4_p23-r1::gentoo dev-lang/perl: 5.28.2-r1::gentoo dev-lang/python: 2.7.15::gentoo, 3.6.5::gentoo dev-util/cmake: 3.14.6::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.6-r1::gentoo sys-apps/openrc: 0.41.2::gentoo sys-apps/sandbox: 2.13::gentoo sys-devel/autoconf: 2.69-r4::gentoo sys-devel/automake: 1.16.1-r1::gentoo sys-devel/binutils: 2.31.1-r6::gentoo, 2.32-r1::gentoo sys-devel/gcc: 8.3.0-r1::gentoo sys-devel/gcc-config: 2.0::gentoo sys-devel/libtool: 2.4.6-r3::gentoo sys-devel/make: 4.2.1-r4::gentoo sys-kernel/linux-headers: 4.19::gentoo (virtual/os-headers) sys-libs/uclibc-ng: 1.0.31::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-rsync-verify-metamanifest: no sync-rsync-verify-jobs: 1 sync-rsync-verify-max-age: 24 sync-rsync-extra-opts: x-portage location: /usr/local/portage masters: gentoo priority: 0 ACCEPT_KEYWORDS="arm" ACCEPT_LICENSE="@FREE" CBUILD="armv6j-unknown-linux-uclibceabihf" CFLAGS="-march=armv6j -mfpu=vfp -mfloat-abi=hard -pipe -O2" CHOST="armv6j-unknown-linux-uclibceabihf" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-march=armv6j -mfpu=vfp -mfloat-abi=hard -pipe -O2" DISTDIR="/usr/portage/distfiles" ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe -march=armv6j -mfpu=vfp -mfloat-abi=hard" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync network-sandbox news nodoc noinfo noman parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe -march=armv6j -mfpu=vfp -mfloat-abi=hard" GENTOO_MIRRORS="http://mirror.its.dal.ca/gentoo" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en" MAKEOPTS="-j8" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="acl arm bzip2 cli crypt cxx dri gdbm iconv ncurses nptl openmp pcre readline seccomp split-usr ssl tcpd uclibc unicode xattr zlib" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="uclibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" L10N="en" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" NETBEANS_MODULES="apisupport cnd groovy gsf harness ide identity j2ee java mobility nb php profiler soa visualweb webcommon websvccommon xml" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-2" POSTGRES_TARGETS="postgres10 postgres11" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby24" USERLAND="GNU" VIDEO_CARDS="exynos fbdev omap dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Created attachment 587070 [details] failed build log as requested
Also affecting me on python 2.7.10 ... build.log to follow.
Created attachment 589594 [details] build.log from armv7a catalyst stage3 build
Comment on attachment 587070 [details] failed build log as requested Please do not compress build logs under 1 MB. It just makes it harder to view them in a web browser.
I am unable to produce either failure on amd64/glibc.
(In reply to Mike Gilbert from comment #9) > I am unable to produce either failure on amd64/glibc. (In reply to David Flogeras from comment #4) > I apologize, I completely failed to mention this is related to uclibc-ng, > and blueness is in on it. > > sys-libs/uclibc-ng: 1.0.31::gentoo This is a peculiarity for uclibc, it doesn't even manifest on musl. Further investigation tends to suggest it's missing -ltinfow in addition to -lncursesw in LDFLAGS possibly? There is a patch for pkg-config but it would appear that it's still not using that for ncurses link detection somewhere/somehow.
(In reply to Michael 'veremitz' Everitt from comment #10) > This is a peculiarity for uclibc, it doesn't even manifest on musl. > > Further investigation tends to suggest it's missing -ltinfow in addition to > -lncursesw in LDFLAGS possibly? There is a patch for pkg-config but it would > appear that it's still not using that for ncurses link detection > somewhere/somehow. ie. eg. http://dpaste.com/3SP0BDV#line-941 (also in build log, paste will expire)
(In reply to Michael 'veremitz' Everitt from comment #11) It does seem like -ltinfo or -ltinfow is missing, but I'm baffled as to why it doesn't show up on glibc.
Here, the problem shows up on glibc (amd64 and x86) with all 3.6 and 3.7 python versions up to (including) 3.6.9 and 3.7.4-r1
Note: The patch from comment #1 fixes the problem. IMHO, this patch should not be modified to dynamically detect the library: This would break if the user compiles ncurses with a toggled tinfo. So IMHO, what is needed is instead a new USE-flag tinfo with a proper USE-depend and a conditional application of the patch.
(In reply to Martin Väth from comment #14) > Note: The patch from comment #1 fixes the problem. > > IMHO, this patch should not be modified to dynamically detect the library: > This would break if the user compiles ncurses with a toggled tinfo. > So IMHO, what is needed is instead a new USE-flag tinfo with a proper > USE-depend and a conditional application of the patch. Should be feasible with a conditional based on eg. has_version ncurses[tinfo] perhaps .. ?
(In reply to Michael 'veremitz' Everitt from comment #15) > has_version ncurses[tinfo] No, since this is practically the same as compile-time testing, only. For instance, if a user adds sys-libs/ncurses tinfo to package.use and then runs emerge -NDu @world, then only ncurses will be recompiled, but portage will not recompile python and will not even warn that python will be broken afterwards.
(In reply to Martin Väth from comment #16) > (In reply to Michael 'veremitz' Everitt from comment #15) > > has_version ncurses[tinfo] > > No, since this is practically the same as compile-time testing, only. > > For instance, if a user adds > sys-libs/ncurses tinfo > to package.use and then runs emerge -NDu @world, then only ncurses will be > recompiled, but portage will not recompile python and will not even warn > that python will be broken afterwards. Mmm, fair point .. but there should be some link between the ncurses Python USE flag then .. Outta my dept here now .. but I see your point! :)
We need someone to submit a patch uptream on this. Happy to backport from there.
(In reply to Mike Gilbert from comment #18) > We need someone to submit a patch uptream on this. Happy to backport from > there. I will take this up then. also of interest to some, konsolebox's comment at https://bugs.gentoo.org/457530#c20 for 'matching' USE flag/rebuild scenarios.
Filed https://bugs.python.org/issue38318 .
Created attachment 591532 [details] PoC patch python2.7 This is a slightly less-crude compile-time patch which works for me for Python2.7. Will adjust shortly for 3.x
Created attachment 591538 [details] PoC patch python3.6 Similarly crude compile-time patch for python3.6. I'm deliberately ignoring the separate issues of recompiling python when [tinfo] is toggled, and any issue that may be arising from linker configuration options that may be enabled/disabled for uclibc-ng on my local x86_64/ARM systems at this point. These patches /shouldn't/ affect these scenarios, nor glibc, as far as I can tell however, although will await review.
(In reply to Michael 'veremitz' Everitt from comment #21) > Created attachment 591532 [details] > PoC patch python2.7 > > This is a slightly less-crude compile-time patch which works for me for > Python2.7. Will adjust shortly for 3.x That doesn't account for tinfow.
(In reply to Michał Górny from comment #23) > (In reply to Michael 'veremitz' Everitt from comment #21) > > Created attachment 591532 [details] > > PoC patch python2.7 > > > > This is a slightly less-crude compile-time patch which works for me for > > Python2.7. Will adjust shortly for 3.x > > That doesn't account for tinfow. That's a fair assertion.. I will further re-work the logic a bit, and check that works too.
Created attachment 591632 [details, diff] Python_2.7 patch take 2 Builds correctly on x86_64 with uclibc-ng, and on ARMv6zk (also with uclibc-ng.
Created attachment 591634 [details, diff] Python_3.6 patch take 2 Corresponding patch for v3.6
Testing on armv6j-unknown-linux-uclibceabihf (rpi). python:2.7 compiled with above patch, and import curses works as expected. Trying with python:3.6, and I got a failure I can't understand. Attaching build.log, emerge --info as above (same system).
Created attachment 591682 [details] Failed build log with latest patch on python:3.6
(In reply to David Flogeras from comment #27) > Testing on armv6j-unknown-linux-uclibceabihf (rpi). > > python:2.7 compiled with above patch, and import curses works as expected. > > Trying with python:3.6, and I got a failure I can't understand. Attaching > build.log, emerge --info as above (same system). It doesn't /look/ like a build system failure .. but I'll compare my logs later and see if I spot anything. If you repeat the build does it halt at the same point?
Hmm, nm. Emerging again and it finished, probably unrelated to this. In any case, python3 with above patch works for me, including importing curses module. Thanks for the effort!
(In reply to David Flogeras from comment #30) > Hmm, nm. Emerging again and it finished, probably unrelated to this. > > In any case, python3 with above patch works for me, including importing > curses module. Thanks for the effort! Super, thanks for the verification! :] enjoy!
I can confirm that the two patches work on a i586-gentoo-linux-uclibc system with the default/linux/x86/17.0/uclibc/hardened profile
Patches from #c25 and #c26 apply and build correctly with python-2.7.16 and python-3.6.9 respectively, as recently stabilised, FYI.
Is there any chance of getting the patches applied to the ebuilds in the tree? I'm continuously facing the issue for both Python 3 and 2, although with fewer error messages. If the patches are manually applied I can successfully merge Python, but it is a hassle...
*** Bug 644470 has been marked as a duplicate of this bug. ***
