Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 683148 - >=net-fs/samba-4.10.0_rc1 on arm/x86: emerge hangs during configure phase
Summary: >=net-fs/samba-4.10.0_rc1 on arm/x86: emerge hangs during configure phase
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: ARM Linux
: Normal normal (vote)
Assignee: Gentoo's SAMBA Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 704998
  Show dependency tree
 
Reported: 2019-04-12 12:55 UTC by Peter Serbe
Modified: 2020-05-31 20:52 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
build log (build.log,87.88 KB, text/plain)
2019-04-12 12:55 UTC, Peter Serbe
Details
build log on ~x86 system (build.log,88.58 KB, text/x-log)
2019-05-09 06:49 UTC, raffaele_at_gentoo
Details
ps -faux > hung.txt while configure has hung (hung.txt,7.46 KB, text/plain)
2020-01-09 17:01 UTC, Peter Serbe
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Peter Serbe 2019-04-12 12:55:11 UTC
Created attachment 572560 [details]
build log

Starting with 4.10.0rc1 the installation of Samba on my various Arm devices hangs during the compile phase. I have three of them, a Raspi 1 (armv6-j) an Raspi 3 and a LXC on my router (both armv7-a). Samba-4.9.x works flawlessly. 

I have tried many thing, including to install from the straight sources - and I have noticed, that I could provoke similar hang-ups when using a configure with parallel threads, e.g. "./configure -j4". This is strongly discouraged by the Samba team, as it seems to provoke those hangs. 

To further investigate the issue, I put the samba ebuild in my local overlay and exchanged the wav-configure by a plain standard configure... Here's the corresponding diff: 

--------------------------------------------------------------------------------
circe /usr # diff portage/net-fs/samba/samba-4.10.2.ebuild local/overlay/net-fs/samba/samba-4.10.1.ebuild
136d135
<       "${FILESDIR}/${PN}-4.10.0-disable_gnutls_build_fix.patch"
225a225,226
> #     CPPFLAGS="-I${SYSROOT}${EPREFIX}/usr/include/et ${CPPFLAGS}" \
> #             waf-utils_src_configure ${myconf[@]}
227c228
<               waf-utils_src_configure ${myconf[@]}
---
>       econf ${myconf[@]}
--------------------------------------------------------------------------------

This is a very dubious hack - but it build successfully. In my eyes this looks like a bug in waf-utils_src_configure. Numerous experiments with installing and removing various packages did change the configure item, last seen in the log. But I could not overcome the problem.

My samba use flags are these:
net-fs/samba client addc winbind smbclient ldb netapi -system-mitkrb5 -system-heimdal syslog addns ldap gnutls json -cups python3_6
This means, I've got a Samba AD-DC with embedded Heimdal...

This is the outcome of emerge --info:
--------------------------------------------------------------------------------
Portage 2.3.62 (python 3.6.5-final-0, default/linux/arm/17.0/armv7a, gcc-8.2.0, glibc-2.28-r6, 4.19.27-gentoo-r1 armv7l)
=================================================================
System uname: Linux-4.19.27-gentoo-r1-armv7l-ARMv7_Processor_rev_4_-v7l-with-gentoo-2.6
KiB Mem:     2062180 total,    678084 free
KiB Swap:    3834876 total,   3806460 free
Timestamp of repository gentoo: Thu, 11 Apr 2019 02:45:01 +0000
Head commit of repository gentoo: 8f2e01f0aed68693bde85e9d149937f49a546299
sh bash 4.4_p23-r1
ld GNU ld (Gentoo 2.30 p5) 2.30.0
distcc 3.3.2 armv7a-unknown-linux-gnueabihf [enabled]
app-shells/bash:          4.4_p23-r1::gentoo
dev-lang/perl:            5.26.2::gentoo
dev-lang/python:          2.7.15::gentoo, 3.6.5::gentoo
dev-util/cmake:           3.9.6::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.6-r1::gentoo
sys-apps/openrc:          0.38.3-r1::gentoo
sys-apps/sandbox:         2.13::gentoo
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r4::gentoo
sys-devel/automake:       1.16.1-r1::gentoo
sys-devel/binutils:       2.30-r4::gentoo, 2.31.1-r4::gentoo
sys-devel/gcc:            8.2.0-r6::gentoo
sys-devel/gcc-config:     2.0::gentoo
sys-devel/libtool:        2.4.6-r3::gentoo
sys-devel/make:           4.2.1-r4::gentoo
sys-kernel/linux-headers: 4.19::gentoo (virtual/os-headers)
sys-libs/glibc:           2.28-r6::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://pandora.home.serbe.ch/gentoo-portage
    priority: -1000
    sync-rsync-extra-opts: 
    sync-rsync-verify-metamanifest: no
    sync-rsync-verify-max-age: 24
    sync-rsync-verify-jobs: 1

PSeOverlay
    location: /usr/local/overlay
    masters: gentoo
    priority: 10

ACCEPT_KEYWORDS="arm"
ACCEPT_LICENSE="* -@EULA"
CBUILD="armv7a-unknown-linux-gnueabihf"
CFLAGS="-O2 -pipe -march=armv7-a -mfpu=neon-vfpv4 -mfloat-abi=hard"
CHOST="armv7a-unknown-linux-gnueabihf"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/bind"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=armv7-a -mfpu=neon-vfpv4 -mfloat-abi=hard"
DISTDIR="/usr/portage/distfiles"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -pipe -march=armv7-a"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distcc distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe -march=armv7-a"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="de_DE.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j12 -l2"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X aac acl aio alsa arm armv5te armv6 armv6t2 bash-completion berkdb branding bzip2 caps cli crypt cxx dbus dri evdev fat fbdev flac fortran gdbm git glamor glint gnuplot gnutls gtk iconv jpeg kerberos keyboard lame lzma lzo modules mouse mp3 mplayer ncurses nls nptl openmp pam pcre pdf perl png python quota readline samba seccomp server session sound sqlite ssl startup-notification suid tcpd truetype udev unicode xa xattr xorg zlib" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_ARM="edsp thumb thumb2 v4 v5 v6 v7 vfp" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" NETBEANS_MODULES="apisupport cnd groovy gsf harness ide identity j2ee java mobility nb php profiler soa visualweb webcommon websvccommon xml" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-1" POSTGRES_TARGETS="postgres9_5 postgres10" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby24" USERLAND="GNU" VIDEO_CARDS="exynos fbdev omap dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
--------------------------------------------------------------------------------
Comment 1 raffaele_at_gentoo 2019-05-09 06:49:02 UTC
Created attachment 575580 [details]
build log on ~x86 system
Comment 2 raffaele_at_gentoo 2019-05-09 06:50:07 UTC
Same here on ~x86, build log attached
Comment 3 raffaele_at_gentoo 2019-05-09 06:52:25 UTC
$emerge --info

Portage 2.3.66 (python 2.7.16-final-0, default/linux/x86/17.0/desktop, gcc-9.1.0, glibc-2.29-r2, 5.0.9-gentoo i686)
=================================================================
System uname: Linux-5.0.9-gentoo-i686-Intel-R-_Pentium-R-_4_CPU_3.40GHz-with-gentoo-2.6
KiB Mem:     3102176 total,    646856 free
KiB Swap:    3076344 total,   3053456 free
Timestamp of repository gentoo: Wed, 08 May 2019 00:45:01 +0000
Head commit of repository gentoo: 34067c2af12900901ad9c5e1d8bb69e86f010ab9
sh bash 5.0_p7
ld GNU ld (Gentoo 2.32 p2) 2.32.0
app-shells/bash:          5.0_p7::gentoo
dev-java/java-config:     2.2.0-r4::gentoo
dev-lang/perl:            5.28.2-r1::gentoo
dev-lang/python:          2.7.16::gentoo, 3.6.8::gentoo
dev-util/cmake:           3.14.3::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.6-r1::gentoo
sys-apps/openrc:          0.41.2::gentoo
sys-apps/sandbox:         2.17::gentoo
sys-devel/autoconf:       2.13-r1::gentoo, 2.69-r4::gentoo
sys-devel/automake:       1.16.1-r1::gentoo
sys-devel/binutils:       2.32-r1::gentoo
sys-devel/gcc:            8.3.0-r1::gentoo, 9.1.0::gentoo
sys-devel/gcc-config:     2.0::gentoo
sys-devel/libtool:        2.4.6-r5::gentoo
sys-devel/make:           4.2.1-r4::gentoo
sys-kernel/linux-headers: 5.0-r1::gentoo (virtual/os-headers)
sys-libs/glibc:           2.29-r2::gentoo
Repositories:

gentoo
    location: /home/portage
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000
    sync-rsync-verify-jobs: 1
    sync-rsync-verify-metamanifest: yes
    sync-rsync-extra-opts: 
    sync-rsync-verify-max-age: 24

x-portage
    location: /usr/local/portage
    masters: gentoo
    priority: 0

ACCEPT_KEYWORDS="x86 ~x86"
ACCEPT_LICENSE="*"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=native -O2 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=native -O2 -pipe -fomit-frame-pointer"
DISTDIR="/home/tmp/distfiles"
EMERGE_DEFAULT_OPTS="--keep-going --quiet-build"
ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR"
FCFLAGS="-O2 -march=i686 -pipe"
FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles ipc-sandbox merge-sync multilib-strict network-sandbox news nodoc parallel-fetch pid-sandbox preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userpriv usersandbox usersync"
FFLAGS="-O2 -march=i686 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en it"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/home/tmp/"
USE="X a52 aac acl acpi alsa berkdb branding bzip2 cairo cdda cdr cli consolekit crypt cxx dbus dri dts dvdr emboss encode exif fam flac gdbm gif glamor gpm gtk iconv jpeg lcms ldap libnotify libtirpc mad mng mp3 mp4 mpeg mtp ncurses nls nptl ogg opengl openmp pam pango pcre pdf png policykit ppds qt5 readline sdl seccomp spell ssl startup-notification svg tcpd tiff truetype udev udisks unicode upower usb vorbis wxwidgets x264 x86 xattr xcb xml xv xvid zlib" ABI_X86="32" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2 sse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev" KERNEL="linux" L10N="en it" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" NETBEANS_MODULES="apisupport cnd groovy gsf harness ide identity j2ee java mobility nb php profiler soa visualweb webcommon websvccommon xml" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-1" POSTGRES_TARGETS="postgres9_5 postgres10" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby24" USERLAND="GNU" VIDEO_CARDS="intel i915" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, INSTALL_MASK, LC_ALL, MAKEOPTS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 4 James L. Hammons 2019-07-09 00:50:45 UTC
Was running into the same exact hang as Peter in comment #1 (on x86), I was able to get it past the config hang by using his ebuild patch against version 4.10.5 and get it to successfully build and install.

Not sure what's causing the hang, but it sure looks like WAF is to blame.
Comment 5 Will Simoneau 2019-07-09 12:25:22 UTC
Same problem here too on sparc with MAKEOPTS="-j128 -l180" while attempting to build samba-4.10.2. Downgrading to samba-4.8.6-r2 - along with the requisite downgrades to talloc, tevent, tdb, and ldb - resulted in a successful build.
Comment 6 Lars Wendler (Polynomial-C) gentoo-dev 2019-07-09 13:43:44 UTC
Unfortunately the waf-utils.eclass is in "maintainer-needed" state and I don't know WAF good enough to investigate this issue as thoroughly as necessary.

Changing "waf-utils_src_configure" to plain "econf" is a very ugly workaround and nothing I wanna put into our official ebuilds because samba requires WAF.
Comment 7 Xavier Miller (RETIRED) gentoo-dev 2019-09-30 19:35:17 UTC
(In reply to Will Simoneau from comment #5)
> Same problem here too on sparc with MAKEOPTS="-j128 -l180" while attempting
> to build samba-4.10.2. Downgrading to samba-4.8.6-r2 - along with the
> requisite downgrades to talloc, tevent, tdb, and ldb - resulted in a
> successful build.

Hi, same problem here (armv7).
Downgrading samba to 4.8.6-r2 is working for me.
Comment 8 Xavier Miller (RETIRED) gentoo-dev 2019-10-01 07:08:16 UTC
samba-4.9.13 is OK too
Comment 9 Ben Kohler gentoo-dev 2020-01-09 15:40:40 UTC
I'm trying to reproduce this, without success so far.  Can you guys try to gather some more details, like perhaps a process tree from "ps faux" while the configure is hung?
Comment 10 Peter Serbe 2020-01-09 17:01:43 UTC
Created attachment 602828 [details]
ps -faux > hung.txt while configure has hung

Here is the requested information.
The CPU utilization is varying - but the process is hung. In this case it hung here:

Checking for _pread                                                                             : not found
Checking for __pread                                                                            : not found
Checking for pwrite                                                                             : ok
Checking for _pwrite                                                                            :

The exact position, where it hangs is depending, e.g. on Makeopts, i.e. the number of parallel building threads. But even with a lot of experimentation I could not find a clear pattern.
Comment 11 Xavier Miller (RETIRED) gentoo-dev 2020-01-09 21:24:19 UTC
Hi,

It happens also on arm (armv7). Don't know why x86 was added on the title.

For me (armv7, Marvel Armada388, helios4), now, it's stuck some lines after __dup :

Checking for _dup                                                                               : not found 
Checking for __dup                                                                              : not found 
Checking for _dup2                                                                              : not found 
Checking for __dup2                                                                             : ok 
Checking for endmntent                                                                          : ok 
Checking for execl                                                                              : ok 
Checking for _facl                                                                              : not found 
Checking for __facl                                                                             : not found 
Checking for _fchdir                                                                            : 

root      6207  0.0  0.1   8404  3364 pts/1    Ss   21:49   0:00  \_ -bash
root      2198  6.3  3.1  76308 65956 pts/1    SN+  22:16   0:26  |   \_ /usr/bin/python2.7 -b /usr/lib/python-exec/python2.7/emerge -1 samba
root      2765  0.0  0.2  12320  5860 pts/1    SN+  22:17   0:00  |       \_ /usr/bin/python2.7 /usr/lib/portage/python2.7/pid-ns-init 2766
root      2766  0.0  0.2  12320  6028 pts/4    SNs+ 22:17   0:00  |           \_ /usr/bin/python2.7 /usr/lib/portage/python2.7/pid-ns-init 250 250 250 18 0,1,2 /usr/bin/sandbox [net-fs/samba-4.11.4] sandbox /usr/lib/portage/python2.7/ebuild.sh configure
portage   2769  0.0  0.0   1960  1312 pts/4    SN+  22:17   0:00  |               \_ [net-fs/samba-4.11.4] sandbox /usr/lib/portage/python2.7/ebuild.sh configure
portage   2770  0.0  0.2  11740  6116 pts/4    SN+  22:17   0:00  |                   \_ /bin/bash /usr/lib/portage/python2.7/ebuild.sh configure
portage   2786  0.0  0.2  11868  5504 pts/4    SN+  22:17   0:00  |                       \_ /bin/bash /usr/lib/portage/python2.7/ebuild.sh configure
portage   2807  0.0  0.0   7312  1840 pts/4    SN+  22:17   0:00  |                           \_ tee -a /var/tmp/portage/net-fs/samba-4.11.4/temp/build-.arm.log
portage   2908  9.8  1.7 3116124 36444 pts/4   SNl+ 22:17   0:36  |                           \_ /usr/bin/python3.6 /var/tmp/portage/net-fs/samba-4.11.4/work/samba-4.11.4/buildtools/bin/waf --prefix=/usr --libdir=/usr/lib --enable-fhs --sysconfdir=/etc --localstatedir=/var --with-modulesdir=/usr/lib/samba --with-piddir=/run/samba --bundled-libraries=NONE --builtin-libraries=NONE --disable-rpath --disable-rpath-install --nopyc --nopyo --without-acl-support --without-ad-dc --without-dnsupdate --without-ads --disable-cephfs --without-cluster-support --disable-cups --without-dmapi --without-fam --without-gpgme --without-json --disable-iprint --without-pam --without-quotas --without-syslog --without-systemd --without-winbind --disable-python --disable-avahi --with-system-mitkrb5 --without-lttng --without-ldap --without-profiling-data --with-shared-modules= configure
portage   2910  0.0  0.3  14148  8048 pts/4    SN+  22:17   0:00  |                               \_ /usr/bin/python3.6 -c #! /usr/bin/env python # encoding: utf-8 # Thomas Nagy, 2016-2018 (ita)  import os, sys, traceback, base64, signal try: ?import cPickle except ImportError: ?import pickle as cPickle  try: ?import subprocess32 as subprocess except ImportError: ?import subprocess  try: ?TimeoutExpired = subprocess.TimeoutExpired except AttributeError: ?class TimeoutExpired(Exception): ??pass  def run(): ?txt = sys.stdin.readline().strip() ?if not txt: ??# parent process probably ended ??sys.exit(1) ?[cmd, kwargs, cargs] = cPickle.loads(base64.b64decode(txt)) ?cargs = cargs or {}  ?if not 'close_fds' in kwargs: ??# workers have no fds ??kwargs['close_fds'] = False  ?ret = 1 ?out, err, ex, trace = (None, None, None, None) ?try: ??proc = subprocess.Popen(cmd, **kwargs) ??try: ???out, err = proc.communicate(**cargs) ??except TimeoutExpired: ???if kwargs.get('start_new_session') and hasattr(os, 'killpg'): ????os.killpg(proc.pid, signal.SIGKILL) ???else: ????proc.kill() ???out, err = proc.communicate() ???exc = TimeoutExpired(proc.args, timeout=cargs['timeout'], output=out) ???exc.stderr = err ???raise exc ??ret = proc.returncode ?except Exception as e: ??exc_type, exc_value, tb = sys.exc_info() ??exc_lines = traceback.format_exception(exc_type, exc_value, tb) ??trace = str(cmd) + '\n' + ''.join(exc_lines) ??ex = e.__class__.__name__  ?# it is just text so maybe we do not need to pickle() ?tmp = [ret, out, err, ex, trace] ?obj = base64.b64encode(cPickle.dumps(tmp)) ?sys.stdout.write(obj.decode()) ?sys.stdout.write('\n') ?sys.stdout.flush()  while 1: ?try: ??run() ?except KeyboardInterrupt: ??break  
portage   2911  1.3  0.3  14248  8272 pts/4    SN+  22:17   0:04  |                               \_ /usr/bin/python3.6 -c #! /usr/bin/env python # encoding: utf-8 # Thomas Nagy, 2016-2018 (ita)  import os, sys, traceback, base64, signal try: ?import cPickle except ImportError: ?import pickle as cPickle  try: ?import subprocess32 as subprocess except ImportError: ?import subprocess  try: ?TimeoutExpired = subprocess.TimeoutExpired except AttributeError: ?class TimeoutExpired(Exception): ??pass  def run(): ?txt = sys.stdin.readline().strip() ?if not txt: ??# parent process probably ended ??sys.exit(1) ?[cmd, kwargs, cargs] = cPickle.loads(base64.b64decode(txt)) ?cargs = cargs or {}  ?if not 'close_fds' in kwargs: ??# workers have no fds ??kwargs['close_fds'] = False  ?ret = 1 ?out, err, ex, trace = (None, None, None, None) ?try: ??proc = subprocess.Popen(cmd, **kwargs) ??try: ???out, err = proc.communicate(**cargs) ??except TimeoutExpired: ???if kwargs.get('start_new_session') and hasattr(os, 'killpg'): ????os.killpg(proc.pid, signal.SIGKILL) ???else: ????proc.kill() ???out, err = proc.communicate() ???exc = TimeoutExpired(proc.args, timeout=cargs['timeout'], output=out) ???exc.stderr = err ???raise exc ??ret = proc.returncode ?except Exception as e: ??exc_type, exc_value, tb = sys.exc_info() ??exc_lines = traceback.format_exception(exc_type, exc_value, tb) ??trace = str(cmd) + '\n' + ''.join(exc_lines) ??ex = e.__class__.__name__  ?# it is just text so maybe we do not need to pickle() ?tmp = [ret, out, err, ex, trace] ?obj = base64.b64encode(cPickle.dumps(tmp)) ?sys.stdout.write(obj.decode()) ?sys.stdout.write('\n') ?sys.stdout.flush()  while 1: ?try: ??run() ?except KeyboardInterrupt: ??break  


ps -faux:
Comment 12 Peter Serbe 2020-01-09 23:19:49 UTC
> It happens also on arm (armv7). Don't know why x86 was added on the title.

Well, raffaele_at_gentoo reported the hangup to happen on i386, too. 

I have the problem on several armv7-a machines, which serve as my Samba Domain Controllers. On RaspberryPi, on my TurrisOmnia router (in a VM) and on a Cubietruck.
Comment 13 Lars Wendler (Polynomial-C) gentoo-dev 2020-01-10 10:56:41 UTC
Just to be sure... this also happens on samba-4.11.x for you, right?
Comment 14 Xavier Miller (RETIRED) gentoo-dev 2020-01-10 11:06:44 UTC
(In reply to Lars Wendler (Polynomial-C) from comment #13)
> Just to be sure... this also happens on samba-4.11.x for you, right?

Yes, and on all versions >= 4

Version 3 compiles and run fine.
Comment 15 Ben Kohler gentoo-dev 2020-01-10 13:55:21 UTC
(In reply to Xavier Miller from comment #14)
> (In reply to Lars Wendler (Polynomial-C) from comment #13)
> > Just to be sure... this also happens on samba-4.11.x for you, right?
> 
> Yes, and on all versions >= 4
> 
> Version 3 compiles and run fine.

(In reply to Xavier Miller from comment #8)
> samba-4.9.13 is OK too

(In reply to Xavier Miller from comment #7)
> Downgrading samba to 4.8.6-r2 is working for me.
Comment 16 Peter Serbe 2020-01-10 14:00:59 UTC
(In reply to Lars Wendler (Polynomial-C) from comment #13)
> Just to be sure... this also happens on samba-4.11.x for you, right?

For me the issue started with 4.10.0 on, so it is there on 4.10.x and 4.11.x. IIRC it started with the first 4.10.0_rc I tried.
Comment 17 Xavier Miller (RETIRED) gentoo-dev 2020-01-10 14:12:49 UTC
(In reply to Ben Kohler from comment #15)
> (In reply to Xavier Miller from comment #14)
> > (In reply to Lars Wendler (Polynomial-C) from comment #13)
> > > Just to be sure... this also happens on samba-4.11.x for you, right?
> > 
> > Yes, and on all versions >= 4
> > 
> > Version 3 compiles and run fine.
> 
> (In reply to Xavier Miller from comment #8)
> > samba-4.9.13 is OK too
> 
> (In reply to Xavier Miller from comment #7)
> > Downgrading samba to 4.8.6-r2 is working for me.

Sorry... >= 4.10 is NOK (so is 4.11)
but <= 4.9 is OK as you pointed out
Comment 18 Ben Kohler gentoo-dev 2020-02-10 15:34:55 UTC
Ok I can reproduce this pretty reliably in an x86 qemu VM now.  I was not able to repro at all in a chroot.  I'll see if I can figure anything out here.
Comment 19 Ben Kohler gentoo-dev 2020-02-10 17:49:49 UTC
What I know so far:

I can reproduce this every single attempt on my 4-cpu "qemu-system-i386 enable-kvm -cpu pentium2 -smp 4" VM.  It's autodetecting 4 configure jobs here, not respecting MAKEOPTS.  It hangs on "Checking for getgrset" or "Checking for getmntent" every time.

Tried moving up to python3.7, no change.  Tried FEATURES="-sandbox -userpriv -usersandbox -pid-sandbox -network-sandbox -ipc-sandbox" to rule out most common portage-induced issues, no change.

I can get it to work if I append "--jobs 1" to myconf (to be passed to waf configure).  I can get it to fail if I pass any job count higher than 1.

The configure phase doesn't seem to actually go faster with 4 jobs, there are 4 python processes but all but 1 are idle.  
On my i5-2310 CPU (amd64 where configure succeeds):
 configure --jobs 4: 1m11s
 configure --jobs 1: 1m9s

So I'm tempted to just force 1 configure job.
Comment 20 Larry the Git Cow gentoo-dev 2020-02-10 18:40:01 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=3da2f8152d78ab8128d35edc8764af1541403a46

commit 3da2f8152d78ab8128d35edc8764af1541403a46
Author:     Ben Kohler <bkohler@gentoo.org>
AuthorDate: 2020-02-10 18:36:39 +0000
Commit:     Ben Kohler <bkohler@gentoo.org>
CommitDate: 2020-02-10 18:38:41 +0000

    net-fs/samba: disable broken parallel configure
    
    When parallel configure works, it was slower than --jobs 1 anyway.
    
    Closes: https://bugs.gentoo.org/683148
    Package-Manager: Portage-2.3.88, Repoman-2.3.20
    Signed-off-by: Ben Kohler <bkohler@gentoo.org>

 net-fs/samba/samba-4.11.6-r2.ebuild | 2 ++
 1 file changed, 2 insertions(+)
Comment 21 Peter Serbe 2020-02-10 19:51:12 UTC
You guys are faster on closing the bug than I can run the patched ebuild on my Raspi... ;-) But anyway: I can confirm, that the proposed patch fixes the hanging configure on Arm.

But what still leaves me a bit unsure, is, that at least the Gentoo supplied WAF-tools seem to ignore even at global MAKEOPTS="-j1" in make.conf. Even more so, as quite a few of those Arm-based boards have only a very restricted amount of memory on board, which is known to be problematic for parallel builds. 

But Thank You, Ben, for fixing it!
Comment 22 Ben Kohler gentoo-dev 2020-02-10 19:57:30 UTC
What we found is that waf-utils eclass is translating your MAKEOPTS jobs to a job parameter for the compile phase (https://gitweb.gentoo.org/repo/gentoo.git/tree/eclass/waf-utils.eclass#n111) but not for configure phase.

Forcing --jobs 1 in the ebuild is what we want here, regardless of whether more than one cpu is detected OR more than 1 job is already set in MAKEOPTS.

But it may be true for the general case that we want waf-utils eclass to start passing MAKEOPTS in the configure phase... that would be a new bug report I think.
Comment 23 Peter Serbe 2020-02-10 20:07:22 UTC
I think by myselft, that the waf-utils-eclass should default any configure-option to 1 job. It did hit Samba - but possibly it could happen to pop up on some other package using WAF, too. 
On the other hand: a search would immediately point to this bug report - and make on trying out to add that --job 1 in the ebuild...
Comment 24 Tomáš Mózes 2020-02-11 04:23:31 UTC
Thanks Ben for pushing this forward.