I tried to emerge sys-devel/llvm (sys-devel/llvm-6.0.1::gentoo) but the compilation failed. This happened twice, first in the context of emerging xfce-base/xfce4-meta which depends on llvm, and then doing it again separately. The main error message was ERROR: sys-devel/llvm-6.0.1::gentoo failed (compile phase): ninja -v -j4 -l0 failed This is not very explicit about what kind of failure, and the log output was unrevealing (unless I missed a message). This is on a Raspberry Pi 3B (1Gb RAM, 0.5Gb swap) running a recently installed and updated instance of Gentoo for ARM64. Both times the compilation took unusually long. The second time, when I tried to connect by slogin the client timed out repeatedly. On the console session simple commands were possible like ls and ps and uptime (load of 7), but response was extremely sluggish, and the last visible compilation step was not seen to finish for a long time. I speculate without proof that the complex build system used by llvm ran the machine out of RAM, and the various processes were thrashing. There was no out-of-memory error message (that I noticed), but a timeout in the build system is credible, though not supported by the call trace or other messages. I guess that if I want XFCE I'm going to have to cross-compile llvm. I do have a virtual machine running Gentoo for x86_64, with more memory and a cross-compile toolchain (to compile the kernel). I'm not sure what, if anything, the developers can do about the failure, but bugs should be reported... Log files etc. to follow.
emerge --info '=sys-devel/llvm-6.0.1::gentoo' Portage 2.3.49 (python 3.6.5-final-0, default/linux/arm64/17.0, gcc-7.3.0, glibc-2.26-r7, 4.14.78-v8+ aarch64) ================================================================= System Settings ================================================================= System uname: Linux-4.14.78-v8+-aarch64-with-gentoo-2.4.1 KiB Mem: 992040 total, 841068 free KiB Swap: 524284 total, 520176 free Timestamp of repository gentoo: Thu, 01 Nov 2018 00:45:01 +0000 Head commit of repository gentoo: e02d02af70be8c4edd8d3ff42ea8be8d681557e5 sh bash 4.4_p12 ld GNU ld (Gentoo 2.30 p2) 2.30.0 app-shells/bash: 4.4_p12::gentoo dev-lang/perl: 5.24.3-r1::gentoo dev-lang/python: 2.7.15::gentoo, 3.6.5::gentoo dev-util/cmake: 3.9.6::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.4.1-r2::gentoo sys-apps/openrc: 0.38.3::gentoo sys-apps/sandbox: 2.13::gentoo sys-devel/autoconf: 2.13::gentoo, 2.69-r4::gentoo sys-devel/automake: 1.15.1-r2::gentoo sys-devel/binutils: 2.30-r2::gentoo sys-devel/gcc: 7.3.0-r3::gentoo sys-devel/gcc-config: 1.8-r1::gentoo sys-devel/libtool: 2.4.6-r3::gentoo sys-devel/make: 4.2.1-r4::gentoo sys-kernel/linux-headers: 4.13::gentoo (virtual/os-headers) sys-libs/glibc: 2.26-r7::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 sync-rsync-verify-metamanifest: no sync-rsync-extra-opts: sync-rsync-verify-jobs: 1 sync-rsync-verify-max-age: 24 ACCEPT_KEYWORDS="arm64" ACCEPT_LICENSE="* -@EULA" CBUILD="aarch64-unknown-linux-gnu" CFLAGS="-O2 -pipe -march=native" CHOST="aarch64-unknown-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe -march=native" DISTDIR="/usr/portage/distfiles" ENV_UNSET="DBUS_SESSION_BUS_ADDRESS DISPLAY PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" FCFLAGS="-O2 -pipe -march=native" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync multilib-strict news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe -march=native" GENTOO_MIRRORS="http://gentoo.osuosl.org/ http://gentoo.cs.utah.edu/" LDFLAGS="-Wl,-O1 -Wl,--as-needed" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="acl arm64 berkdb bzip2 cli crypt cxx dri fortran gdbm iconv ipv6 libtirpc multilib ncurses nls nptl openmp pam pcre readline seccomp ssl tcpd unicode xattr zlib" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon plan sheets stage words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_ARM="edsp neon thumb thumb2 v4 v5 v6 v7 v8 vfp vfp-d32 vfpv3 vfpv4" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6 php7-1" POSTGRES_TARGETS="postgres9_5 postgres10" PYTHON_SINGLE_TARGET="python3_6" PYTHON_TARGETS="python2_7 python3_6" RUBY_TARGETS="ruby21 ruby23" USERLAND="GNU" VIDEO_CARDS="fbdev dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, MAKEOPTS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Created attachment 553916 [details] Compilation environment variables
Created attachment 553918 [details] The build log
Created attachment 553920 [details] Standard output+error from emerge, includes backtrace at end.
emerge --pqv $PACKAGE [ebuild N ] sys-devel/llvm-6.0.1 USE="libffi ncurses -debug -doc -gold -libedit -test -xar -xml" LLVM_TARGETS="(AArch64) BPF -AMDGPU -ARM -Hexagon -Lanai -MSP430 -Mips -NVPTX -PowerPC -Sparc -SystemZ -X86 -XCore"
Please see 'dmesg' and verify that you haven't run out of memory.
Jim, You have MAKEOPTS unset. Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, MAKEOPTS, PORTAGE_BINHOST, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS So you get ninja -v -j4 -l0 because the build system count cores. -j4 with C++ and 1G RAM is a bit optimistic. Try MAKEOPTS="-j1" emerge -1av sys-devel/llvm Lots of things work with MAKEOPTS="-j4" but you will need a per package override for others. Its a feature rather than a bug.
Created attachment 553996 [details] dmesg output, just the oom-killer invocation @michal, good call. Confirmed that it ran the machine out of RAM+swap. I noticed that when I executed "uptime" the compilation "coincidentally" collapsed. There were other oom kills in dmesg caused by invoking cc1plus. I noticed something else: ninja -j4. My /etc/portage/make.conf lacks any -j4 in MAKEOPTS, in fact no MAKEOPTS at all. The Gentoo Handbook (for x86_64) suggests that I could speed up compilation by enabling parallel make, but earlier I got a botched emerge (cause not confirmed); I took out -j4 and did it over successfully. (I have 4 cores.) But I noticed that fairly often I get parallel make anyway. Hmm, maybe parallel make is the default and the Handbook doesn't reflect that. I'm going to try -j2 or -j1 and see if it's honored.
From my experience with -j4 and C++, even 4G of RAM is insufficient (especially when building chromium I sometimes need to go down to -j2). With 1G RAM, I'm afraid -j1 is the best option nowadays.
(note: my observations are relevant to amd64, arm64 may be a little different)
@Roy_Bamford, thanks for your hint. I explicitly set MAKEOPTS="-j2" and it was honored in the command line of ninja. llvm has 1572 items to compile, link, etc. With -j4 it died after finishing 150 items. With -j2 it died after 1297 items. It looks like one of the parallel items (compilations) died, the other one tried to finish up, but it also died. But I could be wrong. I'm going to try -j1, but I'm not getting my hopes up. I'll let this one run overnight just for completeness. A disappointing pre-conclusion: the Raspberry Pi 3B is unable to emerge llvm due to insufficient resources, and I've confirmed that llvm is a dependency (possibly indirect) of x11-base/xorg-server, so the Respberry Pi is unable to emerge the X-Server without cross-compiling on a more powerful machine, e.g. Intel. I remember one time when I was ordering a i386 PC with 640Kb RAM, and I was debating whether to splurge and upgrade the memory to 1024Kb for I think $100.
Jim, A bug is really the wrong place for this tutorial but since I have your attention ... In make.conf set MAKEOPTS="-j4" or even -j5. In /etc/portage/env/ make two files called MAKEOPTS-j1 MAKEOPTS-j2 with the content # use this for things that fail with any parallel make at all MAKEOPTS="-j1" # use this for things that fail at higher parallel makes than -j2 MAKEOPTS="-j2" It should be clear which is which. Make a file /etc/portage/package.env containing # *** list things that need -j2 here *** app-text/libmwaw MAKEOPTS-j2 dev-libs/boost MAKEOPTS-j2 dev-libs/libixion MAKEOPTS-j2 games-emulation/sdlmame MAKEOPTS-j2 media-sound/mumble MAKEOPTS-j2 media-tv/kodi MAKEOPTS-j2 sys-devel/clang MAKEOPTS-j2 sys-devel/gcc MAKEOPTS-j2 # *** list things that need -j1 here *** app-office/libreoffice MAKEOPTS-j1 dev-lang/rust MAKEOPTS-j1 nodistcc dev-java/icedtea MAKEOPTS-j1 nodistcc net-libs/webkit-gtk MAKEOPTS-j1 sys-devel/binutils MAKEOPTS-j1 sys-devel/llvm MAKEOPTS-j1 nodistcc The content of /etc/portage/env/nodistcc is left as an exercise for the reader, if you use distcc. Portage will apply the settings as you expect, so you don't have to remember. There is also thriving support for arm64 on Pi3 in the Arm forum at forums.gentoo.org and in #gentoo-arm on freenode.
(In reply to Jim Carter from comment #11) > A disappointing pre-conclusion: the Raspberry Pi 3B is unable to emerge > llvm due to insufficient resources, and I've confirmed that llvm is > a dependency (possibly indirect) of x11-base/xorg-server, so the > Respberry Pi is unable to emerge the X-Server without cross-compiling > on a more powerful machine, e.g. Intel. You can disable llvm dependency with USE flags. Fox example you probably want to add USE="-llvm" for mesa or USE="-glamor" for xorg-server.
Thanks to all who helped on this. Here's a followup on the final outcome, which was successful. I should have seen this from the beginning, but my swap space was skimpy (it was going to be an audio-video performance node). I added a swap file of 2.5Gib for a total of 4.0Gib (1Gib RAM + 0.5Gib swap partition + 2.5Gib supplementary file), and all memory-related issues were solved. After 78 hours of compilation (with -j1) and working through a few dependency tangles, I now have a XFCE desktop and Firefox running on the Raspberry Pi 3B. The highest total memory commitment that I saw was about 3Gib. In the biggest compilations the compiler was definitely thrashing. I would have liked to use distcc, but the lion's share of the time was spent running clang and rustc, and as far as I can see, to make them available through distcc would be beyond my ability.
