emerge fails despite 4 GB RAM free total used free shared buffers cached Mem: 4046244 3238624 807620 0 679008 1571924 -/+ buffers/cache: 987692 3058552 Swap: 0 0 0 * Checking for at least 4 gibibytes RAM ... [ !! ] * There is NOT at least 4 gibibytes RAM * * Space constrains set in the ebuild were not met! * The build will most probably fail, you should enhance the space * as per failed tests. * * ERROR: dev-python/pypy-1.9-r1 failed (pretend phase): * Build requirements not met! * * Call stack: * ebuild.sh, line 85: Called pkg_pretend * pypy-1.9-r1.ebuild, line 36: Called check-reqs_pkg_pretend * check-reqs.eclass, line 105: Called check-reqs_pkg_setup * check-reqs.eclass, line 96: Called check-reqs_output * check-reqs.eclass, line 237: Called die * The specific snippet of code: * [[ ${EBUILD_PHASE} == "pretend" && -z ${I_KNOW_WHAT_I_AM_DOING} ]] && \ * die "Build requirements not met!" * * If you need support, post the output of `emerge --info '=dev-python/pypy-1.9-r1'`, * the complete build log and the output of `emerge -pqv '=dev-python/pypy-1.9-r1'`. * The complete build log is located at '/var/log/portage/dev-python:pypy-1.9-r1:20120701-110256.log'. * For convenience, a symlink to the build log is located at '/mnt/md3/cache/portage/dev-python/pypy-1.9-r1/temp/build.log'. * The ebuild environment file is located at '/mnt/md3/cache/portage/dev-python/pypy-1.9-r1/temp/die.env'. * Working directory: '/mnt/md3/cache/portage/dev-python/pypy-1.9-r1' * S: '/mnt/md3/cache/portage/dev-python/pypy-1.9-r1/work/pypy-1.9' Portage 2.1.11.3 (hardened/linux/amd64, gcc-4.4.7, glibc-2.14.1-r3, 3.4.4-hardened x86_64) ================================================================= System uname: Linux-3.4.4-hardened-x86_64-Intel-R-_Core-TM-2_Quad_CPU_Q6600_@_2.40GHz-with-gentoo-2.1 Timestamp of tree: Sat, 30 Jun 2012 00:45:01 +0000 ccache version 3.1.7 [enabled] app-shells/bash: 4.2_p29 dev-java/java-config: 2.1.12 dev-lang/python: 2.7.3-r2, 3.2.3-r1 dev-util/ccache: 3.1.7 dev-util/cmake: 2.8.8-r3 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.1-r1 sys-apps/openrc: 0.8.3-r1 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.69 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.5, 1.12.1 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.4.7, 4.7.1 sys-devel/gcc-config: 1.7.3 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r3 sys-kernel/linux-headers: 3.4 (virtual/os-headers) sys-libs/glibc: 2.14.1-r3 Repositories: gentoo x-portage ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA PUEL dlj-1.1 ETQW RTCW-ETEULA googleearth AdobeFlash-10.3 Oracle-BCLA-JavaSE" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/bind /var/lib/hsqldb" CONFIG_PROTECT_MASK="${EPREFIX}/etc/gconf /etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.4/ext-active/ /etc/php/cgi-php5.4/ext-active/ /etc/php/cli-php5.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe" DISTDIR="/mnt/md3/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs buildpkg ccache config-protect-if-modified distlocks ebuild-locks fixlafiles metadata-transfer news parallel-fetch parse-eapi-ebuild-head protect-owned sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv webrsync-gpg" FFLAGS="-O2 -pipe" LANG="en_GB.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed" LINGUAS="en_GB" MAKEOPTS="-j2" PKGDIR="/mnt/md3/cache/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/mnt/md3/cache" PORTDIR="/mnt/md3/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X a52 aac acpi aio alsa amd64 amr apng berkdb bzip2 cairo caps chm cli consolekit cracklib crypt cxx dbus device-mapper dlz dri dts dvb eix enca exif extensions extras faad ffmpeg flac gdbm glib glibc-omitfp glitz gnutls gtk gudev handbook hardened iconv jit jpeg jpeg2k justify lcms lm_sensors logrotate lzma matroska mikmod minizip mmx mng modplug modules mp3 mpeg mudflap multilib mysqli ncurses nodrm nptl nsplugin ogg openal opengl openmp pam pax_kernel pcre pdf pic png pppd qt qt3support qt4 rar readline rtsp sandbox schroedinger session slang spell sqlite sqlite3 sse sse2 sse3 ssl ssse3 symlink theora threads tiff truetype unicode urandom usb userpriv v4l2 vlc vorbis vpx x264 xcb xcomposite xml xorg xv xvid zlib" ALSA_CARDS="virtuoso" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en_GB" PHP_TARGETS="php5-3" PYTHON_TARGETS="python3_2 python2_7" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
echo $((1024 * 1024 * 4)) 4194304 that's the number it's looking for. How exact that requirement is for building pypy might be a different question but the check is behaving correctly.
Memory: 4045804k/4980736k available (4815k kernel code, 787400k absent, 147532k reserved, 1694k data, 440k init) Some memory seems to be reserved on my PC. Can be memory requirement changed to e.g. 3840 MB?
(In reply to comment #2) > Memory: 4045804k/4980736k available (4815k kernel code, 787400k absent, > 147532k reserved, 1694k data, 440k init) > Some memory seems to be reserved on my PC. > Can be memory requirement changed to e.g. 3840 MB? You can use swap(in file) for building this package, because i think that portage requires as much memory for some important reasons
I guess the requirement could be lowered slightly to take into account the memory requirements of the kernel (and hardware address space usage), or conversely, raised quite a bit if 4GB is indeed the minimum of free memory required.
I would not expect building pypy on an amd64 system with 4GB of physical memory and no swap to succeed (although I'd not mind somebody trying to confirm this). On my own ~amd64 host (with 8G of physical ram and some swap) the translator process grows to a little less than 4G of resident memory, and then fires off a make subprocess that runs some parallel (in your case 2) gcc processes that will consume a fair bit of memory too. Even if you run next to nothing else I would expect the translator process to get oom-killed once the parallel gccs get going. If you add some swap you may be alright (or the build may grind to a halt, depending on how much else you have occupying memory). If you can confirm the build succeeds with this little ram we can lower the requirement (just copy the pypy ebuild to an overlay and edit the requirement to something like 2G for testing, and run "ebuild /path/to/edited/ebuild manifest" afterwards). But pypy really is very ram-hungry and I do not expect this to actually work. Perhaps we should add a pypy-bin for people with memory-starved hosts?
Would it be possible to enable installation of 32-bit Python on amd64.
I failed to emerge pypy 1.9-r1 on 64-bit intel with 16G of physical RAM. There was out-of-mem error during installation phase ==== Fatal error during initialization: out of memory Generation of Grammar and PatternGrammar pickles failed ==== Should I open new bug? emerge --info follows ===== Portage 2.1.10.11 (default/linux/amd64/10.0/desktop, gcc-4.5.3, glibc-2.12.2-r0, 2.6.39-gentoo-r3 x86_64) ================================================================= System Settings ================================================================= System uname: Linux-2.6.39-gentoo-r3-x86_64-Intel-R-_Core-TM-_i5-2500K_CPU_@_3.30GHz-with-gentoo-11.9 Timestamp of tree: Mon, 30 Jul 2012 23:00:01 +0000 app-shells/bash: 4.1_p9 dev-lang/python: 2.7.3-r2 dev-util/cmake: 2.8.4-r1 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.0.3 sys-apps/openrc: 0.8.3-r1 sys-apps/sandbox: 2.4 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.1 sys-devel/binutils: 2.21.1-r1 sys-devel/gcc: 4.4.5, 4.5.3-r1 sys-devel/gcc-config: 1.4.1-r1 sys-devel/libtool: 2.4-r1 sys-devel/make: 3.82-r1 sys-kernel/linux-headers: 2.6.39 (virtual/os-headers) sys-libs/glibc: 2.12.2 Repositories: gentoo zugaina rebutia kde-sunset sunrise fuck-off ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -march=native -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /usr/share/openvpn/easy-rsa" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -march=native -pipe" DISTDIR="/mnt/big/open/distr/linux/gentoo/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="ftp://ftp.corbina.ru/gentoo" LANG="en_GB.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en en_GB" MAKEOPTS="--jobs 8 --load-average=10" PKGDIR="/usr/portage/packages" PORTAGE_COMPRESS="lzma" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/tmp/" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage/layman/zugaina /usr/local/portage/layman/rebutia /usr/local/portage/layman/kde-sunset /usr/local/portage/layman/sunrise /usr/local/portage/general.overlay" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="3dnow 3dnowext X a52 aac aalib acpi airplay alsa amd64 ao asf berkdb branding bzip2 cairo cdda cdr cli crypt cue cups cxx dbus djbfft dri dts dvd dvdr emboss encode exif ffmpeg firefox flac fortran gdbm gif gpm gtk iconv jpeg kpathsea lcms ldap libcaca libnotify mac mad matroska mmap mmx mng modules mp3 mp4 mpeg mudflap multilib musepack ncurses nophonehome nptl ofa ogg opengl openmp pango pcre pdf png ppds pppd qt3support qt4 readline sdl session sndfile speex spell sse sse2 ssl ssse3 startup-notification static-libs svg tcpd theora threads tiff truetype udev udisks umfpack unicode upower usb vcd vocoder vorbis wavpack wxwidgets x264 xcb xml xv xvfb xvid xvmc zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en en_GB" PHP_TARGETS="php5-3" PYTHON_TARGETS="python3_2 python2_7" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga neomagic nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
I'd need a build log to stand a chance of debugging that, but please provide it in a new bug, as this one is mostly about the check for required memory. One thing: IIRC USE=sandbox on pypy is currently broken and will fail at the stage your build fails, and I *think* with that error message. So if you're building with USE=sandbox try toggling that off (note: leave FEATURES=sandbox enabled! this is about the pypy-specific sandbox feature, not the portage one).
(In reply to comment #8) Yes, turning off sandbox USE-flag helps Yes, I filed new bug #429372
(In reply to comment #3) > (In reply to comment #2) > > Memory: 4045804k/4980736k available (4815k kernel code, 787400k absent, > > 147532k reserved, 1694k data, 440k init) > > Some memory seems to be reserved on my PC. > > Can be memory requirement changed to e.g. 3840 MB? > > You can use swap(in file) for building this package, because i think that > portage requires as much memory for some important reasons This doesn't appear to be the case. On my 32-bit x86 box with 1 GB of ram and a 2 GB swap partition, I received a message that there is not at least 2 gigabytes RAM. Adding another 2 GB swap file to the mix didn't change anything. The check for RAM seems to ignore swap. # free -g total used free shared buffers cached Mem: 1 0 1 0 0 0 -/+ buffers/cache: 0 1 Swap: 4 0 4 # emerge -1 pypy Calculating dependencies... done! >>> Verifying ebuild manifests >>> Running pre-merge checks for dev-python/pypy-1.9-r1 * Checking for at least 2 gigabytes RAM ... [ !! ] * There is NOT at least 2 gigabytes RAM
(In reply to comment #10) > (In reply to comment #3) > > (In reply to comment #2) > > > Memory: 4045804k/4980736k available (4815k kernel code, 787400k absent, > > > 147532k reserved, 1694k data, 440k init) > > > Some memory seems to be reserved on my PC. > > > Can be memory requirement changed to e.g. 3840 MB? > > > > You can use swap(in file) for building this package, because i think that > > portage requires as much memory for some important reasons > > This doesn't appear to be the case. On my 32-bit x86 box with 1 GB of ram > and a 2 GB swap partition, I received a message that there is not at least 2 > gigabytes RAM. Adding another 2 GB swap file to the mix didn't change > anything. The check for RAM seems to ignore swap. swap != RAM. The check doesn't validate swap for the very simple reason that without 2G real RAM, the build is going to take ages swapping. Seriously.
Fair enough, although now I've got to figure out why gentoo only sees 2000 mb of the 2048 I've got installed.
FWIW, pypy does tolerably little swapping (4 hours for me) with 3GB physical RAM on my amd64 so lowering the check to 3.something would be reasonable.
(In reply to comment #13) > FWIW, pypy does tolerably little swapping (4 hours for me) with 3GB physical > RAM on my amd64 > > so lowering the check to 3.something would be reasonable. If we were to do that, we'd need to implement also a swap check. 3 gigs of RAM is the amount around which I no longer have swap.
Let's WONTFIX it with the following rationale: Please remember that check-reqs is purely a primitive tool that we use in order to reduce the likeliness of time-consuming builds failing due to lack of memory or disk space. It's nowhere close to perfect or even good solution. But it's something that works well enough in most of the cases while keeping the overhead low. As such a tool it can't handle all the special cases properly. I would really prefer if we avoided adding more special cases to it since the amount of work resulting for the developer will be much grater than the benefit for users. For the special cases, there's the I_KNOW_WHAT_I_AM_DOING flag. And involving 'some' swap is a special case indeed. I don't really want to get into whether people believe that check-reqs should consider the amounts of memory that are necessary for swap-less build, or with 'acceptable' amounts of swapping. Some people will even want to have a switch for that. After all, we'll at least need to have some tools that would monitor the memory use and pinpoint peek memory requirements and analyze the access patterns to see how much swapping is 'tolerable'. I don't know of such a tool. I suspect it would make builds much slower. Then, there are all the special cases like zram that makes RAM/swap measures quite blurry. I doubt that it is a good idea to start going deeper... If you really want this, please start by suggesting the necessary improvements for check-reqs. After the developer who maintains the eclass reviews them, I believe you'll need to convince the Gentoo community as well.
