portage-2.2.20 seems to gobble memory. portage 14986 0.0 0.0 4192 356 pts/18 SN+ 21:54 0:00 [dev-python/cryptography-0.9.1] sandbox /usr/lib/portage/python3.4/ebuild.sh compile portage 14988 0.0 0.0 123428 408 pts/18 SN+ 21:54 0:00 /bin/bash /usr/lib/portage/python3.4/ebuild.sh compile portage 15005 0.0 0.0 124276 472 pts/18 SN+ 21:54 0:00 /bin/bash /usr/lib/portage/python3.4/ebuild.sh compile portage 15163 19.0 93.7 41117032 15428912 pts/18 DN+ 21:54 2:35 /usr/bin/python3.4 setup.py build root 15326 0.0 0.0 112580 2100 pts/23 S+ 22:08 0:00 grep --color=auto portage 41G of ram before i was able to kill it. have two other machines i haven't been able to kill this process yet. all three servers are updating cryptography Reproducible: Always Actual Results: pmap indicates ever growing anonymous memory allocation. ltrace confirms unending cycle of mmap used to allocate large amounts of pages. if the process is not quickly killed, it will kill the server by consuming all ram and cpu making terminals useless. # emerge --info Portage 2.2.18 (python 3.4.3-final-0, default/linux/amd64/13.0, gcc-4.8.4, glibc-2.20-r2, 3.13.1-gentoo-VSE x86_64) ================================================================= System uname: Linux-3.13.1-gentoo-VSE-x86_64-Intel-R-_Xeon-R-_CPU_X5355_@_2.66GHz-with-gentoo-2.2 KiB Mem: 16460176 total, 15444292 free KiB Swap: 25165820 total, 24080584 free Timestamp of repository gentoo: Wed, 08 Jul 2015 07:00:01 +0000 sh bash 4.3_p39 ld GNU ld (Gentoo 2.25 p1.2) 2.25 app-shells/bash: 4.3_p39::gentoo dev-lang/perl: 5.20.2-r1::gentoo dev-lang/python: 3.4.3::gentoo dev-util/cmake: 3.2.2::gentoo dev-util/pkgconfig: 0.28-r2::gentoo sys-apps/baselayout: 2.2::gentoo sys-apps/openrc: 0.13.9::gentoo sys-apps/sandbox: 2.6-r1::gentoo sys-devel/autoconf: 2.13::gentoo, 2.69-r1::gentoo sys-devel/automake: 1.9.6-r4::gentoo, 1.10.3-r1::gentoo, 1.11.6-r1::gentoo, 1.12.6::gentoo, 1.13.4::gentoo, 1.14.1::gentoo, 1.15::gentoo sys-devel/binutils: 2.23::gentoo, 2.23.1::gentoo, 2.23.2::gentoo, 2.25-r1::gentoo sys-devel/gcc: 4.6.4::gentoo, 4.7.2-r1::gentoo, 4.7.3::gentoo, 4.8.4::gentoo, 4.9.2::gentoo sys-devel/gcc-config: 1.8::gentoo sys-devel/libtool: 2.4.6-r1::gentoo sys-devel/make: 4.1-r1::gentoo sys-kernel/linux-headers: 4.0::gentoo (virtual/os-headers) sys-libs/glibc: 2.20-r2::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 bluelabs location: /usr/local/portage masters: gentoo priority: 1001 ACCEPT_KEYWORDS="amd64 ~amd64 ~x86" ACCEPT_LICENSE="*" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt /var/bind" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.6/ext-active/ /etc/php/cgi-php5.6/ext-active/ /etc/php/cli-php5.6/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--quiet-build" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs buildpkg candy config-protect-if-modified distlocks downgrade-backup ebuild-locks fixlafiles merge-sync metadata-transfer news parallel-fetch preserve-libs protect-owned sandbox sfperms unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.utf8" LC_ALL="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j8" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/usr/vport" USE="3dnow acl acpi aio alsa amd64 aspell audit authdaemond bash-completion bashlogger bcmath berkdb bzip2 caps cli cracklib crypt cryptsetup ctypes-python curl curlwrappers cxx dbus dlz exif expat fam fortran gd gdbm gmp hardened hwdb iconv icu idn inotify iproute2 ipv6 iscsi json kerberos keymap lcms ldap libssh2 logrotate lzma lzo mem-scramble messages minizip mmx mmxext modules multilib multislot multitarget ncurses netlink nettle network-cron nls nptl nptlonly openexr openldap openmp openssl optimization pam pcapnav pch pcntl pcre pcre16 plugins pmu postgres postproc pth python python3 readline realtime rfc3779 sasl script secure-delete security session sockets socks5 soundex spell sqlite sqlite3 srp sse sse2 sse4 ssl strong-optimization svg symlink threads tools udev unicode unlock-notify upcall urandom usb utils xattr xml xmlreader xmlrpc xmlwriter xsl zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="alias auth_basic authn_file authz_groupfile authz_user auth_digest autoindex cgi dav dav_fs dav_lock deflate dir include log_config mime proxy proxy_http rewrite setenvif userdir filter unique_id authz_host headers authn_core unixd authz_core socache_shmcb expires env ext_filter" APACHE2_MPMS="worker" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="none" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer pdfimport" LINGUAS="en en_US" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-4" PYTHON_SINGLE_TARGET="python3_4" PYTHON_TARGETS="python3_4" RUBY_TARGETS="ruby19 ruby20" USERLAND="GNU" VIDEO_CARDS="none" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON note; running "python setup.py build" by hand results in a normal build for this package
I need a build log.
For debugging purposes, please put a limit on the amount of memory the portage user can consume. This should cause a more useful error message to appear in the log. You can create such a limit by adding the following to /etc/security/limits.conf. # limit portage to 2G of memory portage soft as 2097152
As far as I can tell this is triggered by setuptools going stupid if one of the dependencies in the setup.py is missing or too old. I've observed it a few times before ...
Got the same error so I'm uploading my error.log.
Created attachment 409290 [details] build error log
(In reply to dashmz from comment #5) > Created attachment 409290 [details] > build error log Which cffi do you have installed?
(In reply to Justin Lecher from comment #6) > (In reply to dashmz from comment #5) > > Created attachment 409290 [details] > > build error log > > Which cffi do you have installed? dev-python/cffi-1.1.2
(In reply to dashmz from comment #7) > (In reply to Justin Lecher from comment #6) > > (In reply to dashmz from comment #5) > > > Created attachment 409290 [details] > > > build error log > > > > Which cffi do you have installed? > > dev-python/cffi-1.1.2 That's strange, because this is exactly the package pip tries to install. Could you please test to reemerge cffi and cryptograpy?
While rebuilding cffi, cryptography is automatically scheduled for rebuild. But unfortunately it again ate all my memory. The following packages are causing rebuilds: (dev-python/cffi-1.2.1:0/1.2.1::gentoo, ebuild scheduled for merge) causes rebuilds for: (dev-python/cryptography-1.0:0/0::gentoo, ebuild scheduled for merge) (dev-python/pyzmq-14.7.0:0/0::gentoo, ebuild scheduled for merge)
Tried it again with 2G of memory for portage: see attachment 2 [details]g-memory-portage.txt
Created attachment 410654 [details] error log with 2g memory
(In reply to dashmz from comment #11) > Created attachment 410654 [details] > error log with 2g memory So now it appears pycparser is missing on your system. What special things are you doing? Are you in a virtualenv while running emerge? I just installed cryptography on a stage3 chroot without any issue.
I got dev-python/pycparser-2.14 installed and I am not running emerge in a virtualenv. Just trying to do an world update.
Is this still a problem? If so, please do the following cd /var/tmp/portage/dev-python/cryptography*/work/cryptography python setup.py --requires and paste the output here
