| Summary: | dev-python/lockfile-{0.9.1-r1,0.10.2} can't install due to very high memory requirements | ||
|---|---|---|---|
| Product: | Gentoo Linux | Reporter: | Marcin Mirosław <bug> |
| Component: | Current packages | Assignee: | Python Gentoo Team <python> |
| Status: | RESOLVED TEST-REQUEST | ||
| Severity: | normal | CC: | alexander, dustin |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
# dmesg [ 0.000000] Out of memory in UB 108: OOM killed process 2525 (python3.4) score 0 vm:3254864kB, rss:2057260kB, swap:466960kB [ 0.000000] Out of memory in UB 108: OOM killed process 3150 (python3.4) score 0 vm:5220936kB, rss:4157420kB, swap:463504kB [ 0.000000] Out of memory in UB 108: OOM killed process 3831 (python3.4) score 0 vm:10332724kB, rss:8344828kB, swap:457360kB [ 0.000000] Out of memory in UB 108: OOM killed process 4571 (python3.4) score 0 vm:10332568kB, rss:8342296kB, swap:454604kB [ 0.000000] Out of memory in UB 108: OOM killed process 5235 (python3.4) score 0 vm:10332764kB, rss:8337504kB, swap:454608kB [ 0.000000] Out of memory in UB 108: OOM killed process 5767 (python3.4) score 0 vm:10332756kB, rss:8336628kB, swap:451936kB [ 0.000000] Out of memory in UB 108: OOM killed process 6854 (python3.4) score 0 vm:41003604kB, rss:30628828kB, swap:450996kB 30GB is also too low:/ The same with python3.3. please try using python_targets_python2_7 only for this package. This is likely a bug in setuptools; you are not the first to run into it. It shows up in random packages when we miss a dependency somewhere. In a stage3 chroot it installs fine. No clue what package is missing here. (In reply to Mike Gilbert from comment #4) Yeah. I reproduced this bug only when I hadn't installed dev-python/pbr and run "ebuild lockfile-0.10.2.ebuild compile" (emerge --nodeps ... should do the same trick) :) (In reply to Alexander Tsoy from comment #6) > (In reply to Mike Gilbert from comment #4) > Yeah. I reproduced this bug only when I hadn't installed dev-python/pbr and > run "ebuild lockfile-0.10.2.ebuild compile" (emerge --nodeps ... should do > the same trick) :) But pbr is in the depgraph of lockfile so it should be fine. (In reply to Justin Lecher from comment #7) Yes. My previous comment was just a confirmation that missing deps can cause such strange behaviour. (In reply to Justin Lecher from comment #7) > (In reply to Alexander Tsoy from comment #6) > > (In reply to Mike Gilbert from comment #4) > > Yeah. I reproduced this bug only when I hadn't installed dev-python/pbr and > > run "ebuild lockfile-0.10.2.ebuild compile" (emerge --nodeps ... should do > > the same trick) :) > > But pbr is in the depgraph of lockfile so it should be fine. Are you sure? I just untarred stage3: # emerge -pv dev-python/lockfile These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild N ] dev-python/lockfile-0.9.1-r1::gentoo USE="-doc {-test}" PYTHON_TARGETS="python2_7 python3_4 (-pypy) -python3_3" 0 KiB Total: 1 package (1 new), Size of downloads: 0 KiB marcinm / # emerge -pv --emptytree --tree dev-python/lockfile|grep pbr marcinm / # or I'm checking this wrongly:) When I was hitted by this problem I've seen circular dependency while trying emerge dev-python/pip (if I remember correctly name of needed package). And now I can't reproduce problem:( (In reply to Marcin Mirosław from comment #9) > Are you sure? I just untarred stage3: > # emerge -pv dev-python/lockfile > > These are the packages that would be merged, in order: > > Calculating dependencies... done! > [ebuild N ] dev-python/lockfile-0.9.1-r1::gentoo USE="-doc {-test}" > PYTHON_TARGETS="python2_7 python3_4 (-pypy) -python3_3" 0 KiB Can't be: # emerge -pvt dev-python/lockfile These are the packages that would be merged, in reverse order: Calculating dependencies... done! [nomerge ] dev-python/lockfile-0.10.2::gentoo USE="-doc {-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" [nomerge ] dev-python/pbr-1.6.0::gentoo USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" [ebuild N ] dev-python/pip-7.1.2::gentoo PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB [ebuild N ] dev-python/CacheControl-0.11.5::gentoo USE="-redis {-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB [ebuild N ] dev-python/lockfile-0.10.2::gentoo USE="-doc {-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB [ebuild N ] dev-python/pbr-1.6.0::gentoo USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB [ebuild N ] dev-python/requests-2.7.0::gentoo PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB [ebuild N ] dev-python/chardet-2.3.0::gentoo PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB [ebuild N ] dev-python/retrying-1.3.3::gentoo PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB [ebuild N ] dev-python/html5lib-0.999999::gentoo USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB [ebuild N ] dev-python/progress-1.2::gentoo PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB [ebuild N ] dev-python/ipaddress-1.0.14::gentoo PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB [ebuild N ] dev-python/six-1.9.0-r1::gentoo USE="-doc {-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB [ebuild N ] dev-python/packaging-15.3::gentoo USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB [ebuild N ] dev-python/setuptools-18.2-r2::gentoo USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB [ebuild N ] dev-python/distlib-0.2.1::gentoo PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB [ebuild N ] dev-python/colorama-0.3.3::gentoo USE="-examples" PYTHON_TARGETS="python2_7 python3_4 -pypy -python3_3" 0 KiB > or I'm checking this wrongly:) > When I was hitted by this problem I've seen circular dependency while trying > emerge dev-python/pip (if I remember correctly name of needed package). > And now I can't reproduce problem:( All circulars have been removed here. I'm seeing the same - there's an infinite loop in the build:
...
Traceback (most recent call last):
File "/usr/lib64/python3.4/site-packages/setuptools/sandbox.py", line 108, in dump
return pickle.dumps(type), pickle.dumps(exc)
_pickle.PicklingError: Can't pickle <class 'setuptools.sandbox.UnpickleableException'>: it's not the same object as setuptools.sandbox.UnpickleableException
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.4/site-packages/setuptools/sandbox.py", line 108, in dump
return pickle.dumps(type), pickle.dumps(exc)
_pickle.PicklingError: Can't pickle <class 'setuptools.sandbox.UnpickleableException'>: it's not the same object as setuptools.sandbox.UnpickleableException
During handling of the above exception, another exception occurred:
...
which is likely causing the OOM. I, too, manually installed lockfile to get around the circular dependency.
Emerging pbr with --nodeps, then re-emerging lockfile does not help (I get another OOM).
Which versions of pbr, nose, sphinx functools32 are installed? dev-python/pbr - 0.8.2-r1 dev-python/nose - not installed dev-python/sphinx - not installed functools32 - no such ebuild (In reply to Dustin J. Mitchell from comment #13) > dev-python/pbr - 0.8.2-r1 You are mixing arch adn ~arch which is not supported. please upgrade to latest pbr and try again. > dev-python/nose - not installed > dev-python/sphinx - not installed > functools32 - no such ebuild dev-python/functools32 (In reply to Justin Lecher from comment #14) > (In reply to Dustin J. Mitchell from comment #13) > > dev-python/pbr - 0.8.2-r1 > > You are mixing arch adn ~arch which is not supported. please upgrade to > latest pbr and try again. I'm not sure what you mean; from eix pbr is at the latest: [I] dev-python/pbr Available versions: ~0.8.2 0.8.2-r1{tbz2} ~0.10.8 ~0.11.0 ~1.0.1 ~1.1.1 ~1.2.0 ~1.3.0 {test PYTHON_TARGETS="python2_7 python3_3 python3_4"} Installed versions: 0.8.2-r1{tbz2}(05:19:32 PM 05/18/2015)(-test PYTHON_TARGETS="python2_7 -python3_3 -python3_4") pbr isn't mentioned under /etc/portage, and it's a non-testing version that's installed. > dev-python/functools32 knuth ~ # eix dev-python/functools32 No matches found. (In reply to Dustin J. Mitchell from comment #15) > (In reply to Justin Lecher from comment #14) > > (In reply to Dustin J. Mitchell from comment #13) > > > dev-python/pbr - 0.8.2-r1 > > > > You are mixing arch adn ~arch which is not supported. please upgrade to > > latest pbr and try again. > > I'm not sure what you mean; from eix pbr is at the latest: > > [I] dev-python/pbr > Available versions: ~0.8.2 0.8.2-r1{tbz2} ~0.10.8 ~0.11.0 ~1.0.1 > ~1.1.1 ~1.2.0 ~1.3.0 {test PYTHON_TARGETS="python2_7 python3_3 python3_4"} > Installed versions: 0.8.2-r1{tbz2}(05:19:32 PM 05/18/2015)(-test > PYTHON_TARGETS="python2_7 -python3_3 -python3_4") > > pbr isn't mentioned under /etc/portage, and it's a non-testing version > that's installed. > > > dev-python/functools32 > > knuth ~ # eix dev-python/functools32 > No matches found. Please sync your tree. pbr-1.8.0 is the latest version and functools32 was added " Wed Sep 2 13:01:45 2015 +0" Indeed, setting pbr to accept ~amd64 successfully merges (In reply to Justin Lecher from comment #14) > (In reply to Dustin J. Mitchell from comment #13) > > dev-python/pbr - 0.8.2-r1 > > You are mixing arch adn ~arch which is not supported. please upgrade to > latest pbr and try again. While upgrading to ~arch is a useful troubleshooting step, we do in fact support mixing arch and ~arch. It sounds like we need to adjust a dependency somewhere if a certain version of pbr is required. Is this still a problem? If so, please do the following cd /var/tmp/portage/dev-python/lockfile*/work/lockfile* python setup.py --requires and paste the output here Sorry, I solved it in the previous comments so I assume that info is no longer useful for you :( |
Calculating dependencies... done! [ebuild N ] dev-python/lockfile-0.10.2::gentoo USE="-doc {-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB Total: 1 package (1 new), Size of downloads: 0 KiB Would you like to merge these packages? [Yes/No] >>> Verifying ebuild manifests >>> Emerging (1 of 1) dev-python/lockfile-0.10.2::gentoo * lockfile-0.10.2.tar.gz SHA256 SHA512 WHIRLPOOL size ;-) ... [ ok ] >>> Unpacking source... >>> Unpacking lockfile-0.10.2.tar.gz to /var/tmp/portage/dev-python/lockfile-0.10.2/work >>> Source unpacked in /var/tmp/portage/dev-python/lockfile-0.10.2/work >>> Preparing source in /var/tmp/portage/dev-python/lockfile-0.10.2/work/lockfile-0.10.2 ... >>> Source prepared. >>> Configuring source in /var/tmp/portage/dev-python/lockfile-0.10.2/work/lockfile-0.10.2 ... >>> Source configured. >>> Compiling source in /var/tmp/portage/dev-python/lockfile-0.10.2/work/lockfile-0.10.2 ... * python3_4: running distutils-r1_run_phase distutils-r1_python_compile /usr/bin/python3.4 setup.py build /var/tmp/portage/dev-python/lockfile-0.10.2/temp/environment: line 1341: 5767 Killed "${@}" * ERROR: dev-python/lockfile-0.10.2::gentoo failed (compile phase): * (no error message) * * Call stack: * ebuild.sh, line 93: Called src_compile * environment, line 3492: Called distutils-r1_src_compile * environment, line 866: Called _distutils-r1_run_foreach_impl 'distutils-r1_python_compile' * environment, line 297: Called python_foreach_impl 'distutils-r1_run_phase' 'distutils-r1_python_compile' * environment, line 3059: Called multibuild_foreach_variant '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'distutils-r1_python_compile' * environment, line 2176: Called _multibuild_run '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'distutils-r1_python_compile' * environment, line 2174: Called _python_multibuild_wrapper 'distutils-r1_run_phase' 'distutils-r1_python_compile' * environment, line 549: Called distutils-r1_run_phase 'distutils-r1_python_compile' * environment, line 857: Called distutils-r1_python_compile * environment, line 736: Called esetup.py 'build' * environment, line 1345: Called die * The specific snippet of code: * "${@}" || die * * If you need support, post the output of `emerge --info '=dev-python/lockfile-0.10.2::gentoo'`, * the complete build log and the output of `emerge -pqv '=dev-python/lockfile-0.10.2::gentoo'`. * The complete build log is located at '/var/tmp/portage/dev-python/lockfile-0.10.2/temp/build.log'. * The ebuild environment file is located at '/var/tmp/portage/dev-python/lockfile-0.10.2/temp/environment'. * Working directory: '/var/tmp/portage/dev-python/lockfile-0.10.2/work/lockfile-0.10.2' * S: '/var/tmp/portage/dev-python/lockfile-0.10.2/work/lockfile-0.10.2' VPS has 8GB memory and it's still to low to install this package. Reproducible: Always # emerge --info Portage 2.2.20.1 (python 3.4.1-final-0, default/linux/amd64/13.0, gcc-4.8.5, glibc-2.20-r2, 2.6.32-40-pve x86_64) ================================================================= System uname: Linux-2.6.32-40-pve-x86_64-Intel-R-_Xeon-R-_CPU_E5620_@_2.40GHz-with-gentoo-2.2 KiB Mem: 8388608 total, 8303548 free KiB Swap: 524288 total, 507584 free Timestamp of repository gentoo: Fri, 18 Sep 2015 06:00:01 +0000 sh bash 4.3_p39 ld GNU ld (Gentoo 2.24 p1.4) 2.24 app-shells/bash: 4.3_p39::gentoo dev-lang/perl: 5.20.2::gentoo dev-lang/python: 2.7.9-r1::gentoo, 3.4.1::gentoo dev-util/pkgconfig: 0.28-r2::gentoo sys-apps/baselayout: 2.2::gentoo sys-apps/openrc: 0.17::gentoo sys-apps/sandbox: 2.6-r1::gentoo sys-devel/autoconf: 2.69::gentoo sys-devel/automake: 1.14.1::gentoo, 1.15::gentoo sys-devel/binutils: 2.24-r3::gentoo sys-devel/gcc: 4.8.5::gentoo sys-devel/gcc-config: 1.7.3::gentoo sys-devel/libtool: 2.4.6::gentoo sys-devel/make: 4.1-r1::gentoo sys-kernel/linux-headers: 3.18::gentoo (virtual/os-headers) sys-libs/glibc: 2.20-r2::gentoo Repositories: gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://backup/gentoo-portage priority: -1000 ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=core2 -mtune=native -fno-unwind-tables -fno-asynchronous-unwind-tables" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe -march=core2 -mtune=native -fno-unwind-tables -fno-asynchronous-unwind-tables" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs collision-protect config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.UTF-8" LC_ALL="en_US.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j16" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_EXTRA_OPTS="-O" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" USE="acl amd64 bash-completion caps cli cracklib crypt cxx dri iconv logrotate mmap mmx mmxext modules multilib ncurses nls nptl openmp pam pcre readline seccomp session sse sse2 sse3 ssl ssse3 threads threadsafe unicode urandom vim vim-pager vim-syntax zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MPMS="prefork" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en pl" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby20 ruby21" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, USE_PYTHON