Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 560752 - dev-python/lockfile-{0.9.1-r1,0.10.2} can't install due to very high memory requirements
Summary: dev-python/lockfile-{0.9.1-r1,0.10.2} can't install due to very high memory r...
Status: RESOLVED TEST-REQUEST
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-09-18 08:35 UTC by Marcin Mirosław
Modified: 2015-11-04 02:01 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Marcin Mirosław 2015-09-18 08:35:17 UTC
Calculating dependencies... done!
[ebuild  N     ] dev-python/lockfile-0.10.2::gentoo  USE="-doc {-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB

Total: 1 package (1 new), Size of downloads: 0 KiB

Would you like to merge these packages? [Yes/No] 

>>> Verifying ebuild manifests

>>> Emerging (1 of 1) dev-python/lockfile-0.10.2::gentoo
 * lockfile-0.10.2.tar.gz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                 [ ok ]
>>> Unpacking source...
>>> Unpacking lockfile-0.10.2.tar.gz to /var/tmp/portage/dev-python/lockfile-0.10.2/work
>>> Source unpacked in /var/tmp/portage/dev-python/lockfile-0.10.2/work
>>> Preparing source in /var/tmp/portage/dev-python/lockfile-0.10.2/work/lockfile-0.10.2 ...
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/dev-python/lockfile-0.10.2/work/lockfile-0.10.2 ...
>>> Source configured.
>>> Compiling source in /var/tmp/portage/dev-python/lockfile-0.10.2/work/lockfile-0.10.2 ...
 * python3_4: running distutils-r1_run_phase distutils-r1_python_compile
/usr/bin/python3.4 setup.py build
/var/tmp/portage/dev-python/lockfile-0.10.2/temp/environment: line 1341:  5767 Killed                  "${@}"
 * ERROR: dev-python/lockfile-0.10.2::gentoo failed (compile phase):
 *   (no error message)
 * 
 * Call stack:
 *     ebuild.sh, line   93:  Called src_compile
 *   environment, line 3492:  Called distutils-r1_src_compile
 *   environment, line  866:  Called _distutils-r1_run_foreach_impl 'distutils-r1_python_compile'
 *   environment, line  297:  Called python_foreach_impl 'distutils-r1_run_phase' 'distutils-r1_python_compile'
 *   environment, line 3059:  Called multibuild_foreach_variant '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'distutils-r1_python_compile'
 *   environment, line 2176:  Called _multibuild_run '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'distutils-r1_python_compile'
 *   environment, line 2174:  Called _python_multibuild_wrapper 'distutils-r1_run_phase' 'distutils-r1_python_compile'
 *   environment, line  549:  Called distutils-r1_run_phase 'distutils-r1_python_compile'
 *   environment, line  857:  Called distutils-r1_python_compile
 *   environment, line  736:  Called esetup.py 'build'
 *   environment, line 1345:  Called die
 * The specific snippet of code:
 *       "${@}" || die
 * 
 * If you need support, post the output of `emerge --info '=dev-python/lockfile-0.10.2::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=dev-python/lockfile-0.10.2::gentoo'`.
 * The complete build log is located at '/var/tmp/portage/dev-python/lockfile-0.10.2/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/dev-python/lockfile-0.10.2/temp/environment'.
 * Working directory: '/var/tmp/portage/dev-python/lockfile-0.10.2/work/lockfile-0.10.2'
 * S: '/var/tmp/portage/dev-python/lockfile-0.10.2/work/lockfile-0.10.2'



VPS has 8GB memory and it's still to low to install this package.

Reproducible: Always




# emerge --info
Portage 2.2.20.1 (python 3.4.1-final-0, default/linux/amd64/13.0, gcc-4.8.5, glibc-2.20-r2, 2.6.32-40-pve x86_64)
=================================================================
System uname: Linux-2.6.32-40-pve-x86_64-Intel-R-_Xeon-R-_CPU_E5620_@_2.40GHz-with-gentoo-2.2
KiB Mem:     8388608 total,   8303548 free
KiB Swap:     524288 total,    507584 free
Timestamp of repository gentoo: Fri, 18 Sep 2015 06:00:01 +0000
sh bash 4.3_p39
ld GNU ld (Gentoo 2.24 p1.4) 2.24
app-shells/bash:          4.3_p39::gentoo
dev-lang/perl:            5.20.2::gentoo
dev-lang/python:          2.7.9-r1::gentoo, 3.4.1::gentoo
dev-util/pkgconfig:       0.28-r2::gentoo
sys-apps/baselayout:      2.2::gentoo
sys-apps/openrc:          0.17::gentoo
sys-apps/sandbox:         2.6-r1::gentoo
sys-devel/autoconf:       2.69::gentoo
sys-devel/automake:       1.14.1::gentoo, 1.15::gentoo
sys-devel/binutils:       2.24-r3::gentoo
sys-devel/gcc:            4.8.5::gentoo
sys-devel/gcc-config:     1.7.3::gentoo
sys-devel/libtool:        2.4.6::gentoo
sys-devel/make:           4.1-r1::gentoo
sys-kernel/linux-headers: 3.18::gentoo (virtual/os-headers)
sys-libs/glibc:           2.20-r2::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://backup/gentoo-portage
    priority: -1000

ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=core2 -mtune=native -fno-unwind-tables -fno-asynchronous-unwind-tables"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=core2 -mtune=native -fno-unwind-tables -fno-asynchronous-unwind-tables"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs collision-protect config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j16"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_EXTRA_OPTS="-O"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
USE="acl amd64 bash-completion caps cli cracklib crypt cxx dri iconv logrotate mmap mmx mmxext modules multilib ncurses nls nptl openmp pam pcre readline seccomp session sse sse2 sse3 ssl ssse3 threads threadsafe unicode urandom vim vim-pager vim-syntax zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MPMS="prefork" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="en pl" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby20 ruby21" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, USE_PYTHON
Comment 1 Marcin Mirosław 2015-09-18 08:39:32 UTC
# dmesg 
[    0.000000] Out of memory in UB 108: OOM killed process 2525 (python3.4) score 0 vm:3254864kB, rss:2057260kB, swap:466960kB
[    0.000000] Out of memory in UB 108: OOM killed process 3150 (python3.4) score 0 vm:5220936kB, rss:4157420kB, swap:463504kB
[    0.000000] Out of memory in UB 108: OOM killed process 3831 (python3.4) score 0 vm:10332724kB, rss:8344828kB, swap:457360kB
[    0.000000] Out of memory in UB 108: OOM killed process 4571 (python3.4) score 0 vm:10332568kB, rss:8342296kB, swap:454604kB
[    0.000000] Out of memory in UB 108: OOM killed process 5235 (python3.4) score 0 vm:10332764kB, rss:8337504kB, swap:454608kB
[    0.000000] Out of memory in UB 108: OOM killed process 5767 (python3.4) score 0 vm:10332756kB, rss:8336628kB, swap:451936kB
[    0.000000] Out of memory in UB 108: OOM killed process 6854 (python3.4) score 0 vm:41003604kB, rss:30628828kB, swap:450996kB


30GB is also too low:/
Comment 2 Marcin Mirosław 2015-09-18 08:59:23 UTC
The same with python3.3.
Comment 3 Alex Xu (Hello71) 2015-09-22 23:34:19 UTC
please try using python_targets_python2_7 only for this package.
Comment 4 Mike Gilbert gentoo-dev 2015-09-22 23:47:57 UTC
This is likely a bug in setuptools; you are not the first to run into it. It shows up in random packages when we miss a dependency somewhere.
Comment 5 Justin Lecher (RETIRED) gentoo-dev 2015-09-23 06:38:26 UTC
In a stage3 chroot it installs fine. No clue what package is missing here.
Comment 6 Alexander Tsoy 2015-09-23 07:44:10 UTC
(In reply to Mike Gilbert from comment #4)
Yeah. I reproduced this bug only when I hadn't installed dev-python/pbr and run "ebuild lockfile-0.10.2.ebuild compile" (emerge --nodeps ... should do the same trick) :)
Comment 7 Justin Lecher (RETIRED) gentoo-dev 2015-09-23 07:58:43 UTC
(In reply to Alexander Tsoy from comment #6)
> (In reply to Mike Gilbert from comment #4)
> Yeah. I reproduced this bug only when I hadn't installed dev-python/pbr and
> run "ebuild lockfile-0.10.2.ebuild compile" (emerge --nodeps ... should do
> the same trick) :)

But pbr is in the depgraph of lockfile so it should be fine.
Comment 8 Alexander Tsoy 2015-09-23 08:10:58 UTC
(In reply to Justin Lecher from comment #7)

Yes. My previous comment was just a confirmation that missing deps can cause such strange behaviour.
Comment 9 Marcin Mirosław 2015-09-23 15:30:07 UTC
(In reply to Justin Lecher from comment #7)
> (In reply to Alexander Tsoy from comment #6)
> > (In reply to Mike Gilbert from comment #4)
> > Yeah. I reproduced this bug only when I hadn't installed dev-python/pbr and
> > run "ebuild lockfile-0.10.2.ebuild compile" (emerge --nodeps ... should do
> > the same trick) :)
> 
> But pbr is in the depgraph of lockfile so it should be fine.

Are you sure? I just untarred stage3:
# emerge -pv dev-python/lockfile

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] dev-python/lockfile-0.9.1-r1::gentoo  USE="-doc {-test}" PYTHON_TARGETS="python2_7 python3_4 (-pypy) -python3_3" 0 KiB

Total: 1 package (1 new), Size of downloads: 0 KiB

marcinm / # emerge -pv --emptytree --tree dev-python/lockfile|grep pbr
marcinm / #

or I'm checking this wrongly:)
When I was hitted by this problem I've seen circular dependency while trying emerge dev-python/pip (if I remember correctly name of needed package).
And now I can't reproduce problem:(
Comment 10 Justin Lecher (RETIRED) gentoo-dev 2015-09-23 16:24:22 UTC
(In reply to Marcin Mirosław from comment #9)
> Are you sure? I just untarred stage3:
> # emerge -pv dev-python/lockfile
> 
> These are the packages that would be merged, in order:
> 
> Calculating dependencies... done!
> [ebuild  N     ] dev-python/lockfile-0.9.1-r1::gentoo  USE="-doc {-test}"
> PYTHON_TARGETS="python2_7 python3_4 (-pypy) -python3_3" 0 KiB

Can't be:
# emerge -pvt dev-python/lockfile

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
[nomerge       ] dev-python/lockfile-0.10.2::gentoo  USE="-doc {-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3"
[nomerge       ]  dev-python/pbr-1.6.0::gentoo  USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3"
[ebuild  N     ]   dev-python/pip-7.1.2::gentoo  PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB
[ebuild  N     ]    dev-python/CacheControl-0.11.5::gentoo  USE="-redis {-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB
[ebuild  N     ]     dev-python/lockfile-0.10.2::gentoo  USE="-doc {-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB
[ebuild  N     ]      dev-python/pbr-1.6.0::gentoo  USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB
[ebuild  N     ]     dev-python/requests-2.7.0::gentoo  PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB
[ebuild  N     ]      dev-python/chardet-2.3.0::gentoo  PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB
[ebuild  N     ]    dev-python/retrying-1.3.3::gentoo  PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB
[ebuild  N     ]    dev-python/html5lib-0.999999::gentoo  USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB
[ebuild  N     ]    dev-python/progress-1.2::gentoo  PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB
[ebuild  N     ]    dev-python/ipaddress-1.0.14::gentoo  PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB
[ebuild  N     ]    dev-python/six-1.9.0-r1::gentoo  USE="-doc {-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB
[ebuild  N     ]    dev-python/packaging-15.3::gentoo  USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB
[ebuild  N     ]     dev-python/setuptools-18.2-r2::gentoo  USE="{-test}" PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB
[ebuild  N     ]    dev-python/distlib-0.2.1::gentoo  PYTHON_TARGETS="python2_7 python3_4 -pypy -pypy3 -python3_3" 0 KiB
[ebuild  N     ]    dev-python/colorama-0.3.3::gentoo  USE="-examples" PYTHON_TARGETS="python2_7 python3_4 -pypy -python3_3" 0 KiB

> or I'm checking this wrongly:)
> When I was hitted by this problem I've seen circular dependency while trying
> emerge dev-python/pip (if I remember correctly name of needed package).
> And now I can't reproduce problem:(

All circulars have been removed here.
Comment 11 Dustin J. Mitchell 2015-09-28 21:43:21 UTC
I'm seeing the same - there's an infinite loop in the build:

...
Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/setuptools/sandbox.py", line 108, in dump
    return pickle.dumps(type), pickle.dumps(exc)
_pickle.PicklingError: Can't pickle <class 'setuptools.sandbox.UnpickleableException'>: it's not the same object as setuptools.sandbox.UnpickleableException

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/setuptools/sandbox.py", line 108, in dump
    return pickle.dumps(type), pickle.dumps(exc)
_pickle.PicklingError: Can't pickle <class 'setuptools.sandbox.UnpickleableException'>: it's not the same object as setuptools.sandbox.UnpickleableException

During handling of the above exception, another exception occurred:
...

which is likely causing the OOM.  I, too, manually installed lockfile to get around the circular dependency.

Emerging pbr with --nodeps, then re-emerging lockfile does not help (I get another OOM).
Comment 12 Justin Lecher (RETIRED) gentoo-dev 2015-09-29 06:42:09 UTC
Which versions of pbr, nose, sphinx functools32 are installed?
Comment 13 Dustin J. Mitchell 2015-09-29 12:17:33 UTC
dev-python/pbr - 0.8.2-r1
dev-python/nose - not installed
dev-python/sphinx - not installed
functools32 - no such ebuild
Comment 14 Justin Lecher (RETIRED) gentoo-dev 2015-09-29 12:38:23 UTC
(In reply to Dustin J. Mitchell from comment #13)
> dev-python/pbr - 0.8.2-r1

You are mixing arch adn ~arch which is not supported. please upgrade to latest pbr and try again.

> dev-python/nose - not installed
> dev-python/sphinx - not installed
> functools32 - no such ebuild

dev-python/functools32
Comment 15 Dustin J. Mitchell 2015-09-29 12:55:05 UTC
(In reply to Justin Lecher from comment #14)
> (In reply to Dustin J. Mitchell from comment #13)
> > dev-python/pbr - 0.8.2-r1
> 
> You are mixing arch adn ~arch which is not supported. please upgrade to
> latest pbr and try again.

I'm not sure what you mean; from eix pbr is at the latest:

[I] dev-python/pbr
     Available versions:  ~0.8.2 0.8.2-r1{tbz2} ~0.10.8 ~0.11.0 ~1.0.1 ~1.1.1 ~1.2.0 ~1.3.0 {test PYTHON_TARGETS="python2_7 python3_3 python3_4"}
     Installed versions:  0.8.2-r1{tbz2}(05:19:32 PM 05/18/2015)(-test PYTHON_TARGETS="python2_7 -python3_3 -python3_4")

pbr isn't mentioned under /etc/portage, and it's a non-testing version that's installed.

> dev-python/functools32

knuth ~ # eix dev-python/functools32
No matches found.
Comment 16 Justin Lecher (RETIRED) gentoo-dev 2015-09-29 13:00:15 UTC
(In reply to Dustin J. Mitchell from comment #15)
> (In reply to Justin Lecher from comment #14)
> > (In reply to Dustin J. Mitchell from comment #13)
> > > dev-python/pbr - 0.8.2-r1
> > 
> > You are mixing arch adn ~arch which is not supported. please upgrade to
> > latest pbr and try again.
> 
> I'm not sure what you mean; from eix pbr is at the latest:
> 
> [I] dev-python/pbr
>      Available versions:  ~0.8.2 0.8.2-r1{tbz2} ~0.10.8 ~0.11.0 ~1.0.1
> ~1.1.1 ~1.2.0 ~1.3.0 {test PYTHON_TARGETS="python2_7 python3_3 python3_4"}
>      Installed versions:  0.8.2-r1{tbz2}(05:19:32 PM 05/18/2015)(-test
> PYTHON_TARGETS="python2_7 -python3_3 -python3_4")
> 
> pbr isn't mentioned under /etc/portage, and it's a non-testing version
> that's installed.
> 
> > dev-python/functools32
> 
> knuth ~ # eix dev-python/functools32
> No matches found.

Please sync your tree. pbr-1.8.0 is the latest version and functools32 was added " Wed Sep 2 13:01:45 2015 +0"
Comment 17 Dustin J. Mitchell 2015-09-29 13:34:08 UTC
Indeed, setting pbr to accept ~amd64 successfully merges
Comment 18 Mike Gilbert gentoo-dev 2015-09-29 13:40:47 UTC
(In reply to Justin Lecher from comment #14)
> (In reply to Dustin J. Mitchell from comment #13)
> > dev-python/pbr - 0.8.2-r1
> 
> You are mixing arch adn ~arch which is not supported. please upgrade to
> latest pbr and try again.

While upgrading to ~arch is a useful troubleshooting step, we do in fact support mixing arch and ~arch.

It sounds like we need to adjust a dependency somewhere if a certain version of pbr is required.
Comment 19 Justin Lecher (RETIRED) gentoo-dev 2015-11-03 09:15:16 UTC
Is this still a problem?

If so, please do the following

cd /var/tmp/portage/dev-python/lockfile*/work/lockfile*
python setup.py --requires

and paste the output here
Comment 20 Dustin J. Mitchell 2015-11-04 02:01:40 UTC
Sorry, I solved it in the previous comments so I assume that info is no longer useful for you :(