Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 445236 - sys-apps/portage deadlocks due to orphan lock-helper.py process
Summary: sys-apps/portage deadlocks due to orphan lock-helper.py process
Status: RESOLVED CANTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Unspecified (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-11-29 18:54 UTC by Kerin Millar
Modified: 2012-11-29 19:25 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
Last used make.conf (make.conf.txt,543 bytes, text/plain)
2012-11-29 18:57 UTC, Kerin Millar
Details
lsof | grep /var/tmp/portage (lsof-var-tmp-portage.txt,2.92 KB, text/plain)
2012-11-29 19:02 UTC, Kerin Millar
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Kerin Millar 2012-11-29 18:54:22 UTC
In the course of conducting a new installation, I have found that portage reproducibly deadlocks during an emerge -e @world run.

I started with the stage3-amd64-20121107.tar.bz2 tarball, switched to python-2.7 and unmasked gcc-4.6 (as I always do). After installing gcc-4.6, I wanted to rebuild the other packages, so I ran:

emerge -e @world --exclude sys-devel/gcc

Thus far, I have reproduced the issue with the following combinations.

--jobs 2 and MAKEOPTS="-j12"
--jobs 4 and MAKEOPTS="-j6"

Here's an example from a recent run:

Calculating dependencies... done!
>>> Verifying ebuild manifests
>>> Starting parallel fetch
>>> Emerging (1 of 122) sys-libs/ncurses-5.9-r2
>>> Emerging (2 of 122) sys-libs/zlib-1.2.7
>>> Emerging (3 of 122) app-arch/xz-utils-5.0.4
>>> Emerging (4 of 122) sys-apps/sed-4.2.1-r1
>>> Emerging (5 of 122) sys-devel/gnuconfig-20120818
>>> Emerging (6 of 122) virtual/libiconv-0
>>> Emerging (7 of 122) dev-libs/expat-2.1.0-r2
>>> Emerging (8 of 122) app-arch/bzip2-1.0.6-r3
>>> Emerging (9 of 122) sys-devel/make-3.82-r3
>>> Emerging (10 of 122) sys-apps/findutils-4.4.2-r1
>>> Emerging (11 of 122) sys-apps/diffutils-3.2
>>> Emerging (12 of 122) sys-devel/patch-2.6.1
>>> Emerging (13 of 122) sys-devel/gcc-config-1.7.3
>>> Emerging (14 of 122) sys-libs/timezone-data-2012f
>>> Emerging (15 of 122) app-misc/editor-wrapper-4
>>> Emerging (16 of 122) dev-libs/popt-1.16-r1
>>> Emerging (17 of 122) sys-devel/binutils-config-3-r3
>>> Emerging (18 of 122) dev-libs/libffi-3.0.11
>>> Emerging (19 of 122) app-misc/mime-types-8
>>> Emerging (20 of 122) virtual/libintl-0
>>> Emerging (21 of 122) sys-apps/kbd-1.15.3
>>> Emerging (22 of 122) app-arch/tar-1.26
>>> Emerging (23 of 122) app-arch/gzip-1.5
>>> Emerging (24 of 122) sys-apps/which-2.20
>>> Emerging (25 of 122) sys-devel/autoconf-wrapper-12
>>> Installing (25 of 122) sys-devel/autoconf-wrapper-12
>>> Emerging (26 of 122) app-portage/portage-utils-0.10
>>> Emerging (27 of 122) sys-devel/automake-wrapper-7
>>> Installing (27 of 122) sys-devel/automake-wrapper-7
>>> Emerging (28 of 122) app-arch/lbzip2-2.2
>>> Installing (26 of 122) app-portage/portage-utils-0.10
>>> Installing (28 of 122) app-arch/lbzip2-2.2
>>> Installing (2 of 122) sys-libs/zlib-1.2.7
>>> Installing (5 of 122) sys-devel/gnuconfig-20120818
>>> Installing (6 of 122) virtual/libiconv-0
>>> Installing (3 of 122) app-arch/xz-utils-5.0.4
>>> Installing (4 of 122) sys-apps/sed-4.2.1-r1
>>> Installing (8 of 122) app-arch/bzip2-1.0.6-r3
>>> Installing (7 of 122) dev-libs/expat-2.1.0-r2
>>> Installing (9 of 122) sys-devel/make-3.82-r3
>>> Installing (12 of 122) sys-devel/patch-2.6.1
>>> Installing (13 of 122) sys-devel/gcc-config-1.7.3
>>> Installing (14 of 122) sys-libs/timezone-data-2012f
>>> Installing (15 of 122) app-misc/editor-wrapper-4
>>> Installing (10 of 122) sys-apps/findutils-4.4.2-r1
>>> Installing (16 of 122) dev-libs/popt-1.16-r1
>>> Installing (17 of 122) sys-devel/binutils-config-3-r3
>>> Installing (19 of 122) app-misc/mime-types-8
>>> Installing (20 of 122) virtual/libintl-0
>>> Installing (11 of 122) sys-apps/diffutils-3.2
>>> Installing (18 of 122) dev-libs/libffi-3.0.11
>>> Installing (1 of 122) sys-libs/ncurses-5.9-r2
>>> Installing (21 of 122) sys-apps/kbd-1.15.3
>>> Installing (24 of 122) sys-apps/which-2.20
>>> Installing (23 of 122) app-arch/gzip-1.5
>>> Installing (22 of 122) app-arch/tar-1.26
>>> Emerging (29 of 122) sys-apps/texinfo-4.13
>>> Emerging (30 of 122) sys-libs/readline-6.2_p1
>>> Emerging (31 of 122) sys-apps/coreutils-8.16
>>> Emerging (32 of 122) app-misc/pax-utils-0.4
>>> Emerging (33 of 122) sys-apps/file-5.11
>>> Emerging (34 of 122) sys-libs/cracklib-2.8.19
>>> Emerging (35 of 122) virtual/libffi-3.0.11
>>> Emerging (36 of 122) net-misc/rsync-3.0.9-r2
>>> Emerging (37 of 122) sys-apps/module-init-tools-3.16-r2
>>> Emerging (38 of 122) sys-process/procps-3.2.8_p11
>>> Emerging (39 of 122) sys-devel/m4-1.4.16
>>> Installing (39 of 122) sys-devel/m4-1.4.16
>>> Emerging (40 of 122) dev-libs/gmp-5.0.2_p1
>>> Emerging (41 of 122) sys-devel/bison-2.4.3
>>> Installing (41 of 122) sys-devel/bison-2.4.3
>>> Emerging (42 of 122) virtual/yacc-0
>>> Installing (42 of 122) virtual/yacc-0
>>> Installing (32 of 122) app-misc/pax-utils-0.4
>>> Installing (30 of 122) sys-libs/readline-6.2_p1
>>> Installing (33 of 122) sys-apps/file-5.11
>>> Installing (35 of 122) virtual/libffi-3.0.11
>>> Installing (29 of 122) sys-apps/texinfo-4.13
>>> Installing (34 of 122) sys-libs/cracklib-2.8.19
>>> Installing (37 of 122) sys-apps/module-init-tools-3.16-r2
>>> Installing (38 of 122) sys-process/procps-3.2.8_p11
>>> Installing (36 of 122) net-misc/rsync-3.0.9-r2
>>> Installing (31 of 122) sys-apps/coreutils-8.16
>>> Installing (40 of 122) dev-libs/gmp-5.0.2_p1
>>> Emerging (43 of 122) app-admin/eselect-1.3.1
>>> Emerging (44 of 122) sys-apps/debianutils-4.3.2
>>> Emerging (45 of 122) sys-apps/sandbox-2.5
>>> Emerging (46 of 122) virtual/modutils-0
>>> Emerging (47 of 122) sys-apps/gawk-4.0.1
>>> Emerging (48 of 122) dev-libs/mpfr-3.0.1_p4
>>> Emerging (49 of 122) app-shells/bash-4.2_p37
>>> Emerging (50 of 122) app-admin/eselect-vi-1.1.7-r1

At this point, the process tree looks like:

screen
  zsh
    bash
      emerge
        python2.7

And the python2.7 process looks like this:

/usr/bin/python2.7 /var/tmp/portage/._portage_reinstall_.czcWaN/bin/lock-helper.py /var/tmp/portage/app-admin/eselect-vi-1.1.7-r1

If I Ctrl+C then --resume, it hangs again. Thus far, that's been on eselect-vi.

I shall attach make.conf and lsof output, as requested in #gentoo-portage.

Incidentally, the last time I ran through this procedure was in setting up a system with almost identical specs. At that time, I think portage-2.1.10.44 was current and I did not encounter any locking issues.
Comment 1 Kerin Millar 2012-11-29 18:57:57 UTC
Created attachment 330976 [details]
Last used make.conf
Comment 2 Kerin Millar 2012-11-29 19:02:32 UTC
Created attachment 330978 [details]
lsof | grep /var/tmp/portage
Comment 3 Kerin Millar 2012-11-29 19:03:47 UTC
Just to add that I didn't start with FEATURES="parallel-install". That was merely something I recently tried to see if if influences the behaviour of portage with respect to this issue. Apparently, it doesn't.
Comment 4 Zac Medico gentoo-dev 2012-11-29 19:09:07 UTC
Are you sure that pid 21225 is not just an orphan process that needs to be killed? It seems likely, because the emerge pid 24497 that's trying to lock the same file has a higher pid.
Comment 5 Kerin Millar 2012-11-29 19:22:05 UTC
Well spotted. You're absolutely right. Sorry for the false alarm.
Comment 6 Zac Medico gentoo-dev 2012-11-29 19:25:18 UTC
No problem.