Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 396375 - sys-apps/portage-2.1.10.41: does not obey parallel-install when --jobs is enabled
Summary: sys-apps/portage-2.1.10.41: does not obey parallel-install when --jobs is ena...
Status: RESOLVED DUPLICATE of bug 262145
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-12-29 02:08 UTC by Matthew Thode ( prometheanfire )
Modified: 2011-12-29 03:14 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matthew Thode ( prometheanfire ) archtester Gentoo Infrastructure gentoo-dev Security 2011-12-29 02:08:00 UTC
this occurs if parallel-install is enabled disabled or absent.  --jobs also needs to be enabled to process more then one job at a time.  This is especially bad if you compile and install to binary.


Reproducible: Always

Steps to Reproduce:
1. set --jobs=5
2. emerge world
3. watch as it tries to install every package at the same time.
Actual Results:  
>>> Emerging binary (1 of 61) app-arch/bzip2-1.0.6-r3
>>> Emerging binary (2 of 61) sys-devel/gnuconfig-20110202
>>> Emerging binary (3 of 61) sys-apps/gawk-3.1.8
>>> Emerging binary (4 of 61) sys-apps/kbd-1.15.3
>>> Emerging binary (5 of 61) sys-apps/net-tools-1.60_p20110409135728
>>> Emerging binary (6 of 61) sys-apps/paxctl-0.5
>>> Emerging binary (7 of 61) sys-devel/patch-2.6.1
>>> Emerging binary (8 of 61) app-arch/tar-1.23-r2
>>> Emerging binary (9 of 61) app-arch/xz-utils-5.0.3
>>> Emerging binary (10 of 61) app-arch/gzip-1.4
>>> Emerging binary (11 of 61) sys-apps/which-2.20
>>> Emerging binary (12 of 61) sys-apps/diffutils-3.0
>>> Emerging binary (13 of 61) net-misc/dhcpcd-5.2.12
>>> Emerging binary (14 of 61) sys-apps/mlocate-0.23.1-r1
>>> Emerging binary (15 of 61) sys-apps/util-linux-2.19.1-r1
>>> Emerging binary (16 of 61) app-shells/bash-4.1_p9
>>> Emerging binary (17 of 61) sys-apps/texinfo-4.13
>>> Emerging binary (18 of 61) sys-process/psmisc-22.14
>>> Emerging binary (19 of 61) sys-devel/make-3.82-r1
>>> Emerging binary (20 of 61) sys-apps/findutils-4.4.2
>>> Emerging binary (21 of 61) virtual/os-headers-0
>>> Installing (13 of 61) net-misc/dhcpcd-5.2.12
>>> Emerging binary (22 of 61) sys-process/procps-3.2.8_p11
>>> Installing (14 of 61) sys-apps/mlocate-0.23.1-r1
>>> Emerging binary (23 of 61) sys-apps/module-init-tools-3.16-r1
>>> Emerging binary (24 of 61) sys-process/htop-0.9-r2
>>> Emerging binary (25 of 61) sys-apps/sed-4.2.1
>>> Emerging binary (26 of 61) net-misc/rsync-3.0.8
>>> Emerging binary (27 of 61) app-portage/eix-0.23.3-r1
>>> Installing (24 of 61) sys-process/htop-0.9-r2
>>> Installing (27 of 61) app-portage/eix-0.23.3-r1
>>> Installing (4 of 61) sys-apps/kbd-1.15.3
>>> Installing (1 of 61) app-arch/bzip2-1.0.6-r3
>>> Emerging binary (28 of 61) dev-libs/elfutils-0.149
>>> Emerging binary (29 of 61) sys-apps/grep-2.5.4-r1
>>> Installing (28 of 61) dev-libs/elfutils-0.149
>>> Installing (2 of 61) sys-devel/gnuconfig-20110202
>>> Installing (5 of 61) sys-apps/net-tools-1.60_p20110409135728
>>> Installing (3 of 61) sys-apps/gawk-3.1.8
>>> Installing (6 of 61) sys-apps/paxctl-0.5
>>> Installing (7 of 61) sys-devel/patch-2.6.1
>>> Installing (8 of 61) app-arch/tar-1.23-r2
>>> Installing (9 of 61) app-arch/xz-utils-5.0.3
>>> Installing (10 of 61) app-arch/gzip-1.4
>>> Installing (11 of 61) sys-apps/which-2.20
>>> Installing (12 of 61) sys-apps/diffutils-3.0
...

Expected Results:  
--jobs should only run one install at a time when parallel-install is disabled/absent.
--jobs should only run how ever many jobs are set number of installs at a time when parallel-install is set.
Comment 1 Zac Medico gentoo-dev 2011-12-29 02:30:05 UTC
For system packages and their deep dependencies, the code from bug 256616 leads to the confusing output because the merge of these packages is delayed. It's not really doing more than 5 extraction/pkg_setup phases at a time, but after extraction/pkg_setup, the "Installing" part is delayed for safety reasons related to bug 256616.
Comment 2 Matthew Thode ( prometheanfire ) archtester Gentoo Infrastructure gentoo-dev Security 2011-12-29 02:32:42 UTC
The real problem might be the packages don't wait for their deps to finish installing before going on to emerge themselves.

I noticed this because I had to emerge -uDNav world a few times before it would go all the way through.  It failed at random places

The problem of parallel-install not being obeyed still remains.
Comment 3 Matthew Thode ( prometheanfire ) archtester Gentoo Infrastructure gentoo-dev Security 2011-12-29 02:37:17 UTC
(In reply to comment #1)
> For system packages and their deep dependencies, the code from bug 256616 leads
> to the confusing output because the merge of these packages is delayed. It's
> not really doing more than 5 extraction/pkg_setup phases at a time, but after
> extraction/pkg_setup, the "Installing" part is delayed for safety reasons
> related to bug 256616.

well, maybe parallel-install is being obeyed but hidden behind the display bug (saying it is installing things when it is not).
Comment 4 Zac Medico gentoo-dev 2011-12-29 02:43:24 UTC
(In reply to comment #3)
> well, maybe parallel-install is being obeyed but hidden behind the display bug
> (saying it is installing things when it is not).

It's not, but there's a reason for it. For system packages and their deep dependencies, parallel-install is effectively disabled. This is another side-ffect from the code for bug 256616 and bug 259954.

The output is confusing, but it's not a display bug. The "Installing" message indicates exactly when something is going to be installed. It's just delayed due bug 256616 and bug 259954.
Comment 5 Zac Medico gentoo-dev 2011-12-29 02:45:55 UTC
People ask about this behavior frequently. Here's the relevant section from the FAQ:

When packages are built in parallel with the --jobs option, why aren't some packages installed immediately after they have finished building? They are installed only after a long delay.

As a safety precaution, installation actions for system packages and their deep dependecies are executed only when no other packages are building. This behavior is required in order to avoid cases like bug 256616 (unspecified system dependencies) and bug 259954 (temporarily unsatisfied system dependencies).
Comment 6 Zac Medico gentoo-dev 2011-12-29 03:14:16 UTC

*** This bug has been marked as a duplicate of bug 262145 ***