Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 247370 - portage locking is wrong for parallel fetching of distfiles
Summary: portage locking is wrong for parallel fetching of distfiles
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Unclassified (show other bugs)
Hardware: All All
: High normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS, REGRESSION
Depends on:
Blocks: 210077
  Show dependency tree
 
Reported: 2008-11-18 14:40 UTC by Vincent Legoll
Modified: 2008-11-22 05:55 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Vincent Legoll 2008-11-18 14:40:55 UTC
portage locking prevents parallel fetching of distfiles.

SPECULATION:
Maybe the locking should occur on the distfiles instead of the packages themselves, and only after the checksums or actual file presence is
confirmed to be wrong.

Reproducible: Always

Steps to Reproduce:
1.emerge -DN world in one window
2. emerge -DNf in another, this one is waiting for lock files of packages that are currently being built, this should not happen.

Actual Results:  
the emerge -DNf wait for locks to be released on packages that are already present

Expected Results:  
the fetchonly should skip the files that are already there
Comment 1 Vincent Legoll 2008-11-18 14:42:25 UTC
emerge --version
Portage 2.2_rc14 (default/linux/amd64/2008.0, gcc-4.2.4, glibc-2.8_p20080602-r0, 2.6.27.4 x86_64)
Comment 2 Jeroen Roovers (RETIRED) gentoo-dev 2008-11-18 15:39:37 UTC
Looks like an unusual test to me - AFAIK having parallel emerge processes isn't the same as having parallel subprocesses (and I surely have seen some weird behaviour running parallel emerge calls that I wouldn't want to blame sys-apps/portage for).
Comment 3 Vincent Legoll 2008-11-18 16:05:18 UTC
Maybe this will become a "wont fix" but this had been working smoothly for ages and broke recently... and as it is annoying me, I wanted to report the problem...
Comment 4 Zac Medico gentoo-dev 2008-11-18 17:50:38 UTC
This case is similar to bug 245231. Like that case, it makes sense to create a separate temporary directory for --fetchonly mode.

However, I'm curious about why you run a separate emerge instance for fetching. That's supposed to be what FEATURES=parallel-fetch is for. It's enabled by default now, and /var/log/emerge-fetch.log should show the output so you can see that it's working.

In the future I'd like to add support for something like --jobs that's for parallel fetching instead of parallel building, which will allow you to control the number of fetches that will be automatically performed in parallel. It could be called --fetch-jobs.
Comment 5 Zac Medico gentoo-dev 2008-11-20 02:44:55 UTC
It's fixed in svn r12006 to share the code from bug #245231.
Comment 6 Zac Medico gentoo-dev 2008-11-22 05:55:49 UTC
This is fixed in 2.2_rc15.