Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 213863 - net-libs/libwww-5.4.0-r7 parallel make fails
Summary: net-libs/libwww-5.4.0-r7 parallel make fails
Status: RESOLVED WORKSFORME
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Library (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: No maintainer - Look at https://wiki.gentoo.org/wiki/Project:Proxy_Maintainers if you want to take care of it
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-03-18 19:20 UTC by Duncan
Modified: 2012-03-19 09:15 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Duncan 2008-03-18 19:20:22 UTC
This one is probably a dup of bug #167354, which was closed as NEEDINFO, with the reporter saying it worked with MAKEOPTS=-j1 later.  I'm running MAKEOPTS="-j -l15", so there's lots of room for parallel make errors and I got one here.  Compiling with -j1 eliminates the issue here as it did for him, so it sure looks like a parallel make problem.  Let's see if I can deliver a more usable log segment, however. =8^)

Along about line 803 of the emerge log (long lines truncated but target object noted in brackets)...

mv -f .deps/HTWriter.Tpo .deps/HTWriter.Plo
 x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../.. [THInit.o]
HTBind.c: In function 'HTBind_add':
HTBind.c:195: warning: pointer targets in assignment differ in signedness
HTBind.c: In function 'HTBind_getFormat':
HTBind.c:457: warning: pointer targets in assignment differ in signedness
 x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../.. [HTBInit.o]
mv -f .deps/HTHeader.Tpo .deps/HTHeader.Plo
 x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../.. [HTMLPDTD.o]
 x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../.. [HTFile.o]
 x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../.. [HTStyle.o]
 x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../.. [HTBind.o]
mv -f .deps/HTWriter.Tpo .deps/HTWriter.Plo
 x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../.. [HTFSave.o]
mv -f .deps/HTSocket.Tpo .deps/HTSocket.Plo
 x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../.. [HTDescpt.o]
mv -f .deps/HTAAUtil.Tpo .deps/HTAAUtil.Plo
mv: cannot stat `.deps/HTWriter.Tpo': No such file or directory
 x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../.. [HTGuess.o]
 x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../.. [HTPlain.o]
 x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../.. [HTIcons.o]
 x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../.. [HText.o]
mv -f .deps/HTCookie.Tpo .deps/HTCookie.Plo
 x86_64-pc-linux-gnu-gcc -DHAVE_CONFIG_H -I. -I../.. [HTTeXGen.o]
mv -f .deps/HTSChunk.Tpo .deps/HTSChunk.Plo
mv -f .deps/HTTPRes.Tpo .deps/HTTPRes.Plo
mv -f .deps/HTXParse.Tpo .deps/HTXParse.Plo
make[8]: *** [HTWriter.lo] Error 1
make[8]: *** Waiting for unfinished jobs....

So, it tries to move HTWriter.Tpo to HTWriter.Plo, twice, the second time failing as it has already been moved, thereby triggering the error.  Apparently there are two dependencies on the file, with parallel make causing them to both try to move it at ~ the same time, under the right conditions.

Hopefully that's enough to sort things.  Let me know if you need full emerge --info and/or the full merge log, but it's ~amd64 on no-multilib here, if it makes a difference.

Duncan
Comment 1 Matthew Gregory Sr. 2008-11-16 02:04:42 UTC
files that would have to be patched to enable -j*

========begin file list==========
ComLine/src/Makefile.in
Library/src/SSL/Makefile.in
Library/src/Makefile.in
Library/cvs2sql/Makefile.in
Library/Examples/Makefile.in
LineMode/src/Makefile.in
PICS-client/src/Makefile.in
Robot/src/Makefile.in
aclocal.m4
autom4te.cache/output.0
autom4te.cache/output.1
autom4te.cache/traces.0
config/ltmain.sh
config/ltconfig
configure
modules/md5/Makefile.in
modules/expat/xmltok/Makefile.in
modules/expat/xmlparse/Makefile.in
=======================end file list==================

so instead I'm adding -j1 to the emake call in r8 of the ebuild.

bug #246978 fixes this bug

please close.
Comment 2 Duncan 2008-11-16 06:05:00 UTC
(In reply to comment #1)
> so instead I'm adding -j1 to the emake call in r8 of the ebuild.

> bug #246978 fixes this bug
> 
> please close.

Except it doesn't fix it, it works around it by adding -j1... which is acceptable in the short term, but doesn't fix the bug.

AFAIK the suggested solution, if a package maintainer doesn't have the time or skills to fix it properly and has thus simply instituted a workaround, is to use the whiteboard or keywords to indicate status so the bug can be filtered on in searches, but to keep it open until fixed properly.  It /may/ also be assignable to someone with deeper skills in the area (I know flameeyes is asking for that with --as-needed, I'm not sure about -j1) if the maintainer wants to get the open bug off his assignment list, but I'm not sure who on -j1 bugs.

But I'm glad someone's at least getting the workaround in there.  The bug sat for six months without even that, and getting the workaround in there is better than having people keep stumbling on it.

BTW, no offense but you don't have a Gentoo address and I don't know you from Adam.  Are you working with the herd as a kind of proxy maintainer?  You're not listed in metadata.xml as such.  I do see you're doing a lot of work on the package in the other bug.  Thanks.  It looks like the package needed some lovin' so it's good to see it getting it.  But either here or perhaps better there, it'd be useful to those of us with related bugs to give at least a sentence or two telling why we should be trusting someone without a Gentoo email address, who obviously isn't familiar with the "don't close parallel make bugs when all you've done it worked around it using -j1" rule I've seen on some of my other parallel make bugs.

Duncan
Comment 3 Matthew Gregory Sr. 2008-11-16 06:32:46 UTC
(In reply to comment #2)
> (In reply to comment #1)
> > so instead I'm adding -j1 to the emake call in r8 of the ebuild.
> 
> > bug #246978 fixes this bug
> > 
> > please close.
> 
> Except it doesn't fix it, it works around it by adding -j1... which is
> acceptable in the short term, but doesn't fix the bug.
> 
> AFAIK the suggested solution, if a package maintainer doesn't have the time or
> skills to fix it properly and has thus simply instituted a workaround, is to
> use the whiteboard or keywords to indicate status so the bug can be filtered on
> in searches, but to keep it open until fixed properly.  It /may/ also be
> assignable to someone with deeper skills in the area (I know flameeyes is
> asking for that with --as-needed, I'm not sure about -j1) if the maintainer
> wants to get the open bug off his assignment list, but I'm not sure who on -j1
> bugs.
> 
> But I'm glad someone's at least getting the workaround in there.  The bug sat
> for six months without even that, and getting the workaround in there is better
> than having people keep stumbling on it.
> 
> BTW, no offense but you don't have a Gentoo address and I don't know you from
> Adam.  Are you working with the herd as a kind of proxy maintainer?  You're not
> listed in metadata.xml as such.  I do see you're doing a lot of work on the
> package in the other bug.  Thanks.  It looks like the package needed some
> lovin' so it's good to see it getting it.  But either here or perhaps better
> there, it'd be useful to those of us with related bugs to give at least a
> sentence or two telling why we should be trusting someone without a Gentoo
> email address, who obviously isn't familiar with the "don't close parallel make
> bugs when all you've done it worked around it using -j1" rule I've seen on some
> of my other parallel make bugs.
> 
> Duncan
> 

I am fully aware it's a workaround and not a fix, so if you want to keep the bug open then that's fine with me.

I don't expect anyone to trust me, I am not a gentoo dev just a very experienced user.  I cannot commit to portage but I can submit fixes, so that's what I'm doing.  I came into this because I don't like leaving untidy messes behind me and I was working on getting amaya working, which needs webdav in libwww, which wasn't there, etc...

as far as 'trust' goes, feel free to open up the ebuild ;-)
Comment 4 Duncan 2008-11-16 06:55:45 UTC
(In reply to comment #3)

> I don't expect anyone to trust me, I am not a gentoo dev just a very
> experienced user.

Thanks.  Somewhat like me, then, only apparently a level (more?) better, as I couldn't have traced that list of files as you did. =:^)
Comment 5 Matthew Gregory Sr. 2008-11-16 08:33:32 UTC
(In reply to comment #4)
> (In reply to comment #3)
> 
> > I don't expect anyone to trust me, I am not a gentoo dev just a very
> > experienced user.
> 
> Thanks.  Somewhat like me, then, only apparently a level (more?) better, as I
> couldn't have traced that list of files as you did. =:^)
> 

It wasn't too hard since I knew what I was looking for and used recursive grep.  The main reason I didn't go in and actually fix this is that the sources for libwww have been unmaintained since 2002 and this is pretty much a global autoconf/automake change.  Rather than fixing this in gentoo it would be much better to go ahead and fix it in libwww but I have enough on my plate at the moment.  Especially if you consider how many bugfixes are open on that project...
Comment 6 kfm 2008-11-17 05:22:05 UTC
> email address, who obviously isn't familiar with the "don't close parallel make
> bugs when all you've done it worked around it using -j1" rule I've seen on some
> of my other parallel make bugs.

As far as I can tell, the rule in question is based upon a set of best practices suggested by flameeyes:

http://blog.flameeyes.eu/2008/10/29/for-a-parallel-world-theory-lesson-n-2-handling-broken-ebuilds

As he also offers to help fix such bugs in the event that no-one else knows how to do so, I'm taking the liberty of copying him in on this bug :)
Comment 7 Matthew Gregory Sr. 2008-11-17 06:03:48 UTC
(In reply to comment #6)
> > email address, who obviously isn't familiar with the "don't close parallel make
> > bugs when all you've done it worked around it using -j1" rule I've seen on some
> > of my other parallel make bugs.
> 
> As far as I can tell, the rule in question is based upon a set of best
> practices suggested by flameeyes:
> 
> http://blog.flameeyes.eu/2008/10/29/for-a-parallel-world-theory-lesson-n-2-handling-broken-ebuilds
> 
> As he also offers to help fix such bugs in the event that no-one else knows how
> to do so, I'm taking the liberty of copying him in on this bug :)
> 

Much appreciated, especially for the clarification.
Comment 8 Diego Elio Pettenò (RETIRED) gentoo-dev 2008-11-17 07:52:52 UTC
Ohhh libwww... my bane.... :P

I guess I'll have to take a look at this together with bug #246970 ;)
Comment 9 Pacho Ramos gentoo-dev 2012-03-18 17:19:23 UTC
Is this still valid with latest patchset?
Comment 10 Duncan 2012-03-19 09:15:32 UTC
(In reply to comment #9)
> Is this still valid with latest patchset?

RESOLVED/WORKSFORME, as nothing's pulling it in any more here so I'm not rebuilding it regularly any more, but I did two test builds, one fresh and a second one with the updated ccache from the first one, and neither one triggered any parallel-make issues for me.

Thanks. pacho, for the orphaned-package followup. =:^)

Duncan