Bug 487056 – multiprocessing.eclass relies on sane fifos behavior: opening for simultaneous r/w behavior

• Status: RESOLVED FIXED
• Product: Gentoo/Alt

[Description Alan Hourihane 2013-10-05 23:26:00 UTC]

The usage of the read/write fifos at the end of multiprocessing.eclass results in failure on some non-Linux OS's due to POSIX claiming the behaviour is undefined and not guaranteed.

My OS fails with this usage and cannot support the behaviour currently.

http://pubs.opengroup.org/onlinepubs/7908799/xsh/open.html

The above document explicitly says....

O_RDWR
    Open for reading and writing. The result is undefined if this flag is applied to a FIFO.

Any chance we can fix multiprocessing.eclass for other OS's ?

[Comment 1 Alan Hourihane 2013-10-06 13:53:55 UTC]

Created attachment 360230 [details, diff]
multiprocessing.eclass patch

Use two file descriptors for FIFO. Fixes my non-Linux OS.

[Comment 2 Alan Hourihane 2013-10-06 13:54:18 UTC]

Created attachment 360232 [details, diff]
helper-functions.sh patch

Use two file descriptors for FIFO. Fixes my non-Linux OS.

[Comment 3 Alan Hourihane 2013-10-06 13:54:41 UTC]

These two patches fixes the problem here, and it still works on Linux.

Any chance these are o.k. to commit ?

Thanks.

[Comment 4 Alan Hourihane 2013-10-06 13:56:09 UTC]

Oh, forgot to also mention this patch fixes the FIFO permissions to be 600.

[Comment 5 Alan Hourihane 2013-10-06 15:02:09 UTC]

Just to note as well, that the FIFO is still opened R/W, that's because it's the only way to get bash to do non-blocking IO. But because we have a reader and a writer on two seperate file descriptors, we end up just using a write-only and a read-only path. Which is what's desired here.

Thanks.

[Comment 6 Fabian Groffen 2013-10-06 18:39:14 UTC]

Prefix directly gets this eclass from gx86.

Helper functions is a portage bug/patch, will address it in Prefix branch now.

[Comment 7 SpanKY 2013-10-12 21:13:05 UTC]

(In reply to Alan Hourihane from comment #0)

the OS you've chosen is a pos.  but POSIX does make it clear that some OS's are allowed to be moronic, and your patch doesn't look terrible.

http://sources.gentoo.org/eclass/multiprocessing.eclass?r1=1.2&r2=1.3

[Comment 8 Alan Hourihane 2013-10-12 22:21:04 UTC]

Thanks for committing !

[Comment 9 Alan Hourihane 2013-10-12 22:26:13 UTC]

(In reply to SpanKY from comment #7)
> (In reply to Alan Hourihane from comment #0)
> 
> the OS you've chosen is a pos.  but POSIX does make it clear that some OS's
> are allowed to be moronic, and your patch doesn't look terrible.
> 
> http://sources.gentoo.org/eclass/multiprocessing.eclass?r1=1.2&r2=1.3

Oh, 

Did the helpers fix get committed to portage too ?

Thanks.

[Comment 10 SpanKY 2013-10-16 06:56:18 UTC]

(In reply to Alan Hourihane from comment #9)

no, but it has now:
http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commitdiff;h=c651476c03a67e98054335b43e476c55bcab167f

[Comment 11 Arfrever Frehtes Taifersar Arahesis 2013-10-16 21:43:58 UTC]

(In reply to SpanKY from comment #10)
> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commitdiff;
> h=c651476c03a67e98054335b43e476c55bcab167f

Functions other than makeopts_jobs() in bin/helper-functions.sh have consistently private names (with initial "__").

[Comment 12 SpanKY 2013-10-17 02:45:15 UTC]

(In reply to Arfrever Frehtes Taifersar Arahesis from comment #11)

i'm aware.  this code isn't sourced anywhere that seems to matter.