Bash and Python fail when they receive EAGAIN from fork(). Interix does that a lot so using emerge is nearly impossible. After I patched Bash and Python to retry fork() if EAGAIN is received I no longer experience the errors (until new versions are installed and fail in the middle of updating world) Reproducible: Sometimes Steps to Reproduce: 1. Enable Subsystem for Unix-based Applications on Vista 2. Install Gentoo Prefix (on Interix 6) 3. Try to emerge something Actual Results: With high probability you'll receive fork errors like these: Example of error: /usr/portage3/usr/lib/portage/bin/ecompressdir: fork: Resource temporarily unavailable /usr/portage3/usr/lib/portage/bin/misc-functions.sh: fork: Resource temporarily unavailable !!! post install failed; exiting. >>> Failed to emerge sys-apps/findutils-4.5.2, Log file: Another example: >>> Emerging (1 of 52) sys-apps/findutils-4.5.2 * findutils-4.5.2.tar.gz RMD160 SHA1 SHA256 size ;-) ... [ ok ] * checking ebuild checksums ;-) ... [ ok ] * checking auxfile checksums ;-) ... [ ok ] * checking miscfile checksums ;-) ... [ ok ] /usr/portage3/usr/lib/portage/bin/ebuild.sh: fork: Resource temporarily unavailable >>> Failed to emerge sys-apps/findutils-4.5.2 Expected Results: I expect emerge to work [on Interix] without failing. Portage 2.2.00.12182-prefix (default-prefix/windows/interix/6.0/x86, gcc-4.2.4, unavailable, 6.0 x86) ================================================================= System uname: Interix-6.0-x86-32bit Timestamp of tree: Sat, 13 Dec 2008 02:31:32 +0000 app-shells/bash: 3.2_p48 dev-lang/python: 2.5.2-r7 dev-python/pycrypto: 2.0.1-r6 sys-devel/autoconf: 2.13, 2.63 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.2 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.0-r04.5 sys-devel/libtool: 1.5.26 ACCEPT_KEYWORDS="~x86-interix" CBUILD="i586-pc-interix6.0" CFLAGS="" CHOST="i586-pc-interix6.0" CONFIG_PROTECT="/etc" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/term info" CXXFLAGS="" DISTDIR="/usr/portage3/usr/portage/distfiles" EPREFIX="/usr/portage3" FEATURES="collision-protect distlocks fixpackages nostrip parallel-fetch preserve-libs protect-owned sfperms strict unme rge-orphans userfetch" GENTOO_MIRRORS="ftp://mirror.yandex.ru/gentoo-distfiles/" LDFLAGS="" PKGDIR="/usr/portage3/usr/portage/packages" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats -- timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/usr/portage3/var/tmp" PORTDIR="/usr/portage3/usr/portage" PORTDIR_OVERLAY="/usr/portage3/usr/local/portage/layman/Ark-kun" SYNC="rsync://rsync.prefix.freens.org/gentoo-portage-prefix" USE="X bzip2 cracklib gtk gtk2 jpeg midi moonlight ncurses nls png prefix readline slang ssl x86-interix xcomposite zlib " ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alia s authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_use r autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include in fo log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vh ost_alias" ELIBC="Interix" INPUT_DEVICES="keyboard mouse" KERNEL="Interix" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44 780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, MAKEOPTS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Created attachment 175215 [details, diff] fix fork() failing on EAGAIN in Bash
Created attachment 175216 [details, diff] fix fork() failing on EAGAIN in Python
I don't know if EAGAIN is returned on other systems so I filed this as Interix bug.
Did you send these patches upstream? If I look at the manpages for fork on Darwin and Interix they show about the same description for EAGAIN. mduft, do you have any comment on this? I think I didn't encounter this problem, but I'm on old Interix (3.5).
Created attachment 175328 [details] A preload library, which fixes fork the problem seems to appear in vista/server 2008 only here. i have a solution around this, which has been populated before on interopcommunity.com. i'll post the sources for it here. mainly this "solution" preloads every binary (so LD_PRELOAD needs to be set system wide for it to catch the login shell (bash in your case?)) otherwise you could start a bash and do something like this: LD_PRELOAD=/path/to/libfork.so bash -l und use that shell for working. then everything works like a charm, and it removes the need to patch every single program for interix (yeah, there are many of them, not handling EAGAIN). you can build the attached source like this: gcc -O2 -shared fork.c -DLIBC_NAME="\"libc.so.5.2\"" -DINSTALL_LIBNAME="\"`pwd`/libfork.so\"" -D_ALL_SOURCE -o libfork.so -ldl be sure that INSTALL_LIBNAME is correct (an absolute path to the library as it will be loaded), since the library adds itselve back to LD_PRELOAD on every exec which would clean it from the environment. also it is quite hard to replace the library after activating it from interix, since even the compiler and linker are preloaded with it, so the file cannot be replaced, nor can it be moved, since after moving nothing loads anymore ;) so if you happen to rebuild the library, change the -o to some other filename (but leave INSTALL_LIBNAME the same), and rename the two files in explorer (rename, replace won't work, since the lib may still be loaded somewhere), and all new processes will catch the new one. you can add -DDEBUG to the compiler command to enable debug output (be aware, it's much ;)). also you can adjust the FORK_RETRY count in fork.c. note that this only works for fork() and not for vfork(). for a more detailed description on why this is the case, look at the bottom of the source. there is a first implementation try, which is commented out, and described. maybe interesting is the reason for why this happens: the windows kernel returns a "try again" error, whenever it's internal caching mechanisms are busy processing some executable - at least thats what i was told. this has become much worse on vista, since there is some new executable cache... although it _can_ happen on older systems too, it's more unlikely. hope that helps. i guess i won't patch all interix ebuilds to handle fork correctly as long as this workaround works fine. it would be just too much work... Cheers, Markus
Hmmm, Markus, can we somehow get this in a package or something and use an env.d entry to get this automated for vista users?
(In reply to comment #6) > Hmmm, Markus, can we somehow get this in a package or something and use an > env.d entry to get this automated for vista users? hmm, maybe, yes. this would make prefix work 1000times better than the rest of interix ;). but the only way to catch the whole system is to insert the LD_PRELOAD env var into the system variables. the only real problem (besides interix not working properly outside of a prefix installation then... ;)) is preloading the prefix shell. if the env.d's are sourced, this is probably too late, since the shell won't catch the LD_PRELOAD variable (it's already running, right?).
thought a little about it... i guess the best way would be to create a package for prefix, and write out an einfo at the end telling the user (how) to set LD_PRELOAD in the windows system envrionment...
how does this work currently?
(In reply to comment #9) > how does this work currently? > manually :( ATM, i'm still fixing this manually for every vista/server 2008 machine. i plan to create a package this week, and fix bash so it does not need the preload... this way at least after entering prefix, everything should work (and i guess it doesn't matter that the "outside prefix" world is completely b0rked... :))
Markus, can you revisit this thing again?
i will..
the i6fork package is in the tree by now, and it seems to work quite well. however ATM i'm hunting a few server 2008 only bugs, which could be caused by i6fork... all this hackery is causing troubles over troubles - but nothing else would help except patching every package that fork()s... damn
this has been fixed by the user of the suacomp package, which i wrote to fix some of the major problems with the interix libc.
