Unlike Linux, on HP-UX (at least) a file in-use cannot be removed. Although it is possible to rename it. While re-emerging gcc fex I got this error: --- /EPREFIX/usr/lib/gcc/ia64-hp-hpux11.23/4.2.2/ !!! Failed to move /EPREFIX/var/tmp/portage/sys-devel/gcc-4.2.2/image/EPREFIX/usr/lib/gcc/ia64-hp-hpux11.23/4.2.2/libgcc_s.so.0 to /EPREFIX/usr/lib/gcc/ia64-hp-hpux11.23/4.2.2/libgcc_s.so.0 !!! [Errno 26] Text file busy Because python is linked against libgcc_s.so.0, and python is running whenever portage is running ;) This will apply to re-emerging python too. What we need here is sth. like this: 0) each time portage is called (or within that "autocleaning packages"): for f in $(</var/lib/portage/2bremovedlist) do rm -f "${f}" || echo "${f}" >> /var/lib/portage/2bremovedlist.new done mv /var/lib/portage/2bremovedlist{.new,} 1) echo "/file/in/use.removedbyportage" >> /var/lib/portage/2bremovedlist 2) mv "/file/in/use" "/file/in/use.removedbyportage" 3) cp "${D}/file/in/use" to "/file/in/use" 4) -> 0) Without this behaviour we can't say portage works on hpux.
Would this be something that gets fixed in prefix-world or goes upstream to dev-portage?
I like to investigate what I can do here first, but I most probably need some help from the gurus CCed for sure.
In general I don't like that idea at all, however it overlaps partially with something I need to do for preserve-libs (the removal part), so I'm not completely against it.
Have committed a workaround to prefix-overlay's profiles/default-prefix/hpux/profile.bashrc, revision 12938. This enables us to continue without the immediate need for portage to have this fixed, although there still _may_ be race conditions breaking prefix on hpux, which require some manual work to repair.
(In reply to comment #4) > Have committed a workaround to prefix-overlay's > profiles/default-prefix/hpux/profile.bashrc, revision 12938. > > This enables us to continue without the immediate need for portage to have this > fixed, although there still _may_ be race conditions breaking prefix on hpux, > which require some manual work to repair. The first package I tried with this "workaround" failed. >>> Merging sys-libs/ncurses-5.5-r03.2 to / trying to remove old busy text file /home/jolexa/portage/hpia64-11.23/usr/lib/portage/bin/chpathtool.removedbyportage0 trying to remove old busy text file /home/jolexa/portage/hpia64-11.23/usr/lib/portage/bin/tbz2tool.removedbyportage0 backing up text file /home/jolexa/portage/hpia64-11.23/usr/bin/tic (0) backing up text file /home/jolexa/portage/hpia64-11.23/usr/bin/toe (0) backing up text file /home/jolexa/portage/hpia64-11.23/usr/bin/infocmp (0) backing up text file /home/jolexa/portage/hpia64-11.23/usr/bin/clear (0) backing up text file /home/jolexa/portage/hpia64-11.23/usr/bin/tput (0) backing up text file /home/jolexa/portage/hpia64-11.23/usr/bin/tset (0) backing up text file /home/jolexa/portage/hpia64-11.23/usr/bin/tack (0) backing up text file /home/jolexa/portage/hpia64-11.23/usr/lib/libpanel.so (0) backing up text file /home/jolexa/portage/hpia64-11.23/usr/lib/libmenu.so (0) backing up text file /home/jolexa/portage/hpia64-11.23/usr/lib/libform.so (0) backing up text file /home/jolexa/portage/hpia64-11.23/lib/libncurses.so (0) /usr/lib/hpux32/dld.so: Unsatisfied data symbol 'BC' in load module '/home/jolexa/portage/hpia64-11.23/bin/bash'. /usr/lib/hpux32/dld.so: Unsatisfied data symbol 'PC' in load module '/home/jolexa/portage/hpia64-11.23/bin/bash'. /usr/lib/hpux32/dld.so: Unsatisfied data symbol 'UP' in load module '/home/jolexa/portage/hpia64-11.23/bin/bash'. !!! post preinst failed; exiting. !!! FAILED preinst: 2304 Thoughts?
(In reply to comment #5) > /usr/lib/hpux32/dld.so: Unsatisfied data symbol 'BC' in load module > '/home/jolexa/portage/hpia64-11.23/bin/bash'. > Thoughts? More information: Turns out that the workaround broke bash. I had to restore it from a binary elsewhere on the system.
Seems bash is started again somewhere between post_pkg_preinst() and the actual merge. Should work better with r13190. But now the race condition still is in the timeframe between post_pkg_preinst() and the actual merge, because if that just renamed and copied back binary (sharedlib) is started from elsewhere (another user), the original error will occur. This does not happen with that bash started from portage, as this is finished before the actual merge.
(In reply to comment #7) > Should work better with r13190. Confirmed works on my problem package.
Handling this ETXTBSY seems to lead to ugly workarounds. This is even worse than the chflags stuff that we do for FreeBSD. Perhaps it's possible to configure the kernel so that it just works instead of returning ETXTBSY at such an inconvenient time?
unfortunately even if that would be possible, it would not solve this issue, as we're in prefix here, so usually unprivileged, and without any control over the host system... :/
What types of rename calls are possible without triggering ETXTBSY? Can we move a busy file into a new directory or can we rename the parent directory of a file that is busy? It would be nice if we could move the files off into some sort of a "garbage" directory.
(In reply to comment #11) > What types of rename calls are possible without triggering ETXTBSY? Can we move > a busy file into a new directory or can we rename the parent directory of a > file that is busy? It would be nice if we could move the files off into some > sort of a "garbage" directory. > AFAICT anything can be done as long as: *) it is on the same filesystem *) the inode-number stays the same *) the link-count keeps > 0 Have tried some possible commands: *) Rename (mv) directory containing busy-text-file (BTF) *) Rename (mv) BTF within the same fs. *) Move(=rename) BTF into another directory on same filesystem *) Create a hardlink (ln) to BTF (in another directory) *) After having a hardlink, 'rm' or 'cp -f' over original works. Sidenote: A hardlink implies staying on same filesystem. Having tried these IMO it would be safest to create a hardlink somewhere (on the same filesystem), and ensure the merge is done like 'cp -f', not just like 'cp': $ cp --help -f, --force if an existing destination file cannot be opened, remove it and try again BTW, I expect very similar problems on windows, although I'm unsure if Interix handles that internally. But I'm pretty sure cygwin does not.
(In reply to comment #11) > What types of rename calls are possible without triggering ETXTBSY? I'd guess os.rename should work within its usual limitations. > Can we move a busy file into a new directory Only if we could rely on that new dir to be on the same filesystem, and generally that's not reliable unless we create a subdir ourselves (which gets pretty messy IMO, and doesn't provide much benefit over just renaming the file itself). > or can we rename the parent directory of a file that is busy? That's a joke, right? You don't want to rename /usr/lib ... > It would be nice if we could move the files off into some sort of a "garbage" > directory. But probably not doable without risky assumptions about filesystem layout, as we can't physically relocate the file (if we could then we could just as well delete/overwrite it). Out of curiosity, does HPUX support LD_PRELOAD or something similar? I'd rather not work around such weird limitations in the application layer but deal with it in the system layer.
Here's one possible solution that seems pretty good: 1) Before attempting to replace a file, create a temporary hardlink to it in the same directory. The hardlink should be a hidden file and it's name should have a random component to prevent collisions. 2) Replace the file with a new one. It should succeed since the link-count stays greater than zero. 3) Try to unlink the temporary hardlink. If it's busy, record it's path in a log somewhere.
4) Add an emaint command that checks for stale busy files removes them.
(In reply to comment #13) > Out of curiosity, does HPUX support LD_PRELOAD or something similar? AFAICT, HPUX supports LD_PRELOAD. > I'd rather not work around such weird limitations in the application layer but > deal with it in the system layer. Huh! Yes, this could work for ia64-hpux (have successfully tried to preload remove). But on Interix, I was unable to create a shared library doing dlopen/dlsym, which has the same "text file busy" problem and also supports LD_PRELOAD. Although shared libs without doing dlopen/dlsym work.
haubi, didn't you implement this inside portage now?
Nope, not yet. It was the aix-sharedlib handling I did recently.
Created attachment 213275 [details, diff] proposed patch to fix this problem i've been using this for a while now in our local tree without any (obvious) problems. could you have a look at the patch and tell me what you think?
Created attachment 240293 [details, diff] uh - fix for the previous patch. there was a mistake in the previous patch, makeing it fail if a busy file already existed, since the function to check for it's existance ignored the path to the file...
would be cool to get this into portage, as it would remove the need for profile.bashrc for hp-ux and interix. (and it doesn't disturb anybody else, afaict). please review ;)
Created attachment 240455 [details, diff] fixed version of the above patch
I believe this is in.
(In reply to Fabian Groffen from comment #23) > I believe this is in. For the records: Nope, this isn't in. It is the affected platforms HP-UX and Interix being obsolete now.
