Hi, recent asterisk packages have changed the last part of the install phase to read while read d < <(find "${ED}"/var -type d -empty || die "Find failed."); do keepdir "${d#${ED}}" done Previously it was something like: for d in $(find "${ED}"/var -type d -empty || die "Find failed."); do For me the updated line using read results in this error: * ERROR: net-misc/asterisk-18.3.0::tpb-overlay failed (install phase): * Find failed. * * Call stack: * ebuild.sh, line 125: Called src_install * environment, line 3446: Called die * The specific snippet of code: * while read d < <(find "${ED}"/var -type d -empty || die "Find failed."); do If I'm honest, I don't know how to parse the read statement, so I'm struggling to understand what is going wrong. I can confirm that I *do* have some empty directories, and that this command fails if I try and run it from the bash command line. Bash is 5.0.18 and this system is compiled using musl (not sure how that might affect things, but...?) Any thoughts?
Did you test the version from the official Gentoo tree, or just the "tpb-overlay" overlay? We need to have all information at hand before ticket assignment. That is why I ask you to * attach the logs and * paste the emerge info as described on https://wiki.gentoo.org/wiki/Attach_the_logs_to_the_bug_ticket Please reopen this ticket (Status:UNCONFIRMED) afterwards.
Ooops, yes, sorry, it's actually from the gentoo repo, sync'd as of a few minutes before report. I have an embedded build process which for various reasons runs against an older tree and I'm just pulling the latest ebuilds that I need across. On closer inspection, I don't seem to be able to repro this on an amd64/glibc machine, but it occurs at least on x86+arm / musl-1.21 (will test amd64/musl once that build machine frees up) On the x86 machine, the complete spew is curious, at first I get this: Installing file configs/samples/xmpp.conf.sample Updating asterisk.conf * ERROR: net-misc/asterisk-18.3.0::tpb-overlay failed (install phase): * Find failed. * * Call stack: * ebuild.sh, line 125: Called src_install * environment, line 3448: Called die * The specific snippet of code: * ERROR: net-misc/asterisk-18.3.0::tpb-overlay failed (install phase): * Find failed. * * Call stack: * ebuild.sh, line 125: Called src_install * environment, line 3448: Called die * The specific snippet of code: * while read d < <(find "${ED}"/var -type d -empty || die "Find failed."); do * * If you need support, post the output of `emerge --info '=net-misc/asterisk-18.3.0::tpb-overlay'`, * the complete build log and the output of `emerge -pqv '=net-misc/asterisk-18.3.0::tpb-overlay'`. * The complete build log is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/build.log'. * ERROR: net-misc/asterisk-18.3.0::tpb-overlay failed (install phase): * The ebuild environment file is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/environment'. * Find failed. * * Working directory: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * Call stack: * S: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * ebuild.sh, line 125: Called src_install * environment, line 3448: Called die * while read d < <(find "${ED}"/var -type d -empty || die "Find failed."); do * The specific snippet of code: * * If you need support, post the output of `emerge --info '=net-misc/asterisk-18.3.0::tpb-overlay'`, * the complete build log and the output of `emerge -pqv '=net-misc/asterisk-18.3.0::tpb-overlay'`. * The complete build log is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/build.log'. * The ebuild environment file is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/environment'. * ERROR: net-misc/asterisk-18.3.0::tpb-overlay failed (install phase): * Find failed. * Working directory: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * * S: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * Call stack: * ebuild.sh, line 125: Called src_install * environment, line 3448: Called die * while read d < <(find "${ED}"/var -type d -empty || die "Find failed."); do * The specific snippet of code: * * If you need support, post the output of `emerge --info '=net-misc/asterisk-18.3.0::tpb-overlay'`, * the complete build log and the output of `emerge -pqv '=net-misc/asterisk-18.3.0::tpb-overlay'`. * ERROR: net-misc/asterisk-18.3.0::tpb-overlay failed (install phase): * The complete build log is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/build.log'. * Find failed. * The ebuild environment file is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/environment'. * * Working directory: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * Call stack: * S: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * ebuild.sh, line 125: Called src_install * environment, line 3448: Called die * The specific snippet of code: * while read d < <(find "${ED}"/var -type d -empty || die "Find failed."); do * * ERROR: net-misc/asterisk-18.3.0::tpb-overlay failed (install phase): * If you need support, post the output of `emerge --info '=net-misc/asterisk-18.3.0::tpb-overlay'`, * Find failed. * the complete build log and the output of `emerge -pqv '=net-misc/asterisk-18.3.0::tpb-overlay'`. * * The complete build log is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/build.log'. * Call stack: * The ebuild environment file is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/environment'. * Working directory: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * ebuild.sh, line 125: Called src_install * S: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * environment, line 3448: Called die * The specific snippet of code: * while read d < <(find "${ED}"/var -type d -empty || die "Find failed."); do * * If you need support, post the output of `emerge --info '=net-misc/asterisk-18.3.0::tpb-overlay'`, * the complete build log and the output of `emerge -pqv '=net-misc/asterisk-18.3.0::tpb-overlay'`. * ERROR: net-misc/asterisk-18.3.0::tpb-overlay failed (install phase): * The complete build log is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/build.log'. * Find failed. * The ebuild environment file is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/environment'. * * Working directory: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * Call stack: * S: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * ebuild.sh, line 125: Called src_install * environment, line 3448: Called die * The specific snippet of code: * while read d < <(find "${ED}"/var -type d -empty || die "Find failed."); do * * If you need support, post the output of `emerge --info '=net-misc/asterisk-18.3.0::tpb-overlay'`, * the complete build log and the output of `emerge -pqv '=net-misc/asterisk-18.3.0::tpb-overlay'`. * The complete build log is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/build.log'. * ERROR: net-misc/asterisk-18.3.0::tpb-overlay failed (install phase): * The ebuild environment file is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/environment'. * Find failed. * Working directory: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * * S: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * Call stack: * ebuild.sh, line 125: Called src_install * environment, line 3448: Called die * while read d < <(find "${ED}"/var -type d -empty || die "Find failed."); do * The specific snippet of code: * * If you need support, post the output of `emerge --info '=net-misc/asterisk-18.3.0::tpb-overlay'`, * the complete build log and the output of `emerge -pqv '=net-misc/asterisk-18.3.0::tpb-overlay'`. * The complete build log is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/build.log'. * The ebuild environment file is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/environment'. * ERROR: net-misc/asterisk-18.3.0::tpb-overlay failed (install phase): * Working directory: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * S: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * Find failed. * * Call stack: * ebuild.sh, line 125: Called src_install * environment, line 3448: Called die * while read d < <(find "${ED}"/var -type d -empty || die "Find failed."); do * The specific snippet of code: * * If you need support, post the output of `emerge --info '=net-misc/asterisk-18.3.0::tpb-overlay'`, * the complete build log and the output of `emerge -pqv '=net-misc/asterisk-18.3.0::tpb-overlay'`. * The complete build log is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/build.log'. * The ebuild environment file is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/environment'. * Working directory: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' * S: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' /usr/lib/portage/python3.8/isolated-functions.sh: line 236: kill: (12959) - No such process * while read d < <(find "${ED}"/var -type d -empty || die "Find failed."); do * * If you need support, post the output of `emerge --info '=net-misc/asterisk-18.3.0::tpb-overlay'`, * the complete build log and the output of `emerge -pqv '=net-misc/asterisk-18.3.0::tpb-overlay'`. * The complete build log is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/build.log'. * The ebuild environment file is located at '/var/tmp/portage/net-misc/asterisk-18.3.0/temp/environment'. * Working directory: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' /usr/lib/portage/python3.8/isolated-functions.sh: line 236: kill: (12959) - No such process * S: '/var/tmp/portage/net-misc/asterisk-18.3.0/work/asterisk-18.3.0' /usr/lib/portage/python3.8/isolated-functions.sh: line 236: kill: (12959) - No such process /usr/lib/portage/python3.8/isolated-functions.sh: line 236: kill: (12959) - No such process /usr/lib/portage/python3.8/isolated-functions.sh: line 236: kill: (12959) - No such process /usr/lib/portage/python3.8/isolated-functions.sh: line 236: kill: (12959) - No such process /usr/lib/portage/python3.8/isolated-functions.sh: line 236: kill: (12959) - No such process /usr/lib/portage/python3.8/isolated-functions.sh: line 236: kill: (12959) - No such process However, if I check, then I find: # find /var/tmp/portage/net-misc/asterisk-18.3.0/image/var -type d -empty /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/log/asterisk/cel-custom # find /var/tmp/portage/net-misc/asterisk-18.3.0/image/var -name .keep* /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/lib/asterisk/third-party/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/lib/asterisk/astdb/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/lib/asterisk/agi-bin/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/lib/asterisk/moh/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/lib/asterisk/sounds/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/lib/asterisk/phoneprov/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/lib/asterisk/keys/stir_shaken/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/lib/asterisk/firmware/iax/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/lib/asterisk/documentation/thirdparty/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/log/asterisk/cdr-custom/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/log/asterisk/cdr-csv/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/spool/asterisk/recording/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/spool/asterisk/voicemail/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/spool/asterisk/tmp/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/spool/asterisk/system/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/spool/asterisk/monitor/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/spool/asterisk/meetme/.keep_net-misc_asterisk-0 /var/tmp/portage/net-misc/asterisk-18.3.0/image/var/spool/asterisk/dictate/.keep_net-misc_asterisk-0 So that implies that part of the read loop ran?? Note that I don't see this with the "for d in" version? (as in I first noticed this when changing across the 16.x releases where this change was introduced. Only raising the bug because the change hasn't been reverted, so obviously not everyone else is hitting this?) Thanks if someone can either revert the change or help me understand the "< <(..)" syntax so that I can try and write a test case on the command line?
Hi, There are other issues with for d in $(find ...); do loops in general, spaces in names (which isn't the case here), but in general, it's better to not use things that way. net-misc/asterisk-18.3.0::tpb-overlay - this looks like you're using the overlay? Just to be clear, you're copying the ebuilds from ::gentoo into this overlay? So ... re the < <() syntax ... command <filename Is quite common. <(command) Says start command, and then provide a "dummy" filename which represents stdout from that command. So command1 < <(command2) say to run command2, then pipe it's stdout to stdin ... the reason for not using actual | here is because that would re-run command2 repeatedly on every invocation. So the theory is here that command2 (the find) should be executed exactly once ... Which I'm now wondering if that's actually the case, would you mind expanding that into more detail: exec 3< <(find ... || die "Find failed.") white read d <&3; do ... done exec 3<&- And I'm just hoping fd3 won't already be open somewhere else (guard that with [[ -r /proc/self/fd/3 ]] && die "fd=3 already in use" just prior. What I'm wondering is if the find isn't getting re-executed on every run anyway ... in which case we're probably better off with the for d in $(...); do variant, alternatively, the exec syntax above (which has fd number collision risks), or the < <(find ...) needs to move to the end of the done, as in: while read d; do ... done < <(find ...)
Thanks for looking further into this. Remember also that I can only repro this on a musl chroot, not in an amd64 glibc, so this seems affected by something to do with the libc (or being in the chroot) So, I create a basic test setup like this: # mkdir /tmp/test/a/{a,b,c} -p Then I try your code (I think) like this: # exec 3< <(find /tmp/test -type d -empty || die "Find failed.") exec 3< <(find /tmp/test -type d -empty || die "Find failed.") # while read d <&3; do while read d <&3; do > echo $d echo $d > done done /tmp/test/a/a /tmp/test/a/b /tmp/test/a/c # exec 3<&- So my reading of that is that it went as expected? Now the next is somewhat interesting. If I run the following (which may well be invalid anyway??), then on the musl chroot I see: # while read d < <(find /tmp/test -type d -empty || die "Find failed."); do while read d < <(find /tmp/test -type d -empty || die "Find failed."); do > echo $d echo $d > done done /tmp/test/a/a bash: die: command not found /tmp/test/a/a bash: die: command not found /tmp/test/a/a ... this repeats endlessly, but notice that it also seems to be running my || clause (which fails because this is just at the command line, so die is not defined However, on the outside of the chroot (the amd64 glibc), I see: # while read d < <(find /tmp/test -type d -empty || die "Find failed."); do > echo $d > done /tmp/test/a/a /tmp/test/a/a /tmp/test/a/a ... endlessly repeating So unless I'm reading this wrong your suggestion in the post above correctly implements the logic needed. However, the logic in the ebuild happens to re-run the find command each time, presumably it cleans up the dirs one by one, hence it eventually ends? Additionally musl is behaving differently to glibc for some reason? I think I'm happy that your amended version works as expected, caveat I haven't tried it in the context of the ebuild. However, google turns up the suggestion to do this, better? find . -type f -name '*.*' -print0 | while IFS= read -r -d '' file; do printf '%s\n' "$file" done I do a quick test and that appears to perform as expected in the musl environment: # find /tmp/test -type d -empty -print0 | while IFS= read -r -d '' file; do echo $file; done /tmp/test/a/a /tmp/test/a/b /tmp/test/a/c What do you think?
(In reply to Ed Wildgoose from comment #4) > Thanks for looking further into this. Remember also that I can only repro > this on a musl chroot, not in an amd64 glibc, so this seems affected by > something to do with the libc (or being in the chroot) > > So, I create a basic test setup like this: > > # mkdir /tmp/test/a/{a,b,c} -p > > Then I try your code (I think) like this: > > # exec 3< <(find /tmp/test -type d -empty || die "Find failed.") > exec 3< <(find /tmp/test -type d -empty || die "Find failed.") > # while read d <&3; do > while read d <&3; do > > echo $d > echo $d > > done > done > /tmp/test/a/a > /tmp/test/a/b > /tmp/test/a/c > # exec 3<&- > > So my reading of that is that it went as expected? I concur. > Now the next is somewhat interesting. If I run the following (which may well > be invalid anyway??), then on the musl chroot I see: > > > # while read d < <(find /tmp/test -type d -empty || die "Find failed."); do > while read d < <(find /tmp/test -type d -empty || die "Find failed."); do > > echo $d > echo $d > > done > done > /tmp/test/a/a > bash: die: command not found > /tmp/test/a/a > bash: die: command not found > /tmp/test/a/a > ... this repeats endlessly, but notice that it also seems to be running my > || clause (which fails because this is just at the command line, so die is > not defined This states that find is exiting non-zero!! die will function in the context of ebuild where the function is defined. Do you have strace inside your chroot? Please provide: strace find /tmp/test -type d -empty 2>&1 | tee /tmp/find_strace.log Just upload the find_strace.log file. > However, on the outside of the chroot (the amd64 glibc), I see: > # while read d < <(find /tmp/test -type d -empty || die "Find failed."); do > > echo $d > > done > /tmp/test/a/a > /tmp/test/a/a > /tmp/test/a/a > ... endlessly repeating This confirms the theory I though. The find is getting re-executed for every iteration, which is undesired anyway, so needs to be fixed. And using xargs won't help either since keepdir is a shell function and thus won't be accessible by xargs. And the alternatives: while read ..; do done < <(find ...) Is ugly, and to make that safe, something like: while read ... <&3; do done 3< <(find ...) is even uglier ... > So unless I'm reading this wrong your suggestion in the post above correctly > implements the logic needed. However, the logic in the ebuild happens to > re-run the find command each time, presumably it cleans up the dirs one by > one, hence it eventually ends? Additionally musl is behaving differently to > glibc for some reason? Correct. Thus why I'm asking for the strace. > I think I'm happy that your amended version works as expected, caveat I > haven't tried it in the context of the ebuild. > > However, google turns up the suggestion to do this, better? > > find . -type f -name '*.*' -print0 | > while IFS= read -r -d '' file; do > printf '%s\n' "$file" > done This is equivalent to my while ...; do ... done < <(find ...) variant, with the elimination of a sub-shell spawn (ie, mine is "better"). Can still use the IFS= will need to test that though. And to make it fd safe we need to use an otherwise unused fd ... which I believe 3 should be fairly safe since it's only updated for that single command. > I do a quick test and that appears to perform as expected in the musl > environment: > > # find /tmp/test -type d -empty -print0 | while IFS= read -r -d '' file; do > echo $file; done > /tmp/test/a/a > /tmp/test/a/b > /tmp/test/a/c > > What do you think? Please test this one too: while IFS= read -r file; do echo $file done 3< <(find /tmp/test -type d -empty -print0) -r shouldn't be needed since stdin isn't a tty, and -d isn't documented in the manpage. But it's unclear around the \ so rather keep it.
Hi, thanks again for continuing to investigate this. Starting with the last first, I tried this: while IFS= read -r file; do echo $file done 7< <(find /tmp/test -type d -empty -print0) (with both 7 and 3 as fd) and this doesn't work as expected on any of the systems. So on my glibc + musl chroot it seems to stop and expect input from the keyboard? So the strace for strace find /tmp/test -type d -empty 2>&1 | tee /tmp/find_strace.log will be uploaded momentarily, however, note this result: # find /tmp/test -type d -empty 2>&1 ; echo $? /tmp/test/a/a /tmp/test/a/b /tmp/test/a/c 0 If I could gently offer an (uneducated) opinion. To me the find | read is significantly easier to read for a non bash expert. I'm still not quite sure I understand how the < <() is working? Additionally, is this a bash-ism? I notice that gentoo seems to try and keep to generic sh kind of use where possible (eg no arrays)? Not sure if this is a policy though? Finally, the performance of these scripts is important to me (run many builds a day), however, I don't see that this one extra shell invocation is a noticeable performance point. Can I make a (non counting, only putting my hand up) vote in favour of the find | read version? Thanks for listening!
Created attachment 701715 [details] strace from find, taken on amd64/musl
(In reply to Ed Wildgoose from comment #6) > Hi, thanks again for continuing to investigate this. Starting with the last > first, I tried this: > > while IFS= read -r file; do > echo $file > done 7< <(find /tmp/test -type d -empty -print0) You missed <&7 for the read inside the while, in other words: while IFS= read -r file <&7; do > will be uploaded momentarily, however, note this result: > > # find /tmp/test -type d -empty 2>&1 ; echo $? > /tmp/test/a/a > /tmp/test/a/b > /tmp/test/a/c > 0 Which is what I would expect, but the fact that || die triggers tells a different story! > If I could gently offer an (uneducated) opinion. To me the find | read is > significantly easier to read for a non bash expert. I'm still not quite sure > I understand how the < <() is working? Additionally, is this a bash-ism? I > notice that gentoo seems to try and keep to generic sh kind of use where > possible (eg no arrays)? Not sure if this is a policy though? Finally, the > performance of these scripts is important to me (run many builds a day), > however, I don't see that this one extra shell invocation is a noticeable > performance point. Can I make a (non counting, only putting my hand up) > vote in favour of the find | read version? Thanks for listening! I believe this change was proposed by Michał, but I'd need to go confirm this. As far as I know <(command) isn't a bash-ism. In this case it avoids spawning additional sub-shells, and thus a scenario in which intended side effects to environment could simply be discarded. Sub-shells are usually not what the intent was. So what <(command) does is it executes command, and connects it's stdout to a file descriptor named, eg /dev/fd/63, now, if you do exec 3</dev/fd63 it connects fd 3 to that, so basically: exec 3 < <(command) says: execute command and connect it's standard output to the read descriptor number 3. Very cool stuff, so if for example you want to redirect your own stdin through a command you could perform a sequence like: exec 0< <(sed -nre '...') Whict will take your existing stdin, wire that into sed, and then your shell's stdin will read from the pipe that's connected to stdout from the sed command. I am however inclined to just revert to the for x in $(find ...); do ... on the assumption that paths really aught to not contain whitespace characters.
Ooops, sorry. OK, so please correct my next blunder. The following does not produce any output neither on glibc nor musl?? while IFS= read -r file <&7; do echo $file done 7< <(find /tmp/test -type d -empty -print0)
OK, so the < < stuff is now snapping into focus!! I guess I only ever see the reverse, ie the 2>&1, so it really wasn't triggering to see it in reverse and without the &fd bit OK, so I've looked closer at this and it looks ever odder... Notice this result: # while read d < <(find /tmp/test -type d -empty || echo "Find failed."); do > echo $d > done /tmp/test/a/a /tmp/test/a/a /tmp/test/a/a /tmp/test/a/a /tmp/test/a/a vs this result: # while read d < <(find /tmp/test -type d -empty || die "Find failed."); do > echo $d > done /tmp/test/a/a bash: die: command not found /tmp/test/a/a bash: die: command not found /tmp/test/a/a bash: die: command not found /tmp/test/a/a bash: die: command not found /tmp/test/a/a bash: die: command not found /tmp/test/a/a Where "die" is just any command which doesn't exist So the strace for the first option is coming, as this seems like an interesting result? Why is find running constantly? So in conclusion, my best *guess* as to what happens to the original script is: - find runs once for each line read - this means we process one directory with a keep_dir each iteration and gradually the output of find decreases until it's empty. - there is then a change in behaviour with musl, it's not clear to me if the difference is: a) find sometimes returns false b) die *is* actually called on glibc, but doesn't die because it's in a subshell? I haven't looked at the implementation of die, but I'm curious how it can kill both the subshell and the parent process? Perhaps whatever it does works differently on musl and glibc? Perhaps die *is* getting called on glibc, but the output is getting eaten by the loop?? Anyway... Curious
Created attachment 701733 [details] strace of the original style find script on amd64/musl
(In reply to Ed Wildgoose from comment #7) > Created attachment 701715 [details] > strace from find, taken on amd64/musl This exits 0 ... so I'm really not sure why this would try to execute || die ...
Does this shed any insight? # cat /tmp/test2.sh while read d < <(find /tmp/test -type d -empty || die "Find failed."); do echo $d done # strace bash /tmp/test2.sh > /tmp/strace.log 2>&1 Gives loads of /tmp/test/a/a bash: die: command not found /tmp/test/a/a bash: die: command not found /tmp/test/a/a (see attached for strace)
Created attachment 701769 [details] Strace for version with non existing command die
(In reply to Ed Wildgoose from comment #14) > Created attachment 701769 [details] > Strace for version with non existing command die Unfortunately these don't help much without -f. Only thing I can think is that find exits non-zero for some reason, PROBABLY due to being killed with a signal after the first line of output was written (read by the shell). This probably relates to musl vs glibc (no output buffer?) resulting in the pipe(2) being closed or something, either way. I've made alterations to this code in both 18.5.0 and 16.19.0 now. About to generate a PR. It works on AMD64/glibc at least, but I don't have a musl to test with at hand, most certainly not on ARM either way. So would really appreciate if you can test this PR and close if that works.
> Unfortunately these don't help much without -f. Hi, I would be delighted to produce these if you were interested to debug this further? Perhaps send me an email offline if you do actually have interest in this? > I've made alterations to this code in both 18.5.0 and 16.19.0 now. About to > generate a PR. It works on AMD64/glibc at least, but I don't have a musl to > test with at hand, most certainly not on ARM either way. So would really > appreciate if you can test this PR and close if that works. Perfect! Many thanks! I can confirm that this change resolves my previous install problem on musl libc systems. I've jotted a note to the same effect on your pull request I have however, spotted a new "bug". I don't know that it's worth filing a bug report (you can close this current one now). However, could you please add the /var/spool/asterisk/outgoing directory via the ebuild. At present asterisk doesn't own /var/spool/asterisk, hence cannot create the outgoing dir. My solution was as follows If you want a proper bug raising then please close this one and I'll raise it. However, perhaps it's trivial enough? Thanks --- asterisk-18.5.0.ebuild.orig 2021-07-05 19:05:06.879152809 +0000 +++ asterisk-18.5.0.ebuild 2021-07-05 19:07:56.673279183 +0000 @@ -296,7 +296,8 @@ # keep directories diropts -m 0750 -o asterisk -g root - keepdir /var/spool/asterisk/{system,tmp,meetme,monitor,dictate,voicemail,recording} + dodir /var/spool/asterisk + keepdir /var/spool/asterisk/{system,tmp,meetme,monitor,dictate,voicemail,recording,outgoing} diropts -m 0750 -o asterisk -g asterisk keepdir /var/log/asterisk/{cdr-csv,cdr-custom}
Suggestion taken, I've left /var/spool/asterisk root:root, but I have added outgoing as per your suggestion. Will update the PR momentarily, just merge-testing quick.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=3acd744f0c9e81b23694d191a29d9055915d6f1b commit 3acd744f0c9e81b23694d191a29d9055915d6f1b Author: Jaco Kroon <jaco@uls.co.za> AuthorDate: 2021-06-30 19:39:33 +0000 Commit: Joonas Niilola <juippis@gentoo.org> CommitDate: 2021-07-12 10:38:52 +0000 net-misc/asterisk: Version 18.5.0 Closes: https://bugs.gentoo.org/783762 Package-Manager: Portage-3.0.20, Repoman-3.0.2 Signed-off-by: Jaco Kroon <jaco@uls.co.za> Closes: https://github.com/gentoo/gentoo/pull/21488 Signed-off-by: Joonas Niilola <juippis@gentoo.org> net-misc/asterisk/Manifest | 1 + net-misc/asterisk/asterisk-18.5.0.ebuild | 362 +++++++++++++++++++++++++++++++ 2 files changed, 363 insertions(+) Additionally, it has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=44bf94f748876e5ec5b13ea56b788627c2b5a767 commit 44bf94f748876e5ec5b13ea56b788627c2b5a767 Author: Jaco Kroon <jaco@uls.co.za> AuthorDate: 2021-06-30 19:35:30 +0000 Commit: Joonas Niilola <juippis@gentoo.org> CommitDate: 2021-07-12 10:38:47 +0000 net-misc/asterisk: Version 16.19.0 This was based off of previously deleted 16.16.2-r1 which contained a bunch of improvements that subsequently got lost in 16.17 and 16.18 versions. Bug: https://bugs.gentoo.org/783762 Closes: https://bugs.gentoo.org/796317 Package-Manager: Portage-3.0.20, Repoman-3.0.2 Signed-off-by: Jaco Kroon <jaco@uls.co.za> Signed-off-by: Joonas Niilola <juippis@gentoo.org> net-misc/asterisk/Manifest | 1 + net-misc/asterisk/asterisk-16.19.0.ebuild | 362 ++++++++++++++++++++++++++++++ 2 files changed, 363 insertions(+)
