Created attachment 417376 [details] Patch to fix race condition The gentoo's patch for parallel builds is used within buildroot and using the autobuilders it is detected that there is a still a race condition to building openssl in parallel. See the discussion here: http://lists.busybox.net/pipermail/buildroot/2015-November/144686.html The build-shared target depends on do_crypto and link-shared, which will be executed in parallel. do_crypto calls link_a.linux_shared -> link_a.gnu which does SYMLINK_SO; in parallel, link-shared calls symlink.linux_shared which also does SYMLINK_SO. Before the symlink is created, it is rm'ed, but there is a tiny chance that the second one is created after the rm has been called. I have attached the patch that fixes this race condition by forcing the symlink (using 'ln -sf' instead of 'ln -s')
The error originally see during a failure is either: ln: failed to create symbolic link 'libssl.so': File exists ln: creating symbolic link `libcrypto.so': File exists And to update patch originally attached is to remove the 'rm -f' before the symbolic links: diff --git a/dev-libs/openssl/files/openssl-1.0.2d-parallel-build.patch b/dev-libs/openssl/files/openssl-1.0.2d-parallel-build.patch index b7aa0ea..9b74380 100644 --- a/dev-libs/openssl/files/openssl-1.0.2d-parallel-build.patch +++ b/dev-libs/openssl/files/openssl-1.0.2d-parallel-build.patch @@ -118,14 +118,26 @@ https://rt.openssl.org/Ticket/Display.html?id=3738&user=guest&pass=guest LD_LIBRARY_PATH=$$LIBPATH:$$LD_LIBRARY_PATH \ $${SHAREDCMD} $${SHAREDFLAGS} \ -o $$SHLIB$$SHLIB_SOVER$$SHLIB_SUFFIX \ -@@ -122,6 +123,7 @@ +@@ -117,15 +117,15 @@ SYMLINK_SO= \ + prev=$$SHLIB$$SHLIB_SOVER$$SHLIB_SUFFIX; \ + if [ -n "$$SHLIB_COMPAT" ]; then \ + for x in $$SHLIB_COMPAT; do \ +- ( $(SET_X); rm -f $$SHLIB$$x$$SHLIB_SUFFIX; \ +- ln -s $$prev $$SHLIB$$x$$SHLIB_SUFFIX ); \ ++ ( $(SET_X); \ ++ ln -sf $$prev $$SHLIB$$x$$SHLIB_SUFFIX ); \ + prev=$$SHLIB$$x$$SHLIB_SUFFIX; \ done; \ fi; \ if [ -n "$$SHLIB_SOVER" ]; then \ -+ [ -e "$$SHLIB$$SHLIB_SUFFIX" ] || \ - ( $(SET_X); rm -f $$SHLIB$$SHLIB_SUFFIX; \ - ln -s $$prev $$SHLIB$$SHLIB_SUFFIX ); \ + [ -e "$$SHLIB$$SHLIB_SUFFIX" ] || \ +- ( $(SET_X); rm -f $$SHLIB$$SHLIB_SUFFIX; \ +- ln -s $$prev $$SHLIB$$SHLIB_SUFFIX ); \ ++ ( $(SET_X); \ ++ ln -sf $$prev $$SHLIB$$SHLIB_SUFFIX ); \ fi; \ + fi +
`ln -sf` is no more atomic than `rm; ln`. there is still a window where things can fail. you can easily check this: i=0; while [ $((i++)) -lt 500 ]; do ln -sf a b & :; done a good number of those will fail with: ln: failed to create symbolic link ‘b’: File exists the deps need to be fixed up
You are indeed correct, thank you for pointing this out and giving this good example. So I guess my patch is invalid but the bug is still valid.
seems to me the buildroot guys didn't pick up all the Gentoo patches. the 1.0.2d ebuild applies more than just openssl-1.0.2d-parallel.patch. in fact, the race you refer to is most likely fixed by openssl-1.0.2a-parallel-symlinking.patch.
