Specifically mysql_lib_symlinks() has several hard-coded references to ".so". This won't work on macos arches, and as a result the symlinks in /usr/lib aren't created, and thus other software can't link to the mysql libraries. Manually creating the symlinks works around this problem.
ah, nice find. There is a get_libname() function in multilib.eclass that should be used instead.
Is the .so -> get_libname() the only prefix change needed?
I've applied the following patch locally to try to fix this bug. Let me know if you see anything wrong. --- /home/gentoo-cvs/gentoo-x86/eclass/mysql_fx.eclass 2009-02-12 11:15:38.000000000 -0100 +++ eclass/mysql_fx.eclass 2010-12-21 02:11:15.889538967 -0100 @@ -3,7 +3,8 @@ # $Header: /var/cvsroot/gentoo-x86/eclass/mysql_fx.eclass,v 1.22 2009/02/12 05:05:14 robbat2 Exp $ # Author: Francesco Riosa (Retired) <vivo@gentoo.org> -# Maintainer: Luca Longinotti <chtekk@gentoo.org> +# Maintainer: MySQL Team <mysql-bugs@gentoo.org> +# Luca Longinotti <chtekk@gentoo.org> inherit multilib @@ -182,8 +183,8 @@ # library to the best version available. # mysql_lib_symlinks() { - einfo "Updating MySQL .so symlinks" - local d dirlist maxdots soname sonameln reldir + einfo "Updating MySQL get_libname() symlinks" + local d dirlist maxdots libname libnameln reldir reldir="${1}" pushd "${reldir}/usr/$(get_libdir)" &> /dev/null # dirlist must contain the less significative directory left @@ -191,20 +192,20 @@ # waste some time in removing and recreating symlinks for d in $dirlist ; do - for soname in $( find "${d}" -name "*.so*" -and -not -type "l" 2>/dev/null ) ; do + for libname in $( find "${d}" -name "*.get_libname()*" -and -not -type "l" 2>/dev/null ) ; do # maxdot is a limit versus infinite loop maxdots=0 - sonameln=${soname##*/} + libnameln=${libname##*/} # loop in version of the library to link it, similar to how # libtool works - while [[ ${sonameln:0-3} != '.so' ]] && [[ ${maxdots} -lt 6 ]] ; do - rm -f "${sonameln}" - ln -s "${soname}" "${sonameln}" + while [[ ${libnameln:0-3} != 'get_libname()' ]] && [[ ${maxdots} -lt 6 ]] ; do + rm -f "${libnameln}" + ln -s "${libname}" "${libnameln}" (( ++maxdots )) - sonameln="${sonameln%.*}" + libnameln="${libnameln%.*}" done - rm -f "${sonameln}" - ln -s "${soname}" "${sonameln}" + rm -f "${libnameln}" + ln -s "${libname}" "${libnameln}" done done popd &> /dev/null
(In reply to comment #2) > Is the .so -> get_libname() the only prefix change needed? On the macos-x86 machine that I tested on I manually created these symlinks, and after that things worked fine (as in: it runs our large Ruby on Rails test suite without issues), so I'm inclined to say yes.
I think this won't work, because the code assumes libraries are always named like this: libthething(.ver)*.ext while Mach-O objects are named like this: libthething.ext(.ver)* It seems that the snippet here hardcodes "get_libname()" in strings e.g. those passed onto find, is that some editor thing? I suspect it won't work this way.
Hi, I'm the original author, I've forgot why symlinking was necessary in first place, wasn't it mostly related to multi-slot mysql? Build system don't create symlinks now by default and correctly? Also this should disappear for cmake builds in mysql-5.5 right?
(In reply to comment #5) > I think this won't work, because the code assumes libraries are always named > like this: > libthething(.ver)*.ext > while Mach-O objects are named like this: > libthething.ext(.ver)* > > It seems that the snippet here hardcodes "get_libname()" in strings e.g. those > passed onto find, is that some editor thing? I suspect it won't work this way. I wasn't sure if that wasn't the case in some arch and so needed some feedback. I don't understand your question about the "editor thing". (In reply to comment #6) > Hi, I'm the original author, I've forgot why symlinking was necessary in first > place, wasn't it mostly related to multi-slot mysql? Build system don't create > symlinks now by default and correctly? That seems a good reason. The mysql_lib_symlinks function is still the last statement of mysql_src_install in the mysql eclass. > Also this should disappear for cmake builds in mysql-5.5 right? If we don't allow multi-slot mysql, seems like it should. In the meantime I've applied the following patch in the mysql overlay. It still searches for libraries with the hardcoded libthething(.ver)*.ext format. --- /home/gentoo-cvs/gentoo-x86/eclass/mysql_fx.eclass 2009-02-12 11:15:38.000000000 -0100 +++ ../../eclass/mysql_fx.eclass 2010-12-21 06:33:20.941538970 -0100 @@ -3,7 +3,8 @@ # $Header: /var/cvsroot/gentoo-x86/eclass/mysql_fx.eclass,v 1.22 2009/02/12 05:05:14 robbat2 Exp $ # Author: Francesco Riosa (Retired) <vivo@gentoo.org> -# Maintainer: Luca Longinotti <chtekk@gentoo.org> +# Maintainer: MySQL Team <mysql-bugs@gentoo.org> +# Luca Longinotti <chtekk@gentoo.org> inherit multilib @@ -182,8 +183,9 @@ # library to the best version available. # mysql_lib_symlinks() { - einfo "Updating MySQL .so symlinks" - local d dirlist maxdots soname sonameln reldir + local d dirlist maxdots libname libnameln libsuffix reldir + libsuffix=$(get_libname) + einfo "Updating MySQL ${libsuffix} symlinks" reldir="${1}" pushd "${reldir}/usr/$(get_libdir)" &> /dev/null # dirlist must contain the less significative directory left @@ -191,20 +193,20 @@ # waste some time in removing and recreating symlinks for d in $dirlist ; do - for soname in $( find "${d}" -name "*.so*" -and -not -type "l" 2>/dev/null ) ; do + for libname in $( find "${d}" -name "*.${libsuffix}*" -and -not -type "l" 2>/dev/null ) ; do # maxdot is a limit versus infinite loop maxdots=0 - sonameln=${soname##*/} + libnameln=${libname##*/} # loop in version of the library to link it, similar to how # libtool works - while [[ ${sonameln:0-3} != '.so' ]] && [[ ${maxdots} -lt 6 ]] ; do - rm -f "${sonameln}" - ln -s "${soname}" "${sonameln}" + while [[ ${libnameln:0-3} != '${libsuffix}' ]] && [[ ${maxdots} -lt 6 ]] ; do + rm -f "${libnameln}" + ln -s "${libname}" "${libnameln}" (( ++maxdots )) - sonameln="${sonameln%.*}" + libnameln="${libnameln%.*}" done - rm -f "${sonameln}" - ln -s "${soname}" "${sonameln}" + rm -f "${libnameln}" + ln -s "${libname}" "${libnameln}" done done popd &> /dev/null
(In reply to comment #7) > I wasn't sure if that wasn't the case in some arch and so needed some feedback. > I don't understand your question about the "editor thing". + while [[ ${libnameln:0-3} != '${libsuffix}' ]] This looks like a replacement, and does not seem to do the right thing to me. I expect double quotes around ${libsuffix} here.
mmh ${libnameln:0-${#libsuffix}}
also back from Alzheimer: mysql_lib_symlinks() takes all shared libraries in /usr/$libdir/mysql and link them in /usr/$libdir/ where broken build system of dependant packages could find them. --- mysql_fx.eclass.orig2 2010-12-21 21:05:37.178512862 +0100 +++ mysql_fx.eclass 2010-12-21 21:10:01.815789745 +0100 @@ -185,6 +185,7 @@ mysql_lib_symlinks() { local d dirlist maxdots libname libnameln libsuffix reldir libsuffix=$(get_libname) + [[ -z ${libsuffix} ]] && return einfo "Updating MySQL ${libsuffix} symlinks" reldir="${1}" pushd "${reldir}/usr/$(get_libdir)" &> /dev/null @@ -199,7 +200,9 @@ libnameln=${libname##*/} # loop in version of the library to link it, similar to how # libtool works - while [[ ${libnameln:0-3} != '${libsuffix}' ]] && [[ ${maxdots} -lt 6 ]] ; do + while [[ ${libnameln:0-${#libsuffix}} != "${libsuffix}" ]] \ + && [[ ${maxdots} -lt 6 ]] + do rm -f "${libnameln}" ln -s "${libname}" "${libnameln}" (( ++maxdots ))
Created attachment 257707 [details, diff] cumulative patch relative to in-tree mysql_fx.eclass (In reply to comment #5) > I think this won't work, because the code assumes libraries are always named > like this: > libthething(.ver)*.ext > while Mach-O objects are named like this: > libthething.ext(.ver)* > > It seems that the snippet here hardcodes "get_libname()" in strings e.g. those > passed onto find, is that some editor thing? I suspect it won't work this way. > if I get it correctly this UNTESTED patch will handle also .dylib (relative to my latest comment, see attach for the cumulative one) colordiff -u mysql_fx.eclass.orig mysql_fx.eclass --- mysql_fx.eclass.orig 2010-12-21 21:45:09.998285493 +0100 +++ mysql_fx.eclass 2010-12-21 21:56:27.429050625 +0100 @@ -178,12 +178,17 @@ [[ ${want_s} -le ${have_s} ]] && return 0 || return 1 } +__rm_and_ln() { + rm -f "${2}" + ln -s "${1}" "${2}" +} + # # To be called on the live filesystem, reassigning symlinks of each MySQL # library to the best version available. # mysql_lib_symlinks() { - local d dirlist maxdots libname libnameln libsuffix reldir + local d dirlist maxdots libname libnameln libsuffix reldir t libsuffix=$(get_libname) [[ -z ${libsuffix} ]] && return einfo "Updating MySQL ${libsuffix} symlinks" @@ -198,18 +203,37 @@ # maxdot is a limit versus infinite loop maxdots=0 libnameln=${libname##*/} + t=${libnameln//./} # t will be used to count number of dots # loop in version of the library to link it, similar to how # libtool works - while [[ ${libnameln:0-${#libsuffix}} != "${libsuffix}" ]] \ - && [[ ${maxdots} -lt 6 ]] - do - rm -f "${libnameln}" - ln -s "${libname}" "${libnameln}" - (( ++maxdots )) - libnameln="${libnameln%.*}" - done - rm -f "${libnameln}" - ln -s "${libname}" "${libnameln}" + # - [[ $((${#libnameln} - ${#t})) > 1 ]] check is redundant + # because libmysqlclient.dylib libmysqlclient.so behave the same + # - idem for maxdots check + if [[ ${libnameln:0-${#libsuffix}} == "${libsuffix}" ]] \ + && [[ $((${#libnameln} - ${#t})) > 1 ]] + then + # .dylib like naming scheme + libnameln=${libnameln%.${libsuffix}} + t=${libnameln%%.*} + while [[ ${t} != ${libnameln} ]] \ + && [[ ${maxdots} -lt 6 ]] + do + __rm_and_ln "${libname}" "${libnameln}.${libsuffix}" + (( ++maxdots )) + libnameln="${libnameln%.*}" + done + __rm_and_ln "${libname}" "${libnameln}.${libsuffix}" + else + # .so like naming scheme + while [[ ${libnameln:0-${#libsuffix}} != "${libsuffix}" ]] \ + && [[ ${maxdots} -lt 6 ]] + do + __rm_and_ln "${libname}" "${libnameln}" + (( ++maxdots )) + libnameln="${libnameln%.*}" + done + __rm_and_ln "${libname}" "${libnameln}" + fi done done popd &> /dev/null
@prefix: Can you please test Francesco's patch?
May I add another comment? Better avoid to change the behaviour now after years it's in production but I think the algorithm should be different: What this function does now: - it search for *.so* files in under /usr/lib/mysql (which include plugin/) filtering out symlinks - then it recreates a bunch of symlinks in /usr/lib to _all_ these libs looping in (<basename>, <major>, <minor>, <rev> ) What it really should do is: - search for commonly used lib in /usr/lib/mysql, namely libmysqlclient{,_r}.so - detect all their aliases (simlink + real file) - for each alias clone it in /usr/lib pointing to it's `realpath` rationale is: 1) not trying to be smarter than libtool (or equiv.) 2) simplier and more general 3) applications with build system unable to discover mysql shared objects in /usr/lib/mysql should not ever think to link against something else than the most basic client libraries. Or the build systems unable to find mysql lib should all be fixed.
following you on this side track, isn't this the job for a mysql-config, or eselect mysql tool?
Ok, my current dev-db/mysql-5.0.91 doesn't install any libs in $EPREFIX/usr/lib, this has never actually caused *any* problem, since mysql_config --libs nicely returns the usr/lib/mysql dir. Any packages not using libtool to link, will fail at runtime on ELF-platforms, but not on Darwin, where install_names are stored. Francesco's patch won't work, because Mach-O libs have an install_name, which is like soname, but then as absolute path. It needs to be updated as well, or one of Portage's QA checks will bail that they are incorrect. However, I strongly vote not do this copying in the first place, since it's obviously not necessary at all for Darwin, and would be unnecessary for ELF platforms, if the runtime linker would always find the libs, e.g. using ldconfig, or --rpath instructions, as libtool would add. An approach to get us out of this bug, would be to simply skip this step on Darwin platforms. Alternatively, I opt for writing a second function that does symlinking of the couple of libs that exist (probably using scanelf/scanmacho to identify soname/install_name) to usr/lib, Macho-O style. If you really want to unify, I suggest you to use scanelf/scanmacho to read the libdir (use -Ry whatever) and use its output instead of trying to do a lot of magic with bash that relies on assumptions of naming style and more. scantool=scanelf [[ ${CHOST} == *-darwin* ]] && scantool=scanmacho pushd "${ED}"/usr/$(get_libdir) > /dev/null ${scantool} -BRyF "%F:%S" mysql | while read line ; do lib=${line%:*} soname=${line#*:} lib=${lib#${ED}} ln -s ${lib} ${soname##*/} done popd > /dev/null
darn, the lib=${lib#${ED}} should not be there of course, sorry.
(In reply to comment #14) > following you on this side track, isn't this the job for a mysql-config, or > eselect mysql tool? > been there done that ... and failed (unneded added complexity user-side) (Someting about comment #15) scanelf -BRyF "%F:%S" mysql /usr/lib64/mysql/libmysqlclient.so.16.0.0 /usr/lib64/mysql/libmysqlclient.so.16.0.0:libmysqlclient.so.16 one ore more lines like the following if working are a great solution ;) [[ ${CHOST} == *-darwin* ]] && return don't know if this suffice ... anyway I do call me out now, disaster recovery and xmas ask for me
(In reply to comment #15) > Ok, my current dev-db/mysql-5.0.91 doesn't install any libs in > $EPREFIX/usr/lib, this has never actually caused *any* problem, since > mysql_config --libs nicely returns the usr/lib/mysql dir. Any packages not > using libtool to link, will fail at runtime on ELF-platforms, but not on > Darwin, where install_names are stored. Installing mysql-ruby on Darwin (x86-macos) failed for me due to the missing symlinks, which prompted me to file this bug. It could be that mysql-ruby is broken instead. I can't make that assessment.
(In reply to comment #18) > (In reply to comment #15) > > Ok, my current dev-db/mysql-5.0.91 doesn't install any libs in > > $EPREFIX/usr/lib, this has never actually caused *any* problem, since > > mysql_config --libs nicely returns the usr/lib/mysql dir. Any packages not > > using libtool to link, will fail at runtime on ELF-platforms, but not on > > Darwin, where install_names are stored. > > Installing mysql-ruby on Darwin (x86-macos) failed for me due to the missing > symlinks, which prompted me to file this bug. It could be that mysql-ruby is > broken instead. I can't make that assessment. I'd blame mysql-ruby, because it just guesses, however, when given the path to mysqlconfig, it works fine. --- mysql-ruby-2.8.1-r2.ebuild 11 Oct 2010 12:14:46 -0000 1.2 +++ mysql-ruby-2.8.1-r2.ebuild 6 Jan 2011 18:07:53 -0000 @@ -2,7 +2,7 @@ # Distributed under the terms of the GNU General Public License v2 # $Header: /var/cvsroot/gentoo-x86/dev-ruby/mysql-ruby/mysql-ruby-2.8.1-r2.ebuild,v 1.2 2010/10/11 12:14:46 jer Exp $ -EAPI="2" +EAPI="3" USE_RUBY="ruby18 ree18 ruby19" RUBY_FAKEGEM_NAME="mysql" @@ -31,7 +31,7 @@ } each_ruby_configure() { - ${RUBY} extconf.rb || die + ${RUBY} extconf.rb --with-mysql-config "${EPREFIX}/usr/bin/mysqlconfig" || die } each_ruby_compile() {
(In reply to comment #19) > I'd blame mysql-ruby, because it just guesses, however, when given the path to > mysqlconfig, it works fine. Thanks! Applied in mysql-ruby 2.8.2.
I've updated the mysql_lib_symlinks function of the mysql_fx eclass in the mysql overlay. Can anyone with prefix please test it?
Ok, I should get all the mysql-overlay's eclasses in prefix somehow to sort this entire thing. Sorry for not responding for so long.
I've fixed the eclass in the prefix overlay now. I've added symlinks support for macho targets not by trying to do it in a single loop, but with a CHOST conditional, as that looks cleaner to me.
I've moved the eclasses to the tree, so I'm closing this as fixed. Please reopen if you hit any issues.
