Summary: | dev-python/graph-tool-2.2.36 - python -c 'from graph_tool.all import *': ImportError: /usr/lib64/python3.3/site-packages/graph_tool/libgraph_tool_core.so: undefined symbol: _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | Justin Keogh <bugs.gentoo.org> |
Component: | [OLD] Development | Assignee: | Tim Harder <radhermit> |
Status: | RESOLVED FIXED | ||
Severity: | normal | CC: | python |
Priority: | Normal | Keywords: | PATCH |
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 537042 | ||
Attachments: |
build.log.gz
dev-python/graph-tool-2.2.36 build.log boost_py_33_module.tar.gz graph-tool-2.2.36.ebuild.patch graph-tool-2.2.36.ebuild.patch |
Description
Justin Keogh
2015-01-15 21:06:15 UTC
Note MAKEOPTS="${MAKEOPTS} -j1" is hard coded in graph-tool-2.2.36.ebuild, for testing I changed it to -j8 with 32G without issue. $nm -D /usr/lib64/python3.3/site-packages/graph_tool/libgraph_tool_core.so | grep _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE U _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE $ldd -r /usr/lib64/python3.3/site-packages/graph_tool/libgraph_tool_core.so linux-vdso.so.1 (0x000002cf009d9000) libstdc++.so.6 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.1/libstdc++.so.6 (0x000002cefed7c000) libpython3.3.so.1.0 => /usr/lib64/libpython3.3.so.1.0 (0x000002cefe8da000) libboost_iostreams.so.1.55.0 => /usr/lib64/libboost_iostreams.so.1.55.0 (0x000002cefe6bf000) libboost_python-2.7.so.1.55.0 => /usr/lib64/libboost_python-2.7.so.1.55.0 (0x000002cefe46f000) libboost_regex.so.1.55.0 => /usr/lib64/libboost_regex.so.1.55.0 (0x000002cefe162000) libgomp.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.1/libgomp.so.1 (0x000002cefdf47000) libCGAL.so.10 => /usr/lib64/libCGAL.so.10 (0x000002cefdd1b000) libexpat.so.1 => /usr/lib64/libexpat.so.1 (0x000002cefdaed000) libbz2.so.1 => /lib64/libbz2.so.1 (0x000002cefd8dc000) libm.so.6 => /lib64/libm.so.6 (0x000002cefd5df000) libc.so.6 => /lib64/libc.so.6 (0x000002cefd233000) libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/4.9.1/libgcc_s.so.1 (0x000002cefd01b000) /lib64/ld-linux-x86-64.so.2 (0x000002cf009da000) libpthread.so.0 => /lib64/libpthread.so.0 (0x000002cefcdfe000) libdl.so.2 => /lib64/libdl.so.2 (0x000002cefcbf9000) libutil.so.1 => /lib64/libutil.so.1 (0x000002cefc9f6000) libz.so.1 => /lib64/libz.so.1 (0x000002cefc7de000) librt.so.1 => /lib64/librt.so.1 (0x000002cefc5d5000) libpython2.7.so.1.0 => /usr/lib64/libpython2.7.so.1.0 (0x000002cefc1f4000) libicuuc.so.52 => /usr/lib64/libicuuc.so.52 (0x000002cefbe74000) libicui18n.so.52 => /usr/lib64/libicui18n.so.52 (0x000002cefbc62000) libicudata.so.52 => /usr/lib64/libicudata.so.52 (0x000002cefa3f5000) libboost_thread.so.1.55.0 => /usr/lib64/libboost_thread.so.1.55.0 (0x000002cefa1db000) libboost_system.so.1.55.0 => /usr/lib64/libboost_system.so.1.55.0 (0x000002cef9fd6000) undefined symbol: _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE (/usr/lib64/python3.3/site-packages/graph_tool/libgraph_tool_core.so) $c++filt -n _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE boost::python::detail::init_module(PyModuleDef&, void (*)()) $nm -D /usr/lib64/libboost_python-3.3.so | grep _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE 0000000000036700 T _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE Created attachment 394064 [details]
build.log.gz
note this is with -std=gnu++1
I verified via it's build log that graph-tool is being compiled with -std=gnu++11 (without changing the ebuild). By default, boost is not. Created attachment 394066 [details]
dev-python/graph-tool-2.2.36 build.log
Notice that graph-tool is being compiled with -lboost_python-2.7-mt (everywhere) even though my system default is py3.3. I think "-lboost_python-2.7-mt" should be "-lboost_python"
With this in mind, I edited the ebuild to remove py2.7 support completely:
PYTHON_COMPAT=( python{3_3,3_4} )
and recompiled graph-tool. It's build log still shows it's using "-lboost_python-2.7-mt" only.
in: ./work/graph-tool-2.2.36-python3_3/config.log configure:18032: checking for exit in -lboost_python configure:18057: x86_64-pc-linux-gnu-g++ -o conftest -Wall -Wextra -Wno-unused-local-typedefs -Wno-unused-parameter -march=native -O2 -pipe -DNDEBUG -std=gnu++11 -ftemplate-depth-250 -Wno-deprecated -Wno-unknown-pragmas -fvisibility=default -fvisibility-inlines-hidden -fopenmp -I/usr/include/python3.3 -Wl,-O1 -Wl$ /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lboost_python collect2: error: ld returned 1 exit status Created attachment 394068 [details] boost_py_33_module.tar.gz On my system, "-lboost_python-3.3" works (test case attached). This may be related to bug 498552 graph-tool appears to be attempting to use -lboost_python when it should be using -lboost_python-3.3. The ./configure log shows it does not show a check for -lboost_python-3.3. Created attachment 394082 [details, diff]
graph-tool-2.2.36.ebuild.patch
Patch attached, this fixed it for me.
(In reply to Justin Keogh from comment #8) > Created attachment 394082 [details, diff] [details, diff] > graph-tool-2.2.36.ebuild.patch > > Patch attached, this fixed it for me. inherit eutils check-reqs toolchain-funcs python-r1 sigh, I went to the effort of removing it and your trying to add it in again. fyi, eutils is implicitly inherited by python-r1. However, this is not so straight forward. There are devs who 'like' to explicitly inherit eclasses that are already implicitly inherited. So I guess both are right. I'm in the camp that says it's totally redundant and so I leave it out. The first attempt yielded a total system freeze, however on the 2nd attempt, ~/cvsPortage/gentoo-x86/dev-python/graph-tool $ time PYTHON_TARGETS=python3_3 ebuild graph-tool-2.2.36.ebuild clean install yields make[2]: Leaving directory '/mnt/gen2/TmpDir/portage/dev-python/graph-tool-2.2.36/work/graph-tool-2.2.36-python3_3' make[1]: Leaving directory '/mnt/gen2/TmpDir/portage/dev-python/graph-tool-2.2.36/work/graph-tool-2.2.36-python3_3' (delete unhelpful lines) >>> Completed installing graph-tool-2.2.36 into /mnt/gen2/TmpDir/portage/dev-python/graph-tool-2.2.36/image/ (delete more unhelpful lines) real 114m44.059s user 112m38.933s sys 2m45.857s with the ebuild untouched. I for 1 am not convinced that the ebuild is bugged. Thanks, I agree. The problem has returned for me, please disregard my patch for now. I'll keep trying to get to the bottom of the problem. This issue returned for me today when I was trying to debug a segfault (caused by the command below), and I: updated boost to 1.56.0-r1 (may or may not be related) installed graph-tool-9999 (see https://bugs.gentoo.org/show_bug.cgi?id=536810) (note this specific problem, the undefined symbol, did not appear with 9999) re-installed graph-tool-2.2.36 (with the ebuild patched) I'd be interested to know if "/usr/bin/python3.3 -c 'import graph_tool.all as gt'" works for you. Thanks for the information about eutils being implicitly inherited by python-r1, I will remember that next time. well; $ python3.3 -c 'import graph_tool.all as gt' yields python3.3: symbol lookup error: /usr/lib64/python3.3/site-packages/graph_tool/libgraph_tool_core.so: undefined symbol: _ZN5boost6python6detail11init_moduleER11PyModuleDefPFvvE a true runtime error. however this is with 2.2.35 however I'd expect the same from 2.2.36. So the ebuild might work to build and install but a runtime error is something different again. Perhaps your patch has merit. I mentioned in comment #10 that I tried 9999, and the undefined symbol problem didnt exist, and that was correct, but the 9999 ebuild was/is using a outdated git repo from July 2014. After fixing the get repo with bug 536826, I recompiled and now the same 'undefined symbol' error happens. (In reply to Justin Keogh from comment #12) > I mentioned in comment #10 that I tried 9999, and the undefined symbol > problem didnt exist, and that was correct, but the 9999 ebuild was/is using > a outdated git repo from July 2014. > > After fixing the get repo with bug 536826, I recompiled and now the same > 'undefined symbol' error happens. and now for the bad news!! I think filing upstream would be apt here. If you don't or can't I can or redhermit OK, I think I (finally) know what's going on. First, the ebuild is fine the way it is. No patch needed. The problem is upstream almost for sure, here is how I worked around it: $PYTHON_TARGETS="python3_3" emerge boost -1 this obviously installs boost without py2.7 support. then: $emerge graph-tool then during the graph-tool ./configure this will happen: configure:18032: checking for exit in -lboost_python-2.7 configure:18057: x86_64-pc-linux-gnu-g++ -o conftest -Wall -Wextra -Wno-unused-local-typedefs -Wno-unused-parameter -march=native -O2 -pipe -DNDEBUG -std=gnu++11 -ftemplate-depth-250 -Wno-deprecated -Wno-unknown-pragmas -fvisibility=default -fvisibility-inlines-hidden -fopenmp -I/usr/include/python3.3 -Wl,-O1 -Wl,--as-needed -L/usr/lib64 -lpython3.3 conftest.cpp -lboost_python-2.7 -L/usr/lib64 -lpython3.3 -lexpat -lbz2 -lm >&5 /usr/lib/gcc/x86_64-pc-linux-gnu/4.8.3/../../../../x86_64-pc-linux-gnu/bin/ld: cannot find -lboost_python-2.7 collect2: error: ld returned 1 exit status configure:18057: $? = 1 configure: failed program was: | /* confdefs.h */ | #define PACKAGE_NAME "graph-tool" | #define PACKAGE_TARNAME "graph-tool" | #define PACKAGE_VERSION "2.2.36" | #define PACKAGE_STRING "graph-tool 2.2.36" | #define PACKAGE_BUGREPORT "http://graph-tool.skewed.de/tickets" | #define PACKAGE_URL "http://graph-tool.skewed.de" | #define PACKAGE "graph-tool" | #define VERSION "2.2.36" | #define STDC_HEADERS 1 | #define HAVE_SYS_TYPES_H 1 | #define HAVE_SYS_STAT_H 1 | #define HAVE_STDLIB_H 1 | #define HAVE_STRING_H 1 | #define HAVE_MEMORY_H 1 | #define HAVE_STRINGS_H 1 | #define HAVE_INTTYPES_H 1 | #define HAVE_STDINT_H 1 | #define HAVE_UNISTD_H 1 | #define __EXTENSIONS__ 1 | #define _ALL_SOURCE 1 | #define _GNU_SOURCE 1 | #define _POSIX_PTHREAD_SEMANTICS 1 | #define _TANDEM_SOURCE 1 | #define STDC_HEADERS 1 | #define HAVE_DLFCN_H 1 | #define LT_OBJDIR ".libs/" | #define USING_OPENMP 1 | #define HAVE_LIBM 1 | #define HAVE_LIBBZ2 1 | #define HAVE_LIBEXPAT 1 | #define HAVE_PYTHON "3.3" | #define HAVE_BOOST /**/ | #define HAVE_BOOST_PYTHON /**/ | /* end confdefs.h. */ | | /* Override any GCC internal prototype to avoid an error. | Use char because int might match the return type of a GCC | builtin and then its argument prototype would still apply. */ | #ifdef __cplusplus | extern "C" | #endif | char exit (); | int | main () | { | return exit (); | ; | return 0; | } configure:18067: result: no configure:18032: checking for exit in -lboost_python-3-mt That error, seems to cause the last line "checking for exit in -lboost_python-3-mt" to run. With boost py2.7 support, the ./configure log never shows the -lboost_python-3-mt check. now: /usr/bin/python3.3 -c 'import graph_tool' (which also would previously error our with undefined symbol) works. $/usr/bin/python3.3 -c 'import graph_tool ; print(graph_tool.__version__)' 2.2.36 (commit e3865108, Thu Jan 8 17:03:44 2015 +0100) unfortunatly, the original test command segfauls: $/usr/bin/python3.3 -c 'from graph_tool.all import *' Segmentation fault $tail -n 1 /var/log/messages Jan 17 04:56:28 localhost kernel: [24767.038816] python3.3[28655]: segfault at 16ae76 ip 000000000016ae76 sp 00007fff0d7f0a48 error 14 in libgmp.so.10.1.3[7f0cb76f3000+78000] but that's likely a different bug. I'll contact upstream. upstream contacted: http://thread.gmane.org/gmane.comp.science.graph.graph-tool.general/1789 well done Justin most impressive. It appears the recent form from upstream is py2 centric in its install of boost. Possibly a solution will need doing in the boost lib. This is out of more realm of usual practice, I think Mr. Harder might be better at this. the comments from /1789 is not exactly inspiring. "but it seems to be failing in this case" is where it all ends. As for what they're going to do about it, well, nothing planned it seems. He just settles for passively observing the failure then, ...... To my understanding, his suggestion would make the build suddenly receptive to py3.3, but I anticipate it would then be insensitive to py2.7. You (??) fix 1 thing only to break another. My realm doesn't cover C C++ builds in depth, so I am not offering any final fix. In this case I will only comment that the configure script for the build fails to serve py2 and py3, but rather py2 OR py3. Hardly satisfactory for gentoo trying to provide multi impl. builds. Created attachment 394322 [details, diff] graph-tool-2.2.36.ebuild.patch Thanks for the encouragement and help Ian, I think it's sorted out now, the attached patch makes all python versions work. Tested with (both 2.2.36 and 9999): $/usr/bin/python3.3 -c 'import graph_tool ; print(graph_tool.__version__)' 2.2.37dev (commit 201879ac, Fri Jan 9 15:46:17 2015 +0100) $/usr/bin/python2.7 -c 'import graph_tool ; print(graph_tool.__version__)' 2.2.37dev (commit 201879ac, Fri Jan 9 15:46:17 2015 +0100) I filed a duplicate bug 537034 for the 9999 version, the fix is the same, 9999 patch attached over there. Hat tip to #gentoo-dev-help *** Bug 537034 has been marked as a duplicate of this bug. *** well looks good however I shall leave the final commit to Mr. Harder. I wonder if the line --with-boost-python=${EPYTHON: -3}-mt could simply be added to the current content of src_configure() {. I am not clear whether this would suffice with use of python-r1 since it's an eclass I don't normally utilise, but the maintainer does. (You made a 'typo' with src_configure(){, a triviality. It works anyway I believe) Do you mean the lack of the space in "src_configure_py(){"? I see that's different, adding to my mental checklist. I used a second function because when I added it to the current src_configure() ${EPYTHON} didnt re-evaluate for each version. (In reply to Justin Keogh from comment #21) > Do you mean the lack of the space in "src_configure_py(){"? I see that's > different, adding to my mental checklist. > > I used a second function because when I added it to the current > src_configure() ${EPYTHON} didnt re-evaluate for each version. 1. yes. totally trivial. 2. In that case your patch looks spot on (In reply to Ian Delaney from comment #22) > 2. In that case your patch looks spot on Not quite, you're assuming everyone has boost built with threads enabled. Anyway, this should now be fixed in 2.2.36-r1 and the live ebuild. I should also note that this should really be handled in the boost-python m4 script, but we can hack around it for now. |