Recent versions of dev-libs/boost, say 1.42.0, install their libraries at /usr/lib/boost-1_42, with symlinks in /usr/lib that add the version number. app-admin/eselect-boost allows switching between boost versions if one happens to need to install more than one version. With the debug USE flag, debug libraries are installed into /usr/lib/boost-1_42-debug, with exactly the same names as the non-debug ones, and their symlinks in /usr/lib append -debug to the name. Whereas this approach allows people to quickly eselect between release and debug libraries (I don't see when this might be useful), it has one problem: both the debug library names in /usr/lib/boost-1.42-debug and those in /usr/lib break the library name convention for the boost build system. Because of this, and because boost isn't normally used with other systems such as pkg-config, it's hard to have a build system that automatically and portably finds the debug libraries in Gentoo. Reproducible: Always Actual Results: For example, the name for the installed boost_program_options library is libboost_program_options.a for the unversioned name in /usr/lib/boost-1_42-debug libboost_program_options-1_42-debug.a for the versioned name in /usr/lib. Neither can be found by the boost build system in a portable way. Expected Results: If the names for the debug libraries where libboost_program_options-d.a for the unversioned name in /usr/lib/boost-1_42-debug libboost_program_options-d-1_42.a for the versioned name in /usr/lib Then, the boost build system would be able to find them. This would eliminate the need (or the possibility) of eselecting between release and debug version, but does that outweight the need of writing portable Boost.Build jamfiles? The Boost.Build system includes a Jamfile, tools/build/v2/contrib/boost.jam, which allows people that use it to simply add atoms as /boost//program_options in their Jamfiles to have the libraries locates for them by the build system in a portable way. Breaking the naming convention used by the build system, eliminates this possibility. A programmer using boost for developing (and that's the reason for using debug libraries) has to resort either to using unportable or manual means for locating the libraries or keeping a separate copy of boost, not maintained by portage, that keeps the naming convention (that's what I have been doing). While most boost programmers don't actually use the boost build system, there also is no alternative (say, no pkg-config file); but if there was one, it wouldn't be as portable (it would have to be expected that on other systems, pkg-config was working too). But then, if a home directory copy is necessary for developers, why have the ebuild install debug libraries at all? Who uses debug libraries and does the (IMO, questionable) ability of eselecting between debug and release builds outweight this? Consider my point, Miguel Ramos
This also affects CMake's ability to find the boost libraries automatically and makes multi-platform Boost projects even more of a nightmare than they already are. Please fix ASAP.
(In reply to comment #0) > Recent versions of dev-libs/boost, say 1.42.0, install their libraries at > /usr/lib/boost-1_42, with symlinks in /usr/lib that add the version number. > app-admin/eselect-boost allows switching between boost versions if one happens > to need to install more than one version. > > With the debug USE flag, debug libraries are installed into > /usr/lib/boost-1_42-debug, with exactly the same names as the non-debug ones, > and their symlinks in /usr/lib append -debug to the name. > > Whereas this approach allows people to quickly eselect between release and > debug libraries (I don't see when this might be useful), it has one problem: > both the debug library names in /usr/lib/boost-1.42-debug and those in /usr/lib > break the library name convention for the boost build system. > > Because of this, and because boost isn't normally used with other systems such > as pkg-config, it's hard to have a build system that automatically and portably > finds the debug libraries in Gentoo. > > > Reproducible: Always > > Actual Results: > For example, the name for the installed boost_program_options library is > > libboost_program_options.a for the unversioned name in > /usr/lib/boost-1_42-debug This is by design such that every boost version+variation combination can be used by just appending a corresponding '-L...' flag to the CFLAGS. Mainly because most build system look for the name only and ignore boost libraries with the version or tags in the name. > libboost_program_options-1_42-debug.a for the versioned name in /usr/lib. > > Neither can be found by the boost build system in a portable way. > > > Expected Results: > If the names for the debug libraries where > > libboost_program_options-d.a for the unversioned name in > /usr/lib/boost-1_42-debug This is not going to happen for the reason mentioned above. > libboost_program_options-d-1_42.a for the versioned name in /usr/lib I guess we could do that. > Then, the boost build system would be able to find them. This would eliminate > the need (or the possibility) of eselecting between release and debug version, > but does that outweight the need of writing portable Boost.Build jamfiles? Well, the eselect module was thought of a transition step between unslotted and slotted boost versions to make sure that ebuilds depending on boost still work. > > The Boost.Build system includes a Jamfile, tools/build/v2/contrib/boost.jam, > which allows people that use it to simply add atoms as /boost//program_options > in their Jamfiles to have the libraries locates for them by the build system in > a portable way. > Breaking the naming convention used by the build system, eliminates this > possibility. A programmer using boost for developing (and that's the reason for > using debug libraries) has to resort either to using unportable or manual means > for locating the libraries or keeping a separate copy of boost, not maintained > by portage, that keeps the naming convention (that's what I have been doing). Well, we're using boost.build to get that -debug in the name by passing "--buildid=debug". Do you have an alternative?
Hi, and sorry for taking such a long time to respond (even though I use boost on Gentoo every day). When using the Boost.Build system for building boost, as you know, there are three possible layouts for the names of the libraries: system, tagged and versioned. Using "system" and putting everything into /usr/local/lib is the natural default for UNIX, and good for most Linux distributions. Of course, not Gentoo. We want many versions of boost to coexist peacefully. I agree with these purposes and I agree with this: (In reply to comment #2) (...) > This is by design such that every boost version+variation combination can be > used by just appending a corresponding '-L...' flag to the CFLAGS. Mainly > because most build system look for the name only and ignore boost libraries > with the version or tags in the name. ... even though I think being able to eselect between release and debug variants, or getting an application to link to release or debug variants through CFLAGS is a bad idea and only works due to unplanned coincidences. I agree with that for version selection, though. But let's look closely at the part we both agree, (...) > Well, we're using boost.build to get that -debug in the name by passing > "--buildid=debug". > Do you have an alternative? Well, IMO that points to the root of all problems. By using --buildid, you create a new filename layout of your own and your users not only have to account for the three possible filename layouts that boost may have on different platforms, but also account for that new layout you just created for Gentoo. But that's not the only new layout you created. The filenames for the shared and static versions of program_options are, a) using "system" layout: libboost_program_options.a // and likewise for the debug or mt variant (it collides...) libboost_program_options.so.1.42.0 // and likewise for debug or mt b) using "tagged" layout: libboost_program_options.a libboost_program_options.so.1.42.0 libboost_program_options-d.a libboost_program_options-d.so.1.42.0 libboost_program_options-mt.a libboost_program_options-mt.so.1.42.0 libboost_program_options-mt-d.a libboost_program_options-mt-d.so.1.42.0 c) using "versioned" layout: libboost_program_options-gcc44-1_42.a libboost_program_options-gcc44-1_42.so.1.42.0 libboost_program_options-gcc44-d-1_42.a libboost_program_options-gcc44-d-1_42.so.1.42.0 libboost_program_options-gcc44-mt-1_42.a libboost_program_options-gcc44-mt-1_42.so.1.42.0 libboost_program_options-gcc44-mt-d-1_42.a libboost_program_options-gcc44-mt-d-1_42.so.1.42.0 Now, the ebuild and installs, omitting the static libraries, a) on /usr/lib: libboost_program_options-1_42-debug.so // Hum, looks like "versioned" but isn't... (call it "a") libboost_program_options-1_42-debug.so.1.42.0 // " libboost_program_options-1_42.so // " libboost_program_options-1_42.so.1.42.0 // " libboost_program_options-mt-1_42-debug.so // " libboost_program_options-mt-1_42-debug.so.1.42.0 // " libboost_program_options-mt-1_42.so // " libboost_program_options-mt-1_42.so.1.42.0 // " libboost_program_options.so // This one looks like "system", but it's not, this is linked to the non-reentrant runtime libboost_program_options-mt.so // so, it matches "tagged" but only when using the release variant (call it "b") b) on /usr/lib/something: libboost_program_options.so // Again, this is the one that matches "tagged" but only when using libboost_program_options-mt.so // the release variant So, first, you have two new layouts "a" and "b" none of which is an exact match to the layouts generated (and looked for) by Boost.Build. No build system or hand made makefile will find the files of "a" unless it's specifically written for your ebuilds. Trying to manage portable Jamfiles, how can I tell Boost.Build where to find the files? - I can say there's an installation of boost in /usr/lib64 or /usr/lib64/something using the "tagged" layout. This works on release builds, but not on debug builds because "-d" is missing. - I can say there's an installation in /usr/lib64/something-debug using the "system" layout. Works on debug single-threaded builds, but not debug where you must use the reentrant runtime. - It's pointless to try to make it look for your "a" layout. --------- Now, let's discuss it. Allow me a couple of rhetorical questions. Q1 - Why doesn't the "a" layout match the "versioned" layout? It has most of the elements of it, except it lacks the toolset (which afects ABI... now, I know on Linux all compilers follow the gcc ABI, but can we coexist with platforms where that isn't true? ... and even gcc changes ABI sometimes) and except the "-debug" thing. I think the "versioned" layout, exactly as it comes out of Boost.Build, would be the most sensible solution for files in /usr/lib: - different versions don't collide, - different build variants don't collide, - automated build systems (makefiles don't count) already know how to look for them. Q2 - On /usr/lib/something, you named both your debug and release variants libboost_program_options.so. In this manner, you may force building a project with a specific variant using CFLAGS or eselect. Why did you not do the same with the single-threaded / multi-threaded variants? You may think it's silly to be able to choose the mt variant this way, but I think the situation is exactly the same with debug/release variants. Plus, this rhetorical alternative would have an advantage: the file names would follow one of the layouts which is output from Boost.Build, the "system" layout. When "system" is used, one can use -lboost_program_options in all cases. Conversely, only one of this variants must be accessible to ld. By the way, that's what the boost people found to be the most appropriate default on UNIX systems... Because on a user's computer there isn't much point in having the debug variant or the non-mt one (also no point in having several versions if all went well). I don't know if I explained clearly why I say that your "short" layout doesn't match the "tagged" layout: because if it was the "tagged" layout, someone wanting to build a debug multithreaded program would use -mt-d, not just -mt. ---------- My proposals, P0 - Use the "versioned" layout for files in /usr/lib. These should be the real files, everything else symlinks. P1 - includes P0, adds: - on /usr/lib, created by eselect, 2 symlinks for each library, like this: libboost_program_options.a -> libboost_program_options-gcc44-mt-1_42.a libboost_program_options.so -> libboost_program_options-gcc44-mt-1_42.so.1.42.0 - the eselect module would allow either: -- selecting between version, runtime (mt or not) and variant (debug or not). -- simply selecting between versions, always pointing to the non-debug -mt variant of that version (the natural choice). - eliminate the /usr/lib/something subdirs as they seem redundant. That is, it seems that you are giving too many different ways of achieving the same goal. This proposal, P1, would be my choice. P2 - as P1, except: - don't eliminate the /usr/lib/something subdirs, but instead of having two for each version, have one for each version/variant, such as: /usr/lib/boost-1_42 /usr/lib/boost-1_42-mt /usr/lib/boost-1_42-d /usr/lib/boost-1_42-mt-d In each of these, the "system" layout would be used. P3 - includes P0, adds: - on /usr/lib, created by eselect, symlinks for each library variant, using the "tagged" layout, like this: libboost_program_options.so -> libboost_program_options-gcc44-1_42.so.1.42.0 libboost_program_options-mt.so -> libboost_program_options-gcc44-mt-1_42.so.1.42.0 libboost_program_options-d.so -> libboost_program_options-gcc44-d-1_42.so.1.42.0 libboost_program_options-mt-d.so -> libboost_program_options-gcc44-mt-d-1_42.so.1.42.0 - the eselect module would allow selecting only between version (that seems logical to me). - eliminate the /usr/lib/something subdirs. P4 - relates to P3 as P2 does to P1. Now, both P3 and P4 have one disadvantage: in none of these cases you can safely use a simple -lboost_program_options for linking, because the "system" layout is not used. That is, most commonly, some third-party package that wants to -lboost_program_options will want the -mt version, almost for sure. This disadvantage of P3 and P4 is already present in your current design. And that is why the "system" layout is used by default on UNIX and not "tagged": The most natural way of compiling a program on UNIX is using -lboost_program_options and someone who uses -pthread -lboost_program_options will have on Gentoo with your ebuilds a program which is likely to have problems at runtime. P* - Additional ideas: - Don't install non -mt libraries by default (you also don't install static or debug libraries by default). The -mt ones seem sufficient and you get lower CO2 emissions when emerging. - Add a symlink from bjam. It would be wonderful to use the bjam command as on other systems, instead of bjam-1_42. What do you think?
Also, I would go with my P1 or P2, where two sets of filenames exist ("versioned" and "system") but where the "tagged" filename layout is completely avoided. The "tagged" layout has a huge problem: a simpler build system which would look for libboost_program_options.a would never know that this was the library that linked to the non-reentrant runtime. The program would compile correctly and run fine most of the time, but could suffer from serious problems. My point is that linking to -mt when you only needed the non-reentrant version has consequences that are much less serious than linking to the non-mt in a threaded program. Choosing to deprecate the "tagged" layout altogether is a safer alternative and less likely to break dependent packages. In any case, I see no argument for inventing new filename layouts for Gentoo.
And this finally is resolved (I'm actually going to commit this in a minute, USE=debug replaces the libraries directly).
(In reply to comment #5) > And this finally is resolved (I'm actually going to commit this in a minute, > USE=debug replaces the libraries directly). Is your fix the current boost-1.52.0-r2 ebuild? I tried rebuilding with USE=debug and I get only one set of libraries. The libs are slightly bigger than those installed without USE=debug. Is the ebuild installing only debug libraries with USE=debug? What is the reasoning? What use could that be?
