Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 278667 - Switching libuuid/libblkid from e2fsprogs-libs-1.41.8 to util-linux breaks on BSD and Prefix
Summary: Switching libuuid/libblkid from e2fsprogs-libs-1.41.8 to util-linux breaks on...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High major with 1 vote (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 309247
  Show dependency tree
 
Reported: 2009-07-22 11:30 UTC by Alexis Ballier
Modified: 2016-05-18 20:07 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alexis Ballier gentoo-dev 2009-07-22 11:30:59 UTC
As I understand it, libuuid has been moved from e2fsprogs-libs to util-linux, which means that we don't have libuuid anymore on FreeBSD (and is required by a couple of packages).

I'm not sure how to fix this:
- make building libuuid in e2fsprogs-libs conditional on !kernel_linux useflag
- get it from util-linux(?!) even on BSD

The former is what I've done locally but that still leaves e2fsprogs deps broken, I haven't tried the latter and this will probably require to strip down a bit util-linux not to install too many things on BSD.

Any advice on how to handle this is welcome.
Comment 1 Diego Elio Pettenò (RETIRED) gentoo-dev 2009-07-22 11:33:16 UTC
Or we could start looking at what uses libuuid and fix it to use the FreeBSD-style uuid functions that are in the C library like I did for libSM.
Comment 2 Alexis Ballier gentoo-dev 2009-07-22 11:36:36 UTC
(In reply to comment #1)
> Or we could start looking at what uses libuuid and fix it to use the
> FreeBSD-style uuid functions that are in the C library like I did for libSM.

IMHO this is a good long term goal but as it is now a g/fbsd system with e2fsprogs-libs-1.41.8 is half broken.
Comment 3 Diego Elio Pettenò (RETIRED) gentoo-dev 2009-07-22 11:53:51 UTC
What does bring it in? Beside libSM for which I already have a bug open here, too.
Comment 4 Alexis Ballier gentoo-dev 2009-07-22 12:40:10 UTC
(In reply to comment #3)
> What does bring it in? Beside libSM for which I already have a bug open here,
> too.

preserved-rebuild tells me this:
[ebuild   R   ] x11-libs/libSM-1.1.0  USE="ipv6 -debug" 0 kB
[ebuild   R   ] media-sound/mpg123-1.8.1  USE="3dnow 3dnowext ipv6 mmx network oss sdl sse (-alsa) (-altivec) (-jack) -nas (-portaudio) -pulseaudio" 0 kB
[ebuild   R   ] xfce-extra/xfce4-appfinder-4.6.1  USE="-debug" 0 kB
[ebuild   R   ] xfce-extra/xfce4-mixer-4.6.1  USE="oss (-alsa) -debug" 0 kB
[ebuild   R   ] app-office/orage-4.6.1  USE="dbus libnotify -berkdb -debug" 0 kB
[ebuild   R   ] media-libs/libcaca-0.99_beta16  USE="X imlib ncurses opengl slang -doc -mono -nocxx -ruby" 0 kB
[ebuild     U ] media-sound/pulseaudio-0.9.15-r51 [0.9.15-r50] USE="X dbus glib hal libsamplerate oss tcpd test (-alsa) -asyncns -avahi -bluetooth (-caps) -gnome (-jack) (-lirc) (-policykit)" 0 kB
[ebuild   R   ] media-gfx/imagemagick-6.5.2.9  USE="X bzip2 corefonts djvu jpeg perl png raw svg tiff truetype xml zlib -doc -fontconfig -fpx -graphviz -gs -hdri -jbig -jpeg2k -lcms -nocxx -openexr -openmp -q32 -q8 -wmf" 0 kB
[ebuild   R   ] xfce-base/xfce4-session-4.6.1  USE="-debug -gnome" 0 kB
[ebuild   R   ] x11-terms/terminal-0.2.12  USE="dbus -debug -doc -startup-notification" 0 kB


A lot of them are probably libtool injecting libSM deps; and there is also e2fsprogs.
Comment 5 Alexis Ballier gentoo-dev 2009-07-22 14:01:02 UTC
On the other hand, we also lost libblkid in that move (required by latest hal but it claims to need 1.43 while e2fsprogs-libs provide 1.41.x...);
For now I'll -x86-fbsd e2fsprogs*-1.41.8 until we find an agreement on how to handle this.
Comment 6 SpanKY gentoo-dev 2009-07-23 00:48:17 UTC
having e2fsprogs-libs provide the libs for non-linux systems is a short term solution too.  the long term plan is to remove the libs completely from the package leaving only util-linux.

util-linux is not linux-specific ... they actively take patches to make all the utils portable to POSIX systems (when it makes sense).
Comment 7 Alexis Ballier gentoo-dev 2009-07-24 07:08:54 UTC
(In reply to comment #6)
> having e2fsprogs-libs provide the libs for non-linux systems is a short term
> solution too.  the long term plan is to remove the libs completely from the
> package leaving only util-linux.
> 
> util-linux is not linux-specific ... they actively take patches to make all the
> utils portable to POSIX systems (when it makes sense).
> 

The problem is that the whole thing has a lot of stuff that isn't useful with a bsd userland, some of its binaries already exist, some others have equivalent.
The solution I see would be to go compile & install from the shlibs dir and skip the rest of util-linux. Moreover I have to check if we can't live without libblkid/libuuid.
Comment 8 Fabian Groffen gentoo-dev 2009-07-27 18:37:39 UTC
how about a virtual for libuuid and libblkid that is satisfied by util-linux on linux and by a new package that provides both libs (from e2fsprogs?) for everything else?
Comment 9 SpanKY gentoo-dev 2009-07-27 18:57:14 UTC
as i said, relying on e2fsprogs isnt going to solve the problem long term.  either you dont use the library, or it comes from util-linux (whether we choose to split the libs from util-linux is one possibility, but it'll still be from util-linux).
Comment 10 Fabian Groffen gentoo-dev 2009-07-27 19:44:32 UTC
ok, I tried building util-linux-2.16.  The configure script seems to detect that it's on non-linux and that certain components shouldn't be built, however, it feels major work is necessary given the amount of errors spit out and the almost immediate compilation die.  I don't see obvious things in the ebuild that are enabled but that should be disabled for non-Linux.
Comment 11 Marcus Becker 2009-10-27 19:10:01 UTC
(In reply to comment #10)
> ok, I tried building util-linux-2.16.  The configure script seems to detect
> that it's on non-linux and that certain components shouldn't be built, however,
> it feels major work is necessary given the amount of errors spit out and the
> almost immediate compilation die.  I don't see obvious things in the ebuild
> that are enabled but that should be disabled for non-Linux.
> 

I think this is related, on my ~amd64 it pulls in the 2.16.1 util-linux but doesn't build the uuid libs. This breaks e2fsprogs, so it complains cannot find external uuid lib. I cannot provide more info, this is the laptop I post from.
e2fsprogs-1.41.9 and util-linux-2.16.1 the util-linux-2.16.1-r1 works fine, this should be unmasked soon.
Comment 12 Marcus Becker 2009-10-27 19:15:06 UTC
I meant 
util-linux-2.16-r1 works fine
util-linux-2.16.1 doesn't build uuid lib
Comment 13 Fabian Groffen gentoo-dev 2010-04-03 15:50:52 UTC
ok, util-linux needs some patching then to work.  The shlibs depend on the general includes which need help to e.g. figure out the endianness of the platform.  It doesn't look to bad for now, libuuid compiles with a small fix for non-GNU linkers, libblkid suffers from an include which needs to know the platform's byte-order.

I'll see if I can get the two compiling on Darwin and Solaris.
Comment 14 Fabian Groffen gentoo-dev 2010-04-06 20:21:26 UTC
ok, my attempt/hack is in http://overlays.gentoo.org/proj/alt/changeset/57629

It doesn't work on Solaris, since configure is still checking for stuff it doesn't need (crypt in particular) and dies if it isn't available.
Comment 15 Fabian Groffen gentoo-dev 2010-12-06 17:12:49 UTC
Does anybody know if there is a bugtracker or something for util-linux?  The patches are quite piling up here.
Comment 16 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2010-12-06 17:23:57 UTC
(In reply to comment #15)
> Does anybody know if there is a bugtracker or something for util-linux?  The
> patches are quite piling up here.
> 

From the homepage:

Mailing list
http://vger.kernel.org/vger-lists.html#util-linux-ng
This is the util-linux development discussion and bug report mailing list. 

http://userweb.kernel.org/~kzak/util-linux-ng/
Comment 17 SpanKY gentoo-dev 2010-12-06 18:06:10 UTC
yes, there is only the util-linux mailing list.  it's a normal git project in that way.  the maintainer is pretty responsive.  but the homepage is out of date as it is now:
http://vger.kernel.org/vger-lists.html#util-linux

however, in looking at your patches, there are problems:
 - i think the endian handling has been resolved upstream
 - the program_invocation_short_name -> argv[0] is not acceptable
 - binding GNU_LD to --version-script isnt really correct ... better to convert it to the libtool version script handling so that it is portable
Comment 18 Fabian Groffen gentoo-dev 2010-12-06 18:10:40 UTC
(In reply to comment #17)
> yes, there is only the util-linux mailing list.  it's a normal git project in
> that way.  the maintainer is pretty responsive.  but the homepage is out of
> date as it is now:
> http://vger.kernel.org/vger-lists.html#util-linux
> 
> however, in looking at your patches, there are problems:

Ok, I'm working on a whole new set right now.

>  - i think the endian handling has been resolved upstream

I checked git, but couldn't find proof of that.  I'll have to check with git later.  Now I need to get util-linux ASAP compiling, since someone was so kind to migrate e2fstools to gx86, and only the latest version which depends on util-linux.

>  - the program_invocation_short_name -> argv[0] is not acceptable

Ok, do you have a suggestion for an alternative?

>  - binding GNU_LD to --version-script isnt really correct ... better to convert
> it to the libtool version script handling so that it is portable

Do you mean the -version-info stuff of libtool?  I didn't know that was the same.  Indeed that would be much better.

Thanks
Comment 19 SpanKY gentoo-dev 2010-12-06 18:53:24 UTC
hmm, ok, i looked at the history.  i complained recently that we still have crap checking __BYTE_ORDER instead of just relying on the output of AC_C_BIGENDIAN.  so if your patch moves away from __BYTE_ORDER and to that macro (i guess it outputs WORDS_BIGENDIAN?), then please push that change upstream.  you might also want to grep the tree for other __BYTE_ORDER consumers as at least `lscpu` has been updated in git.

the whole code base assumes program_invocation_short_name exists.  changing a few random files to not use it isnt the right approach.  if you wanted to generalize it, add a configure test for the program_invocation_short_name symbol and change all of the uses (so grep the tree) to something like:
static char get_program_invocation_short_name(char *argv[])
{
#ifdef HAVE_PROGRAM_INVOCATION_SHORT_NAME
    return program_invocation_short_name;
#else
    return argv[0];
#endif
}
(stick that in some common/util header)

as for libtool, i'm talking about the options -export-symbols/-export-symbols-regex.  but in reviewing the files, i see they are actually using version info in the files and not just a list of symbols to control the export.  so best way to approach that i think is:
 - add configure test for -Wl,--version-script flag (sandbox has a check for this if you want to use that as a base)
 - if the -Wl,--version-script exists, use it
 - if it doesnt, fall back to -export-symbols by autogenerating the symbol files from the .sym files in the source tree:
blkid.libtool.sym: $(srcdir)/blkid.sym
    $(EGREP) '^[[:space:]]*blkid' $< > $@
libblkid_la_LDFLAGS += -export-symbols $(srcdir)/blkid.sym
Comment 20 Fabian Groffen gentoo-dev 2010-12-06 19:01:19 UTC
(In reply to comment #19)
> hmm, ok, i looked at the history.  i complained recently that we still have
> crap checking __BYTE_ORDER instead of just relying on the output of
> AC_C_BIGENDIAN.  so if your patch moves away from __BYTE_ORDER and to that
> macro (i guess it outputs WORDS_BIGENDIAN?), then please push that change
> upstream.  you might also want to grep the tree for other __BYTE_ORDER
> consumers as at least `lscpu` has been updated in git.

I just checked git, and it seems my patch indeed still applies/is necessary.
I'll have to split the patch on overlays now into a couple of sub-patches each dealing with just a single issue.

> the whole code base assumes program_invocation_short_name exists.  changing a
> few random files to not use it isnt the right approach.  if you wanted to
> generalize it, add a configure test for the program_invocation_short_name
> symbol and change all of the uses (so grep the tree) to something like:
> static char get_program_invocation_short_name(char *argv[])
> {
> #ifdef HAVE_PROGRAM_INVOCATION_SHORT_NAME
>     return program_invocation_short_name;
> #else
>     return argv[0];
> #endif
> }
> (stick that in some common/util header)

Hmmm, I need to check if it isn't used outside of main.  The argv[0] thing was just a hack that worked for the cases I found.

> as for libtool, i'm talking about the options
> -export-symbols/-export-symbols-regex.  but in reviewing the files, i see they
> are actually using version info in the files and not just a list of symbols to
> control the export.  so best way to approach that i think is:
>  - add configure test for -Wl,--version-script flag (sandbox has a check for
> this if you want to use that as a base)
>  - if the -Wl,--version-script exists, use it
>  - if it doesnt, fall back to -export-symbols by autogenerating the symbol
> files from the .sym files in the source tree:
> blkid.libtool.sym: $(srcdir)/blkid.sym
>     $(EGREP) '^[[:space:]]*blkid' $< > $@
> libblkid_la_LDFLAGS += -export-symbols $(srcdir)/blkid.sym

Ok, I'll have to look into this after I finish fixing all the compilation problems on shlib/* only :/
Comment 21 Fabian Groffen gentoo-dev 2011-01-08 20:22:53 UTC
(In reply to comment #19)
> hmm, ok, i looked at the history.  i complained recently that we still have
> crap checking __BYTE_ORDER instead of just relying on the output of
> AC_C_BIGENDIAN.  so if your patch moves away from __BYTE_ORDER and to that
> macro (i guess it outputs WORDS_BIGENDIAN?), then please push that change
> upstream.  you might also want to grep the tree for other __BYTE_ORDER
> consumers as at least `lscpu` has been updated in git.

partial progress:

http://marc.info/?l=util-linux-ng&m=129451378214864&w=2
Comment 22 Fabian Groffen gentoo-dev 2011-01-17 19:17:21 UTC
(In reply to comment #6)
> util-linux is not linux-specific ... they actively take patches to make all the
> utils portable to POSIX systems (when it makes sense).

I highly doubt this.  The rate at which unportable code is added exceeds by far the rate in which I can submit sane fixes.

I think I'll throw away most of the patches here.  util-linux is bound to be a nightmare on non-Linux hosts.

I only know of a requirement for libuuid.  libblkid I haven't encountered iirc, but it's close to impossible to compile on non-Linux hosts these days.  libuuid compiles with the patches I have here now, but it is known to break Darwin 10 hosts, so it should be preferred to use Darwin and Solaris' host provided uuid for packages that want it.

I guess building only libuuid for systems that lack it, is the easiest solution for now, and seems to work with the current patches I have.
Comment 23 SpanKY gentoo-dev 2011-01-18 20:16:09 UTC
i didnt say util-linux goes out of their way to maintain non-Linux compatibility.  i said they take patches from people to improve things.
Comment 24 Fabian Groffen gentoo-dev 2014-01-14 20:48:43 UTC
I feel a virtual for uuid would still be best thing, since FreeBSD, Darwin and Solaris provide their own implementations (and the util-linux one will break system headers).

Next thing would be getopt/getopt-long for which utils-linux is pulled in.  This is currently solved by ||-deps, although not always as clean as it should be.

For me the route of having util-linux in one way or another on non-linux systems seems pointless.  We have uuid and getopt implementations ready.
Comment 25 SpanKY gentoo-dev 2014-01-14 23:16:35 UTC
(In reply to Fabian Groffen from comment #24)

a virtual/libuuid would be fine by me for cases where the OS itself provides it

i've been talking to upstream about making it easier to just build the libraries, so for cases where the OS doesn't provide libuuid, we should be able to handle that too (on non-Linux)
Comment 26 SpanKY gentoo-dev 2014-08-05 10:31:58 UTC
e2fsprogs-libs still includes/updates libuuid/libblkid.  so i could update them to ship things in newer versions so you aren't stuck on an old version ...
Comment 27 SpanKY gentoo-dev 2014-08-05 10:40:50 UTC
Commit message: Re-enable libuuid/libblkid for non-Linux platforms
http://sources.gentoo.org/sys-libs/e2fsprogs-libs/e2fsprogs-libs-1.42.11.ebuild?r1=1.2&r2=1.3