Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 457134 - app-arch/bzip2, sys-libs/zlib: please discuss the possibility of introducing multilib-build.eclass support
Summary: app-arch/bzip2, sys-libs/zlib: please discuss the possibility of introducing ...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords:
Depends on: 473370 473610 474000
Blocks: gx86-multilib 477956 455070 474356
  Show dependency tree
 
Reported: 2013-02-13 17:56 UTC by Michał Górny
Modified: 2013-07-24 00:31 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
bzip2-1.0.6-r4.ebuild (bzip2-1.0.6-r4.ebuild,2.55 KB, text/plain)
2013-06-15 09:29 UTC, Pacho Ramos
Details
zlib-1.2.8-r1.ebuild (zlib-1.2.8.ebuild,2.75 KB, text/plain)
2013-06-21 07:52 UTC, Pacho Ramos
Details
bzip2-1.0.6-r4.ebuild (bzip2-1.0.6-r4.ebuild,2.55 KB, text/plain)
2013-06-21 07:55 UTC, Pacho Ramos
Details
zlib-1.2.8-r1.ebuild (zlib-1.2.8.ebuild,2.79 KB, text/plain)
2013-06-21 10:50 UTC, Pacho Ramos
Details
Diff between bzip2 ebuilds (bzip2.diff,2.41 KB, text/plain)
2013-06-21 16:21 UTC, Pacho Ramos
Details
Diff between zlib ebuilds (zlib.diff,3.63 KB, text/plain)
2013-06-21 16:21 UTC, Pacho Ramos
Details
bzip2-1.0.6-r4.ebuild (bzip2-1.0.6-r4.ebuild,2.50 KB, text/plain)
2013-06-21 16:27 UTC, Pacho Ramos
Details
zlib-1.2.8-r1.ebuild (zlib-1.2.8.ebuild,2.74 KB, text/plain)
2013-06-21 16:32 UTC, Pacho Ramos
Details
bzip2-1.0.6-r4.ebuild (bzip2-1.0.6-r4.ebuild,2.51 KB, text/plain)
2013-06-21 19:17 UTC, Pacho Ramos
Details
zlib-1.2.8-r1.ebuild (zlib-1.2.8.ebuild,2.77 KB, text/plain)
2013-06-21 19:20 UTC, Pacho Ramos
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-13 17:56:05 UTC
A major point of the multilib project would be replacing emul-linux-x86-baselibs with multilib USEflags on the relevant libraries. In a very short view, this includes app-arch/bzip2 and sys-libs/zlib which are indeed @system packages.

There's a few issues that we need to handle and therefore I'd like to ask your opinion before starting to do anything.

The first large-scale issue is that the multilib-build eclass requires EAPI 5 due to PMS restrictions on USE_EXPAND. I believe that you'd like to preserve the lowest EAPI on system packages.

As the relevant packages do not have any run-time dependencies, they don't really need the EAPI 5-requiring features. I could enable older EAPIs for that reason but I don't want to do that unless necessary as that would encourage people to actually write ebuilds with incorrect deps.

Therefore, my first question would be: what are the perspectives? What are your view on the multilib attempt? If we agree on replacing emul-linux packages, would you use the multilib-build eclass?
Comment 1 Karl Lindén 2013-04-03 17:52:08 UTC
(In reply to comment #0)
> A major point of the multilib project would be replacing
> emul-linux-x86-baselibs with multilib USEflags on the relevant libraries. In
> a very short view, this includes app-arch/bzip2 and sys-libs/zlib which are
> indeed @system packages.

Indeed. We would get much cleaner dependencies of other packages migrated to multilib eclasses. Personally I don't really like depending on the emul-linux-x86-* packages as their composition might (will, in the long run, hopefully) change. Depending directly on a package avoids this issue.

> There's a few issues that we need to handle and therefore I'd like to ask
> your opinion before starting to do anything.
> 
> The first large-scale issue is that the multilib-build eclass requires EAPI
> 5 due to PMS restrictions on USE_EXPAND. I believe that you'd like to
> preserve the lowest EAPI on system packages.
> 
> As the relevant packages do not have any run-time dependencies, they don't
> really need the EAPI 5-requiring features. I could enable older EAPIs for
> that reason but I don't want to do that unless necessary as that would
> encourage people to actually write ebuilds with incorrect deps.

This might seem like a bad question... What are the benefits of preserving a low EAPI for system packages?

> Therefore, my first question would be: what are the perspectives? What are
> your view on the multilib attempt? If we agree on replacing emul-linux
> packages, would you use the multilib-build eclass?

Personally, I think the attempt is definetely good. It gives more control to the user when it comes to the packages installed. I would (indirectly) use the multilib-build eclass (through autotools-multilib for example) if I were to replace the emul-linux packages. But the question might not be directed to me.
Comment 2 Samuli Suominen (RETIRED) gentoo-dev 2013-04-03 18:00:43 UTC
zlib for instance is not autotools based package, it has it's own ./configure script, and the multilib headers wrapper is not available yet on multilib-build.eclass (yet). perhaps this is a bit premature, before it's in?

carry on :)
Comment 3 Karl Lindén 2013-04-03 18:20:34 UTC
(In reply to comment #2)
> zlib for instance is not autotools based package, it has it's own
> ./configure script, and the multilib headers wrapper is not available yet on
> multilib-build.eclass (yet). perhaps this is a bit premature, before it's in?
> 
> carry on :)

Yeah, it might be. The thing is that it is hard to replace packages that can be found in in the emul-linux packages with multilib ones without the most basic packages to depend on. Especially when conflicts arise. For example I tried making a multilib ebuild for jack-audio-connection-kit in the pro-audio overlay. The problem was that it needed some sound libraries (libsndfile, alsa-lib, ...) so it would have to depend on emul-linux-x86-soundlibs but that was impossible as the same package was providing libjack.so, which led to a nasty conflict.

The best way to start this type of migration is from the low-level libraries, I think, as that would give us a clean dependency tree from start which is equal to less work later on (with migrating dependencies in revdep packages).
Comment 4 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-04-03 18:47:45 UTC
(In reply to comment #2)
> zlib for instance is not autotools based package, it has it's own
> ./configure script, and the multilib headers wrapper is not available yet on
> multilib-build.eclass (yet). perhaps this is a bit premature, before it's in?

This is *asking*. If I know there's no interest anyway, I won't hurry with providing the necessary facilities.
Comment 5 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-06-11 07:54:04 UTC
Ping. We're waiting here for you, base-system.
Comment 6 Doug Goldstein (RETIRED) gentoo-dev 2013-06-12 00:16:48 UTC
(In reply to Michał Górny from comment #5)
> Ping. We're waiting here for you, base-system.

So the reality is that the 13.0 profiles are EAPI 5, so we already have a requirement on a package manager that support EAPI 5 to use those profiles so I think its ok that we start migrating some packages to EAPI 5.


NOW for the caveat. We need to support upgrading, IMHO so we'll likely keep an older version of a number of packages necessary to complete the upgrade to a package manager that supports EAPI 5. But we can likely do this with a VM that we keep at an old version. Snapshot it and upgrade it and see that it still works and then revert to the snapshot and automate this. Unfortunately, that requires some man power.

Just my 2 cents.
Comment 7 Pacho Ramos gentoo-dev 2013-06-12 08:22:51 UTC
Thanks :)

Regarding upgrading from old setups, this information was recently added to the docs (I haven't tried it myself but...)
http://www.gentoo.org/doc/en/gentoo-upgrading.xml#old_system

Not sure if that would be enough
Comment 8 Pacho Ramos gentoo-dev 2013-06-15 09:29:18 UTC
Created attachment 350984 [details]
bzip2-1.0.6-r4.ebuild

This ebuild supports multilib via multilib-minimal.eclass
Comment 9 SpanKY gentoo-dev 2013-06-17 06:50:00 UTC
i thought i already posted to this bug.  guess not.

i'm not really interested in moving these up to EAPI=5 at this time.  i'm fine with EAPI=4 though.
Comment 10 Pacho Ramos gentoo-dev 2013-06-17 18:35:47 UTC
What is the reason for preferring eapi4 over eapi5? I guess multilib-minimal.eclass could also allow eapi4 like autotools-multilib.eclass... but I don't see a big jump between eapi4 and 5 (as compared between eapi1 and 2 for example)

Thanks!
Comment 11 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-06-20 12:05:00 UTC
I think the attached ebuild will suffice with s/EAPI=5/EAPI=4/. Are we over an agreement to commit that?
Comment 12 Pacho Ramos gentoo-dev 2013-06-21 06:41:37 UTC
Per vapier's comment, he looks ok if kept in eapi4 (that is now possible). Only zlib migration is pending :S
Comment 13 Pacho Ramos gentoo-dev 2013-06-21 07:52:52 UTC
Created attachment 351514 [details]
zlib-1.2.8-r1.ebuild

This is for zlib, could you please look at it? Thanks
Comment 14 Pacho Ramos gentoo-dev 2013-06-21 07:55:06 UTC
Created attachment 351516 [details]
bzip2-1.0.6-r4.ebuild

Updated bzip2 ebuild using eapi4
Comment 15 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-06-21 10:43:31 UTC
Reviewed both and look fine from multilib perspective.
Comment 16 Pacho Ramos gentoo-dev 2013-06-21 10:50:18 UTC
Created attachment 351532 [details]
zlib-1.2.8-r1.ebuild

This adds some missing "die" calls in "cd" commands as pointed by mgorny in IRC
Comment 17 Pacho Ramos gentoo-dev 2013-06-21 16:21:30 UTC
Created attachment 351560 [details]
Diff between bzip2 ebuilds
Comment 18 SpanKY gentoo-dev 2013-06-21 16:21:41 UTC
Comment on attachment 351516 [details]
bzip2-1.0.6-r4.ebuild

(In reply to Pacho Ramos from comment #14)

you should fix the $D vs $ED handling.  which is to say, you should use $ED.

moving to EAPI=4 also means dropping the `die` with dosym & newbin & such

also while you're in there, the static-libs logic could be fixed:
 use static-libs || find "${ED}"/usr -name libbz2.a -delete
Comment 19 Pacho Ramos gentoo-dev 2013-06-21 16:21:45 UTC
Created attachment 351562 [details]
Diff between zlib ebuilds
Comment 20 SpanKY gentoo-dev 2013-06-21 16:26:19 UTC
Comment on attachment 351532 [details]
zlib-1.2.8-r1.ebuild

>	if use minizip ; then
>		cd contrib/minizip || die
>		eautoreconf

the die doesn't hurt, but it doesn't really matter.  if you try to run eautoreconf in $S, it'll fail since there's nothing in there.

>multilib_src_configure() {
>	case ${CHOST} in
>	*-mingw*|mingw*)
>		true
>		;;

drop the true.  there's no need.

>	*)      # not an autoconf script, so can't use econf
>		local uname=$(/usr/share/gnuconfig/config.sub "${CHOST}" | cut -d- -f3) #347167

i think this should be changed to use ${EPREFIX}.  dunno.  it sucks either way (not that it's your fault).

>	if use minizip ; then
>		cd contrib/minizip || die
>		emake
>	fi

change to:
  use minizip && emake -C contrib/minizip

>	sed_macros "${D}"/usr/include/*.h

at least some of these will need to be $ED

>	if use minizip ; then
>		cd contrib/minizip || die
>		emake install DESTDIR="${D}"

emake -C contrib/minizip ...

>	use static-libs || rm -f "${D}"/usr/$(get_libdir)/lib{z,minizip}.{a,la} #419645

$ED

>	if use minizip; then
>		cd contrib/minizip || die
>		dodoc *.txt
>	fi

use minizip && dodoc contrib/minizip/*.txt
Comment 21 Pacho Ramos gentoo-dev 2013-06-21 16:27:56 UTC
Created attachment 351564 [details]
bzip2-1.0.6-r4.ebuild

Updated ebuild fixing that problems
Comment 22 Pacho Ramos gentoo-dev 2013-06-21 16:32:50 UTC
Created attachment 351568 [details]
zlib-1.2.8-r1.ebuild

Updated zlib ebuild
Comment 23 SpanKY gentoo-dev 2013-06-21 16:44:35 UTC
i feel like both of these should be using more $EPREFIX and $ED.  but the prefix guys would know better.

or maybe we just land and let the prefix guys commit after the fact.  not like the ebuilds will make things more broken for them.
Comment 24 Fabian Groffen gentoo-dev 2013-06-21 17:22:39 UTC
(In reply to SpanKY from comment #23)
> i feel like both of these should be using more $EPREFIX and $ED.  but the
> prefix guys would know better.

Correct, they should.

> or maybe we just land and let the prefix guys commit after the fact.  not
> like the ebuilds will make things more broken for them.

We still have copies of these two ebuilds in our own overlay indeed.  Needless to say, I *am* willing to deliver the occasional bits to get the new ebuilds to possibly working for Prefix, compared to definitely not.
Comment 25 Pacho Ramos gentoo-dev 2013-06-21 18:07:36 UTC
Generally, on gnome team we simply let prefix team fix their things. We can commit that ebuilds and, later, prefix team can "prefixy" them (as they are already using different ebuilds from overlay anyway)
Comment 26 Jonathan Callen (RETIRED) gentoo-dev 2013-06-21 19:02:38 UTC
(In reply to Fabian Groffen from comment #24)
> (In reply to SpanKY from comment #23)
> > i feel like both of these should be using more $EPREFIX and $ED.  but the
> > prefix guys would know better.
> 
> Correct, they should.
> 

bzip2-1.0.6-r4.ebuild:

>multilib_src_install() {
>	emake PREFIX="${D}"/usr LIBDIR=$(get_libdir) install

Replace ${D} with ${ED} here

>multilib_src_install_all() {
>	dodoc README* CHANGES bzip2.txt manual.*
>
>	# move "important" bzip2 binaries to /bin and use the shared libbz2.so
>	dodir /bin
>	mv "${D}"/usr/bin/b{zip2,zcat,unzip2} "${D}"/bin/ || die

Replace both ${D} with ${ED} here

zlib-1.2.8-r1.ebuild:

>		echoit ./configure \
>			--shared \
>			--prefix=/usr \
>			--libdir=/usr/$(get_libdir) \
>			${uname:+--uname=${uname}} \
>			|| die

Should be --prefix="${EPREFIX}/usr" --libdir="${EPREFIX}/usr/$(get_libdir)"

>multilib_src_compile() {
>	case ${CHOST} in
>	*-mingw*|mingw*)
>		emake -f win32/Makefile.gcc STRIP=true PREFIX=${CHOST}-
>		sed \
>			-e 's|@prefix@|/usr|g' \\

Use  -e "s|@prefix@|${EPREFIX}/usr|g" , only use one \

>multilib_src_install() {
>	case ${CHOST} in
>	*-mingw*|mingw*)
>		emake -f win32/Makefile.gcc install \
>			BINARY_PATH="${D}/usr/bin" \
>			LIBRARY_PATH="${D}/usr/$(get_libdir)" \
>			INCLUDE_PATH="${D}/usr/include" \
>			SHARED_MODE=1

All the "${D}" in the above emake should be "${ED}"
Comment 27 Pacho Ramos gentoo-dev 2013-06-21 19:17:17 UTC
Created attachment 351594 [details]
bzip2-1.0.6-r4.ebuild

Ebuild with prefix fixes
Comment 28 Pacho Ramos gentoo-dev 2013-06-21 19:20:06 UTC
Created attachment 351598 [details]
zlib-1.2.8-r1.ebuild

Ebuild with prefix fixes
Comment 29 SpanKY gentoo-dev 2013-06-23 04:37:34 UTC
Comment on attachment 351594 [details]
bzip2-1.0.6-r4.ebuild

>		-e 's:ln -s bzip2.1:ln -f -s bzip2.1:g' \

this could probably be written more simply:
  -e 's:ln -s bzip2.1:& -f:g' \

>multilib_src_compile() {
>	bemake -f Makefile-libbz2_so all
>	use static && append-flags -static
>	bemake all

does it make sense to even build multilib programs for /bin ?  seems like a waste of time to produce ABI=x86 when the native ABI is amd64.  maybe current multilib eclass can't handle that though ?

>	local s
>	for v in libbz2.so{,.{${PV%%.*},${PV%.*}}} ; do

not your fault, but there's a var mismatch here.  i think when i wrote it the first time, i was walking the version string alone (which is why the var is called $v), then i changed it to include the libname, so i intended to change it all to $s, but missed the for loop.

>	if ! use static ; then
>		newbin bzip2-shared bzip2

one line it while you're here:
  use static || newbin bzip2-shared bzip2

>	dosym bzip2 /bin/bzcat
>	dosym bzip2 /bin/bunzip2

does prefix take care of this automatically (adding $EPREFIX before the target) ?  or should the target be $EPREFIX/bin/bzcat ?
Comment 30 SpanKY gentoo-dev 2013-06-23 04:38:28 UTC
Comment on attachment 351598 [details]
zlib-1.2.8-r1.ebuild

if prefix guys are happy, i'm fine with it.  so feel free to commit.
Comment 31 Jonathan Callen (RETIRED) gentoo-dev 2013-06-23 06:22:28 UTC
(In reply to SpanKY from comment #29)
> Comment on attachment 351594 [details]
> bzip2-1.0.6-r4.ebuild
> 
> >		-e 's:ln -s bzip2.1:ln -f -s bzip2.1:g' \
> 
> this could probably be written more simply:
>   -e 's:ln -s bzip2.1:& -f:g' \
> 
> >multilib_src_compile() {
> >	bemake -f Makefile-libbz2_so all
> >	use static && append-flags -static
> >	bemake all
> 
> does it make sense to even build multilib programs for /bin ?  seems like a
> waste of time to produce ABI=x86 when the native ABI is amd64.  maybe
> current multilib eclass can't handle that though ?
> 
> >	local s
> >	for v in libbz2.so{,.{${PV%%.*},${PV%.*}}} ; do
> 
> not your fault, but there's a var mismatch here.  i think when i wrote it
> the first time, i was walking the version string alone (which is why the var
> is called $v), then i changed it to include the libname, so i intended to
> change it all to $s, but missed the for loop.
> 
> >	if ! use static ; then
> >		newbin bzip2-shared bzip2
> 
> one line it while you're here:
>   use static || newbin bzip2-shared bzip2
> 
> >	dosym bzip2 /bin/bzcat
> >	dosym bzip2 /bin/bunzip2
> 
> does prefix take care of this automatically (adding $EPREFIX before the
> target) ?  or should the target be $EPREFIX/bin/bzcat ?

Prefix takes care of this automatically for all the do*, new*, and *into wrappers, and will not work properly if any argument starts with ${EPREFIX}.

Note that the ebuilds in the prefix tree contain further changes which are not encapulated here, including patches to make things build/work on exotic systems, like Mac OS X (which requires updating the build system to build libbz2.${PV}.dylib instead of libbz2.so.${PV}, for example).  A separate bug will be filed to track those changes, when they are ready for inclusion in the gentoo-x86 tree.
Comment 32 Pacho Ramos gentoo-dev 2013-06-23 07:21:01 UTC
(In reply to Jonathan Callen from comment #31)
> (In reply to SpanKY from comment #29)
> > Comment on attachment 351594 [details]
> > bzip2-1.0.6-r4.ebuild
> > 
> > >		-e 's:ln -s bzip2.1:ln -f -s bzip2.1:g' \
> > 
> > this could probably be written more simply:
> >   -e 's:ln -s bzip2.1:& -f:g' \
> > 

Didn't know this way to use sed, thanks for the tip, will try in next revision

> > >multilib_src_compile() {
> > >	bemake -f Makefile-libbz2_so all
> > >	use static && append-flags -static
> > >	bemake all
> > 
> > does it make sense to even build multilib programs for /bin ?  seems like a
> > waste of time to produce ABI=x86 when the native ABI is amd64.  maybe
> > current multilib eclass can't handle that though ?
> > 

I think we cannot make it in other way (maybe multilib team could reply)

> > >	local s
> > >	for v in libbz2.so{,.{${PV%%.*},${PV%.*}}} ; do
> > 
> > not your fault, but there's a var mismatch here.  i think when i wrote it
> > the first time, i was walking the version string alone (which is why the var
> > is called $v), then i changed it to include the libname, so i intended to
> > change it all to $s, but missed the for loop.
> > 

OK

> > >	if ! use static ; then
> > >		newbin bzip2-shared bzip2
> > 
> > one line it while you're here:
> >   use static || newbin bzip2-shared bzip2
> > 

Yeah, I tried to not modify much the ebuild ;), but I also prefer the shorter form

> > >	dosym bzip2 /bin/bzcat
> > >	dosym bzip2 /bin/bunzip2
> > 
> > does prefix take care of this automatically (adding $EPREFIX before the
> > target) ?  or should the target be $EPREFIX/bin/bzcat ?
> 
> Prefix takes care of this automatically for all the do*, new*, and *into
> wrappers, and will not work properly if any argument starts with ${EPREFIX}.
> 
> Note that the ebuilds in the prefix tree contain further changes which are
> not encapulated here, including patches to make things build/work on exotic
> systems, like Mac OS X (which requires updating the build system to build
> libbz2.${PV}.dylib instead of libbz2.so.${PV}, for example).  A separate bug
> will be filed to track those changes, when they are ready for inclusion in
> the gentoo-x86 tree.

OK
Comment 33 Pacho Ramos gentoo-dev 2013-06-23 07:43:56 UTC
+  23 Jun 2013; Pacho Ramos <pacho@gentoo.org> package.mask:
+  Mask packages currently included in emul-linux-x86-baselibs that are
+  incorporating native multilib support (as requested by mgorny via IRC for a
+  bit more testing)
+

+*bzip2-1.0.6-r4 (23 Jun 2013)
+
+  23 Jun 2013; Pacho Ramos <pacho@gentoo.org> +bzip2-1.0.6-r4.ebuild:
+  Add ebuild supporting native multilib (#457134, thanks to mgorny and vapier
+  for their help)
+

+*zlib-1.2.8-r1 (23 Jun 2013)
+
+  23 Jun 2013; Pacho Ramos <pacho@gentoo.org> +zlib-1.2.8-r1.ebuild:
+  Add ebuild supporting native multilib (#457134, thanks to mgorny and vapier
+  for their help)
+