61868 2004-08-26 16:33 0000 zlib doesn't create/install shared library 2004-09-01 20:16:40 0000 1 1 1 Unclassified Gentoo Linux Ebuilds unspecified AMD64 Linux RESOLVED FIXED P2 critical --- 1 lorenb@lbtechservices.com lv@gentoo.org 1i5t5.duncan@cox.net rmsand@alienkiss.com lorenb@lbtechservices.com 2004-08-26 16:33:24 0000 I was doing on update on my amd64 box and zlib was upgraded to v1.2.1-r3 (due to a security fix). Something is wrong though because it never builds and/or installs libz.so.1.2.1. I tried using gcc 3.3 to compile. I tried to downgrade. Nothing worked for me. From the emerge: >>> Install zlib-1.2.1-r3 into /var/tmp/portage/zlib-1.2.1-r3/image/ category sys-libs cp libz.a /var/tmp/portage/zlib-1.2.1-r3/image//lib64 cd /var/tmp/portage/zlib-1.2.1-r3/image//lib64; chmod 755 libz.a cd /var/tmp/portage/zlib-1.2.1-r3/image//lib64; if test -f libz.so.1.2.1; then \ rm -f libz.so libz.so.1; \ ln -s libz.so.1.2.1 libz.so; \ ln -s libz.so.1.2.1 libz.so.1; \ (ldconfig || true) >/dev/null 2>&1; \ fi cp zlib.3 /var/tmp/portage/zlib-1.2.1-r3/image//usr/share/man/man3 chmod 644 /var/tmp/portage/zlib-1.2.1-r3/image//usr/share/man/man3/zlib.3 cp zlib.h zconf.h /var/tmp/portage/zlib-1.2.1-r3/image//usr/include chmod 644 /var/tmp/portage/zlib-1.2.1-r3/image//usr/include/zlib.h /var/tmp/portage/zlib-1.2.1-r3/image//usr/include/zconf.h /usr/lib/portage/bin/dolib: libz.so.1.2.1 does not exist chmod: cannot access `libz.so.*': No such file or directory ln: creating symbolic link `/var/tmp/portage/zlib-1.2.1-r3/image//usr/lib64/libz.so' to `/lib64/libz.so.1.2.1': No such file or directory man: prepallstrip: strip: >>> Completed installing into /var/tmp/portage/zlib-1.2.1-r3/image/ Reproducible: Always Steps to Reproduce: 1. emerge zlib Actual Results: An upgraded zlib Expected Results: Most of the services on my machine won't start because they can't find libz.so.1. For example: /usr/sbin/named: error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory Portage 2.0.50-r10 (gcc34-2004.2, gcc-3.4.1, glibc-2.3.4.20040808-r0, 2.6.8-gentoo-r1) ================================================================= System uname: 2.6.8-gentoo-r1 x86_64 AMD Athlon(tm) 64 Processor 3800+ Gentoo Base System version 1.5.3 Autoconf: sys-devel/autoconf-2.59-r4 Automake: sys-devel/automake-1.8.5-r1 ACCEPT_KEYWORDS="amd64 ~amd64" AUTOCLEAN="yes" CFLAGS="-march=k8 -O2 -pipe" CHOST="x86_64-pc-linux-gnu" COMPILER="" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/bind /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs ccache" GENTOO_MIRRORS="http://gentoo.osuosl.org http://distro.ibiblio.org/pub/Linux/distributions/gentoo" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="amd64 apache2 berkdb crypt cups encode foomaticdb gdbm gif imlib jpeg ldap libg++ mysql ncurses nptl pam png procmail python readline samba sasl slang ssl tcpd xml2 zlib" lv@gentoo.org 2004-08-26 21:43:43 0000 ...do you have CONF_LIBDIR set for some reason? lv@gentoo.org 2004-08-26 21:59:38 0000 i think i might have fixed your bug. the fixed ebuild has the following md5sum: ayanami zlib # md5sum zlib-1.2.1-r3.ebuild d90fb61ad477289ef8b45327a8583bf3 zlib-1.2.1-r3.ebuild it'll take at least 15 minutes to make it to rsync lv@gentoo.org 2004-08-26 22:12:45 0000 i'd also like to note that you need portage 2.0.51_pre20 if you want (mostly) working CONF_LIBDIR support lorenb@lbtechservices.com 2004-08-27 02:28:23 0000 I don't know what CONF_LIBDIR does (is that like LD_LIBRARY_PATH?). I'm not even sure where I would set that. If it's just an environment variable, I don't have it set: # echo $CONF_LIBDIR # I just tired out the new ebuild and it doesn't seem to have worked. Here is another snip from my emerge: >>> Install zlib-1.2.1-r3 into /var/tmp/portage/zlib-1.2.1-r3/image/ category sys-libs cp libz.a /var/tmp/portage/zlib-1.2.1-r3/image//lib cd /var/tmp/portage/zlib-1.2.1-r3/image//lib; chmod 755 libz.a cd /var/tmp/portage/zlib-1.2.1-r3/image//lib; if test -f libz.so.1.2.1; then \ rm -f libz.so libz.so.1; \ ln -s libz.so.1.2.1 libz.so; \ ln -s libz.so.1.2.1 libz.so.1; \ (ldconfig || true) >/dev/null 2>&1; \ fi cp zlib.3 /var/tmp/portage/zlib-1.2.1-r3/image//usr/share/man/man3 chmod 644 /var/tmp/portage/zlib-1.2.1-r3/image//usr/share/man/man3/zlib.3 cp zlib.h zconf.h /var/tmp/portage/zlib-1.2.1-r3/image//usr/include chmod 644 /var/tmp/portage/zlib-1.2.1-r3/image//usr/include/zlib.h /var/tmp/portage/zlib-1.2.1-r3/image//usr/include/zconf.h /usr/lib/portage/bin/dolib: libz.so.1.2.1 does not exist chmod: cannot access `libz.so.*': No such file or directory man: prepallstrip: strip: >>> Completed installing into /var/tmp/portage/zlib-1.2.1-r3/image/ I had to go back to the binary I grabbed from the AMD64 2004.2 LiveCD. To try and figure out what the problem is, I manually unpacked the source code and applied the gentoo patches. I then tried to build the library myself using gcc 3.3 and 3.4. Here are my results: # tar jxvf /usr/portage/distfiles/zlib-1.2.1.tar.bz2 # cd zlib-1.2.1 # patch -p1 < /usr/portage/sys-libs/zlib/files/zlib-1.2.1-glibc.patch patching file Makefile.in death zlib-1.2.1 # patch -p1 < /usr/portage/sys-libs/zlib/files/zlib-1.2.1-build-fPIC.patch patching file Makefile.in patching file configure death zlib-1.2.1 # patch -p0 < /usr/portage/sys-libs/zlib/files/zlib-1.2.1-mapfile.patch patching file configure patching file mapfile death zlib-1.2.1 # patch -p0 < /usr/portage/sys-libs/zlib/files/zlib-1.2.1-configure.patch patching file configure # ./configure Building static library libz.a version 1.2.1 with gcc. Checking for unistd.h... Yes. Checking whether to use vs[n]printf() or s[n]printf()... using vs[n]printf() Checking for vsnprintf() in stdio.h... Yes. Checking for return value of vsnprintf()... Yes. Checking for errno.h... Yes. Checking for mmap support... Yes. # make # gcc -o libz.so adler32.o compress.o crc32.o gzio.o uncompr.o deflate.o trees.o zutil.o inflate.o infback.o inftrees.o inffast.o -lc /usr/lib/gcc/x86_64-pc-linux-gnu/3.4.1/../../../crt1.o(.text+0x21): In function `_start': : undefined reference to `main' collect2: ld returned 1 exit status # gcc-config x86_64-pc-linux-gnu-3.3.4 # source /etc/profile # gcc -o libz.so adler32.o compress.o crc32.o gzio.o uncompr.o deflate.o trees.o zutil.o inflate.o infback.o inftrees.o inffast.o -lc /usr/lib/gcc-lib/x86_64-pc-linux-gnu/3.3.4/../../../crt1.o(.text+0x21): In function `_start': : undefined reference to `main' collect2: ld returned 1 exit status lorenb@lbtechservices.com 2004-08-27 02:33:40 0000 Forgot to include in my cut/paste, when trying to build the zlib shared object by hand, I did "make clean" and "make" after I switched from gcc 3.4 to 3.3. But they both still gave the same error. lorenb@lbtechservices.com 2004-08-27 05:57:11 0000 Realized I made a mistake trying to compile zlib by hand, I didn't pass --shared to ./configure. It does work when I do it by hand: # pwd /root/zlib-1.2.1 # make clean rm -f *.o *~ example minigzip \ libz.* foo.gz so_locations \ _match.s maketree contrib/infback9/*.o # ./configure --shared Checking for shared library support... Building shared library libz.so.1.2.1 with gcc. Checking for unistd.h... Yes. Checking whether to use vs[n]printf() or s[n]printf()... using vs[n]printf() Checking for vsnprintf() in stdio.h... Yes. Checking for return value of vsnprintf()... Yes. Checking for errno.h... Yes. Checking for mmap support... Yes. # make gcc -fPIC -O3 -DUSE_MMAP -c -o example.o example.c gcc -fPIC -O3 -DUSE_MMAP -c -o adler32.o adler32.c gcc -fPIC -O3 -DUSE_MMAP -c -o compress.o compress.c gcc -fPIC -O3 -DUSE_MMAP -c -o crc32.o crc32.c gcc -fPIC -O3 -DUSE_MMAP -c -o gzio.o gzio.c gcc -fPIC -O3 -DUSE_MMAP -c -o uncompr.o uncompr.c gcc -fPIC -O3 -DUSE_MMAP -c -o deflate.o deflate.c gcc -fPIC -O3 -DUSE_MMAP -c -o trees.o trees.c gcc -fPIC -O3 -DUSE_MMAP -c -o zutil.o zutil.c gcc -fPIC -O3 -DUSE_MMAP -c -o inflate.o inflate.c gcc -fPIC -O3 -DUSE_MMAP -c -o infback.o infback.c gcc -fPIC -O3 -DUSE_MMAP -c -o inftrees.o inftrees.c gcc -fPIC -O3 -DUSE_MMAP -c -o inffast.o inffast.c ar rc libz.a adler32.o compress.o crc32.o gzio.o uncompr.o deflate.o trees.o zutil.o inflate.o infback.o inftrees.o inffast.o gcc -shared -Wl,-soname,libz.so.1 -Wl,--version-script,mapfile -o libz.so.1.2.1 adler32.lo compress.lo crc32.lo gzio.lo uncompr.lo deflate.lo trees.lo zutil.lo inflate.lo infback.lo inftrees.lo inffast.lo -lc rm -f libz.so libz.so.1 ln -s libz.so.1.2.1 libz.so ln -s libz.so.1.2.1 libz.so.1 gcc -fPIC -O3 -DUSE_MMAP -o example example.o -L. libz.so.1.2.1 gcc -fPIC -O3 -DUSE_MMAP -c -o minigzip.o minigzip.c gcc -fPIC -O3 -DUSE_MMAP -o minigzip minigzip.o -L. libz.so.1.2.1 # ls -la libz.so* lrwxrwxrwx 1 root root 13 Aug 27 08:56 libz.so -> libz.so.1.2.1 lrwxrwxrwx 1 root root 13 Aug 27 08:56 libz.so.1 -> libz.so.1.2.1 -rwxr-xr-x 1 root root 90966 Aug 27 08:56 libz.so.1.2.1 It still fails however when I try and build from the ebuild. Strange. lorenb@lbtechservices.com 2004-08-27 06:14:44 0000 I think I've found the source of the problem. Trying to build using the ebuild again: # emerge -B zlib Calculating dependencies ...done! >>> emerge (1 of 1) sys-libs/zlib-1.2.1-r3 to / >>> md5 src_uri ;-) zlib-1.2.1.tar.bz2 >>> Unpacking source... >>> Unpacking zlib-1.2.1.tar.bz2 to /var/tmp/portage/zlib-1.2.1-r3/work * Applying zlib-1.2.1-glibc.patch... [ ok ] * Applying zlib-1.2.1-build-fPIC.patch... [ ok ] * Applying zlib-1.2.1-mapfile.patch... [ ok ] * Applying zlib-1.2.1-configure.patch... [ ok ] * Applying zlib-1.2.1-CAN-2004-0797.patch... [ ok ] >>> Source unpacked. Checking for shared library support... No shared library support; try without defining CC and CFLAGS Building static library libz.a version 1.2.1 with gcc. It doesn't find shared library support. As you can see in my last comment, when I do this by hand, it works fine. Getting closer to finding a solution. kugelfang@gentoo.org 2004-08-27 07:09:35 0000 *** Bug 61912 has been marked as a duplicate of this bug. *** lv@gentoo.org 2004-08-27 07:14:53 0000 ...can you try doing an rm -rf /var/tmp/portage? and then trying again? i know it's sounds like a strange request, but really that's just the sound of me grasping at straws. ^^; but i think we might be seeing a portage bug here... lorenb@lbtechservices.com 2004-08-27 07:23:06 0000 # rm -r /var/tmp/portage # mkdir /var/tmp/portage # emerge -B zlib Calculating dependencies ...done! >>> emerge (1 of 1) sys-libs/zlib-1.2.1-r3 to / >>> md5 src_uri ;-) zlib-1.2.1.tar.bz2 >>> Unpacking source... >>> Unpacking zlib-1.2.1.tar.bz2 to /var/tmp/portage/zlib-1.2.1-r3/work * Applying zlib-1.2.1-glibc.patch... [ ok ] * Applying zlib-1.2.1-build-fPIC.patch... [ ok ] * Applying zlib-1.2.1-mapfile.patch... [ ok ] * Applying zlib-1.2.1-configure.patch... [ ok ] * Applying zlib-1.2.1-CAN-2004-0797.patch... [ ok ] >>> Source unpacked. Checking for shared library support... No shared library support; try without defining CC and CFLAGS Didn't seem to have any effect. Just for the hell of it (I'm grasping at straws too!) I tried to build zlib 1.1.4-r4. Same error: # emerge -B =sys-libs/zlib-1.1.4-r4 Calculating dependencies ...done! >>> emerge (1 of 1) sys-libs/zlib-1.1.4-r4 to / >>> md5 src_uri ;-) zlib-1.1.4.tar.bz2 >>> Unpacking source... >>> Unpacking zlib-1.1.4.tar.bz2 to /var/tmp/portage/zlib-1.1.4-r4/work * Applying zlib-1.1.4-gzprintf.patch... [ ok ] * Applying zlib-1.1.4-glibc.patch... [ ok ] * Applying zlib-1.1.4-build-fPIC.patch... [ ok ] * Applying zlib-1.1.4-mapfile.patch... [ ok ] >>> Source unpacked. Checking for shared library support... No shared library suppport; try without defining CC and CFLAGS mikko.tiihonen@iki.fi 2004-08-27 08:26:17 0000 Just found a reason why the build won't build a shared library but running configure directly will. This is what I get when I remove some redirects to dev null from the shared library test in configure script. ./configure --shared CC=gcc CFLAGS=-march=k8 -pipe -O2 -fomit-frame-pointer -frename-registers -fweb --param max-delay-slot-insn-search=300 --param max-delay-slot-live-search=1000 --param max-pending-list-length=100 --param max-gcse-passes=4 --param max-gcse-memory=104857600 --param max-crossjump-edges=400 SFLAGS=-march=k8 -pipe -O2 -fomit-frame-pointer -frename-registers -fweb --param max-delay-slot-insn-search=300 --param max-delay-slot-live-search=1000 --param max-pending-list-length=100 --param max-gcse-passes=4 --param max-gcse-memory=104857600 --param max-crossjump-edges=400 Checking for shared library support... /usr/lib/gcc/x86_64-pc-linux-gnu/3.4.1/../../../../x86_64-pc-linux-gnu/bin/ld: ztest1503.o: relocation R_X86_64_PC32 against `getchar@@GLIBC_2.2.5' can not be used when making a shared object; recompile with -fPIC /usr/lib/gcc/x86_64-pc-linux-gnu/3.4.1/../../../../x86_64-pc-linux-gnu/bin/ld: final link failed: Bad value collect2: ld returned 1 exit status No shared library support; try without defining CC and CFLAGS And adding a CFLAGS="$CFLAGS -fPIC" to the build src_compile ./configure line cured the ebuild problem. lv@gentoo.org 2004-08-27 09:34:45 0000 ...what binutils version are you all using? and glibc lorenb@lbtechservices.com 2004-08-27 13:30:31 0000 I've got: binutils-2.14.90.0.8-r1 glibc-2.3.4.20040808 orpheus@primeinteractive.net 2004-08-27 14:10:32 0000 Confirmed, after adding -fPIC to CFLAGS in /etc/make.conf the biuld was succesfull. gcc-3.4.1-r2 binutils-2.15.91.0.2 portage-2.0.51_pre20 lv@gentoo.org 2004-08-27 16:21:17 0000 pavol, your binutils version is known to be broken. stop using -U, i masked that version aug11. the latest stable binutils is binutils-2.15.90.0.1.1-r3 but since loren is using a version of binutils known to work, that cant be it. mikko - when you ran configure by hand, were you using a clean unpatched zlib, or a zlib with all the gentoo patches applied? there is one that patches configure... and it touches that test. :| can you edit the ebuild to not apply the configure patch and see if that fixes it? mikko.tiihonen@iki.fi 2004-08-28 03:07:04 0000 Travis, when running by hand I run the zlib that had been patched with all gentoo patches. (I'm also running binutils 2.15.91.0.2, but I don't think it matters). I now tried removing the zlib-1.2.1-configure.patch from the ebuild but it still fails. Also removing the other patches didn't help. The reason why running configure by hand works and the ebuild doesn't is that if no CFLAGS are defined when the configure is run it defaults to "-O3 -fPIC". But when ebuild is run it of course has some other CFLAGS already defined and they do not include -fPIC. So I would just add the following line to the ebuild as so many other ebuilds already do: use amd64 && append-flags -fPIC lv@gentoo.org 2004-08-28 05:33:41 0000 mikko - i have cflags set and it works fine for me. they dont include -fPIC either, which is why i'm confused. but after looking at the configure it seems that this test is horribly wrong anyways, as shared libs always need to be PIC on amd64 and a few other archs. i'll post a patch that i would like you to try... lv@gentoo.org 2004-08-28 05:35:24 0000 Created an attachment (id=38382) always use -fPIC in shared library test lorenb@lbtechservices.com 2004-08-28 06:03:55 0000 I added the patch to the zlib-1.2.1-r3.ebuild and tried it. It works fine for me. I'm able to build and install zlib again. mikko.tiihonen@iki.fi 2004-08-28 11:35:20 0000 The patch fixes the ebuild also for me. I'm just wondering what could cause the different behaviour. Is it a new bash that runs the tests differently, different version of binutils/gcc/glibc or something else? lv@gentoo.org 2004-08-28 20:10:18 0000 i guess it all depends on the behavior of gcc. for example, hardened gcc builds everything either -fPIC or -fPIE unless told not to. *shrug* anyways, this bug is now fixed in cvs. give it some time to reach rsync :) lv@gentoo.org 2004-09-01 02:24:58 0000 ...closing bug vapier@gentoo.org 2004-09-01 20:16:38 0000 *** Bug 62467 has been marked as a duplicate of this bug. *** 38382 2004-08-28 05:35 0000 always use -fPIC in shared library test zlib-fPIC.patch text/plain LS0tIHpsaWItMS4yLjEub2xkL2NvbmZpZ3VyZQkyMDA0LTA4LTI4IDA4OjE5OjQ5LjAwMDAwMDAw MCAtMDQwMAorKysgemxpYi0xLjIuMS9jb25maWd1cmUJMjAwNC0wOC0yOCAwODozMzo1Ny43NDU4 NjQwNjQgLTA0MDAKQEAgLTczLDcgKzczLDExIEBACiAKIGlmIHRlc3QgIiRnY2MiIC1lcSAxICYm ICgkY2MgLWMgJGNmbGFncyAkdGVzdC5jKSAyPi9kZXYvbnVsbDsgdGhlbgogICBDQz0iJGNjIgot ICBTRkxBR1M9JHtDRkxBR1MtIi1mUElDIC1PMyJ9CisgICNTRkxBR1M9JHtDRkxBR1MtIi1mUElD IC1PMyJ9CisgICMgdGhlIGFib3ZlIGlzIGhvcnJpYmx5IHdyb25nIG9uIGEgZmV3IGFyY2hzIHdo ZXJlIC1mUElDIHNob3VsZCBBTFdBWVMgYmUKKyAgIyB1c2VkIGluIHRoZSBjcmVhdGlvbiBvZiBz aGFyZWQgbGlicmFyaWVzLiB3aXRob3V0IHRoZSBmb2xsb3dpbmcsIHRoZQorICAjIHNoYXJlZCBs aWIgdGVzdCB3aWxsIHNvbWV0aW1lcyBmYWlsIGV2ZW4gd2hlbiBzaGFyZWQgbGlicyAtY2FuLSBi ZSBjcmVhdGVkLgorICBTRkxBR1M9IiR7Q0ZMQUdTLSItTzMifSAtZlBJQyIKICAgQ0ZMQUdTPSIk Y2ZsYWdzIgogICBjYXNlIGAodW5hbWUgLXMgfHwgZWNobyB1bmtub3duKSAyPi9kZXYvbnVsbGAg aW4KICAgTGludXggfCBsaW51eCB8IEdOVSB8IEdOVS8qKQo=