Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 125988

Summary: glibc-2.4 upgrade breaks system
Product: Gentoo Linux Reporter: Juergen Rose <rose>
Component: [OLD] Core systemAssignee: Gentoo Toolchain Maintainers <toolchain>
Status: RESOLVED FIXED    
Severity: normal CC: gentoobugs, ikelos, rockoo, stefan
Priority: High    
Version: 2006.0   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 125909    
Attachments: LD_DEBUG=all patch 2>lddebug.log
LD_DEBUG=all bash 2>lddebug.log
amd64_libssp_simple.so
x86_libssp_simple.so

Description Juergen Rose 2006-03-12 15:00:04 UTC
After upgrading to glibc-2.4 my system several essential programs were broken. Bash, python, gcc, make, readelf, grep and less were broken, they produced relocation errors,e.g.

bash: relocation error: bash: symbol __guard, version GLIBC_2.3.2 not defined
in file libc.so.6 with link time reference

As a consequence also 'emerge' does not work. 'ldconfig' worked without errors, but the relocation errors did not disapear. Temporarely I could reduce my problems with the following steps.
root@cheetah:/usr/src(139)# cd /lib
root@cheetah:/lib(142)# mkdir new_glibc
root@cheetah:/lib(143)# mv *2.4* new_glibc/
root@cheetah:/lib(145)# ldconfig

Now 'emerge' works again:

root@cheetah:/lib(149)# emerge --info

Portage 2.1_pre5-r4 (default-linux/x86/2005.1, gcc-3.4.5-vanilla,
glibc-2.3.6-r3,glibc-2.4-r0, 2.6.16-rc4 i686)
=================================================================
System uname: 2.6.16-rc4 i686 AMD Athlon(tm) XP 1800+
Gentoo Base System version 1.12.0_pre16
distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632)
[disabled]
ccache version 2.4 [disabled]
dev-lang/python:     2.4.2-r1
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1
sys-devel/binutils:  2.16.1-r2
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r3
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=athlon-xp -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/grass60/etc /usr/kde/2/share/config /usr/kde/3.3/env
/usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3.4/env
/usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3.5/env
/usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/kde/3/share/config
/usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config
/var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/texmf/web2c /etc/env.d"
CXXFLAGS="-O2 -march=athlon-xp -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks sandbox sfperms strict"
GENTOO_MIRRORS="http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror
http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/
rsync://ftp.fi.muni.cz/pub/linux/gentoo
ftp://ftp.wh2.tu-dresden.de/pub/mirrors/gentoo
rsync://gd.tuwien.ac.at/opsys/linux/gentoo
ftp://sunsite.informatik.rwth-aachen.de/pub/Linux/gentoo
ftp://ftp.tu-clausthal.de/pub/linux/gentoo http://gentoo.oregonstate.edu"
LINGUAS="de fr"
MAKEOPTS="-j1"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 3dnow 3dnowext X Xaw3d a52 aac alsa ao aotuv apache2 apm arts asf
atlas auctex audiofile automount avi berkdb bitmap-fonts blas bonobo bzip2
cairo cdda cddb cdf cdio cdparanoia cgi cli corba crypt cups curl daap dbus
devmap dga dri dvb dvd dvi dxr3 dynagraph eds effects elf emacs emboss encode
epiphany esd evo evolution exif expat fame ffmpeg fftw fits flac fltk
foomaticdb fortran fpx gdal gdbm geos gif ginac glade gml gmp gnome gnuplot gpm
graphviz grass gs gsl gsm gstreamer gtk gtk2 guile hal hardened hddtemp hdf
hdf5 hlapi icq id3 imagemagick imlib innodb ipv6 jabber jbig jp2 jpeg jpeg2 kde
kexi lame lapack latex lcms ldap libg++ libgda libsamplerate libwww lirc
lm_sensors lzo lzw mad math matroska matrox mikmod mmx mng mod mono motif
mozcalendar mozdevelop mozilla mozsvg mozxmlterm mp3 mp4live mpeg mpeg2 mplayer
musicbrainz mysql mysqli nautilus ncurses netcdf network nfs nls nntp nptl
nptlonly numarray numeric ocaml octave odbc ogdi ogg oggvorbis opengl oss pam
pcre pdf pdflib perl plotutils plugin png posix postgres proj python qhull qt
quicktime readline real reiserfs rhythmbox rle rrdcgi rrdtool samba sdl slang
slp sndfile snmp sox spell sqlite sse sse2 ssl subtitles svg t1lib tcltk tcpd
tetex theora tidy tiff truetype truetype-fonts type1-fonts unicode v4l2
video_cards_mga vorbis win32codecs wmf xemacs xine xml2 xmlreader xmlrpc xmms
xpm xv xvid xvmc zlib zvbi elibc_glibc kernel_linux linguas_de linguas_fr
userland_GNU"
Unset:  ASFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, LANG, LC_ALL, LDFLAGS

Then I made the error to remove the new glibc-2.4 with 'emerge -C glibc-2.4', because all glibc header files were also removed. I.e. 'make' did not work more.
After copying the all missing  glibc-2.3.6 files (headers, locale, some libs ) from a second Gentoo-PC with a similar hard- and software configuration. Now ls, vi, xterm, emacs, bash produce Segmentation faults. So I again desperate.  What is the best way to repair the system?
Comment 1 Jakub Moc (RETIRED) gentoo-dev 2006-03-12 15:03:16 UTC

*** This bug has been marked as a duplicate of 125868 ***
Comment 2 SpanKY gentoo-dev 2006-03-13 19:21:43 UTC
not a dupe
Comment 3 Mike Auty (RETIRED) gentoo-dev 2006-03-14 02:15:27 UTC
Interestingly I've had this on two boxes.  On one of them, only certain binaries failed with the symbol __guard, version GLIBC_2.3.2 error, and were easily rebuilt.  On another, brand new fresh install, every single binary died and I've had to spend the weekend rebuilding it.

As far as I can tell, it's because of the hardened USE flags on the old glibc and then not having them on the latest glibc.  I'm also in the position of running a non-hardened profile with the hardened USE flag set.  This can be pretty dangerous, so I'd be tempted to raise the severity.  For the record here's my emerge --info from the first machine (the one that didn't die).  I'm afraid I can't recover the one from the broken system (because even if it were still here, python wouldn't start).

I'm also a little suprised that such a major package seemed to go into portage without a long masked session, or any warning on GWN or -dev...

Portage 2.1_pre6-r2 (default-linux/x86/2006.0, gcc-4.1.0, glibc-2.4-r0, 2.6.16-rc5 i686)
=================================================================
System uname: 2.6.16-rc5 i686 AMD Athlon(tm) 64 Processor 3400+
Gentoo Base System version 1.12.0_pre16
ccache version 2.4 [enabled]
dev-lang/python:     2.4.2-r1
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1
sys-devel/binutils:  2.16.1-r2
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r3
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=athlon-xp -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/X11/xkb /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -march=athlon-xp -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig ccache confcache cvs distlocks metadata-transfer parallel-fetch sandbox sfperms splitdebug strict"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
MAKEOPTS="-j1"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/overlays/personal /usr/local/overlays/gcc41 /usr/local/overlays/gnome-experimental /usr/local/overlays/xgl-overlay"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 3dnow X aac acl acpi alsa ao apache2 asf audiofile avi bash-completion berkdb bitmap-fonts boundschecking browserplugin bzip2 cairo cdr cjk client crypt cups dbus djvu dlloader dpms dri dvd dvdr dvi emboss encode ffmpeg firefox flac foomaticdb gdbm gif glade glitz gnome gnuplot gnutls graphviz gstreamer gtk gtk2 gtkhtml hal hardened imap java jpeg ldap libg++ libnotify libsexy libwww mad maildir mbox mikmod mimeunicode mmx mmxext mng modplug mozsvg mp3 mpeg ncurses nls nntp nptl nptlonly nvidia ogg opengl pam pdflib perl pic png python quicktime readline real sdl snmp speex spell sse sse2 ssl svg t1lib tcpd theora truetype truetype-fonts type1-fonts udev unicode usb v4l v4l2 vorbis win32codecs wma xml xrandr xscreensaver xv xvid zlib elibc_glibc input_devices_wacom input_devices_mouse input_devices_keyboard input_devices_evdev kernel_linux userland_GNU video_cards_nvidia"
Unset:  ASFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, LANG, LC_ALL, LDFLAGS, LINGUAS
Comment 4 Kevin F. Quinn (RETIRED) gentoo-dev 2006-03-14 09:29:46 UTC
Ways I can think of that give you __guard references in exes

1) exe and/or dsos built with -fstack-protector
2) exe and/or dsos built with hardened gcc
3) gcc was built hardened

(3) is interesting in that if gcc was built USE=hardened, the support libraries (libstc++.so in particular) were built with SSP and reference __guard.  So even if the compiler was switched via gcc-config to a vanilla compiler, the executables still need __guard for the referenced shared libraries.

It might be useful if someone with such a broken system could try:

LD_DEBUG=all command 2>lddebug.log

(where 'command' should obviously be replaced with a real exe that fails).

and attach lddebug.log here.
Comment 5 Svetlio 2006-03-14 10:32:59 UTC
Created attachment 82115 [details]
LD_DEBUG=all patch 2>lddebug.log

I hope I did the attachement right, this is my first post here (~x86)
Comment 6 Kevin F. Quinn (RETIRED) gentoo-dev 2006-03-14 11:10:25 UTC
hmm; seems that ld-linux.so2 just doesn't have the guard symbol - as if the ssp compatibility didn't happen although it needed to.

Can you see what /lib/ld-linux.so.2 points to; just to confirm it points ot the 2.4 version of ld.so?

readlink /lib/ld-linux.so.2
or
ls -l /lib/ld-linux.so.2

and also paste the output of:

readelf -s /lib/ld-linux.so.2 | grep __guard

Comment 7 Svetlio 2006-03-14 11:43:12 UTC
> ls -l /lib/ld-linux.so.2

lrwxrwxrwx 1 root root 9 Mar 13 10:18 /lib/ld-linux.so.2 -> ld-2.4.so

 
> readelf -s /lib/ld-linux.so.2 | grep __guard

17: 00019ea8     4 OBJECT  GLOBAL DEFAULT   14 __guard@@GLIBC_2.3.2
394: 00019ea8     4 OBJECT  LOCAL  HIDDEN   14 __guard_local
515: 00019ea8     4 OBJECT  GLOBAL DEFAULT   14 __guard

Thanks for your time and your help!

Svetlio
 

Comment 8 Svetlio 2006-03-14 12:05:10 UTC
Wow, I was a bit in a hurry to post here, but I just realized that people are talking about hardened use flag in the posts above, and I don't have it in my USE...

Maybe I should post my emerge --info here...

All my appologies if I'm posting to the wrong place and waste your time..

Here is the output of emerge --info:
Gentoo Base System version 1.12.0_pre16
Portage 2.1_pre6-r1 (default-linux/x86/2005.1, gcc-3.4.5, glibc-2.4-r0, 2.6.14-gentoo-r5 i686)
=================================================================
System uname: 2.6.14-gentoo-r5 i686 AMD Athlon(tm) processor
dev-lang/python:     2.3.5, 2.4.2-r1
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.59-r7
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1
sys-devel/binutils:  2.16.1-r2
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r3
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=athlon-tbird -O2 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/texmf/web2c /etc/env.d"
CXXFLAGS="-march=athlon-tbird -O2 -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks metadata-transfer sandbox sfperms strict"
GENTOO_MIRRORS="http://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/ http://mir.zyrianes.net/gentoo/ http://gentoo.mirror.solnet.ch http://gentoo.inode.at/ http://ftp.du.se/pub/os/gentoo"
LINGUAS="en fr bg"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 3dnow X a52 aac acl alsa apm arts avi berkdb bitmap-fonts crypt cups dri dvd dvdr dvdread eds emboss encode exif flac foomaticdb fortran gdbm gif gpm gtk gtk2 imlib jpeg kde libg++ libwww mad matroska mikmod mmx motif mp3 mpeg ncurses nls nptl nvidia ogg oggvorbis opengl pam pdf pdflib perl pic png python qt quicktime readline sdl spell ssl tcpd truetype truetype-fonts type1-fonts vorbis win32codecs xml2 xmms xv xvid zlib elibc_glibc kernel_linux linguas_en linguas_fr linguas_bg userland_GNU"
Unset:  ASFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, LANG, LC_ALL, LDFLAGS
  

Comment 9 Kevin F. Quinn (RETIRED) gentoo-dev 2006-03-14 13:30:16 UTC
Hmm - interesting - it looks like libc.so can pick up symbols from ld.so because ld-linux.so.2 is NEEDED by libc.so.  However the executable (patch in your example) doesn't list ld-linux.so.2 as NEEDED, because no doubt when patch was built it found everything it needed in libc.so.  If a library isn't in an objects NEEDED list, it isn't scanned for symbols, it seems.
Comment 10 Juergen Rose 2006-03-14 14:25:57 UTC
Created attachment 82149 [details]
LD_DEBUG=all bash 2>lddebug.log

LD_DEBUG=all bash 2>lddebug.log
Comment 11 Juergen Rose 2006-03-14 14:28:04 UTC
After repairing my system returning to glibc-2.3.6-r3 compiled with gcc-3.4.5 (Gentoo Hardened 3.4.5-r1, ssp-3.4.5-1.0, pie-8.7.9) using bash for command. i.e.
doing 'LD_DEBUG=all bash 2>lddebug.log' the attachted file was created.
Normally I am using hardened gcc, but glibc-2.4 was created with nonhardened gcc, because hardened gcc failed to compile glibc-2.4 .
Comment 12 Juergen Rose 2006-03-14 14:31:36 UTC
I have:
root@cheetah:/root(13)# ls -l /lib/ld-linux.so.2
lrwxrwxrwx 1 root root 11 Mar 14 01:48 /lib/ld-linux.so.2 -> ld-2.3.6.so*
root@cheetah:/root(14)# readelf -s /lib/ld-linux.so.2 | grep __guard
   368: 000174c0     4 OBJECT  LOCAL  DEFAULT   18 __guard
   382: 00010560   166 FUNC    LOCAL  DEFAULT    9 __guard_setup
Comment 13 Svetlio 2006-03-15 16:55:29 UTC
A workaround for the patch problem:
 1.re-emerge patch with no patches (comment out the "epatch" related lines in the ebuild, save the ebuild and execute ebuild patch-*.ebuild digest)
 2.re-emerge patch with the patches (revert the changes in the ebuild, save and execute ebuild patch-*.ebuild digest )

=> 2 min later everything works like a charm :-)

Regards,
Svetlio
Comment 14 SpanKY gentoo-dev 2006-03-16 20:34:35 UTC
ok, glibc-2.4-r1 should resolve this

if you guys have a broken system and are unable to sync/re-emerge glibc, then you can grab prebuilt lib's to work around this issue:
# bb
# wget http://dev.gentoo.org/~vapier/amd64_libssp_simple.so
# mv amd64_libssp_simple.so /lib/libssp_simple.so
# echo /lib/libssp_simple.so > /etc/ld.so.preload

for x86 users, just change the "amd64" to "x86"

once you've installed glibc-2.4-r1, then just delete /lib/libssp_simple.so and /etc/ld.so.preload and you should be all set

thanks to solar for this little hack workaround
Comment 15 SpanKY gentoo-dev 2006-10-10 20:37:39 UTC
Created attachment 99338 [details]
amd64_libssp_simple.so

moving out of my ~ space on dev.g.o ...
Comment 16 SpanKY gentoo-dev 2006-10-10 20:37:55 UTC
Created attachment 99339 [details]
x86_libssp_simple.so

moving out of my ~ space on dev.g.o ...