Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 85555 - Emerging glibc-2.3.4.20041102-r1 makes system unusable
Summary: Emerging glibc-2.3.4.20041102-r1 makes system unusable
Status: RESOLVED WORKSFORME
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: x86 Linux
: High critical (vote)
Assignee: Gentoo Toolchain Maintainers
URL:
Whiteboard:
Keywords:
: 86400 (view as bug list)
Depends on:
Blocks:
 
Reported: 2005-03-16 12:44 UTC by a
Modified: 2005-08-03 09:01 UTC (History)
10 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
/etc/make.conf file (make.conf,14.31 KB, text/plain)
2005-03-16 13:04 UTC, a
Details
Packages installed in the system (qpkg -I) (package-installed.txt,2.42 KB, text/plain)
2005-03-17 02:00 UTC, a
Details
/etc/make.conf file from the machine that didn't break (make.conf,14.17 KB, text/plain)
2005-03-17 12:57 UTC, a
Details
Installed software on machine that didn't break (qpkg -I) (installed.txt,7.06 KB, text/plain)
2005-03-17 13:00 UTC, a
Details
Just for reference on another machine that went well (make.conf) (make.conf,14.20 KB, text/plain)
2005-03-17 17:16 UTC, a
Details
Debugging with ltrace and gdb (debug.txt,5.64 KB, text/plain)
2005-03-21 12:23 UTC, Lars T. Mikkelsen
Details
Broken libpthread-0.10.so (libpthread-0.10.so.bz2,33.79 KB, application/x-bzip2)
2005-03-21 23:25 UTC, Lars T. Mikkelsen
Details
make.conf from the machine that worked on the 2nd try (make.conf,14.36 KB, text/plain)
2005-03-25 17:24 UTC, a
Details
Packages installed (equery -C l) (installed.txt,3.03 KB, text/plain)
2005-03-25 18:35 UTC, a
Details

Note You need to log in before you can comment on or make changes to this bug.
Description a 2005-03-16 12:44:40 UTC
After I emerged glibc-2.3.4.20041102-r1 (http://packages.gentoo.org/ebuilds/?glibc-2.3.4.20041102-r1) in the recent emerge -uD world, the system is now pretty unusable causing alot of segmetation faults here and there.

From 'ls' to 'python', when I try to issue those commands, they go segmentation fault, so I cannot even run emerge anymore. Some commands still work, but from what I see from 'ldd /bin/ls', and I use whereis command to look for depended libraries, 'linux-gate.so.1' is the only library that does not seem to be found, and same for python.

This is like the second time glibc update messed up my system really bad.
Hope the Gentoo team take care of glibc a bit more and not make these happen in the future.

And I really need a quick remedy to this problem.


Reproducible: Always
Steps to Reproduce:
1. emerge -uD world (including glibc-2.3.4.20041102-r1)
2.
3.

Actual Results:  
The emerge itself seems to have completed but the following updates within 
the 'emerge -uD world' had stopped since 'python' now goes segmentation fault.

Other programs such as 'ls' also goes segmentation fault.

Expected Results:  
Should not mess up glibc libraries and programs should not start seg fault'ing.
Comment 1 a 2005-03-16 12:48:26 UTC
The system I'm running that caused this bug is, x86 tree on stable and development-sources kernel without any sort of X installations.

And, I have't rebooted the system yet, as it is on a remote location and from a fear that it may not come back.
Comment 2 Ciaran McCreesh 2005-03-16 12:54:28 UTC
emerge info? Or at least your make.conf?
Comment 3 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-16 13:01:56 UTC
Can you try to strace ls/whatever?  Also, do you still have a working g++?
Comment 4 a 2005-03-16 13:04:42 UTC
Created attachment 53647 [details]
/etc/make.conf file
Comment 5 Jakub Moc (RETIRED) gentoo-dev 2005-03-16 13:13:20 UTC
Just a note: linux-gate.so.1 is a common name for kernel DSO (dynamically shared object), this "library" is purely virtual, provided by kernel and unrelated to glibc. 
Comment 6 a 2005-03-16 13:18:07 UTC
I now changed the -march to pentium3 in the make.conf but it was pentium4 when I compiled glibc.

Since I cannot run emerge anymore, I've collected information that would normally be displayed by emerge info in a shorter form. For package versions, I'm just picking up the latest version emerge logged at /var/log/emerge.log.

System uname: Linux cat 2.6.10 #1 Thu Jan 20 00:52:36 JST 2005 i686 Intel(R) Celeron(R) CPU 2.40GHz GenuineIntel GNU/Linux
Gentoo Base System version: 1.4.16
Python: (not sure, can't run python nor emerge, but should be as recent as possible)
distcc: [enabled]
ccache version 2.3 [enabled]
sys-devel/autoconf: 2.59-r6
sys-devel/automake: 1.7.9-r1
sys-devel/binutils: 2.15.92.0.2-r1
sys-devel/libtool: 1.5.10-r4
virtual/os-headers:  (not sure)
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-march=pentium3 -O2 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CXXFLAGS="-march=pentium3 -O2 -pipe -fomit-frame-pointer"
FEATURES="autoaddcvs autoconfig buildpkg ccache distlocks sandbox sfperms distcc"
GENTOO_MIRRORS="ftp://ftp.ecc.u-tokyo.ac.jp/GENTOO http://gentoo.oregonstate.edu http://www.ibiblio.org/pub/Linux/distributions/gentoo"
MAKEOPTS="-j7"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.asia.gentoo.org/gentoo-portage"

I do not have any package.* file in /etc/portage.
Comment 7 a 2005-03-16 13:20:32 UTC
Yes, run without arguments and g++ and gcc asks for input files and they don't seg fault.

I did not have strace installed, so I can't do that.
Comment 8 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-16 13:33:06 UTC
What gcc version?
Comment 9 a 2005-03-16 19:23:18 UTC
gcc should be latest, up till I emerged glibc.

Version is
gcc (GCC) 3.3.5  (Gentoo Linux 3.3.5-r1, ssp-3.3.2-3, pie-8.7.7.1)
Comment 10 a 2005-03-16 19:38:06 UTC
I just picked up some random commands for test and these are results...
Also I can still ssh into the box.

Hope someone can figure out what is going on.

command still working - cp, rm, mv, nano, g++, gcc, chmod, chown, ifconfig, kill, ping, nmap, cd, pwd, ssh, less, more, dmesg, last, whois, find, ps, df, route, wc, wget, gzip, screen

seg faults - ls, python, vi, perl, make, tar, tail, sleep, date
Comment 11 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-16 21:16:44 UTC
Ok, its prob librt.so flaking around - give me some time and I will build you another one.
Comment 12 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-17 01:06:55 UTC
Can you get this ls and binaries:

  http://dev.gentoo.org/~azarah/bins/i586/ls.bz2
  http://dev.gentoo.org/~azarah/bins/i586/tar.bz2

and see if they work by you (note, its just the /bin/{ls,tar} binaries, not gentoo binary packages) ?

If so, then replace your versions with them, and get the coreutils/python/make packages there, and:

  tar -jxpf <name>.tbz2 -C /

which should hopefully get you up and running.
Comment 13 a 2005-03-17 01:28:59 UTC
I just 'wget' your bz2 files and did 'bzip2 -d theFiles', 'chmod 755 theFiles' and run them, but they also caused segmentation faults.

Is it safe if I download stage 3 ball and overwrite the system?
And maybe I try to re-emerge everything that gets outdated again.
Comment 14 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-17 01:46:29 UTC
Ok, try this .. get:

   http://dev.gentoo.org/~azarah/bins/i586/librt-2.3.4.so.bz2

Then:

  # bunzip librt-2.3.4.so.bz2
  # chmod a+rx librt-2.3.4.so
  # mv librt-2.3.4.so /lib
  # ldconfig

And see if cp/whatever works then ?
Comment 15 a 2005-03-17 01:51:06 UTC
I replaced the librt-2.3.4.so with the one you provided and done chmod and ldconfig (ldconfig didn't print anything out upon running), but it seems 'ls' still seg faults. cp has been working after the glibc update.
Comment 16 a 2005-03-17 01:56:38 UTC
Just noticed, it may not be related, but it still looks weired, but I run depscan.sh just for no reason and I get

 * Caching service dependencies...
 * No scripts to process!
/var/lib/init.d/depcache: /var/lib/init.d/depcache: No such file or directory

And, sure there is no depcache file in that place...
I think this command also should be working too.

I also run 'rc-status', and it only showed 

Runlevel: default

single line and no more.

I really haven't done anything special on this box, just been keeping it running as a router and doing world updates twice a week automated. And the last update halted after glibc got emerged.
Comment 17 a 2005-03-17 02:00:26 UTC
Created attachment 53685 [details]
Packages installed in the system (qpkg -I)
Comment 18 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-17 02:37:58 UTC
You did not perhaps change the CHOST before the update?  I googled a bit and that might be an issue.

Also, I know it might not do anything, but can you try:

  # export LD_ASSUME_KERNEL=2.2.5
  # ls
Comment 19 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-17 02:43:27 UTC
Another alternative is also booting from a livecd, and then getting:

  http://dev.gentoo.org/~azarah/bins/i586/glibc-2.3.4.20041102-r1.tbz2

and then mount your root to say /mnt/root, and do:

  tar -jxpf glibc-2.3.4.20041102-r1.tbz2 -C /mnt/root/

Then chroot and check if things still is wonky.  If still, then also try going out of the chroot, get:

  http://dev.gentoo.org/~azarah/bins/i586/coreutils-5.2.1-r4.tbz2

Also:

  tar -jxpf coreutils-5.2.1-r4.tbz2 -C /mnt/root/

and have a look what ls does in the chroot.
Comment 20 a 2005-03-17 02:51:25 UTC
CHOST has been 'i686-pc-linux-gnu' since the initial installation and I never touched it, or I should have for glibc update?

I did
export LD_ASSUME_KERNEL=2.2.5
and things have changed a bit.

'ls' now claims

ls: error while loading shared libraries: librt.so.1: cannot open shared object file: No such file or directory

although, I can see librt.so.1 in the /lib dir, and even tried LD_LIBRARY_PATH set to /lib, but still claims the same.

And some other commands that was working also now claims similar problems, as in 'find', 

find: error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory

Unsetting LD_ASSUME_KERNEL, puts it back into the siuation before.

Also I have 2 more Gentoo box running, which still they're in normal state, since I haven't had them updated to THE glibc yet (though they get updated within a day or 2 automatically, maybe I should not?), so if I have to copy or bring some files, I can do that from those machines. Their CFLAG and CHOST are both same as I posted in the comment for 'emerge info' running on x86 cpu.
Comment 21 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-17 03:04:17 UTC
Try to do: readlink /lib/librt.so.1
And also do while LD_ASSUME_KERNEL is set: ldd /bin/ls
Comment 22 a 2005-03-17 03:10:31 UTC
# readlink /lib/librt.so.1
librt-2.3.4.so.older

This '.older' is the original one I just moved, to get it replaced with the one you provided.

And as for 'ldd', after I set LD_KERNEL_ASSUME, now 'sh' complains about missing shared object and 'ldd' won't run anymore.

# export LD_ASSUME_KERNEL=2.2.5

# ldd /bin/ls
/bin/sh: error while loading shared libraries: libdl.so.2: cannot open shared object file: No such file or directory

# sh
sh: error while loading shared libraries: libdl.so.2: cannot open shared object file: No such file or directory

I posted this issue on the Gentoo forum, but I don't get much response, and it's wiered it's not happening to other people.

The last emerge installed

sys-apps/texinfo-4.8
sys-libs/db-1.85-r2
sys-libs/glibc-2.3.4.20041102-r1

And the system broke.

I sort of tried to track any sign of intrusion (if they're good at it, I know they delete any trace) but by looking at chkrootkit, history, log or ssh attempt, they weren't really found.
Comment 24 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-17 03:12:57 UTC
The librt.so.1 should point to the new librt-2.3.4.so I asked you to test ...
Comment 25 a 2005-03-17 03:17:10 UTC
Ok. I just moved .older file out of the way to root's home, and executed 'ldconfig', and readlink to librt.so.1 now points to librt-2.3.4.so the one you provided sitting in /lib, but still 'ls' won't work and setting LD_ASSUME_KERNEL makes it complain error about missing shared object.
Comment 26 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-17 03:21:03 UTC
What profile are you using btw?
Comment 27 a 2005-03-17 03:22:25 UTC
Sorry, but what you mean by which profile?
Comment 28 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-17 03:23:19 UTC
No matter about the profile thing - its only amd64 that sets nptl in newer profiles.  Any possibility you can try the chroot thing ?
Comment 29 a 2005-03-17 03:24:57 UTC
Ok, I was going to visit the place where the machine is tomorrow, there's a Gentoo or Knoppix cd lying around there, so I try that chroot with those live-cd.

Tx for help.
Comment 30 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-17 03:28:35 UTC
Sorry for the issues :(  It should work fine, and the only place I have seen this before is a non-nptl to nptl conversion.
Comment 31 a 2005-03-17 03:37:59 UTC
I think the last time glibc messed up my whole 3 machines was probably something to do with ntpl, when I set that in USE flag and re-emerged glibc.

At that time there was problem with libpthread and I think I just replaced it with a sane one and things got working again, and I never use ntpl since then.

It's still better case, since the last one, I couldn't even ssh into the box.
I write what happanes in chroot, tomorrow.
Comment 32 a 2005-03-17 12:56:59 UTC
Ok, I just fearlessly emerged glibc on another different Gentoo machine last night, and surprisingly the system never went unusable.

Also there seems to be that this problem is happenening to other people too according to the forum thread I just posted above.

I post the condition of the machine that actually made the glibc to work.
This machine is not a router, but more like a desktop like built machine in terms of software structure.

# emerge info

Portage 2.0.51.19 (default-linux/x86/2004.3, gcc-3.3.5, glibc-2.3.4.20041102-r1, 2.6.10-gentoo-r6 i686)
=================================================================
System uname: 2.6.10-gentoo-r6 i686 Intel(R) Pentium(R) 4 CPU 2.80GHz
Gentoo Base System version 1.4.16
Python:              dev-lang/python-2.3.4-r1 [2.3.4 (#1, Feb  8 2005, 04:10:11)]
distcc 2.16 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
ccache version 2.3 [enabled]
dev-lang/python:     2.3.4-r1
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.5, 1.7.9-r1, 1.9.4, 1.6.3, 1.4_p6, 1.8.5-r3
sys-devel/binutils:  2.15.92.0.2-r1
sys-devel/libtool:   1.5.10-r4
virtual/os-headers:  2.4.22-r1
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-march=pentium3 -O2 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-march=pentium3 -O2 -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig buildpkg ccache distlocks sandbox sfperms"
GENTOO_MIRRORS="ftp://ftp.ecc.u-tokyo.ac.jp/GENTOO http://gentoo.oregonstate.edu http://www.ibiblio.org/pub/Linux/distributions/gentoo"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.asia.gentoo.org/gentoo-portage"
USE="x86 X alsa anthy apache2 apm avi berkdb bitmap-fonts cddb cdr crypt cups curl dba dedicated dvd emboss exif fam flac font-server foomaticdb fortran gd gdbm gif gnome gpm gstreamer gtk gtk2 iconv imap imlib jpeg libg++ libwww mad maildir mikmod mmx motif moznocompose moznoirc moznomail mozplaintext mozsvg mp3 mpeg mysql ncurses nls nopop3d opengl pam pcntl pcre pdflib perl png python quicktime readline samba scanner sdl session spell sse ssl svg svga tcltk tcpd tiff truetype truetype-fonts type1-fonts usb userlocales xml2 xmms xv zlib"
Unset:  ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL, LDFLAGS, PORTDIR_OVERLAY
Comment 33 a 2005-03-17 12:57:54 UTC
Created attachment 53720 [details]
/etc/make.conf file from the machine that didn't break
Comment 34 a 2005-03-17 13:00:01 UTC
Created attachment 53721 [details]
Installed software on machine that didn't break (qpkg -I)
Comment 35 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-17 13:43:28 UTC
Can you remember what glibc versions on both was before the update?  There is really no diff between glibc-2.3.4.20041102 and glibc-2.3.4.20041102-r1 other than one patch fixing a tls issue (should not really hit this if not using nvidia-glx with nptl glibc).  Also, I see on the forums some have this issue with 2.3.4.2005* as well, or do I remember incorrectly?
Comment 36 Andy Wang 2005-03-17 14:45:52 UTC
I ran across this problem.
ldd still worked.  ldd'ed a broken binary (/usr/bin/python) and noticed somethign odd.  I built a 2.4.1 compatible non-nptl glibc to support some binary applications I have and they were being linked in at runtime for some reason.  Deleted that glibc install and the problem was solved.  Do you have a custom built glibc installed somewhere for this purpose?  If so, that might explain it.
Comment 37 Lars T. Mikkelsen 2005-03-17 15:44:11 UTC
I've got the same problem as Hideki described - i.e. all programs linked to libpthread.so.0 (like python, ls, tar, make etc.) seg fault. However, the problem occured when updating glibc from 2.3.4.20050125 to 2.3.4.20050125-r1.

The following is an equivalent to emerge --info made by hand:

Portage 2.0.51.19 (default-linux/x86/2005.0, gcc-3.4.3, glibc-2.3.4.20050125-r1, 2.6.10 i686)
=================================================================
System uname: 2.6.10 i686 VIA Samuel 2
Gentoo Base System version 1.6.10
Python:              dev-lang/python-2.3.5
distcc 2.18.3 i586-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
dev-lang/python:     2.3.5
sys-devel/autoconf:  2.59-r6, 2.13
sys-devel/automake:  1.7.9-r1, 1.8.5-r3, 1.9.5, 1.5, 1.4_p6, 1.6.3
sys-devel/binutils:  2.15.92.0.2-r6, 2.15.92.0.2-r1
sys-devel/libtool:   1.5.10-r5
virtual/os-headers:  2.6.11
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CFLAGS="-O2 -march=i586 -fomit-frame-pointer"
CHOST="i586-pc-linux-gnu"
CONFIG_PROTECT="/etc /var/qmail/control /usr/share/config /usr/kde/2/share/config /usr/kde/3/share/config"
CONFIG_PROTECT_MASK="/etc/gconf"
CXXFLAGS="-O2 -march=i586 -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperms"
GENTOO_MIRRORS="ftp://ftp.uni-erlangen.de/pub/mirrors/gentoo ftp://vlaai.snt.utwente.nl/pub/os/linux/gentoo/ http://ftp.uni-erlangen.de/pub/mirrors/gentoo http://gentoo.tiscali.nl/gentoo/"
LANG=en_DK.UTF-8
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="apache2 -arts dba gd -gtk -gtk2 -kde -gnome mysql -qt -X"
Unset:  ASFLAGS, CBUILD, CTARGET, LC_ALL, LDFLAGS, PORTDIR_OVERLAY
Comment 38 a 2005-03-17 17:12:15 UTC
The glibc that was installed on those systems before this upgrade was

glibc-2.3.4.20040808-r1

and not the 20041102 series.
I don't think 2.3.4-20041102 ever became stable, unless that has been stable for only like a few days and I missed it in the auto-update.

I tried upgrading another different Gentoo and that one went well too...
They have same CHOST CFLAG and using stable tree on x86.
Comment 39 a 2005-03-17 17:16:32 UTC
Created attachment 53741 [details]
Just for reference on another machine that went well (make.conf)

This one also had the 20040808-r1 glibc before update with gcc version 3.3.5.

# emerge info

Portage 2.0.51.19 (default-linux/x86/2004.3, gcc-3.3.5,
glibc-2.3.4.20041102-r1,
 2.6.10-gentoo-r6 i686)
=================================================================
System uname: 2.6.10-gentoo-r6 i686 Transmeta Efficeon(tm) Processor TM8000
Gentoo Base System version 1.4.16
Python: 	     dev-lang/python-2.3.4-r1 [2.3.4 (#1, Mar 13 2005,
07:58:34)
]
distcc 2.16 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [enabled]

ccache version 2.3 [enabled]
dev-lang/python:     2.3.4-r1
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.8.5-r3, 1.5, 1.9.4, 1.6.3, 1.7.9-r1, 1.4_p6
sys-devel/binutils:  2.15.92.0.2-r1
sys-devel/libtool:   1.5.10-r4
virtual/os-headers:  2.4.22-r1
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-march=pentium3 -O2 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config
/usr/lib/X11/xkb /usr/share/config /var/bind /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-march=pentium3 -O2 -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig buildpkg ccache distcc distlocks sandbox
sfperms"
GENTOO_MIRRORS="ftp://ftp.ecc.u-tokyo.ac.jp/GENTOO
http://gentoo.oregonstate.edu
http://www.ibiblio.org/pub/Linux/distributions/gentoo"
MAKEOPTS="-j7"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.asia.gentoo.org/gentoo-portage"
USE="x86 X alsa anthy apm berkdb crypt curl fam foomaticdb fortran gdbm gnome
gpm gtk gtk2 imlib jpeg lcd libg++ libwww mmx motif mozsvg mp3 ncurses nls
opengl pam pcntl pcre perl png python readline sdl sse ssl svg svga tcpd
truetype-fonts unicode userlocales xml2 xv zlib"
Unset:	ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL, LDFLAGS, PORTDIR_OVERLAY
Comment 40 Neil Bothwick 2005-03-17 17:39:57 UTC
I have installed the new glibc on three machines. The first two, an amd64 and a P4 had no problems. I then installed on a VIA C3 box and all hell broke loose.
The symptoms were as described, above, segfaults on many programs. I untarred the package of the previous version and all is well. Here is emerge info fro the box that broke.

Portage 2.0.51.19 (default-linux/x86/2004.0, gcc-3.4.3, glibc-2.3.4.20050125-r1, 2.6.11-gentoo-r3 i686)
=================================================================
System uname: 2.6.11-gentoo-r3 i686 VIA Samuel 2
Gentoo Base System version 1.6.10
Python:              dev-lang/python-2.3.5 [2.3.5 (#1, Feb 18 2005, 16:57:19)]
distcc 2.18.3 i586-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
dev-lang/python:     2.3.5
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.5, 1.6.3, 1.4_p6, 1.8.5-r3, 1.7.9-r1, 1.9.5
sys-devel/binutils:  2.15.92.0.2-r6
sys-devel/libtool:   1.5.14
virtual/os-headers:  2.6.8.1-r2
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CFLAGS="-march=c3 -Os -pipe -fomit-frame-pointer"
CHOST="i586-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-march=c3 -Os -pipe -fomit-frame-pointer"
DISTDIR="/mnt/portage/distfiles"
FEATURES="autoaddcvs autoconfig buildpkg ccache distlocks sandbox sfperms"
GENTOO_MIRRORS="ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ ftp://ftp.mirror.ac.uk/sites/www.ibiblio.org/gentoo/ ftp://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/"
LDFLAGS="-Wl,-O1"
MAKEOPTS="-j2"
PKGDIR="/mnt/portage/packages/desatio"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/mnt/portage/local"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 3dnow apache2 apm avi berkdb bitmap-fonts crypt cups curl emboss encode fam font-server foomaticdb fortran gdbm gif gtk2 imlib jpeg ldap libg++ libwww mad mbox mikmod mmx motif mp3 mpeg mysql ncurses oggvorbis opengl oss pam pdflib perl png ppds python quicktime readline samba slang ssl svga tcpd tiff truetype-fonts type1-fonts usb xml xml2 xmms xv zlib linguas_en_GB"
Unset:  ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL
Comment 41 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-17 18:40:47 UTC
Try:

  # SEGFAULT_SIGNALS="all" LD_SHOW_AUXV=1 /bin/ls

If that shows nothing, do:

  # LD_SHOW_AUXV=1 /bin/echo
  # SEGFAULT_SIGNALS="all" /bin/ls

If still nothing of interest, do:

  # SEGFAULT_USE_ALTSTACK=1 SEGFAULT_SIGNALS="all" /bin/ls


(on machines that ls still segfaults ...)
Comment 42 Ries 2005-03-18 00:34:31 UTC
My mashine also segfaults, its a Via C3 (eden, the type without cmov). I noticed that the two others with problems, comment #37 and #40 also has C3's. May the thing ignored the make.conf and compiled to a i686? cmov "bug" ?
Comment 43 Lars T. Mikkelsen 2005-03-18 00:53:56 UTC
Running the command Martin suggested gives the following output:

# SEGFAULT_SIGNALS="all" LD_SHOW_AUXV=1 /bin/ls
AT_SYSINFO:      0xffffe400
AT_SYSINFO_EHDR: 0xffffe000
AT_HWCAP:    fpu de tsc msr cx8 mtrr pge mmx
AT_PAGESZ:       4096
AT_CLKTCK:       100
AT_PHDR:         0x8048034
AT_PHENT:        32
AT_PHNUM:        9
AT_BASE:         0xb7fea000
AT_FLAGS:        0x0
AT_ENTRY:        0x80498b0
AT_UID:          0
AT_EUID:         0
AT_GID:          0
AT_EGID:         0
AT_SECURE:       0
AT_PLATFORM:     i686
Segmentation fault

It says i686, so maybe Ries is right about something being compiled to i686? However, this doesn't explain Hideki's problem on a Celeron (comment #6).
Comment 44 Gordon Innes 2005-03-18 03:39:40 UTC
Hi, I had the same problem but with slightly less damage. Emerge mostly worked except fo anything that uses strip-flags from flag-o-matic. I was able to remerge coreutils and tar with the static use flag which stopped them from seg faulting. I then downloaded the binary glibc from bugzilla and I'm currently doing an emerge -e system with the problem glibc masked.

FYI here's my emerge info:

Portage 2.0.51.19 (default-linux/x86/2004.0, gcc-3.3.5, glibc-2.3.4.20041102-r1, 2.6.7-gentoo-r8 i686)
=================================================================
System uname: 2.6.7-gentoo-r8 i686 Pentium III (Coppermine)
Gentoo Base System version 1.4.16
Python:              dev-lang/python-2.3.4-r1 [2.3.4 (#1, Mar 17 2005, 18:37:08)]
dev-lang/python:     2.3.4-r1
sys-devel/autoconf:  2.59-r6, 2.13
sys-devel/automake:  1.7.9-r1, 1.8.5-r3, 1.5, 1.4_p6, 1.6.3, 1.9.4
sys-devel/binutils:  2.15.92.0.2-r1
sys-devel/libtool:   1.5.10-r4
virtual/os-headers:  2.4.22-r1
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-O2 -march=pentium3 -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/bind /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -march=pentium3 -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache distlocks fixpackages nostrip sandbox sfperms"
GENTOO_MIRRORS="ftp://ftp.heanet.ie/pub/gentoo/ http://194.117.158.29/mirrors/gentoo/"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="x86 acl apache2 apm avi berkdb bitmap-fonts cdr crypt dvb dvd emboss encode fam flac font-server foomaticdb fortran gcj gdbm gif gtk2 imlib java jpeg junit kerberos ldap libg++ libwww mad maildir mikmod mmx motif mp3 mpeg mysql ncurses nls oggvorbis pam pdflib perl png python qt quicktime readline samba sdl slang spell sse ssl svga tcltk tcpd tiff truetype truetype-fonts type1-fonts xml2 xmms zlib"
Unset:  ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL, LDFLAGS


Hope this helps!

G.
Comment 45 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-18 05:37:49 UTC
Lars, so the glibc I compiled here (2.3.4.20041102-r1) works fine, but the one compiled on that box, but same version/revision segfaults?
Comment 46 Lars T. Mikkelsen 2005-03-18 06:05:06 UTC
Martin, glibc-2.3.4.20050125-r1 seg faults on my system. I just tried to copy /lib/libpthread-0.10.so (and nothing else) from glibc-2.3.4.20041102-r1.tbz2 (comment #19) and then the programs stop seg faulting.

I'm currently emerge'ing glibc-2.3.4.20041102-r1 to see if it will seg fault.
Comment 47 Jack Grey 2005-03-18 09:01:54 UTC
This left my system unbootable, I had to go to backups to recover. I can provide more information on request.
Comment 48 Carsten Lohrke (RETIRED) gentoo-dev 2005-03-18 09:29:00 UTC
I'm hit, too. Wonderful, it seems that it's not even possible to rollback a binary tarball.

bzip2: I/O or other error, bailing out.  Possible reason follows.
bzip2: Broken pipe
        Input file = /usr/portage/distfiles/freetype-2.1.9.tar.bz2, output file = (stdout)
/usr/lib/portage/bin/ebuild.sh: line 1874: 24066 Exit 1                  bzip2 -dc "${DISTDIR}/${x}"
     24067 Segmentation fault      | tar xf - ${tarvars}

Comment 49 Ries 2005-03-18 09:31:42 UTC
It's the glibc-2.3.4.20050125-r1 that breaks my system, but comment #46 works for me. 
Comment 50 Lars T. Mikkelsen 2005-03-18 09:48:26 UTC
Carsten, try to replace your /lib/libpthread-0.10.so with http://www.cs.aau.dk/~ltm/libpthread-0.10.so (the file is from Martins tarball).
Comment 51 Carsten Lohrke (RETIRED) gentoo-dev 2005-03-18 09:55:26 UTC
Lars: Oh thanks, all "fine". I couldn't fall back to the previous version via emerge/python. but had to unpack the binary tarball manually. I consider this very unpleasant. Portage should just work even if the glibc is dead.
Comment 52 Lars T. Mikkelsen 2005-03-18 14:05:36 UTC
Follow up on comment #46: After emerge'ing glibc-2.3.4.20041102-r1 I don't get any seg faults.
Comment 53 Jack Grey 2005-03-18 18:44:52 UTC
More oddities around glibc-2.3.4.20041102-r1 hosing up the system to which I applied it:

1. The emerge created a host.conf in my /etc directory. Once I restored my lib directories, my resolver did not like it, complaining about the line "mdns off". Renaming the file to host.conf.huh cleared this up.
2. The rights on /dev/null were changed from crwxrwxrwx to crwr-xr-x (wow)
3. The emerge installed a new /etc/init.d/nscd file. It was identical to my previous one. Just because, I guess :-)
4. I had file system corruption problems on two different reiserfs file systems that have never had a problem previously. Removing affected directories ( which happened to be in use for write when the new libs were installed, for example, /var/tmp/portage/glibc* ) and reiserfsck cleared this up. (ouch!)
5. I didn't know I was really hosed until I restarted the system, where boot hung at "entering runlevel 3".

Are these perhaps some side effect of using a system-level diff utility to generate the package? It's hard to understand #3 above in particular.
Comment 54 SpanKY gentoo-dev 2005-03-18 19:14:07 UTC
/etc/host.conf differs between glibc versions which is why you got that error

/dev/null changes permissions because of different factors of build utilities when running as root

every version of glibc installs the nscd init.d script

i *highly* doubt reiserfs corruption is because of glibc
Comment 55 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-19 00:13:54 UTC
Ok, I put up a libpthread (linuxthreads) for those that might have difficulty untarring:

  http://dev.gentoo.org/~azarah/bins/i586/libpthread-0.10.so.bz2
Comment 56 Peter Volkov (RETIRED) gentoo-dev 2005-03-19 02:20:08 UTC
Well. What are your thoughts about mentioned in the forum magic nptl nptlonly USE flag combinations? Can this be related?
Comment 57 Jakub Moc (RETIRED) gentoo-dev 2005-03-19 03:18:55 UTC
Peter, I don
Comment 58 Jakub Moc (RETIRED) gentoo-dev 2005-03-19 03:18:55 UTC
Peter, I don´t know. I tried glibc-2.3.4.20041102-r1 with USE="nptl -nptlonly" and USE="nptl nptlonly" and both works for me. I don´t volunteer for trying USE="-nptl -nptlonly" though - I have had enough troubles with gcc recompile that screwed up my system after successful glibc upgrade (Bug 85490).

I have been using USE="nptl -nptlonly" before, now I am staying with USE="nptl nptlonly", don´t want to play the Russian roulette any more this time. :-)
Comment 59 Lars T. Mikkelsen 2005-03-19 18:10:30 UTC
I've been investigating this issue further. It seems to be setlocale(3) that triggers this bug. The following seg faults on the broken glibc, but not on the non-broken glibc:

$ cat setlocale.c 
#include <locale.h>

int main(int argc, char **argv)
{
  setlocale(LC_ALL, "");

  return 0;
}
$ gcc -lpthread -o setlocale setlocale.c
$ ./setlocale
Segmentation fault
$

Furthermore, it only seg faults if some of the LANG or LC_* environment variables are set.

I think setlocale()'s use of pthread_mutex_lock() and pthread_mutex_unlock() is somehow causing the problem. In the program above the seg fault occurs in textdomain().
Comment 60 Julio Cazares 2005-03-19 23:04:40 UTC
Hi.  My system do not finish compiling glibc-2.3.4.20041102-r1,  It crashes so badly that I suspect is a CPU division by 0 or something,  it crashes where it is installing the language libraries of glibc.

Also is strange that it happens on uniprocessor systems, dual is not affected.
 
Do ACPI and APIC may be involved in the compile problem?

What do you think?


I believe that maybe my crash problem save me of Hideki problems.








Portage 2.0.51.19 (default-linux/x86/2004.2, gcc-3.3.5, glibc-2.3.4.20040808-r1, 2.4.28-gentoo-r8 i686)
=================================================================
System uname: 2.4.28-gentoo-r8 i686 Intel(R) Pentium(R) III CPU             1200MHz
Gentoo Base System version 1.4.16
Python:              dev-lang/python-2.3.4-r1 [2.3.4 (#1, Feb  7 2005, 11:44:00)]
ccache version 2.3 [enabled]
dev-lang/python:     2.3.4-r1
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.5, 1.6.3, 1.7.9-r1, 1.4_p6, 1.8.5-r3, 1.9.4
sys-devel/binutils:  2.15.92.0.2-r1
sys-devel/libtool:   1.5.10-r4
virtual/os-headers:  2.4.22-r1
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-O2 -march=pentium3 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/bind /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -march=pentium3 -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperm sfperms strict"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/Linux/distributions/gentoo"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X acpi apache2 apm arts avi berkdb bitmap-fonts cdr crypt cups curl dga directfb doc dvb dvd emboss encode fam firebird flash font-server foomaticdb fortran gdbm gif gnome gpm gps gstreamer gtk gtk2 imlib informix ipv6 java javascript jpeg kde libg++ libwww mad maildir mikmod mmx motif mp3 mpeg mysql ncurses nls oggvorbis oggvorvis opengl oss pam pdflib perl png python qt quicktime readline samba sasl sdl slang spell sse ssl svga tcltk tcpd tetex tiff truetype truetype-fonts type1-fonts unicode usb wmf x86 xml xml2 xmms xv zlib"
Unset:  ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL, LDFLAGS, PORTDIR_OVERLAY

Comment 61 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-20 23:39:46 UTC
The original reporter have USE="-nptl -nptlonly", so USE="nptl nptlonly" might be the only magical combination if any.

Can anybody plug a debugger in there somewhere?  I err, cant get it to break, so its difficult to try and debug.

Also, anybody who have some time, patience, and had this happen twice on the same machine, can you try without changing anything from what it was when you got the problem originally, try with glibc-2.3.4.20041102 (note, the one that is still ~x86)?
Comment 62 a 2005-03-21 12:12:48 UTC
Sorry but I messed up the system further by trying to copy previous glibc binary package over to the system from another system, but it failed and now ssh don't work, so I can't test the chroot method anymore.

I extracted the binary package (since tar won't work) and scp'ed the files over to the broken machine, but during transfer ssh hung.
So, I suppose copying the previous glibc files over will break the system even further.

Not sure if this is anything related, but the kernel that made it to work are both gentoo-developement-sources and the one that messed up the glibc is the development-sources for my case.
Comment 63 Lars T. Mikkelsen 2005-03-21 12:23:08 UTC
Created attachment 54086 [details]
Debugging with ltrace and gdb

I finally got the broken glibc-2.3.4.20050125-r1 compiled with FEATURES=nostrip
to get some debug symbols (btw. compiling glibc on a VIA C3 takes 6+ hours :-)
). I did some debugging of /bin/date - the results are in the attached file.
The only thing I'm able to get out of this debugging, is that the problem
somehow seems related to the mutex locking/unlocking in pthreads.

Does anyone have ideas for further debugging? I'm able to switch between a
broken and non-broken system quickly, by replacing /lib/libpthread-0.10.so, so
emerge'ing new programs isn't an issue.
Comment 64 Lars T. Mikkelsen 2005-03-21 12:58:09 UTC
Martin, I'm now trying to do an 'USE="nptl nptlonly" emerge glib' to see if that will fix the problem. I will report the result in ~12 hours.
Comment 65 Lars T. Mikkelsen 2005-03-21 13:56:02 UTC
I just noticed that the glibc ebuild is unset'ing the LANGUAGE, LANG, and LC_ALL environment variables. I have, however, set LC_CTYPE in my environment - could this trigger a faulty build of glibc?

Hideki, do you have any LC_ environment variables set (i.e. what is the output of 'set | grep LC_') on your system?
Comment 66 a 2005-03-21 14:25:24 UTC
I'm using Japanese as my primary language, so I set LC_ALL variable as "ja_JP.SJIS" on one of the machine, but the one that broke only acts as a router, so I never fiddled with language settings, except putting 'userlocale' USE flag.

LC_ALL variables are set as,
machine that broke (Celeron) - POSIX
machine that worked 1 (Pentium) - ja_JP.SJIS
machine that worked 2 (Efficeon) - POSIX

So, I'm not sure if this matters to the cause.

On addition, about locale settings, I set 'userlocale' USE on all of the 3 machines, but I only made new locales (which is ja_JP.SJIS) out of localedef on the 2 machines that didn't break.
Comment 67 a 2005-03-21 14:28:16 UTC
Forgot to say, but the LC_ALL variable is set as ja_JP.SJIS on /etc/xprofile, which is loaded inside X environment, so the locale is still set as POSIX in situation like when I ssh in or when I run world updates on machine 2.
Comment 68 Lars T. Mikkelsen 2005-03-21 14:37:51 UTC
Okay, I might be chasing a dead end then. I was just wondering since the seg faults seems to occur in setlocale() and there's a "large" amount of non-US people reporting this bug - at least 1xjp, 2xdk, 1xcz, and 1xde as far as I can see.
Comment 69 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-21 14:45:26 UTC
I am pretty sure LC_ALL should override them all, but as I am not locale cluded up, I am not sure.  Another test might be to remerge on the same machine with all those LC_* variables unset.

PS: can anybody still try the ~x86 glibc-2.3.4.20041102?  I want to see if it may be the tls patch we added, although it should not be.
Comment 70 Lars T. Mikkelsen 2005-03-21 14:55:33 UTC
Martin, I currently have the 2.3.4.20050125-r1 version installed (which seg faults) - I havn't had the 20041102-r1 installed. Should I try to revert to 20041102 or 20050125? And what's most important to test:
1) USE="nptl nptlonly" emerge =glib-2.3.4.20050125-r1
2) LC_CTYPE= emerge =glib-2.3.4.20050125-r1
3) emerge =glib-2.3.4.20050125 / emerge =glib-2.3.4.20041102
Comment 71 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-21 16:09:12 UTC
Erk, no, don't revert!  Downgrading glibc is bad if you already recompiled stuff against the newer glibc.
Comment 72 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-21 16:13:54 UTC
You can try glibc-2.3.4.20050125-r1 with this patch to see if removing that patch makes a diff:

-----
Index: sys-libs/glibc/glibc-2.3.4.20050125-r1.ebuild
===================================================================
RCS file: /var/cvsroot/gentoo-x86/sys-libs/glibc/glibc-2.3.4.20050125-r1.ebuild,v
retrieving revision 1.29
diff -u -r1.29 glibc-2.3.4.20050125-r1.ebuild
--- sys-libs/glibc/glibc-2.3.4.20050125-r1.ebuild       19 Mar 2005 22:57:39 -0000      1.29
+++ sys-libs/glibc/glibc-2.3.4.20050125-r1.ebuild       22 Mar 2005 00:13:09 -0000
@@ -1100,6 +1100,8 @@

        [[ $(gcc-major-version) == "4" ]] || GLIBC_PATCH_EXCLUDE="${GLIBC_PATCH_EXCLUDE} 5040_all_2.3.4-gcc4.patch"

+       GLIBC_PATCH_EXCLUDE="${GLIBC_PATCH_EXCLUDE} 5060_all_fix-dl-next-tls-modid-assert.patch"
+
        toolchain-glibc_src_unpack

        case $(tc-arch) in
Comment 73 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-21 16:17:32 UTC
Also Lars, please keep that broken libpthread around if you get it fixed in case we need to test something (can you attach it here btw if for i586/i686 - sorry forgot) ?
Comment 74 Lars T. Mikkelsen 2005-03-21 23:21:45 UTC
Emerge'ing with USE="nptl nptlonly" builds a non-broken glibc! However, I'm not able to break it by copying the old libpthread-0.10.so back in place.

I'm now emerge'ing with 5060_all_fix-dl-next-tls-modid-assert.patch disabled.
Comment 75 Lars T. Mikkelsen 2005-03-21 23:25:40 UTC
Created attachment 54132 [details]
Broken libpthread-0.10.so
Comment 76 Jakub Moc (RETIRED) gentoo-dev 2005-03-22 01:50:37 UTC
Lars, this is what I have on my nptlonly system (glibc-2.3.4.20041102-r1)

# locate libpthread
/lib/libpthread-2.3.4.so
/lib/libpthread.so.0
/usr/lib/libpthread.so
/usr/lib/libpthread_nonshared.a
/usr/lib/libpthread.a

Cannot find any libpthread-0.10.so
Comment 77 Lars T. Mikkelsen 2005-03-22 04:56:36 UTC
Ahh... when using USE="nptl nptlonly", I havn't got a libpthread-0.10.so - I didn't notice that in the hurry this morning. This seems to be the explanation why USE="-nptl -nptlonly" breaks and USE="nptl nptlonly" doesn't. Thanks for noticing this, Jakub!
Comment 78 Lars T. Mikkelsen 2005-03-22 06:00:50 UTC
Martin, I just finished emerge'ing with 5060_all_fix-dl-next-tls-modid-assert.patch disabled - it doesn't fix the problem. :-(
Comment 79 Jakub Moc (RETIRED) gentoo-dev 2005-03-22 07:29:08 UTC
Checked on a USE="nptl -nptlonly" system (glibc-2.3.4.20040808-r1) and no libpthread-0.10.so either:

# locate libpthread
/lib/libpthread-2.3.4.so
/lib/libpthread.so.0
/usr/lib/libpthread.so
/usr/lib/libpthread_nonshared.a
/usr/lib/libpthread.a
Comment 80 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-22 09:30:09 UTC
Weird one - anything under /lib/tls ?
Comment 81 Martin Schlemmer (RETIRED) gentoo-dev 2005-03-22 09:31:16 UTC
Actually not - it did not yet support the dual nptl/linuxthreads install.
Comment 82 Chetan Sarva 2005-03-23 23:56:48 UTC
Also having this problem however it cropped while emerging glibc and the emerge failed. 

make[2]: Leaving directory `/var/tmp/portage/glibc-2.3.4.20041102-r1/work/glibc-2.3.3/localedata'
make[1]: Leaving directory `/var/tmp/portage/glibc-2.3.4.20041102-r1/work/glibc-2.3.3'
 ^[[32;01m*^[[0m Installing man pages and docs...
/usr/lib/portage/bin/ebuild.sh: line 1874: 16710 Segmentation fault      env LD_LIBRARY_PATH="${D}/$(get_libdir)" ${x} >/dev/null

!!! ERROR: sys-libs/glibc-2.3.4.20041102-r1 failed.
!!! Function src_install, Line 1008, Exitcode 139
!!! simple run test (ls) failed
!!! If you need support, post the topmost build error, NOT this status message.

My current version is glibc-2.3.4.20040808-r1.

Portage 2.0.51.19 (default-linux/x86/2004.3, gcc-3.3.5, glibc-2.3.4.20040808-r1, 2.6.9-gentoo-r1 i686)
=================================================================
System uname: 2.6.9-gentoo-r1 i686 Intel(R) Xeon(TM) CPU 2.40GHz
Gentoo Base System version 1.4.16
Python:              dev-lang/python-2.2.3-r5,dev-lang/python-2.3.4 [2.3.4 (#1, Nov  6 2004, 18:40:48)]
ccache version 2.3 [enabled]
dev-lang/python:     2.2.3-r5, 2.3.4
sys-devel/autoconf:  2.59-r6, 2.13
sys-devel/automake:  1.7.9-r1, 1.8.5-r3, 1.5, 1.4_p6, 1.6.3, 1.9.4
sys-devel/binutils:  2.15.92.0.2-r1
sys-devel/libtool:   1.5.2-r7
virtual/os-headers:  2.4.19-r1, 2.4.22-r1
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-O2 -march=pentium4 -fomit-frame-pointer -funroll-loops -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -march=pentium4 -fomit-frame-pointer -funroll-loops -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig ccache distlocks sandbox sfperms"
GENTOO_MIRRORS="http://mirror.datapipe.net/gentoo/ http://gentoo.mirrors.pair.com/ http://www.gtlib.cc.gatech.edu/pub/gentoo"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.us.gentoo.org/gentoo-portage/"
USE="x86 apache2 berkdb curl gdbm innodb mysql ncurses nls pam readline ssl tcltk xml2 zlib"
Unset:  ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL, LDFLAGS
Comment 83 S.Caglar Onur 2005-03-24 03:09:06 UTC
make[2]: Leaving directory `/var/tmp/portage/glibc-2.3.4.20041102-r1/work/glibc-2.3.3/localedata'
make[1]: Leaving directory `/var/tmp/portage/glibc-2.3.4.20041102-r1/work/glibc-2.3.3'
 * Installing man pages and docs...
/usr/portage/sys-libs/glibc/glibc-2.3.4.20041102-r1.ebuild: line 1006:  8301 Par
Comment 84 S.Caglar Onur 2005-03-24 03:09:06 UTC
make[2]: Leaving directory `/var/tmp/portage/glibc-2.3.4.20041102-r1/work/glibc-2.3.3/localedata'
make[1]: Leaving directory `/var/tmp/portage/glibc-2.3.4.20041102-r1/work/glibc-2.3.3'
 * Installing man pages and docs...
/usr/portage/sys-libs/glibc/glibc-2.3.4.20041102-r1.ebuild: line 1006:  8301 Parçalama ar&#305;zas&#305;    env LD_LIBRARY_PATH="${D}/$(get_libdir)" ${x} >/dev/null

!!! ERROR: sys-libs/glibc-2.3.4.20041102-r1 failed.
!!! Function src_install, Line 1008, Exitcode 139
!!! simple run test (ls) failed
!!! If you need support, post the topmost build error, NOT this status message.

Same problem here with USE="nptl -nptlonly", now trying with USE="nptl nptlonly"
Comment 85 Derek Brost 2005-03-24 10:18:23 UTC
I have three x86 systems of which two hit this bug:
    Sys#1 using linuxthreads; from glibc-2.3.4.20040808-r1 upgrading to glibc-2.3.4.20041102-r1 segfaults the same as comment #81
    Sys#2 same as Sys#1... same segfault
    Sys#3 using NPTL; upgraded glibc with no problems
Comment 86 SpanKY gentoo-dev 2005-03-24 22:26:07 UTC
comments #81, #82, and #83 are unrelated to this bug, see Bug 86465
Comment 87 SpanKY gentoo-dev 2005-03-24 22:54:32 UTC
glibc-2.3.4.20041102 was the first version of glibc to introdue the dual linuxthreads/nptl fun

linuxthreads has: (USE=-nptl -nptlonly)
/lib/libpthread-0.10.so
/lib/libpthread.so.0 -> libpthread-0.10.so

nptl has: (USE=nptl nptlonly)
/lib/libpthread-2.3.4.so
/lib/libpthread.so.0 -> libpthread-2.3.4.so

nptl + linuxthreads should (of course) be: (USE=nptl -nptlonly)
/lib/libpthread-0.10.so
/lib/libpthread.so.0 -> libpthread-0.10.so
/lib/tls/libpthread-2.3.4.so
/lib/tls/libpthread.so.0 -> libpthread-2.3.4.so
Comment 88 SpanKY gentoo-dev 2005-03-25 09:54:27 UTC
*** Bug 86400 has been marked as a duplicate of this bug. ***
Comment 89 a 2005-03-25 17:18:16 UTC
I've now done a clean fresh install on the machine that broke from the stage3 set for 2004.3 release.

The glibc came with the release was 20040808-r1, and with the same exact setting as the one that went broke (as in USE on glibc, CFLAG, CXXFLAG, CHOST), I emerged the 20041102 (not -r1) the one you wanted to try and suprisingly this time, it won't cause anymore segmentation fault.

The CFLAG is Pentium3 on the emerge info I posted on the earlier post, but as I mentioned, it was Pentium4 when compiled glibc when it went broke.

This is 'emerge info' from the current enviroment,

Portage 2.0.51.19 (default-linux/x86/2004.3, gcc-3.3.5, glibc-2.3.4.20041102-r0, 2.6.11.4 i686)
=================================================================
System uname: 2.6.11.4 i686 Intel(R) Celeron(R) CPU 2.40GHz
Gentoo Base System version 1.4.16
Python:              dev-lang/python-2.3.4-r1 [2.3.4 (#1, Mar 24 2005, 05:12:40)]
distcc 2.16 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [enabled]
ccache version 2.3 [enabled]
dev-lang/python:     2.3.4-r1
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.8.5-r3, 1.5, 1.7.9-r1, 1.6.3, 1.4_p6, 1.9.4
sys-devel/binutils:  2.15.92.0.2-r1
sys-devel/libtool:   1.5.2-r5
virtual/os-headers:  2.4.22-r1
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-march=pentium4 -O2 -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/bind /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-march=pentium4 -O2 -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig buildpkg ccache distcc distlocks sandbox sfperms"
GENTOO_MIRRORS="ftp://ftp.ecc.u-tokyo.ac.jp/GENTOO http://gentoo.oregonstate.edu http://www.ibiblio.org/pub/Linux/distributions/gentoo"MAKEOPTS="-j5"PKGDIR="/usr/portage/packages"PORTAGE_TMPDIR="/var/tmp"PORTDIR="/usr/portage"SYNC="rsync://rsync.asia.gentoo.org/gentoo-portage"USE="x86 berkdb crypt fortran gdbm libg++ libwww mp3 ncurses nls pam pcre perl python readline ssl tcpd userlocales xml2 zlib"Unset:  ASFLAGS, CBUILD, CTARGET, LANG, LC_ALL, LDFLAGS, PORTDIR_OVERLAY

I try to build -r1 glibc from now and see if it breaks.
Comment 90 a 2005-03-25 17:24:31 UTC
Created attachment 54487 [details]
make.conf from the machine that worked on the 2nd try
Comment 91 a 2005-03-25 18:35:14 UTC
Created attachment 54491 [details]
Packages installed (equery -C l)

Compiled 20041102-r1 and it's working fine...

I put the packages installed just after 20041102-r1 got completed emerging.
Comment 92 Henk Poley 2005-03-27 09:13:02 UTC
I've got an error message that looks a lot like #48

Tar and bzip2 still work outside emerge (I can extract /usr/portage/distfiles/glibc-2.3.3.tar.bz2). Have been trying to get it to work within emerge/portage too.

Extracted the glibc from http://dev.gentoo.org/~avenj/bins/i686/ 
Compiled tar and bzip2 by hand.
Extracted portage_rescue
Rebooted.

Nothing of these things helped, but export "LD_ASSUME_KERNEL=2.2.5 emerge glibc" seems to work. Does this dynamic loader assumption have any negative consequences on compiling glibc?

---
MFL root # emerge glibc
Calculating dependencies ...done!
>>> emerge (1 of 1) sys-libs/glibc-2.3.4.20041102-r1 to /
>>> md5 src_uri ;-) glibc-2.3.3.tar.bz2
>>> md5 src_uri ;-) glibc-manpages-2.3.4.tar.bz2
>>> md5 src_uri ;-) glibc-infopages-2.3.4.tar.bz2
>>> md5 src_uri ;-) glibc-2.3.4-branch-update-20041102.patch.bz2
>>> Unpacking source...
>>> Unpacking glibc-2.3.3.tar.bz2 to /var/tmp/portage/glibc-2.3.4.20041102-r1/work

bzip2: I/O or other error, bailing out.  Possible reason follows.
bzip2: Broken pipe
        Input file = /usr/portage/distfiles/glibc-2.3.3.tar.bz2, output file = (stdout)
/usr/lib/portage/bin/ebuild.sh: line 1857: 15078 Exit 1                  bzip2 -dc "${DISTDIR}/${x}"
     15079 Segmentation fault      | tar xf - ${tarvars}

!!! ERROR: sys-libs/glibc-2.3.4.20041102-r1 failed.
!!! Function unpack, Line 382, Exitcode 139
!!! failure unpacking glibc-2.3.3.tar.bz2
!!! If you need support, post the topmost build error, NOT this status message.
Comment 93 Henk Poley 2005-03-27 15:14:07 UTC
Of course this wouldn't work. Emerge died with a make error message. Then I tried emerging tar with the LD_ASSUME_KERNEL=2.2.5 trick, and now emerge works normaly again. I'm trying to emerge glibc now.

Error message of "LD_ASSUME_KERNEL=2.2.5 emerge glibc":
---
.././scripts/mkinstalldirs /var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/sunrpc/rpcsvc
CPP='i686-pc-linux-gnu-gcc -E -x c-header'  /var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/elf/ld-linux.so.2 --library-path /var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/math:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/elf:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/dlfcn:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/nss:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/nis:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/rt:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/resolv:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/crypt:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/linuxthreads /var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/sunrpc/rpcgen -Y ../scripts -c rpcsvc/bootparam_prot.x -o /var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/sunrpc/xbootparam_prot.T
/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/sunrpc/rpcgen: error while loading shared libraries: /var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/sunrpc/rpcgen: cannot open shared object file: No such file or directory
make[2]: *** [/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/sunrpc/xbootparam_prot.stmp] Error 127
make[2]: *** Waiting for unfinished jobs....
mkdir /var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/sunrpc/rpcsvc
make[2]: *** Waiting for unfinished jobs....
CPP='i686-pc-linux-gnu-gcc -E -x c-header'  /var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/elf/ld-linux.so.2 --library-path /var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/math:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/elf:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/dlfcn:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/nss:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/nis:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/rt:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/resolv:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/crypt:/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/linuxthreads /var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/sunrpc/rpcgen -Y ../scripts -h rpcsvc/bootparam_prot.x -o /var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/sunrpc/rpcsvc/bootparam_prot.T
make[2]: *** Waiting for unfinished jobs....
/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/sunrpc/rpcgen: error while loading shared libraries: /var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/sunrpc/rpcgen: cannot open shared object file: No such file or directory
make[2]: *** [/var/tmp/portage/glibc-2.3.4.20041102-r1/work/build-default-i686-pc-linux-gnu-linuxthreads/sunrpc/rpcsvc/bootparam_prot.stmp] Error 127
make[2]: Leaving directory `/var/tmp/portage/glibc-2.3.4.20041102-r1/work/glibc-2.3.3/sunrpc'
make[1]: *** [sunrpc/others] Error 2
make[1]: Leaving directory `/var/tmp/portage/glibc-2.3.4.20041102-r1/work/glibc-2.3.3'
make: *** [all] Error 2

!!! ERROR: sys-libs/glibc-2.3.4.20041102-r1 failed.
!!! Function src_compile, Line 741, Exitcode 2
!!! (no error message)
!!! If you need support, post the topmost build error, NOT this status message.
Comment 94 Henk Poley 2005-03-27 16:04:33 UTC
The last thing worked. "emerge tar" now compiles into a working binary every time.

What I did (mostly):
http://dev.gentoo.org/~avenj/bins/ replace anything on your system that might not work.
"LD_ASSUME_KERNEL=2.2.5 emerge tar" tar should work now
emerge glibc

Try some other stuff, like rebuilding the packages you replaced with the Avenj tarballs.
Comment 95 Jeremy Huddleston (RETIRED) gentoo-dev 2005-07-14 02:47:43 UTC
Is this still a problem for anyone?
Comment 96 Henk Poley 2005-07-15 01:58:59 UTC
Not a problem for me anymore. But I did learn that documentation on how to  
handle such an 'emergency' was (is?) a bit lacking. 
 
Maybe a documentation bug should be opened? 
Comment 97 Julien Allanos (RETIRED) gentoo-dev 2005-07-15 13:08:33 UTC
I didn't experience this for a long time. Removing myself from CC.
Comment 98 Jeremy Huddleston (RETIRED) gentoo-dev 2005-07-15 14:52:01 UTC
ok, I'm going to close it then... as for a documentation bug, you might like to
talk with the Gentoo Documentation team about contributing documentation for that
Comment 99 Ed Davison 2005-08-03 09:00:47 UTC
I actually started getting the error message described by the original poster 2
days ago.  I think it started when I untarred a stage3 tarball onto my system as
a "jumpstart" to rebuild a hacked system.

Since that time I have been able to get back to a somewhat stable state by doing
a quickpkg of glibc on another system and then emerging that on the unstable system.

Now, however, I am back to seg fault on many basic commands, gawk, ls, rm, etc.

I am using glibc 2.3.5 (and was before I untarred the stage3 tarball as well).

Nothing in this thread so far has helped ...