Some binary distros like mandrake and suse patches their glibcs with x86_64 optimized strings and an x86_64 optimized libm to improve performance. I've extracted those from an mandrake SRPM. The x86_64 strings patch works perfectly and gives a major improvement in memory copy performance. The libm patches don't build currently because of unresolved symbols. I found a small C program on a suse mailing-list to measure glibc memory copy performance: http://lists.suse.com/archive/suse-amd64/2005-Mar/0220.html With the glibc 2.3.5 currently in gentoo: isidor ~ # ./memcpy 2200 1000 1048576 Memory to memory copy rate = 1291.600098 MBytes / sec. Block size = 1048576. With glibc 2.3.5 + amd64 optimized strings: isidor ~ # ./memcpy 2200 1000 1048576 Memory to memory copy rate = 2389.321777 MBytes / sec. Block size = 1048576. That's an improvement of over 1000mb/s! Suse 9.3 also gives about 2300mb/s out of the box. This has been discussed on the mailinglist in this thread: http://marc.theaimsgroup.com/?t=112213015500003&r=1&w=2 The patches are, and should be applied in this order according to the .spec file: glibc-2.3.3-x86_64-new-libm.patch glibc-2.3.5-libm-ulps.patch <- Don't know if this is needed or related. It also modifies files for other arches. glibc-2.3.3-x86_64-pow10-aliases.patch glibc-2.3.5-x86_64-new-libm-aliasing.patch glibc-2.3.3-amd64-string.patch
Created attachment 64301 [details, diff] glibc-2.3.3-x86_64-new-libm.patch
Created attachment 64302 [details, diff] glibc-2.3.5-libm-ulps.patch
Created attachment 64303 [details, diff] glibc-2.3.3-x86_64-pow10-aliases.patch
Created attachment 64304 [details, diff] glibc-2.3.5-x86_64-new-libm-aliasing.patch
Created attachment 64305 [details, diff] glibc-2.3.3-amd64-string.patch
Created attachment 64306 [details] memcpy.c Utility to measure memory copy performance. Bugfixed version from Matt Randolph on the mailing-list.
I'm sorry to report that using the string patches causes nano to crash with a bus error when searching. With the patch it crashes, without it doesn't crash. A backtrace doesn't turn up many useful things, first 3 lines: #0 0x00002aaaaad93ff0 in malloc_usable_size () from /lib/libc.so.6 #1 0x00002aaaaad9483a in malloc_trim () from /lib/libc.so.6 #2 0x00002aaaaad964c3 in free () from /lib/libc.so.6 #3 0x00002aaaaad975bc in realloc () from /lib/libc.so.6 emerge --info: Portage 2.0.51.22-r2 (default-linux/amd64/2005.0, gcc-3.4.4, glibc-2.3.5-r1, 2.6.12-gentoo-r6 x86_64) ================================================================= System uname: 2.6.12-gentoo-r6 x86_64 AMD Athlon(tm) 64 Processor 3500+ Gentoo Base System version 1.6.13 distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled] ccache version 2.4 [enabled] dev-lang/python: 2.4.1-r1 sys-apps/sandbox: 1.2.11 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6 sys-devel/binutils: 2.16.1 sys-devel/libtool: 1.5.18-r1 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="amd64 ~amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=k8 -O2 -pipe -ftracer -ffast-math -fno-unsafe-math-optimizations" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/texmf/web2c /etc/env.d" CXXFLAGS="-march=k8 -O2 -pipe -ftracer -ffast-math -fno-unsafe-math-optimizations -fvisibility-inlines-hidden" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig ccache distlocks sandbox sfperms unsermake userpriv usersandbox" GENTOO_MIRRORS="ftp://ftp.snt.utwente.nl/pub/os/linux/gentoo http://pandemonium.tiscali.de/pub/gentoo/" LDFLAGS="-Wl,-O1 -Wl,--sort-common -s" LINGUAS="en nl" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://192.168.0.2/gentoo-portage" USE="amd64 16bit X a52 aac alsa aotuv arts artswrappersuid audiofile avi bash-completion bitmap-fonts browserplugin bzip2 cairo cdparanoia cdr crypt cscope dts dv dvd dvdread encode fam ffmpeg flac font-server foomaticdb fortran gcj gd gif glitz gpm graphviz gstreamer guile hal imagemagick imlib java joystick jpeg kde kdeenablefinal libwww live lm_sensors logitech-mouse lzo lzw lzw-tiff mad mmap mng mono motif mozsvg mp3 mpeg ncurses network nls nptl nptlonly nsplugin nvidia ogg oggvorbis opengl oss pdflib perl png python qt quicktime readline rtc sblive sdl spell ssl subversion svg tcpd tetex tga theora tiff truetype truetype-fonts type1-fonts unicode usb userlocales utf8 v4l2 vim-with-x visualization vorbis xanim xine xml2 xpm xv xvid xvmc zlib linguas_en linguas_nl userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LANG, LC_ALL Before adding this patch, more testing should be done I thing.
Actually I hade a crash with nano too with a bus error but I didn't know it was realted to this. Does it happen everytime you search in nano? I recompiled nano and I haven't experienced any craches since. Perhaps some things needs to be recompiled against this new libc?
I could crash it reproducibly in one text file (wine config) when searching for "alsa". I did recompile nano and ncurses, before recompiling nano it wiped the file when crashing, lost my fstab by this (had a backup of course). Before recompiling it also crashed when trying to save.
seems like a good idea to put these on hold then pending further testing
Thought I might as well include my opinions about including these patches that I sent to the mailinglist here: Larger changes to glibc usually breaks stuff and requires recompiling. Remember what happened when glibc 2.3.4.20041102 was marked stable? A lot of people got various problems and crashes and needed to recompile or even reinstall. The switch to NPTL only in glibc 2.4 will cause this kind of problems for LT users too and 2.4 will also itself be incompatible with 2.3. So perhaps these patches could be added to the glibc 2.4 snapshots (2.3.5.20050421 and 2.3.5.20050722) and 2.4 when it's released (if they are still relevant by then) because people will have to recompile then anyway when they upgrade. A USE-flag could be a good solution if they are added to a glibc 2.3 ebuild now. Any news on the build error btw?
> Larger changes to glibc usually breaks stuff and requires recompiling. Remember > what happened when glibc 2.3.4.20041102 was marked stable? A lot of people got > various problems and crashes and needed to recompile or even reinstall. Mmm... actually, I don't... there were some problems resulting from having a leftover nptl libpthread.so in /lib... is that what you're referring to? That has been the major upgrade problem that I've seen... > The > switch to NPTL only in glibc 2.4 will cause this kind of problems for LT users > too and 2.4 will also itself be incompatible with 2.3. Yes, it'll cause the same problem, but it's not linuxthreads related. It's tls related, and our linuxthreads glibc doesn't include tls by default except in 2.3.5 and later (if you enable USE=linuxthreads-tls). Also, 2.4 will be compatible with programs built agains 2.3. Hell, you can run almost any program compiled against glibc back to 2.0, and it'll run fine with the current glibc. The only major problem is the tls issue with errno, but that is worked around by having USE="-nptlonly -linuxthreads-tls" and using LD_ASSUME_KERNEL=2.4.1 for those offenders. > So perhaps these patches could be added to the glibc 2.4 snapshots > (2.3.5.20050421 and 2.3.5.20050722) and 2.4 when it's released (if they are > still relevant by then) because people will have to recompile then anyway when > they upgrade. > A USE-flag could be a good solution if they are added to a glibc 2.3 ebuild now. Uhm... no, I'm not going to make a USE="break-my-glibc" use flag. Sorry I haven't gotten around to looking at these more... the bug reports have slightly detered my interest =/
(In reply to comment #12) > > Mmm... actually, I don't... there were some problems resulting from having a > leftover nptl libpthread.so in /lib... is that what you're referring to? That > has been the major upgrade problem that I've seen... > There where several users on the forums that got segfaults and various other problems. Perhaps it was because of that? I don't know. I thought it was because the newer version where somehow incompatible with the older and needed programs to be rebuilt against it. At least that was the solution suggested by many forum users. > > Uhm... no, I'm not going to make a USE="break-my-glibc" use flag. > I was more thinking about a glibc-amd64experimental flag or such. :) But I have a theory on the nano problem that I'm going to investigate more.
I am about to rebuild my entire system with less agressive CFLAGS (more info on that here: http://forums.gentoo.org/viewtopic-t-367682-highlight-.html). I am willing to try these patches if it will help you/Gentoo (please advise how to apply them/ebuild etc). I've used the stage 1/3 guide, and are at the moment using: sys-libs/glibc-2.3.5-r1 sys-devel/binutils-2.15.92.0.2-r10 sys-libs/libstdc++-v3-3.3.6 sys-devel/gcc-3.4.4 sys-apps/portage-2.0.51.22-r2
Following a hint of Simon, I can report the following: Using a stable version of nano (1.2.5) with the patches made the problem go away. So the problem is likely to be in in the unstable version of nano.
It seems like the amd64 strings patch exposes a bug in the 1.3.X development branch of nano causing crashes when searching. The 1.2.X stable branch works though. I came to the conclusion because nano 1.3.8 would suffer from the same problem in suse 9.3 while 1.2.5 was stable. I'm going to report this problem to the nano-devel mailinglist. @Christopher The only patch you can use currently is glibc-2.3.3-amd64-string.patch, the other don't build yet. If you want to try it you can use the ebuild from my overlay, it's based on the latest 2.3.5-r1 ebuild + the strings patch. The libm patches are also included but commented out in the ebuild. http://snigel.no-ip.com/~nxsty/linux/glibc-overlay.tar.bz2 1. Untar it in /usr/local/portage/sys-libs 2. Edit make.conf and make sure you have PORTDIR_OVERLAY="/usr/local/portage" 3. Add =app-editors/nano-1.3* to /etc/portage/package.mask, nano will be crashy otherwise. 4. emerge glibc nano It's running perfectly stable for me and I didn't rebuild anything except nano, but you are using this patch at your own risk. Please report any problems you might find here!
(In reply to comment #13) > (In reply to comment #12) > > Uhm... no, I'm not going to make a USE="break-my-glibc" use flag. > > I was more thinking about a glibc-amd64experimental flag or such. :) either the patches works and we include them or they dont and we reject them
(In reply to comment #16) > It seems like the amd64 strings patch exposes a bug in the 1.3.X development > branch of nano causing crashes when searching. try a cvs checkout before reporting a bug ... we've tracked down a few possible size issues since 1.3.7 in nano and not all are in the 1.3.8 release
Simon - I've no reemerged glibc and gcc with your glibc-overlay, and ran the memcpy test before and after - result: Old glibc: ./memcp 2000 1000 1048576 Memory to memory copy rate = 1599.618652 MBytes / sec. Block size = 1048576. Your glibc-overlay: ./memcp 2000 1000 1048576 Memory to memory copy rate = 2160.129395 MBytes / sec. Block size = 1048576. Just thought I'd report back. Will get on with my complete rebuild of my system, using your glibc-overlay.
(In reply to comment #18) > (In reply to comment #16) > > It seems like the amd64 strings patch exposes a bug in the 1.3.X development > > branch of nano causing crashes when searching. > > try a cvs checkout before reporting a bug ... we've tracked down a few possible > size issues since 1.3.7 in nano and not all are in the 1.3.8 release 1.3.8-cvs still crashes, though now with a segmentation fault instead of a bus error. Got a useful backtrace this time, will submit a bug about this.
Using only strings patch produces very reproducable (every time) bug in java-vm running azureus stable branch: /usr/bin/azureus: line 54: 13944 Segmentation fault java -cp $CLASSPATH -Djava.library.path="${AZDIR}" org.gudy.azureus2.ui.swt.Main "$1" If you recieved an error about a missing java class, you need to setup your classpath correctly. This should do the trick (as root): java-config --add-system-classpath=junit,log4j,commons-cli,systray4j env-update && source /etc/profile Currently, your classpath (including azureus additions) is: swt.jar:swt-pi.jar:swt-mozilla.jar:seda.jar:Azureus2.jar:. # java-config -L [sun-jre-bin-1.4.2.08] "Sun JRE 1.4.2.08" (/etc/env.d/java/20sun-jre-bin-1.4.2.08) [blackdown-jre-1.4.2.02] "Blackdown JRE 1.4.2.02" (/etc/env.d/java/20blackdown-jre-1.4.2.02) [blackdown-jdk-1.4.2.02] "Blackdown JDK 1.4.2.02" (/etc/env.d/java/20blackdown-jdk-1.4.2.02) * # emerge azureus-bin -pv These are the packages that I would merge, in order: Calculating dependencies ...done! [ebuild R ] net-p2p/azureus-bin-2.1.0.4 +gtk +kde 0 kB Total size of downloads: 0 kB Reverting to stable glibc-2.3.5 branch remedies this completely. I don't use nano so I can't comment on that. I did the attached memcpy test, patched glibc gained ~800MB/sec more bandwidth. Real life test emerging glibc: w/o patch 44 minutes with patch 38 minutes at the same average system use.
The Azureus guys pretty explicitly say that you should be using the sun-jre to run. Does this error come up if you use sun-jdk?
Gabriel, this would be clearly waste of the time and resources, because the only change I have is patching the glibc, it doesn't take a rocket scientist to see where the culprit may lay here, azureus working fine with gentoo's stable glibc *and* blackdown java-vm. Regards
When I try building following the "glibc-overlay" instructions, glibc fails to link: uild-amd64-x86_64-pc-linux-gnu-linuxthreads/elf/ld.so -lgcc /usr/lib/gcc/x86_64-pc-linux-gnu/3.4.4/../../../../x86_64-pc-linux-gnu/bin/ld: warning: creating a DT_TEXTREL in object. /var/tmp/portage/glibc-2.3.5-r1/work/build-amd64-x86_64-pc-linux-gnu-linuxthreads/libc_pic.os: In function `add_module': gconv_conf.c:(.text+0x1f32): undefined reference to `__GI_memcmp' /usr/lib/gcc/x86_64-pc-linux-gnu/3.4.4/../../../../x86_64-pc-linux-gnu/bin/ld: /var/tmp/portage/glibc-2.3.5-r1/work/build-amd64-x86_64-pc-linux-gnu-linuxthreads/libc_pic.os: relocation R_X86_64_PC32 against `__GI_memcmp' can not be used when making a shared object; recompile with -fPIC /usr/lib/gcc/x86_64-pc-linux-gnu/3.4.4/../../../../x86_64-pc-linux-gnu/bin/ld: final link failed: Bad value collect2: ld returned 1 exit status distcc[20157] ERROR: compile (null) on localhost failed make[1]: *** [/var/tmp/portage/glibc-2.3.5-r1/work/build-amd64-x86_64-pc-linux-gnu-linuxthreads/libc.so] Error 1 make[1]: Leaving directory `/var/tmp/portage/glibc-2.3.5-r1/work/glibc-2.3.5' make: *** [all] Error 2 !!! ERROR: sys-libs/glibc-2.3.5-r1 failed. !!! Function toolchain-glibc_src_compile, Line 232, Exitcode 2 !!! (no error message) !!! If you need support, post the topmost build error, NOT this status message.
Dario: your logic is flawed. A change in a library can reveal or hide a bug in an application that uses it. The library in question may still conform to the specifications, but in one instance, the bug doesn't present a noticable problem to the user. This can be seen with the nano bug referenced above. Still, would you mind trying the sun-jre
Jeremy, I respect your opinion, but I still think you are on the wrong foot here. Anyways, for the sake of an argument I repeated the test with sun-java-vm with patched glibc-2.3.5 (the strings patch only). Here are the results; /usr/bin/azureus: line 54: 13944 Segmentation fault java -cp $CLASSPATH -Djava.library.path="${AZDIR}" org.gudy.azureus2.ui.swt.Main "$1" If you recieved an error about a missing java class, you need to setup your classpath correctly. This should do the trick (as root): java-config --add-system-classpath=junit,log4j,commons-cli,systray4j env-update && source /etc/profile Currently, your classpath (including azureus additions) is: swt.jar:swt-pi.jar:swt-mozilla.jar:seda.jar:Azureus2.jar:. # java-config -L [sun-jre-bin-1.4.2.08] "Sun JRE 1.4.2.08" (/etc/env.d/java/20sun-jre-bin-1.4.2.08) * [blackdown-jre-1.4.2.02] "Blackdown JRE 1.4.2.02" (/etc/env.d/java/20blackdown-jre-1.4.2.02) [blackdown-jdk-1.4.2.02] "Blackdown JDK 1.4.2.02" (/etc/env.d/java/20blackdown-jdk-1.4.2.02) Please take note that this happens on opening Azureus, GUI gets shown for a brief moment and then it dies, every time. Since both sun-java-vm (prior test blackdown-java-vm) and Azureus I have installed are binary packages, there isn't much I can try to do next (except recompiling azureus e.g. using non-binary ebuild, or try with unstable branches). Other thing to note here is that I haven't got any problems whatsoever with other packages and I am running with patched glibc for two days now.
Simon - I'm back ;) I've rebuilt my entire system with your glibc-overlay and nothing bogus to report yet.. my X window with gnome-light works, opera works, azureus works (I should say I've installed the package from azureus.sf.net manually (v2.3.0.4) _and_ I'm using a self-made sun-jre-bin_x86-1.5.0.04.ebuild to get java to work in opera and firefox-bin (32-bit)) If I can post any info/results you might want, just say so. emerge info: (entire system & world rebuild): Portage 2.0.51.22-r2 (default-linux/amd64/2005.0, gcc-3.4.4, glibc-2.3.5-r1, 2. 6.12-gentoo-r8-ct1 x86_64) ================================================================= System uname: 2.6.12-gentoo-r8-ct1 x86_64 AMD Athlon(tm) 64 Processor 3200+ Gentoo Base System version 1.6.13 ccache version 2.4 [enabled] dev-lang/python: 2.3.5-r1, 2.4.1-r1 sys-apps/sandbox: 1.2.11 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.5 sys-devel/binutils: 2.15.92.0.2-r10 sys-devel/libtool: 1.5.18-r1 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=athlon64 -O2 -pipe -fweb -frename-registers -ftracer" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/ share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb / usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/splash /etc/terminfo /etc/env.d" CXXFLAGS="-march=athlon64 -O2 -pipe -fweb -frename-registers -ftracer - fvisibility-inlines-hidden" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig ccache distlocks fixpackages sandbox sfperms strict" GENTOO_MIRRORS="http://mirror.pudas.net/gentoo http://gentoo.osuosl.org http:// www.ibiblio.org/pub/Linux/distributions/gentoo" LANG="en_US.UTF-8" LC_ALL="nb_NO.UTF-8" LDFLAGS="-Wl,-O1" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="amd64 7zip S3TC X Xaw3d a52 aac aalib acpi aim alsa apache2 apm avi bash- completion bcmath berkdb bitmap-fonts browserplugin bzip2 cardbus cdda cddb cdparanoia cdr chroot clamav clamd crypt cups curl directfb doc dvd dvdr encode esd ethereal exif fam fat fbcon fbsplash ffmpeg firefox flac font-server foomaticdb fortran ftp gif gimp gimpprint gnome gnome-print gphoto2 gpm gstreamer gtk gtk2 hal howl icq ieee1394 imagemagick imlib innodb ipv6 jabber java javascript jce jcs jpeg jpeg2k lame ldap libclamav lm_sensors lzw lzw-tiff mad maildir mikmod mng mp3 mp4live mpeg mpeg2 mpeg4 msn msnextras mysql mysqli mythtv nagios-dns nagios-game nagios-ntp nagios-ping nagios-ssh ncurses nls no- old-linux nptl nptlonly nsplugin ntfs nvidia odbc offensive ogg oggvorbis opengl openssl oss pam pda pdflib perl png ppds python qt quicktime rar readline real reiserfs rtc samba sdl sftplogging slang snmp spell sqlite sse-filters ssl subtitles svg svgz sysfs tcpd theora tidy tiff toolbar truetype truetype-fonts type1-fonts unicode urandom usb userlocales utf8 vcd vfat vhosts vim-with-x vorbis wma123 wmf xfs xine xinetd xml xml2 xmlrpc xmms xpm xscreensaver xv xvid yahoo zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LINGUAS
@Dario: Could you try with the latest azureus-bin and/or sun jre from unstable? @Brian: What's your emerge --info? Have you tried disabling distcc? @Christopher: Is your azureus 32bit also?
Okay. I've updated my overlay because I finally got the x86_64 libm patches to build! It's running here stable and I haven't experienced any regressions. The fix was apparently to use -p1 -E when patching, don't aks my why. http://snigel.no-ip.com/~nxsty/linux/glibc-overlay.tar.bz2 Changes: *Updated to use the latest glibc-2.3.5-r1 ebuild. *Updated the amd64 strings patch from mandrake (should contain some fixes according to the changelog) *The x86_64 libm patches are now applied and builds.
Created attachment 65840 [details, diff] glibc-2.3.3-amd64-string.patch updated
I was finally able to compile using the updated overlay. The problem appeared to be one of my CFLAGS, using a plain set worked fine. See my emerge info below. Before patching glibc: memcopy 2000 2000 200000 1601 MB/s After patching glibc: memcopy 2000 2000 200000 1745 MB/s Not terribly impressive, but I switched glibc from CFLAGS="-Os -march=k8 -ffast-math -fomit-frame-pointer -funit-at-a-time -frename-registers -mtune=athlon64 -fno-ident -pipe" to CFLAGS="-O2 -pipe", so I would expect some speed drop from that. I'll try switching around the CFLAGS a bit and see if I get better results. Portage 2.0.51.22-r2 (default-linux/amd64/2005.1, gcc-3.4.4, glibc-2.3.5-r1, 2.6.11-ck10 x86_64) ================================================================= System uname: 2.6.11-ck10 x86_64 AMD Athlon(tm) 64 Processor 3000+ Gentoo Base System version 1.6.13 distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [enabled] ccache version 2.4 [enabled] dev-lang/python: 2.3.5, 2.4.1-r1 sys-apps/sandbox: 1.2.12 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6 sys-devel/binutils: 2.16.1 sys-devel/libtool: 1.5.18-r1 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="amd64 ~amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib64/mozilla/defaults/pref /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig buildpkg ccache digest distcc distlocks nodoc noinfo sfperms strict" GENTOO_MIRRORS="http://gentoo.mirrors.pair.com/ http://mirror.datapipe.net/gentoo http://mirror.datapipe.net/gentoo http://gentoo.osuosl.org/ http://gentoo.llarian.net/" MAKEOPTS="-j6" PKGDIR="/varsrc/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage" USE="amd64 X alsa avi berkdb bitmap-fonts bzlib cdparanoia cdr chroot crypt cups curl dedicated dga dillo dio dnd dvd dvdr dvdread eds encode esd faac faad fam ffmpeg flac foomaticdb fortran freetype gb gd gdbm gif gimp gimpprint ginac glut gnome gpm gs gstreamer gtk gtk2 gtkhtml imagemagick imlib imlib2 jikes joystick jpeg kde lcd lesstif libdsk lzw lzw-tiff mad maildir matrox mbox mcal md5sum mikmod mmap mng motif mozilla moznocompose moznoirc moznomail mozp3p mozsvg mozxmlterm mp3 mpeg mpeg4 mplayer music native ncurses net network nptl offensive ofx ogg openal opengl pam parse-clocks pdf pdflib perl physfs pic pie png ppds python qt quicktime readline rogue sdl slang sox spell ssl svg tcltk tcpd theora threads tiff transcode truetype-fonts type1 type1-fonts usb userlocales v4l v4l2 videos vorbis wmf wxwindows xface xft xine xml2 xmms xosd xpm xprint xscreensaver xv xvid xvmc yv12 zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS, LINGUAS
The culrpit in my earlier build failure with the glibc-overlay appears to be "-Os". "-O2" in CFLAGS works fine, so it looks to me like the patched glibc is incompatible with AMD64 when -Os is used with gcc 3.4.4. This is the first trouble I've had in months with using -Os, even though for awhile (gcc 3.4.?) -Os wasn't recommended for AMD64 (I use it because I have a 512K cache socket 754 and I don't want to stress my limited memory bandwidth). I think I'm happy with the 9% memcopy improvement this patch provides for me. I can't really quantify how much better -Os is than -O2 for my system anyway.
Before patching: ./a.out 2000 2000 200000 Memory to memory copy rate = 1621.270264 MBytes / sec. Block size = 200000. After patching: ./a.out 2000 2000 200000 Memory to memory copy rate = 1752.453979 MBytes / sec. Block size = 200000. I only recompiled glib and the memcpy app here. Don't know if recompiling gcc and/or other things would make a difference. Haven't had any problems so far. emerge info: Portage 2.0.51.22-r2 (default-linux/amd64/2005.0, gcc-3.4.4, glibc-2.3.5-r1, 2.6.12-morph6 x86_64) ================================================================= System uname: 2.6.12-morph6 x86_64 AMD Athlon(tm) 64 Processor 3200+ Gentoo Base System version 1.12.0_pre5 ccache version 2.4 [enabled] dev-lang/python: 2.3.4-r1, 2.4.1-r1 sys-apps/sandbox: 1.2.12 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6 sys-devel/binutils: 2.16.1 sys-devel/libtool: 1.5.18-r1 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="amd64 ~amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=athlon64 -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/splash /etc/terminfo /etc/env.d" CXXFLAGS="-march=athlon64 -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig ccache distlocks fixpackages sandbox sfperms strict userpriv usersandbox" GENTOO_MIRRORS="ftp://ftp.easynet.nl/mirror/gentoo http://gentoo.osuosl.org" LC_ALL="en_GB.UTF-8" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="amd64 X alsa apache2 avi berkdb bitmap-fonts bzip2 cdr crypt cups curl dba dvd dvdr emul-linux-x86 encode esd fam foomaticdb fortran gd gdbm gif gpm gstreamer gtk gtk2 imlib ipv6 java jikes jpeg libcaca libwww lzw lzw-tiff mad motif mozilla mp3 mpeg mysql ncurses nls nptl ogg oggvorbis openal opengl openmotif pam pdflib perl png python quicktime readline rtp sdl slang spell sqlite ssl symlink tcpd tiff truetype truetype-fonts type1-fonts unicode usb userlocales vorbis xine xml xml2 xmms xpm xv zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LANG, LDFLAGS, LINGUAS
@Brian: -Os isn't supposed to be used for glibc compilations anyway (the ebuild should change it to -O2) so you are likely to be hitting bug #77264. I updated my overlay with a fix for that bug, the latest 2.3.5-r1 ebuild and added a changelog: http://snigel.no-ip.com/~nxsty/linux/glibc-overlay.tar.bz2 This one should compile with your original CFLAGS.
(In reply to comment #28) > @Christopher: > Is your azureus 32bit also? No I believe I downloaded the AMD64 version. I use sun-jdk 1.5.0_04 amd64 version for most java (which is not much), and my own sun-jre-bin_x86 for java support in opera and mozilla-firefox-bin (both 32 bit...) # java-config -L [sun-jre-bin_x86-1.5.0.04] "Sun JRE 1.5.0.04" (/etc/env.d/java/20sun-jre- bin_x86-1.5.0.04) [sun-jdk-1.5.0.04] "Sun JDK 1.5.0.04" (/etc/env.d/java/20sun-jdk-1.5.0.04) *
(In reply to comment #34) I updated my > overlay with a fix for that bug, the latest 2.3.5-r1 ebuild and added a changelog: > http://snigel.no-ip.com/~nxsty/linux/glibc-overlay.tar.bz2 I recompiled glibc using your overlay now, and it compiled just fine. I recompiled memcpy.c and now I got this: $ ./memcp 2000 1000 1048576 Memory to memory copy rate = 2167.753662 MBytes / sec. Block size = 1048576.
I was able to use "-Os" in my make.conf with the new overlay. Same speed as the previous overlay I tried (which included libm I think), ~1745MB/s (keep in mind this is a DDR333 socket 754 system). I've been using the parameters "2000 2000 200000" throughout so far to have a relative benchmark, but if I use "2000 1000 1048576" I get ~2170MB/s.
Just installed the glibc overlay on my dual 1.8Ghz Opteron, 1GB registered DDR400. Before overlay: average of about 1250 MB/s After overlay: memcopy_speed 1800 1000 1048576 Memory to memory copy rate = 1840.629272 MBytes / sec. Block size = 1048576. Approx 47% increase! Seems to work much better on dual-channel memory systems.
> I was able to use "-Os" in my make.conf with the new overlay. Ok, I'm adding #77264 as a depend then.
Could anyone with a reproducible nano/azurius crash try the updated overlay with the new strings-patch? http://snigel.no-ip.com/~nxsty/linux/glibc-overlay.tar.bz2 Here is my report to nano-devel btw: http://lists.gnu.org/archive/html/nano-devel/2005-08/msg00008.html
applied the new overlay, no nano search bug on my build. will compile older overlay to see if i can reproduce earlier nano bug. [I--] [ ] sys-libs/glibc-2.3.5-r1 [I--] [ ] app-editors/nano-1.3.7
nano search bug fixed on my build. Will compile with *-string.patch 2005-07-25 to see if i can reproduce earlier nano bug. [I--] [ ] sys-libs/glibc-2.3.5-r1 [I--] [ ] app-editors/nano-1.3.7
Could not reproduce nano bug with *-string.patch 2005-07-25. I don't use azurius.
New overlay and the nano bug is gone, too bad this didn't come out yesterday when I trashed my xorg.conf by accident...
Minor request with regard to the glibc-overlay tarballs: could we get them with a date in the filename (-0815, etc), and maybe an internal Changelog.overlay? This is assuming there will be additional patched-in changes in the future.
Latest glibc overlay has proved that Azureus bug I have reported before is gone. Also, it seems that strings patch produces even more "juice" on my system; Before patch: ~/test/memcpy $ ./a.out 2210 2000 2097152 Memory to memory copy rate = 1466.128784 MBytes / sec. Block size = 2097152. After patch: ~/test/memcpy $ ./a.out 2210 2000 2097152 Memory to memory copy rate = 2514.519531 MBytes / sec. Block size = 2097152.
I've updated the overlay with the suggestions. The patches are now dated (I just took the dates of the bugzilla) and the changelog is now in files/changelog.overlay. The ebuild and patches are otherwise the same so there is no need to recompile. I'm thinking about posting this to the forums to gain some more testers. Some performance numbers from an intel nocona machine would be interesting!
Yes, a post in the forums is a good idea, I only stumbled on this one while looking for an older glibc bug report, and boosting the memcpy test from ~ 1200MBytes to ~ 2300 sure is impressive! Also it wouldn't hurts to have more victi... err volunteers to see if some programs are affected (none for me by the way).
Recently got nano-1.3.8 crashes again, with the glibc-2.3.3-amd64-string-20050813.patch. It just crashes less, but there still is a problem apparantly. Maybe it is fixed in current nano-CVS, but I've got some weird make errors on that.
I've updated the overlay and posted it to the unsupported software forum: http://forums.gentoo.org/viewtopic-t-376943.html
I have just installed this patched version of glibc on my AMD64 machine. memcopy gives me the following result: Code: Original glibc in gentoo: ./memcpy 2200 1000 1048576 Memory to memory copy rate = 1134.367310 MBytes / sec. Block size = 1048576. Recompiled with patches: ./memcpy 2200 1000 1048576 Memory to memory copy rate = 2294.247314 MBytes / sec. Block size = 1048576. azureus doesn't work, it gives me a segmentation fault: Code: /usr/bin/azureus: line 47: 10159 Segmentation fault java -cp $(java-config -p systray4j,azureus-bin 2>/dev/null) -Djava.library.path="${AZDIR}" org.gudy.azureus2.ui.swt.Main "$1" Java: Code: java-config -L [blackdown-jre-1.4.2.02] "Blackdown JRE 1.4.2.02" (/etc/env.d/java/20blackdown-jre-1.4.2.02) [blackdown-jdk-1.4.2.02] "Blackdown JDK 1.4.2.02" (/etc/env.d/java/20blackdown-jdk-1.4.2.02) * My emerge -info: Code: smaug log # emerge --info Portage 2.0.51.22-r2 (default-linux/amd64/2005.0, gcc-3.4.4, glibc-2.3.5-r1, 2.6.13-gentoo-devil x86_64) ================================================================= System uname: 2.6.13-gentoo-devil x86_64 AMD Athlon(tm) 64 Processor 3800+ Gentoo Base System version 1.12.0_pre7 dev-lang/python: 2.3.5, 2.4.1-r1 sys-apps/sandbox: 1.2.12 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6 sys-devel/binutils: 2.16.1 sys-devel/libtool: 1.5.20 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="amd64 ~amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O3 -march=athlon64 -ffast-math -funroll-loops -funit-at-a-time -fpeel-loops -ftracer -funswitch-loops -fomit-frame-pointer -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib64/mozilla/defaults/pref /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/splash /etc/terminfo /etc/env.d" CXXFLAGS="-O3 -march=athlon64 -ffast-math -funroll-loops -funit-at-a-time -fpeel-loops -ftracer -funswitch-loops -fomit-frame-pointer -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig ccache distlocks sandbox sfperms strict" GENTOO_MIRRORS="http://ftp.du.se/pub/os/gentoo/ http://mirror.pudas.net/gentoo/ http://gentoo.eliteitminds.com http://gentoo.mirror.icd.hu/ ftp://mir.zyrianes.net/gentoo/" LDFLAGS="-Wl,-O1 -Wl,--enable-new-dtags -Wl,--sort-common -Wl,--strip-all" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="X alsa amd64 avi bash-completion bitmap-fonts bzlib cdr crypt cups curl dvd dvdr eds encode esd fam fbcon flac foomaticdb fortran gd gif gnome gpm gstreamer gtk gtk2 hal howl imlib imlib2 ipv6 java jpeg lzw lzw-tiff mad mozilla mp3 mpeg ncurses nls nptl nvidia ogg opengl pam pdflib perl png python quicktime readline samba sdl snmp spell sqlite ssl sysvipc tcpd tiff truetype-fonts type1-fonts usb userlocales vorbis xine xml xml2 xmms xpm xv zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LINGUAS Anything else that i need to post just tell me and i will post it. /Jocke W aka Devils
It seems that Azureus bug persists here also :( Actually, I have found when it segfaults, on checking complete file download. On normal startup, when no checks are needed it works, also when downloading/uploading. As stated above, the same behaviour can be reproduced with both Blackdown and Sun Java VM as backend. Regards
Wouldn't this be more of an Azureus bug rather than glibc? If nothing but Azureus and nano have problems, I'm thinking it's not the patch's fault.
I have now played aroudn a bit with azureus. I downloaded the Amd64 version from Azureus homepage, and AMD64 SUN Java from SUN, and now i got Azureus working. Just my 2cents...
How about puting the strings patch on hold then and focus on the libm patches? So we can at least conclude that they are regression free and work on the strings patch later. I've uploaded a new overlay tarball that no longer applies it: http://snigel.no-ip.com/~nxsty/linux/glibc-overlay2.tar.bz2
I did som benchmarks in ubench and nbench and was suprised to see that the results with a patched glibc only was slightly lower than with a vanilla glibc. I don't think mandriva and suse would ship these if they didn't acctually improve anything so are we doing something wrong? Anyway here are the results from ubench (I didn't save the nbench results for some reason): Unpatched glibc: Ubench CPU: 127201 Ubench MEM: 182503 -------------------- Ubench AVG: 154852 Patched glibc: Ubench CPU: 121404 Ubench MEM: 180816 -------------------- Ubench AVG: 151110 So I did some changes to my overlay to see if I could improve things. I added a 2005-09-24 branch update from the stable 2.3 branch which fixes a lot of bugs (should be done anyway) and changed --enable-kernel to 2.6.9 to match mandriva's spec-file. After that results got a little bit better but it's still not good: Ubench CPU: 121947 Ubench MEM: 181289 -------------------- Ubench AVG: 151618 The updated overlay is at the usual place. It pulls the branch update and patch tarball from my webserver (I had to replace a patch that rejected).
Btw, a glibc with the libm patches only does not show this performance regression but instead gives similar results to an unpatched glibc in ubench.
on the boinc mailinglist, i have talked about x86_64 setiathome performance, and a guy, said an amd developer had told him, that suse, redhat(buy version) and solaris came with a special library, which made x86_64 in 64bit mode become as fast, and faster than 32bit mode, and i believe this is it, since libm is the math functions. im going to try this.
now i tested.. and these patches truly are the ones who is needed for setiathome performance to be as good, and better than amd64 32bit.. a setiathome unit took me over 7 hours on my amd64 3200+ 64bit gentoo without this.. this is MUCH more than on 32bit - in 32bit it took like.. 3 hours.. with the patches (i guess libm is what acutally causes it) it takes 2 hours and 30 minutes.. which is an extremely significant boost, also many other things uses libm, this gives nice performance boost all over the place, frankly, i do not know how gentoo amd64 did not have it before, and also, i dont know why this doesent get HIGHEST prioity under security.. this is an extremely vital thing, since suse and mandrive has it,, gentoo cant not have it.. :)
I added the patch lines to 2.3.5-r2 and it compiled nicely and seems to be working fine (for me, anyway). Is there a new overlay due soon? How goes the war on getting it into portage?
> Is there a new overlay due soon? I'm planning adding some gcc4 fixes and perhaps some other patches unrelated to these. Any suggestions? > > How goes the war on getting it into portage? The libm patches could probably go in right away. There is no reported regressions against them and they give seti and gimp a nice performance boost. I don't know about the string routines patch since it makes azureus and nano crash for a few people and also shows some performance regressions in ubench an nbench. But it's up to the devs!
the problems with azureus seems to be related to java 1.4, so untill java 1.5 goes stable in portage, i guess those should not be merged.
(In reply to comment #62) > the problems with azureus seems to be related to java 1.4, so untill java 1.5 > goes stable in portage, i guess those should not be merged. I just wanted to clearify that the problems are with the string routines patch and not the libm patches.
indeed, the libm patch seems perfectly safe (and suse uses to too) libm is also the most important one, since it does the extreme speedup.
Compiled and tested with memcpy the new glibc on an Athlon 64 4400+ X2. It looks good. Old glibc: ========== Memory to memory copy rate = 1743.700073 MBytes / sec. Block size = 1048576. New glibc: ========== Memory to memory copy rate = 2303.999268 MBytes / sec. Block size = 1048576. edmondo@balrog ~ $ emerge info Portage 2.0.53_rc6 (default-linux/amd64/2005.1, gcc-3.4.4, glibc-2.3.5-r2, 2.6.14-rc3 x86_64) ================================================================= System uname: 2.6.14-rc3 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 4400+ Gentoo Base System version 1.12.0_pre9 dev-lang/python: 2.3.4-r1, 2.4.2 sys-apps/sandbox: 1.2.13 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1 sys-devel/binutils: 2.16.1 sys-devel/libtool: 1.5.20 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="amd64 ~amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=k8 -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-march=k8 -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig distlocks sandbox sfperms strict" GENTOO_MIRRORS="http://mirror.switch.ch/mirror/gentoo/ http://gentoo.mirror.solnet.ch http://gentoo.osuosl.org/" MAKEOPTS="-j3" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="amd64 X alsa avi berkdb bitmap-fonts cdr crypt cups curl dvd dvdr dvdread eds emboss encode esd fam ffmpeg foomaticdb fortran gif gpm gstreamer gtk gtk2 hal imagemagick imlib ipv6 jpeg kde lzw lzw-tiff mad mp3 mpeg mplayer ncurses nls nptl nptlonly ogg oggvorbis opengl pam pdflib perl png python qt quicktime readline samba sdl spell ssl subtitles tcpd theora threads tiff truetype-fonts type1-fonts usb userlocales visualization vorbis xine xml2 xpm xv xvid zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTDIR_OVERLAY
Hi, I also tested the glibc overlay, My system is a MSI K8MM-V, AThlon 64 2800 Newcastle, 1 GB Kingston ddr400 These is memcpy with gentoo
Hi, I also tested the glibc overlay, My system is a MSI K8MM-V, AThlon 64 2800 Newcastle, 1 GB Kingston ddr400 These is memcpy with gentoo´s glibc: ./memcpy.bin 2200 1000 1048576 Memory to memory copy rate = 1164.361694 MBytes / sec. Block size = 1048576 ./memcpy.bin 2200 1000 1048576 Memory to memory copy rate = 1171.167603 MBytes / sec. Block size = 1048576 This is memcpy with the overlay glibc: ./memcpy.bin 2200 1000 1048576 Memory to memory copy rate = 2160.607178 MBytes / sec. Block size = 1048576. ./memcpy.bin 2200 1000 1048576Memory to memory copy rate = 2189.406250 MBytes / sec. Block size = 1048576. So, there is a nice improvement, I had to update Java to sun-jre-1.5 and everything has worked so far, I´m going to try an emerge world later.
This hasn't moved for a while. Has glibc 2.3.6 in portage already containing this patches?
No 2.3.6 in portage does not have these patches. But there is a lenghty thread in the "unsupported softwate" gentoo forums: http://forums.gentoo.org/viewtopic-t-376943.html with links to the up-to-date overlay (2.3.6-r1 is available there with the patches)
well, I'll try and report here, if anyone cares for it's inclusion on oficial glibc.
Gentoo != official glibc
s/official glibc/gentoo's official glibc
Tested both appears to work flawlessly here. In addition the math library enhancements appear to have provided a considerable speed increase on folding at home on AMD64. They should try to atleast put the math enhancements into the offical ~amd64 tree. emerge info Portage 2.0.51.22-r3 (default-linux/amd64/2005.0, gcc-3.4.4, glibc-2.3.6-r1, 2.6.14-gentoo-r4 x86_64) ================================================================= System uname: 2.6.14-gentoo-r4 x86_64 AMD Athlon(tm) 64 Processor 3700+ Gentoo Base System version 1.12.0_pre11 ccache version 2.3 [enabled] dev-lang/python: 2.3.5-r2, 2.4.2 sys-apps/sandbox: 1.2.12 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1 sys-devel/binutils: 2.16.1 sys-devel/libtool: 1.5.20 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=k8 -msse3 -pipe -O2" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib64/mozilla/defaults/pref /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/splash /etc/terminfo /etc/env.d" CXXFLAGS="-march=k8 -msse3 -pipe -O2" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig ccache distlocks sandbox sfperms strict" GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/Linux/distributions/gentoo" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="amd64 X a52 aac aalib acl acpi alsa apm audiofile avi bash-completion berkdb bitmap-fonts bonobo bzip2 ccache cdr crypt css cups curl dts dvd dvdr dvdread eds emboss encode esd ethereal exif expat extensions fam ffmpeg flac foomaticdb fortran gd gdbm gif glut gnome gpm gstreamer gtk gtk2 gtkhtml hal idn imagemagick imlib ipv6 java joystick jpeg kde kdenablefinal lcms lm_sensors lzo lzw lzw-tiff mad mjpeg mng mozilla mp3 mpeg ncurses nls nvidia offensive ogg opengl oss pam pcre pdflib perl pic png python qt quicktime readline real recode rtc samba scanner sdl session spell ssl svg tcltk tcpd tiff truetype truetype-fonts type1-fonts udev usb userlocales vorbis xine xml xml2 xmms xpm xv xvid xvmc zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS, LINGUAS, MAKEOPTS
*** Bug 115833 has been marked as a duplicate of this bug. ***
I've installed the patches too, had some problems with azureus but an upgrade from 1.4.2-blackdown to sun-java 1.5 fixed them. Results: glibc 2.3.5-r2 w/o AMD64 patches: ./a.out 2200 1000 1048576 Memory to memory copy rate = 1673.262573 MBytes / sec. Block size = 1048576. glibc 2.3.6-r1 w/ AMD64 patches: ./a.out 2200 1000 1048576 Memory to memory copy rate = 2387.131836 MBytes / sec. Block size = 1048576. I hope this get's into portage soon!
sorry, forgot my emerge info: Password: Portage 2.0.53 (default-linux/amd64/2005.1, gcc-3.4.4, glibc-2.3.6-r1, 2.6.14-ck6 x86_64) ================================================================= System uname: 2.6.14-ck6 x86_64 AMD Athlon(tm) 64 Processor 3000+ Gentoo Base System version 1.6.13 ccache version 2.3 [enabled] dev-lang/python: 2.4.2 sys-apps/sandbox: 1.2.12 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1 sys-devel/binutils: 2.16.1 sys-devel/libtool: 1.5.20 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=k8 -O2 -pipe -msse3 -fno-ident" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/env.d" CXXFLAGS="-march=k8 -O2 -pipe -msse3 -fno-ident" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig ccache distlocks sandbox sfperms strict" GENTOO_MIRRORS="ftp://pandemonium.tiscali.de/pub/gentoo/ ftp://ftp.tu-clausthal.de/pub/linux/gentoo/ ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/ ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage /usr/local/fluidportage/trunk" SYNC="rsync://rsync.de.gentoo.org/gentoo-portage" USE="amd64 X a52 aac alsa apache2 audiofile avi bash-completion bitmap-fonts bzip2 cairo cdb cdr crypt cups curl dvd dvdr eds emboss encode exif expat fam ffmpeg flac foomaticdb fortran gd gif glibc-omitfp glut gpm gstreamer gtk gtk2 idn imagemagick imlib java javascript jpeg junit lcms libwww lzw lzw-tiff mad matroska mng mp3 mpeg musepack mysql ncurses nls nvidia ogg oggvorbis opengl pam pcre pdflib perl php png python quicktime readline samba sdl spell sqlite ssl tcpd tetex tiff truetype truetype-fonts type1-fonts udev usb userlocales vorbis xine xml xml2 xpm xv xvid zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS, LINGUAS
ive been running both the libm and string patch since Dec 18th and have yet to encounter any issues here at all i dont run azureus on this computer but the issues others were talking about i see on x86 the memcopy microbench i dont think is all that great so i wont post its numbers i will how ever say i am seeing somethin around 12% decrease in compile times Portage 2.1_pre3-r1 (default-linux/amd64/2005.0, gcc-4.0.2, glibc-2.3.6-r1, 2.6.15-rc7 x86_64) ================================================================= System uname: 2.6.15-rc7 x86_64 AMD Athlon(tm) 64 Processor 2800+ Gentoo Base System version 1.6.13 dev-lang/python: 2.3.5-r2, 2.4.2 sys-apps/sandbox: 1.2.12 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1 sys-devel/binutils: 2.16.1 sys-devel/libtool: 1.5.20 virtual/os-headers: 2.6.11-r3 ACCEPT_KEYWORDS="amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=athlon64 -O3 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib64/mozilla/defaults/pref /usr/share/X11/xkb /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-march=athlon64 -O3 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig distlocks fixpackages sandbox sfperms" GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo" LANG="en_US.UTF-8" LDFLAGS="-Wl,-O1" MAKEOPTS="" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/mdhd/portage.local" SYNC="rsync://vox.net/gentoo-portage" USE="amd64 X a52 aac alsa audiofile avi berkdb bitmap-fonts bzip2 canna cdr cjk cli crypt curl dmx dri dv dvb dvd dvdread eds emboss esd exif expat fam fbcon fbdev flac foomaticdb gd glut gmp gnome gstreamer gtk gtk2 idn imagemagick imlib ipv6 jpeg kde lcms libwww live lua lzw lzw-tiff mad matroska mhash mng motif mozilla mp3 mpeg ncurses nls nptl nptlonly ogg oggvorbis openal opengl pam pcre pdflib perl php png python qt quicktime readline real recode samba sdl speex spell ssl svg tcpd theora tiff truetype truetype-fonts type1-fonts udev unicode usb userlocales v4l v4l2 vcd vorbis wmf xine xinerama xml2 xmms xpm xv zlib elibc_glibc kernel_linux userland_GNU" Unset: ASFLAGS, CTARGET, LC_ALL, LINGUAS
I added Simon's overlay and emerged glibc: [ebuild R ] sys-libs/glibc-2.3.6-r1 USE="nls nptl nptlonly pic userlocales -build -erandom -glibc-compat20 -glibc-omitfp -hardened -linuxthreads-tls -nomalloccheck -profile" 0 kB [1] I got a respectable improve using memcpy and got no strange crashes in software. I know this improves memcpy() but what kind of applications are expected to gain more from this? sam ~ # emerge info Portage 2.1_pre3-r1 (default-linux/amd64/2005.1, gcc-3.4.5, glibc-2.3.6-r1, 2.6.14-gentoo-r5 x86_64) ================================================================= System uname: 2.6.14-gentoo-r5 x86_64 AMD Turion(tm) 64 Mobile Technology MT-32 Gentoo Base System version 1.12.0_pre12 ccache version 2.4 [enabled] dev-lang/python: 2.3.4-r1, 2.4.2 sys-apps/sandbox: 1.2.17 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1 sys-devel/binutils: 2.16.1-r1 sys-devel/libtool: 1.5.22 virtual/os-headers: 2.6.11-r3 ACCEPT_KEYWORDS="amd64 ~amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=athlon64 -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib64/mozilla/defaults/pref /usr/share/config /var/bind /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/splash /etc/terminfo /etc/texmf/web2c /etc/env.d" CXXFLAGS="-march=athlon64 -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig ccache cvs distlocks sandbox sfperms sign strict" GENTOO_MIRRORS=" http://felisberto.net/pub/gentoo http://ftp.ua.pt/pub/gentoo " LINGUAS="pt" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.pt.gentoo.org/gentoo-portage" USE="amd64 X aalib acpi alsa apache2 arts audiofile avi bash-completion berkdb bitmap-fonts bluetooth bzip2 cdparanoia cdr crypt cups curl dbus doc dv dvb dvd dvdr dvdread dvi eds emacs emboss encode esd ethereal evo examples exif expat extras fam fbsplash ffmpeg flac foomaticdb fortran gd gdbm geoip gif glut gmp gnome gnome-print gpm gstreamer gtk gtk2 hal howl idn imagemagick imlib ipv6 java jpeg junit kde kdeenablefinal lcms libgda libwww lzw lzw-tiff mad mikmod mng motif mozilla mp3 mpeg mplayer musepack ncurses new-login nls nptl nptlonly objc odbc offensive ogg oggvorbis openal opengl pam pcmcia pcre pdflib perl php pic plotutils png postgres python qt quicktime readline samba sdl source spell sqlite ssl tcltk tcpd tetex threads tiff truetype truetype-fonts type1-fonts udev unicode usb userlocales vcd vorbis wmf xine xml2 xmms xpm xv xvid zeroconf zlib elibc_glibc kernel_linux linguas_pt userland_GNU" Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS
If anyone still cares about this bug. :) There is a easy way to fix the segfaults that I've included in my overlay for some time and I thought I might as well post it here. Just rm sysdeps/x86_64/strncmp.S after the patching. This is what Suse SRPMS does.
I really care about this bug :-) I tested now all the patches + the removal of the offending files you mentioned. Nano works fine here. To me, it seems all issues are sorted out, quite a few gentoo-devs have succesfully played with them, so I'd vote for putting them into the tree. Afaik, there are no issues remaining
also confirmed here that ~amd64 nano problem no longer occurs with proposed removal of file.
So is this going to be added to gentoo tree?
Not really interesting, but it also allow me to be on the CC :) Before : Memory to memory copy rate = 1422.135010 MBytes / sec. Block size = 1048576. After : Memory to memory copy rate = 2366.425781 MBytes / sec. Block size = 1048576. And so far so good here :) Thank you very much for the overlay !!!
Oops, I got into some troubles : At boot time in local.start I have : echo ondemand > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor which result in a direct system crash... My system autoload DVB modules, and with that glibc : vdr: [10403] receiver on device 1 thread started (pid=10264, tid=10403) Unable to handle kernel NULL pointer dereference at 000000000000000a RIP: <ffffffff801253d9>{try_to_wake_up+57} PGD 3ba7a067 PUD 37325067 PMD 0 Oops: 0000 [1] PREEMPT last sysfs file: /class/firmware/0000:00:06.0/loading CPU 0 Modules linked in: dvb_ttpci l64781 ves1820 stv0299 tda8083 stv0297 sp8870 ttpci_eeprom saa7146_vv video_buf v4l1_compat v4l2_common videodev saa7146 ves1x93 dvb_core snd_seq_midi snd_usb_audio snd_usb_lib snd_hwdep snd_cs46xx snd_via82xx snd_ac97_codec snd_ac97_bus snd_mpu401_uart snd_rawmidi binfmt_misc nvidia agpgart Pid: 10264, comm: vdr Tainted: P 2.6.16-rc2-mm1 #1 RIP: 0010:[<ffffffff801253d9>] <ffffffff801253d9>{try_to_wake_up+57} RSP: 0018:ffff8100372f5cd0 EFLAGS: 00010002 RAX: ffff8100372f5fd8 RBX: 000000000000000a RCX: 0000000000000020 RDX: 0000000000000000 RSI: 000000000000000f RDI: 000000000000000a RBP: ffff8100372f5d00 R08: ffff8100372f4000 R09: 0000000000000001 R10: 00000000ffffffff R11: 0000000000000001 R12: 0000000000000000 R13: 0000000000000000 R14: ffff8100372f5cd8 R15: ffff810039c306c0 FS: 00002b4ef4a04350(0000) GS:ffffffff8053c000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 000000000000000a CR3: 000000003737b000 CR4: 00000000000006e0 Process vdr (pid: 10264, threadinfo ffff8100372f4000, task ffff81003b6977f0) Stack: ffffffff8856f44c 0000000000000246 ffffc200005ff7f8 ffff810039c30980 ffffc2000061f5a8 ffffc2000061f5b0 0000000000000000 ffffffff803f908f ffffc2000061f5b0 ffffffff803f9247 Call Trace: <ffffffff8856f44c>{:dvb_ttpci:av7110_stop_feed+700} <ffffffff803f908f>{__mutex_unlock_slowpath+47} <ffffffff803f9247>{.text.lock.mutex+15} <ffffffff88518db8>{:dvb_core:dmx_section_feed_stop_filtering+168} <ffffffff885164ad>{:dvb_core:dvb_dmxdev_feed_stop+109} <ffffffff8851683a>{:dvb_core:dvb_dmxdev_filter_start+474} <ffffffff88517109>{:dvb_core:dvb_demux_do_ioctl+729} <ffffffff88516e30>{:dvb_core:dvb_demux_do_ioctl+0} <ffffffff88515786>{:dvb_core:dvb_usercopy+214} <ffffffff80178460>{chrdev_open+0} <ffffffff8016e119>{__dentry_open+329} <ffffffff803f9672>{__up_wakeup+53} <ffffffff80182bc4>{do_ioctl+116} <ffffffff80182e9d>{vfs_ioctl+685} <ffffffff80182f1d>{sys_ioctl+77} <ffffffff8010abe6>{system_call+126} Code: 48 8b 07 21 c6 48 85 f6 0f 84 ad 00 00 00 48 83 7f 48 00 0f RIP <ffffffff801253d9>{try_to_wake_up+57} RSP <ffff8100372f5cd0> CR2: 000000000000000a <6>note: vdr[10264] exited with preempt_count 2 vdr: [10391] System Time = Sat Feb 11 20:52:00 2006 (1139687520) vdr: [10391] Local Time = Sat Feb 11 20:48:55 2006 (1139687335) vdr: [10387] PANIC: watchdog timer expired - exiting! I am back to my 2.3.5-r2, if I would like to use the patched glibc, shoud I recompil my kernel, vdr, and ??? Thank you
Correct me if I'm wrong, but DVB and frequency scaling have nothing to do with glibc. This is kernel stuff - if your kernel crashes, that's certainly not the fault of glibc
Sorry for not being clear : I got those crash only with the new glibc from the overlay, remerging the old glibc and no crash. Should I recompil the kernel after this glibc upgrade ?
I tried echo ondemand > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor on my system -> no crash (with new glibc) And for DVB: kernel modules have absolutely nothing to do with glibc. They don't use glibc functions, they are not linked to it - statically or dynamically. It's - to me - absolutely impossible for a glibc change to cause kernel panic. It could however be that userspace programs causes trouble. If you can make out the program that casues trouble, try recompiling it against new glibc (I'm not talking about the modules)
I have reinstalled the patched glibc and compiled a new kernel : this way I don't have my governor patch anymore and also no DVB crash :) Thank you very much !
Created attachment 81968 [details, diff] libm patch against glibc 2.4 libm patch against glibc 2.4, also slightly updated, from the latest suse SRPM.
Created attachment 81969 [details, diff] Strings patch against glibc 2.4 Strings patch against glibc 2.4, from the latest suse SRPM.
*** Bug 125946 has been marked as a duplicate of this bug. ***
Comment on attachment 81969 [details, diff] Strings patch against glibc 2.4 this is still broken
(In reply to comment #91) > (From update of attachment 81969 [details, diff] [edit]) > this is still broken > Are you talking about the build problem with hardened, or is there some other problem that I'm not aware of?
Comment on attachment 81968 [details, diff] libm patch against glibc 2.4 this patch is also wrong
updated patches will be in glibc-2.4 patchset, just disabled by default
