Steps to reproduce: 1. emerge gawk 2. yes 22 | awk '{ print $0, length($1), length($2) }' Actual results: 22 2 0 22 2 0 *** glibc detected *** double free or corruption (fasttop): 0x080968d8 *** Aborted Expected results: "22 2 0\n" forever gawk-3.1.4-r4 works fine. Reproduced on two separate machines, both x86. An amd64 does not have this problem. Gentoo Base System version 1.6.14 Portage 2.1_rc4-r2 (default-linux/x86/2005.1, gcc-3.4.6/vanilla, glibc-2.3.6-r3, 2.6.13.3-cube2 i686) ================================================================= System uname: 2.6.13.3-cube2 i686 Intel(R) Pentium(R) 4 CPU 2.40GHz distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled] ccache version 2.3 [disabled] dev-lang/python: 2.4.2 dev-python/pycrypto: 2.0.1-r5 dev-util/ccache: 2.3 dev-util/confcache: [Not Present] sys-apps/sandbox: 1.2.17 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1 sys-devel/binutils: 2.16.1-r2 sys-devel/libtool: 1.5.22 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-march=pentium4 -O2 -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/X11/xkb /var/qmail/alias /var/qmail/control" CONFIG_PROTECT_MASK="/etc/eselect/compiler /etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-march=pentium4 -O2 -pipe" DISTDIR="/usr/portage-distfiles" EMERGE_DEFAULT_OPTS="--alphabetical" FEATURES="autoconfig buildpkg distlocks metadata-transfer noinfo parallel-fetch sandbox sfperms strict userpriv usersandbox" GENTOO_MIRRORS=" ftp://ftp.gg3.net/pub/linux/gentoo ftp://ftp.ecc.u-tokyo.ac.jp/GENTOO http://distfiles.gg3.net/" LANG="en_US.utf8" LINGUAS="en" MAKEOPTS="-j2" PKGDIR="/usr/portage-ss/packages" PORTAGE_RSYNC_EXTRA_OPTS="--progress" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude='/distfiles' --exclude='/local' --exclude='/packages'" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/overlay/maildir /usr/local/overlay/chutz" SYNC="rsync://rsync.gg3.net/gentoo-portage" USE="x86 X acl acpi alsa avi berkdb bitmap-fonts bonobo bzip2 canna caps cid cjk cli crypt cscope cups dga dri dvd eds emboss fontconfig foomaticdb gd gdbm gif gnutls gpm gstreamer gtk gtk2 hpn icc imap imlib ipv6 isdnlog jpeg junit ldap libg++ libwww lirc live mad maildir matroska mikmod mmx moznoirc moznomail mp3 mpeg ncurses nls nocardbus nptl nptlonly offensive ogg oggvorbis opengl pam pcre perl plotutils png ppds pppd python quicktime readline reflection samba savedconfig sdl session speedo spell spl sse ssl tcpd tiff truetype truetype-fonts type1-fonts unicode usb userlocales utf8 vim-pager vim-with-x vorbis wmf wxwindows xgetdefault xml xml2 xorg xv xvid zlib elibc_glibc input_devices_keyboard input_devices_mouse input_devices_evdev kernel_linux linguas_en userland_GNU video_cards_i810 video_cards_vesa video_cards_vga" Unset: CTARGET, INSTALL_MASK, LC_ALL, LDFLAGS
The segfault occurs here: $ sed -n '711,715p' node.c: if (n->wstptr != NULL) { free(n->wstptr); n->wstptr = NULL; n->wstlen = 0; } or more precisely, when free() is called.
Seems to work okay on my x86 machine with sys-apps/gawk-3.1.5-r1 installed... Note that although my profile is 2006.0, I haven't yet upgraded, and haven't done an upgrade for some time. So you may want to take my report with a pinch of salt. Portage 2.0.54-r2 (default-linux/x86/2006.0, gcc-3.3.6, glibc-2.3.4.20041102-r1, 2.6.9-gentoo-r9 i686) ================================================================= System uname: 2.6.9-gentoo-r9 i686 Intel(R) Pentium(R) 4 CPU 1400MHz Gentoo Base System version 1.6.14 distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled] ccache version 2.3 [enabled] dev-lang/python: 2.3.5, 2.4.2 dev-python/pycrypto: 2.0-r1 dev-util/ccache: 2.3 dev-util/confcache: [Not Present] sys-apps/sandbox: 1.2.10 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1 sys-devel/binutils: 2.15.92.0.2-r10 sys-devel/libtool: 1.5.20 virtual/os-headers: 2.4.22-r1 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-march=pentium4 -mcpu=pentium4 -O2 -pipe -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/lib/X11/xkb /usr/share/config /var/bind" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-march=pentium4 -mcpu=pentium4 -O2 -pipe -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig ccache distlocks sandbox sfperms strict" GENTOO_MIRRORS="http://gentoo.ITDNet.net/gentoo http://ftp.heanet.ie/pub/gentoo/ http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/ http://ftp.du.se/pub/os/gentoo http://distfiles.gentoo.org/" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/home/ciaran/portage/overlay" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="x86 X acl acpi alsa apache2 apm audiofile avi berkdb bitmap-fonts bzip2 cdb cdr cli crypt cups curl dga dri dvd eds emboss encode esd ethereal exif expat fam ffmpeg flac fluidsynth foomaticdb fortran gd gdbm geoip gif glut gmail gmp gnome gphoto2 gpm gstreamer gtk gtk2 gtkhtml idn imagemagick imlib ipv6 isdnlog jack java jbig joystick jpeg kde kdeenablefinal ladspa lcms libg++ libwww lua mad mhash mikmod mmx mmxext mng motif mozsvg mp3 mpeg multitarget ncurses nls nodrm nptl ogg openal opengl oss pam pcre pdflib perl pic png povray ppds pppd python qt quicktime readline real reflection samba sdl session slang spl sse sse2 ssl svga tcltk tcpd tiff truetype truetype-fonts type1-fonts udev unicode usb v4l vorbis wmf xine xml xml2 xmms xorg xv xvid zlib userland_GNU kernel_linux elibc_glibc" Unset: CTARGET, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTAGE_RSYNC_EXTRA_OPTS, PORTAGE_RSYNC_OPTS
(In reply to comment #2) > Seems to work okay on my x86 machine with sys-apps/gawk-3.1.5-r1 installed... > > Note that although my profile is 2006.0, I haven't yet upgraded, and haven't > done an upgrade for some time. So you may want to take my report with a pinch > of salt. > > Portage 2.0.54-r2 (default-linux/x86/2006.0, gcc-3.3.6, > glibc-2.3.4.20041102-r1, 2.6.9-gentoo-r9 i686) Yep, I wonder if it could be your glibc version that saves you .... Also note that, in contrast to the testcase above, the following actually work fine: yes 22 | awk '{print $0, length($2) }' yes 22 | awk '{print $0, length($1) }' yes 1 | awk '{print $0, length($1), length($2) }' Furthermore, the original awk '{print $0, length($1), length($2) }' dies even with the following input: 1 1 1 22 22 (dies) == cut 22 1 1 1 22 1 (dies) == cut It's interesting that the length actually matters.
All of those examples work for me. I'd be interested to see reports from other people about whether it works for them. Could be a glibc issue rather than a gawk issue, but then again, I would think such a bug would have been spotted in other situations before being taken out of ~x86...
whats with the "gcc-3.4.6/vanilla" ? are you using hardened or something ?
i'm pretty sure this should be fixed with 3.1.5-r2 now ...
(In reply to comment #6) > i'm pretty sure this should be fixed with 3.1.5-r2 now ... Unfortunately it's not. The problem exists on a Celeron 2.2 machine. I build binary packages for that machine in a chroot on an Athlon64. Inside the chroot on the Athlon64 everything is fine. Outside the chroot there is no problem either (with amd64 profile). There is a problem only on the x86. I also recompiled gawk-3.1.5-r2 on the x86 (I did not use the binary package from the other machine) and it still failed. Well, that's not surprising, since the checksums in the CONTENTS file in /var/db/pkg did not change. Oh, and my CFLAGS changed since I opened the bug. It is -march=i686 now.
This thing is actually fun. On the amd64: yes 22 | chroot /mnt/tiger/ /bin/awk '{print $0, length($1), length($2)}' works fine yes 22 | /mnt/tiger/bin/awk '{print $0, length($1), length($2) }' dies Any recommendations?
that would suggest that the libraries in / are screwed up ... or there is a serious mismatch between /mnt/tiger and / what versions of glibc are in /mnt/tiger and in / ?
(In reply to comment #9) > that would suggest that the libraries in / are screwed up ... or there is a > serious mismatch between /mnt/tiger and / The profile is a serious mismatch (amd64 in / and x86 in /mnt/tiger)... > what versions of glibc are in /mnt/tiger and in / ? lion ~ # ldd /mnt/tiger/bin/awk | grep libc libc.so.6 => /lib32/libc.so.6 (0xf7de6000) lion ~ # chroot /mnt/tiger/ q file -v /lib/libc.so.6 sys-libs/glibc-2.3.6-r4 (/lib/libc.so.6) lion ~ # q file -v /lib32/libc.so.6 sys-libs/glibc-2.3.6-r3 (/lib32/libc.so.6)
glibc-2.4 went stable... and the error got more fun with it: $ yes 22 | awk '{ print $0, length($1), length($2) }' | head *** glibc detected *** awk: double free or corruption (fasttop): 0x080985a8 *** ======= Backtrace: ========= /lib/libc.so.6[0xb7ec773f] /lib/libc.so.6(__libc_free+0x8b)[0xb7ec826b] awk(str2wstr+0x1c5)[0x8071455] awk(do_length+0x12e)[0x8056e7e] awk(r_tree_eval+0xa3a)[0x808229a] awk(do_print+0xbd)[0x805a3cd] awk(interpret+0x308)[0x8080088] awk(interpret+0x229)[0x807ffa9] awk(do_input+0x82)[0x806ba42] awk(main+0x10d1)[0x806fcb1] /lib/libc.so.6(__libc_start_main+0xe2)[0xb7e79892] awk[0x804cac1] ======= Memory map: ======== 08048000-08091000 r-xp 00000000 fe:01 6996 /bin/gawk-3.1.5 08091000-08092000 rw-p 00048000 fe:01 6996 /bin/gawk-3.1.5 08092000-080b8000 rw-p 08092000 00:00 0 [heap] b7c00000-b7c21000 rw-p b7c00000 00:00 0 b7c21000-b7d00000 ---p b7c21000 00:00 0 b7d43000-b7d4d000 r-xp 00000000 fe:05 693131 /usr/lib/gcc/i686-pc-linux-gnu/4.1.1/libgcc_s.so.1 b7d4d000-b7d4e000 rw-p 00009000 fe:05 693131 /usr/lib/gcc/i686-pc-linux-gnu/4.1.1/libgcc_s.so.1 b7d58000-b7d59000 rw-p b7d58000 00:00 0 b7d59000-b7e30000 r--p 00000000 fe:05 244134 /usr/lib/locale/en_US.utf8/LC_COLLATE b7e30000-b7e63000 r--p 00000000 fe:05 117033 /usr/lib/locale/en_US.utf8/LC_CTYPE b7e63000-b7e64000 rw-p b7e63000 00:00 0 b7e64000-b7f7c000 r-xp 00000000 fe:01 6954 /lib/libc-2.4.so b7f7c000-b7f7e000 r--p 00117000 fe:01 6954 /lib/libc-2.4.so b7f7e000-b7f80000 rw-p 00119000 fe:01 6954 /lib/libc-2.4.so b7f80000-b7f83000 rw-p b7f80000 00:00 0 b7f83000-b7fa6000 r-xp 00000000 fe:01 6953 /lib/libm-2.4.so b7fa6000-b7fa8000 rw-p 00022000 fe:01 6953 /lib/libm-2.4.so b7fa8000-b7faa000 r-xp 00000000 fe:01 2132 /lib/libdl-2.4.so b7faa000-b7fac000 rw-p 00001000 fe:01 2132 /lib/libdl-2.4.so b7fac000-b7fad000 r--p 00000000 fe:05 243924 /usr/lib/locale/en_US.utf8/LC_NUMERIC b7fad000-b7fae000 r--p 00000000 fe:05 243925 /usr/lib/locale/en_US.utf8/LC_TIME b7fae000-b7faf000 r--p 00000000 fe:05 244139 /usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES b7faf000-b7fb6000 r--s 00000000 fe:05 4044 /usr/lib/gconv/gconv-modules.cache b7fb6000-b7fb7000 rw-p b7fb6000 00:00 0 b7fb7000-b7fd1000 r-xp 00000000 fe:01 6259 /lib/ld-2.4.so b7fd1000-b7fd2000 r--p 00019000 fe:01 6259 /lib/ld-2.4.so b7fd2000-b7fd3000 rw-p 0001a000 fe:01 6259 /lib/ld-2.4.so bf7eb000-bf801000 rw-p bf7eb000 00:00 0 [stack] ffffe000-fffff000 ---p 00000000 00:00 0 [vdso] 22 2 0 22 2 0
OK, the problem is reproducible now. It appears that it has got something to do with the locale. I am normally running with en_US.utf8 and that is when I have the problem. See below: All utf8 locales that I have on that x86 system (three in total, bg_BG, en_US and ja_JP) cause the problem. All non-utf8 locales that I have on that system (bg_BG, C, en_US, ja_JP.eucjp, POSIX) do not cause the problem. chutz@tiger ~ $ yes 22 | head -n3 | env -i LC_CTYPE=en_US awk '{ print $0, length($1), length($2) }' 22 2 0 22 2 0 22 2 0 chutz@tiger ~ $ yes 22 | head -n3 | env -i LC_CTYPE=en_US.utf8 awk '{ print $0, length($1), length($2) }' 22 2 0 22 2 0 *** glibc detected *** awk: double free or corruption (fasttop): 0x0809a398 *** ======= Backtrace: ========= /lib/libc.so.6[0xb7ed7d20] /lib/libc.so.6(__libc_free+0x84)[0xb7ed9364] awk(str2wstr+0x33)[0x80706f3] awk(do_length+0xb3)[0x8059393] awk(r_tree_eval+0x9ff)[0x80830bf] awk(do_print+0x10e)[0x805b63e] awk(interpret+0x50e)[0x80810de] awk(interpret+0x1d6)[0x8080da6] awk(do_input+0x48)[0x806dcb8] awk(main+0xfe4)[0x806fdd4] /lib/libc.so.6(__libc_start_main+0xd8)[0xb7e8a878] awk[0x804ca71] ======= Memory map: ======== 08048000-08091000 r-xp 00000000 fe:01 4846 /bin/gawk-3.1.5 08091000-08092000 rw-p 00049000 fe:01 4846 /bin/gawk-3.1.5 08092000-080b8000 rw-p 08092000 00:00 0 [heap] b7d00000-b7d21000 rw-p b7d00000 00:00 0 b7d21000-b7e00000 ---p b7d21000 00:00 0 b7e2c000-b7e36000 r-xp 00000000 fe:05 438063 /usr/lib/gcc/i686-pc-linux-gnu/4.1.1/libgcc_s.so.1 b7e36000-b7e37000 rw-p 00009000 fe:05 438063 /usr/lib/gcc/i686-pc-linux-gnu/4.1.1/libgcc_s.so.1 b7e41000-b7e74000 r--p 00000000 fe:05 618863 /usr/lib/locale/en_US.utf8/LC_CTYPE b7e74000-b7e75000 rw-p b7e74000 00:00 0 b7e75000-b7f8c000 r-xp 00000000 fe:01 14305 /lib/libc-2.4.so b7f8c000-b7f8e000 r--p 00116000 fe:01 14305 /lib/libc-2.4.so b7f8e000-b7f90000 rw-p 00118000 fe:01 14305 /lib/libc-2.4.so b7f90000-b7f93000 rw-p b7f90000 00:00 0 b7f93000-b7fb6000 r-xp 00000000 fe:01 14304 /lib/libm-2.4.so b7fb6000-b7fb8000 rw-p 00022000 fe:01 14304 /lib/libm-2.4.so b7fb8000-b7fba000 r-xp 00000000 fe:01 14290 /lib/libdl-2.4.so b7fba000-b7fbc000 rw-p 00001000 fe:01 14290 /lib/libdl-2.4.so b7fbe000-b7fbf000 rw-p b7fbe000 00:00 0 b7fbf000-b7fc6000 r--s 00000000 fe:05 2334 /usr/lib/gconv/gconv-modules.cache b7fc6000-b7fc7000 rw-p b7fc6000 00:00 0 b7fc7000-b7fc8000 r-xp b7fc7000 00:00 0 [vdso] b7fc8000-b7fe2000 r-xp 00000000 fe:01 14301 /lib/ld-2.4.so b7fe2000-b7fe3000 r--p 00019000 fe:01 14301 /lib/ld-2.4.so b7fe3000-b7fe4000 rw-p 0001a000 fe:01 14301 /lib/ld-2.4.so bfacd000-bfae2000 rw-p bfacd000 00:00 0 [stack] Aborted
thanks ... i grabbed some patches from Fedora to fix this in 3.1.5-r3
The new version causes double free corruptions for me, anyone else have this problem too?
(In reply to comment #14) > The new version causes double free corruptions for me, anyone else have this > problem too? Yes. I tried gawk 3.1.5g and it doesn't crash the original command in this thread or the command that crashes for me. My crash happens during some step of the openttd build process (around revision 9973) when it generates deps. My crash also doesn't happen in 3.1.5-r2 or plain 3.1.5 without any gentoo patches, it seems.
Created attachment 120676 [details] awk script that fails awk -f progfile < Makefile.dep.tmp > /dev/null fails with *** glibc detected *** awk: double free or corruption (fasttop): 0x080b43f0 *** ======= Backtrace: ========= /lib/libc.so.6[0xa7e85d21] /lib/libc.so.6(__libc_free+0x89)[0xa7e867f9] awk(unref+0x63)[0x8073bf3] awk(reset_record+0xc5)[0x806a025] awk(set_record+0xf)[0x806a0bf] awk(do_input+0x10b)[0x806f43b] awk(main+0xf1f)[0x8072f3f] /lib/libc.so.6(__libc_start_main+0xe6)[0xa7e38896] awk[0x804d321]
Created attachment 120678 [details] input data to trigger crash
