Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 335995 - dev-db/mysql-5.1.50-r1: mysql segfaults and wrong 'access denied' messages. x86 gcc 4.4 miscompile w/ omit-frame-pointer
Summary: dev-db/mysql-5.1.50-r1: mysql segfaults and wrong 'access denied' messages. x...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Server (show other bugs)
Hardware: x86 Linux
: High critical (vote)
Assignee: Gentoo Linux MySQL bugs team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 915000 CVE-2010-1621
  Show dependency tree
 
Reported: 2010-09-04 17:29 UTC by Ondrej Zary
Modified: 2023-10-01 14:34 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
mysqld.err excerpt (mysqld.crash-excerpt.log,1.42 KB, text/plain)
2010-10-10 13:18 UTC, Christian Burger
Details
emerge --info from defective system (defective-system.emerge-info.text,4.12 KB, text/plain)
2010-10-10 13:19 UTC, Christian Burger
Details
paludis --info from working system (working-system.paludis-info.text,1.64 KB, text/plain)
2010-10-10 13:20 UTC, Christian Burger
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ondrej Zary 2010-09-04 17:29:50 UTC
Just upgraded from dev-db/mysql-5.0.90-r2 to dev-db/mysql-5.1.50-r1 and having major problems now. It started as non-working roundcube webmail, showing this in error log:

04-Sep-2010 17:50:12] MDB2 Error: insufficient permissions (-27): _doQuery: [Error message: Could not execute statement]
[Last executed query: PREPARE mdb2_statement_mysql_1fcaaea0e0b1b327213dd2325ec8395745ca58896 FROM 'SELECT vars, ip, UNIX_TIMESTAMP(changed) AS changed\n     FROM session\n     WHERE  sess_id=?']
[Native code: 1142]
[Native message: SELECT command denied to user 'roundcube'@'localhost' for table 'session']

I then tested it manually from command-line and also got access denied message. When I recreated the privileges, it worked from command-line - but only until I opened another connection using the same user. The interesting thing is that everything works fine with root user...
While attempting to track down the problem, mysql segfaulted.

Reproducible: Always

Steps to Reproduce:
1. mysql -u root -p -e "create database test;use test;create table test(id int);grant all privileges on test.* to test@localhost identified by 'test';"
2. mysql -u test -ptest -e "use test;select * from test;"
Actual Results:  
ERROR 2013 (HY000) at line 1: Lost connection to MySQL server during query

Expected Results:  
mysql should not crash

/var/log/mysql/mysqld.err contains this:

100904 19:19:53 [Warning] No argument was provided to --log-bin, and --log-bin-index was not used; so replication may break when this MySQL server acts as a master and has his hostname changed!! Please use '--log-bin=mysqld-bin' to avoid this problem.
100904 19:19:54  InnoDB: Started; log sequence number 0 43705
100904 19:19:54 [Note] Event Scheduler: Loaded 0 events
100904 19:19:54 [Note] /usr/sbin/mysqld: ready for connections.
Version: '5.1.50-log'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  Gentoo Linux mysql-5.1.50-r1
100904 19:21:06 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=2097152
read_buffer_size=262144
max_used_connections=1
max_threads=151
threads_connected=1
It is possible that mysqld could use up to
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 118977 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x97fedf8
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0xb361636c thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x1e) [0x84b33de]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0x9845350 = select * from test
thd->thread_id=5
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Comment 1 Markos Chandras (RETIRED) gentoo-dev 2010-09-05 10:42:32 UTC
Did you run revdep-rebuild -X after you upgraded your mysql?
Comment 2 Jasper Moeller 2010-09-05 22:03:49 UTC
Also have that problem (with my ampache database and several others).
Additional steps I've taken:
- revdep-rebuild
- mysql_upgrade as well as mysql_fix_permissions
- recreated data dir and restored from backup (binary files only, since I'm using mysqlhotcopy and not mysqldump...), reran mysql_upgrade etc.
- tried to repair tables
- enabled/disabled max-idx-128 as well as big-tables (my database is rather large)
- reduced buffer sizes in my.cnf to exclude OOM conditions

mysqld does not always crash immediately, somtimes, I'm able to execute one or two selects succesfully (sometime, I had to regrant the privileges). However, sooner or later it dies with signal 11. There is no stacktrace beyond the first EIP:

Information from mysqld.err:
--------------------------------------------------
100905 23:44:34 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=16777216
read_buffer_size=262144
max_used_connections=0
max_threads=151
threads_connected=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 210630 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x8d6b068
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0xbfa1e42c thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x1e) [0x84bc21e]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at (nil) is an invalid pointer
thd->thread_id=0
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
-----------------------------------------------------------------------

I can try to provide more information if necessary, however
since I really need mysql running, I've reverted to mysql-5.0 now, so I'd have to reemerge 5.1 first.
Comment 3 Markos Chandras (RETIRED) gentoo-dev 2010-09-05 22:21:00 UTC
can you provide an emerge --info output?
Comment 4 Jasper Moeller 2010-09-05 22:40:07 UTC
Here it goes:

Portage 2.1.8.3 (default/linux/x86/10.0/desktop, gcc-4.4.3, glibc-2.11.2-r0, 2.6.35-gentoo-r4 i686)
=================================================================
System uname: Linux-2.6.35-gentoo-r4-i686-AMD_Athlon-tm-_Processor-with-gentoo-1.12.13
Timestamp of tree: Sun, 05 Sep 2010 19:45:03 +0000
distcc 3.1 i686-pc-linux-gnu [disabled]
app-shells/bash:     4.0_p37
dev-java/java-config: 1.3.7-r1, 2.1.11
dev-lang/python:     2.4.4-r13, 2.5.4-r3, 2.6.5-r3, 3.1.2-r4
dev-util/cmake:      2.8.1-r2
sys-apps/baselayout: 1.12.13
sys-apps/sandbox:    1.6-r2
sys-devel/autoconf:  2.13, 2.65
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r3, 1.10.3, 1.11.1
sys-devel/binutils:  2.20.1-r1
sys-devel/gcc:       3.4.6-r2, 4.1.2, 4.3.4, 4.4.3-r2
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6b
sys-devel/make:      3.81-r2
virtual/os-headers:  2.6.30-r1
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="*"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O3 -march=athlon -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/eselect/postgresql /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O3 -march=athlon -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests distlocks fixpackages news parallel-fetch protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch"
GENTOO_MIRRORS="http://mirror.switch.ch/ftp/mirror/gentoo/ http://mirror.jamit.de/gentoo/ http://de-mirror.org/distro/gentoo/ http://mirror.netcologne.de/gentoo/ http://mirror.cambrium.nl/pub/os/linux/gentoo/"
LANG="C"
LC_ALL="C"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
USE="X a52 aac acl acpi alsa bash-completion berkdb bluetooth branding bzip2 cairo cdr cli consolekit cracklib crypt cups cxx dbus dri dts dvd dvdr encode exif fam firefox flac fortran gdbm gif gtk hal iconv ipv6 jpeg lcms ldap libnotify mng modules mp3 mp4 mudflap ncurses nls nptl nptlonly ogg openmp pam pango pcre pdf perl png ppds pppd python qt3support qt4 readline reflection session spl ssl startup-notification svg sysfs tcpd tiff unicode usb vorbis x264 x86 xcb xml xorg xulrunner xvid zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="vga vesa dummy fbdev mga" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" 
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 5 Markos Chandras (RETIRED) gentoo-dev 2010-09-05 22:45:01 UTC
Assigning to maintainer
Comment 6 Ondrej Zary 2010-09-06 07:10:57 UTC
Portage 2.1.8.3 (default/linux/x86/10.0, gcc-4.4.3, glibc-2.11.2-r0, 2.6.34-gentoo-r1-main i686)
=================================================================
System uname: Linux-2.6.34-gentoo-r1-main-i686-Celeron_-Mendocino-with-gentoo-1.12.13
Timestamp of tree: Sun, 05 Sep 2010 19:00:21 +0000
ccache version 2.4 [enabled]
app-shells/bash:     4.0_p37
dev-lang/python:     2.6.5-r3, 3.1.2-r4
dev-util/ccache:     2.4-r7
dev-util/cmake:      2.8.1-r2
sys-apps/baselayout: 1.12.13
sys-apps/sandbox:    1.6-r2
sys-devel/autoconf:  2.65
sys-devel/automake:  1.10.3, 1.11.1
sys-devel/binutils:  2.20.1-r1
sys-devel/gcc:       4.4.3-r2
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6b
sys-devel/make:      3.81-r2
virtual/os-headers:  2.6.30-r1
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="* -@EULA"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O3 -march=i686 -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /var/bind"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/eselect/postgresql /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O3 -march=i686 -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps y"
FEATURES="assume-digests buildpkg ccache collision-protect cvs distlocks fixpackages metadata-transfer news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch userpriv usersandbox"
GENTOO_MIRRORS="ftp://ftp.fi.muni.cz/pub/linux/gentoo/ ftp://gentoo.mirror.web4u.cz/ ftp://gentoo.inode.at/source/ ftp://gd.tuwien.ac.at/opsys/linux/gentoo/"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en sk"
MAKEOPTS="-j1"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/var/lib/layman/sunrise /usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="apache2 berkdb bzip2 caps chroot cli cracklib crypt cxx dri gdbm gmp iconv idn imap jpeg mbox mmx modules mudflap ncurses nls nptl nptlonly pam pcre pdflib perl png pppd readline reflection samba session slang spl sse ssl sysfs tcpd truetype unicode urandom x86 xattr xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="alias auth_basic authn_anon authn_default authn_file authz_default authz_groupfile authz_host authz_owner authz_user autoindex cgi dbd deflate dir env expires ext_filter filter headers info log_config logio mime mime_magic negotiation rewrite setenvif status unique_id usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en sk" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga neomagic nouveau nv r128 radeon savage sis tdfx trident vesa via vmware voodoo" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 7 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-09-06 08:20:00 UTC
(In reply to comment #1)
I can't reproduce using your instructions at all.
Did you run mysql_upgrade after upgrading?

(In reply to comment #2)
> Also have that problem (with my ampache database and several others).
> Additional steps I've taken:
> - enabled/disabled max-idx-128 as well as big-tables (my database is rather
> large)
How big is "large" in this case? You shouldn't need big-tables or max-idx-128 until you're well beyond 200GiB of database. If you ARE using such a large database, please attach your my.cnf. What InnoDB setup are you using? Stock, InnoDB plugin, XtraDB?

> I can try to provide more information if necessary, however
> since I really need mysql running, I've reverted to mysql-5.0 now, so I'd have
> to reemerge 5.1 first.
Can both of you please try gcc-4.4.4?
Comment 8 Jasper Moeller 2010-09-06 09:09:51 UTC
(In reply to comment #7)
 
> (In reply to comment #2)
> > Also have that problem (with my ampache database and several others).
> > Additional steps I've taken:
> > - enabled/disabled max-idx-128 as well as big-tables (my database is rather
> > large)
> How big is "large" in this case? You shouldn't need big-tables or max-idx-128
> until you're well beyond 200GiB of database. If you ARE using such a large
> database, please attach your my.cnf. What InnoDB setup are you using? Stock,
> InnoDB plugin, XtraDB?
> 

No, it's not that big (max 250MiB, I think) - this was just a test because the issue with max-idx has already been mentioned. Most databases still are MyISAM, fot the only InnoDB one I've used the stock setup (i.e. default configuration). I've only bothered to reproduce the problem with the MyISAM DBs, though, since these are the important ones

> > I can try to provide more information if necessary, however
> > since I really need mysql running, I've reverted to mysql-5.0 now, so I'd have
> > to reemerge 5.1 first.
> Can both of you please try gcc-4.4.4?
> 

Sure, will take a while, though.
Comment 9 Jasper Moeller 2010-09-06 17:52:43 UTC
Ok, tried it with gcc-4.4.4-r1. Run mysql_upgrade after emerge, still got signal 11 (same stacktrace - different pointer, of course ;)).

However, after restarting mysqld after the crash, I haven't been able to reproduce the problem so far (which was different from before, where I could take down mysqld reliably). Unfortunately, I didn't think of restarting mysqld immediately after mysql_upgrade, don't know if this makes a difference, though.

I'll try to run it a while under real life conditions and see if the problem occurs again, fortunately, I've recent backups available...
Comment 10 Jasper Moeller 2010-09-06 19:52:33 UTC
OK, still no go as soon as I try to connect with my php frontend (yes, I did reemerge php) - could this be related? Seems to be a problem as soon as I actually try to manipulate the database.

For the record, stacktrace follows:
----------------------------------------------
100906 21:32:42 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=16777216
read_buffer_size=262144
max_used_connections=1
max_threads=151
threads_connected=1
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 210625 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0xa982098
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0xb279736c thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x1e) [0x84bab5e]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at 0xa9a8a90 = DELETE FROM tmp_playlist_data USING tmp_playlist_data LEFT JOIN tmp_playlist ON tmp_playlist_data.tmp_playlist=tmp_playlist.id WHERE tmp_playlist.id IS NULL
thd->thread_id=5
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
-----------------------------------------------------------

For a last resort (in case the problem is buried somewhere in the binary on-disk format) I've tried to reimport the databases in question from an sql dump which I had prepared with the last working setup - still not working.

I'll have to leave it at that for the moment until I've got new hardware for the server where emerging gcc, mysql and php won't take half a day.
Comment 11 Ondrej Zary 2010-09-06 20:12:53 UTC
I cannot do much testing as this is a production server (and also the only machine here running Gentoo) and I don't want to break it completely. I can compile a different version of mysql and test if it works (Roundcube is a good test case - it breaks immediately).
Comment 12 Simon Koch 2010-09-07 18:47:42 UTC
I'm seeing this same behavior after upgrading from 5.0 to 5.1.  I've noticed a pattern, though.  MySQL segfaults as soon as it gets a query from a client that's not packaged with the server.  In my case, that's mythbackend, jabberd2, and some simple php web pages.  I can execute the same queries (copy/pasted from mysqld.err) in the mysql commandline client with no crashing.  mysqldump worked fine, too.

My hunch is this means there's some library that needs to be rebuilt, but I don't know how to identify it.  revdep-rebuild says everything looks OK.
Comment 13 Ondrej Zary 2010-09-07 19:00:59 UTC
No matter how bad data you send to it, the server must never crash, ever. If it does, then it's a bug. And according to my logs, it never crashed before.
Comment 14 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-09-07 19:17:32 UTC
(In reply to comment #12)
> My hunch is this means there's some library that needs to be rebuilt, but I
> don't know how to identify it.  revdep-rebuild says everything looks OK.
revdep-rebuild is irrelevant with the new @preserved-libs set in Portage. But you did mention PHP there, can you please rebuild PHP and all PECL modules that have database code?

Comment 15 Simon Koch 2010-09-07 20:28:21 UTC
(In reply to comment #14)
> revdep-rebuild is irrelevant with the new @preserved-libs set in Portage. But
> you did mention PHP there, can you please rebuild PHP and all PECL modules that
> have database code?
> 

Isn't preserved-libs a portage 2.2 feature?  I'm still on 2.1.8.3 (stable x86), so I've got to deal with revdep-rebuild.  I rebuilt php, and it still causes mysql to crash.

In distilling down my test script, I did find the real difference between queries that crash it and those that don't.  It's exactly the same as Ondrej's steps to reproduce.  When I connect as root, everything works.  When I connect as a different database user, mysqld segfaults as described.

From a root prompt:
"mysql gcp" no crash
"mysql gcp -uapiuser -p" crash
"mysql gcp -uroot -p" no crash

I get the same results using those username/password combinations in my php script, too.
Comment 16 Ondrej Zary 2010-09-07 20:33:22 UTC
Yes, everything works as root - even Roundcube.
Comment 17 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-09-07 21:12:29 UTC
Can you forcibly dump your 'mysql' database to SQL and then restore it?
And/or force repair on it.
Comment 18 Simon Koch 2010-09-07 22:37:20 UTC
I did a backup/restore of the mysql database using mysqldump, no change in behavior.  I did a mysqlcheck -r as well, still nothing.  Is there something more forceful I should try?
Comment 19 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-09-07 22:44:07 UTC
Hmm, now not sure.
Can we drop -fomit-frame-pointer and add in -ggdb to your CFLAGS+CXXFLAGS, and hopefully get some more useful backtraces? Adding in USE=debug might be useful too.

The fact that root works, but the other users don't does strongly suggest that something is wrong in the authentication path or the auth tables. Can you create user with the same privileges as root, and then slowly drop them until you produce the bug maybe too?
Comment 20 Simon Koch 2010-09-08 01:16:08 UTC
Oh yay, a heisenbug.  I recompiled mysql with CFLAGS="-march=pentium3 -O2 -ggdb -pipe" and USE="debug", and now it doesn't segfault.  I'll slowly move it back to "-march=pentium3 -O3 -fomit-frame-pointer -pipe" and see where it fails.
Comment 21 Simon Koch 2010-09-08 04:09:33 UTC
fomit-frame-pointer looks like the culprit.  I'm back to my original configuration, minus that in CFLAGS and CXXFLAGS.  I've got all the clients running again and mysql is solid.
Comment 22 Ondrej Zary 2010-09-08 06:26:46 UTC
http://bugs.mysql.com/bug.php?id=45205 - not fixed...
Comment 23 Simon Koch 2010-09-08 14:28:35 UTC
(In reply to comment #22)
> http://bugs.mysql.com/bug.php?id=45205 - not fixed...
> 

Unfortunately, the observation that "--disable-profiling causes mysql to not segfault" doesn't seem to apply here.  Neither you nor I have USE="profiling", and I've verified that the ebuild does add "--disable-profiling" to ./configure, at least for my current USE flags.
Comment 24 Marc Elser 2010-09-08 21:19:23 UTC
(In reply to comment #23)
> (In reply to comment #22)
> > http://bugs.mysql.com/bug.php?id=45205 - not fixed...
> > 
> 

Hi just found this bug on bugzilla. I'm also seeing exactly the same problem. Mysql dies when trying to access database using non-root user.

removing -fomit-frame-pointer also solved the problem on my side. You're right the bug is not fixed yet and maybe gentoo should strip -fomit-frame-pointer in the ebuild for pentium3 machines, as my machine also has march=pentium3. Maybe only these CPU's are affected. For the moment I put special CFLAGS && CXXFLAGS in /etc/portage/env/dev-db/mysql so every compile of mysql has the -fomit-frame-pointer stripped.
Comment 25 Grzegorz {NineX} Krzystek 2010-09-08 21:23:51 UTC
i have same problem with march=native on x86 system with xeon cpu
so i think it is not cpu realted

(In reply to comment #24)
> (In reply to comment #23)
> > (In reply to comment #22)
> > > http://bugs.mysql.com/bug.php?id=45205 - not fixed...
> > > 
> > 
> 
> Hi just found this bug on bugzilla. I'm also seeing exactly the same problem.
> Mysql dies when trying to access database using non-root user.
> 
> removing -fomit-frame-pointer also solved the problem on my side. You're right
> the bug is not fixed yet and maybe gentoo should strip -fomit-frame-pointer in
> the ebuild for pentium3 machines, as my machine also has march=pentium3. Maybe
> only these CPU's are affected. For the moment I put special CFLAGS && CXXFLAGS
> in /etc/portage/env/dev-db/mysql so every compile of mysql has the
> -fomit-frame-pointer stripped.
> 

Comment 26 Jasper Moeller 2010-09-08 21:31:03 UTC
Well, I'm running on AMD Athlon K7, guess this will occur on all architectures with a sufficiently recent x86 instruction set. I can try it without vomit-frame-pointer next week or so, when the server has to be upgraded anyway (switching to AMD K10).

Comment 27 Ondrej Zary 2010-09-08 21:33:58 UTC
My CPU is Celeron Mendocino, which is Pentium II. So it's not limited to Pentium 3. Maybe it's limited to 32-bit x86.
Comment 28 Stuart Shelton 2010-09-11 08:56:36 UTC
I'm also finding that, whilst MySQL 5.0 and prior were fine, MySQL 5.1 segfaults on startup.  This is on a non-HT, non-EM64T Northwood Pentium-4 server, with '-fomit-frame-pointer' removed from CFLAGS:

100911  9:41:43 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=16777216
read_buffer_size=262144
max_used_connections=0
max_threads=151
threads_connected=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 133324 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x85c6c98
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0xbf963144 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x1d) [0x8380047]
/usr/sbin/mysqld(handle_segfault+0x1cd) [0x81923b0]
[0xb7819400]
/usr/sbin/mysqld(open_table_def(THD*, st_table_share*, unsigned int)+0x7fd) [0x81d544d]
/usr/sbin/mysqld(get_table_share(THD*, TABLE_LIST*, char*, unsigned int, unsigned int, int*)+0x89) [0x81cded5]
/usr/sbin/mysqld() [0x81ce05a]
/usr/sbin/mysqld(open_table(THD*, TABLE_LIST*, st_mem_root*, bool*, unsigned int)+0x57a) [0x81ceb20]
/usr/sbin/mysqld(open_tables(THD*, TABLE_LIST**, unsigned int*, unsigned int)+0x1c1) [0x81cf181]
/usr/sbin/mysqld(open_and_lock_tables_derived(THD*, TABLE_LIST*, bool)+0x25) [0x81cf55f]
/usr/sbin/mysqld(plugin_init(int*, char**, int)+0x4da) [0x829ab5a]
/usr/sbin/mysqld() [0x8192c9d]
/usr/sbin/mysqld(main+0x279) [0x8194de5]
/lib/libc.so.6(__libc_start_main+0xfe) [0xb73d1bee]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at (nil) is an invalid pointer
thd->thread_id=0
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.

... this is with my existing data loaded.  If I remove /var/lib/mysql and execute 'emerge --config mysql' then the process requests a password, but then fails with:

100910 20:36:53 [ERROR] /usr/sbin/mysqld: Incorrect information in file: './mysql/time_zone.frm'
ERROR: 1033  Incorrect information in file: './mysql/time_zone.frm'
100910 20:36:53 [ERROR] Aborting

100910 20:36:54 [Note] /usr/sbin/mysqld: Shutdown complete

... which occurs after /var/lib/mysql has definitely just been removed, and so time_zone.frm has only just been re-created!

emerge --info:

Portage 2.1.8.3 (default/linux/x86/10.0/server, gcc-4.4.3, glibc-2.11.2-r0, 2.6.35-gentoo-r4 i686)
=================================================================
System uname: Linux-2.6.35-gentoo-r4-i686-Intel-R-_Pentium-R-_4_CPU_2.80GHz-with-gentoo-1.12.13
Timestamp of tree: Sat, 11 Sep 2010 07:15:01 +0000
app-shells/bash:     4.0_p37
dev-java/java-config: 2.1.11
dev-lang/python:     2.6.5-r3, 3.1.2-r4
dev-util/cmake:      2.8.1-r2
sys-apps/baselayout: 1.12.13
sys-apps/sandbox:    1.6-r2
sys-devel/autoconf:  2.65
sys-devel/automake:  1.8.5-r4, 1.10.3, 1.11.1
sys-devel/binutils:  2.20.1-r1
sys-devel/gcc:       4.4.3-r2
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6b
sys-devel/make:      3.81-r2
virtual/os-headers:  2.6.30-r1
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="*"
ACCEPT_PROPERTIES="*"
ANT_HOME="/usr/share/ant"
ARCH="x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-Os -march=pentium4 -mfpmath=sse -mmmx -msse -msse2 -pipe"
CHOST="i686-pc-linux-gnu"
CLEAN_DELAY="5"
COLLISION_IGNORE="/lib/modules"
CONFIG_PROTECT="/etc /etc/env.d /lib/rcscripts/addons /var/bind /var/www/localhost/htdocs/wordpress"
CONFIG_PROTECT_MASK="/etc/X11/xkb /etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CVS_RSH="ssh"
CXXFLAGS="-Os -march=pentium4 -mfpmath=sse -mmmx -msse -msse2 -pipe"
DISTDIR="/usr/portage/distfiles"
EDITOR="/usr/bin/vim"
ELIBC="glibc"
EMERGE_DEFAULT_OPTS="--verbose"
EMERGE_WARNING_DELAY="10"
EPREFIX=""
EROOT="/"
FEATURES="assume-digests collision-protect distlocks fail-clean fakeroot fixpackages metadata-transfer news parallel-fetch protect-owned sandbox sfperms split-elog split-log splitdebug strict unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
GCC_SPECS=""
GDK_USE_XFT="1"
GUILE_LOAD_PATH="/usr/share/guile/1.8"
HOME="/root"
INFOPATH="/usr/share/info:/usr/share/binutils-data/i686-pc-linux-gnu/2.20.1/info:/usr/share/gcc-data/i686-pc-linux-gnu/4.4.3/info"
JAVAC="/etc/java-config-2/current-system-vm/bin/javac"
JAVA_HOME="/etc/java-config-2/current-system-vm"
JDK_HOME="/etc/java-config-2/current-system-vm"
KERNEL="linux"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en_GB en"
LOGNAME="root"
PAGER="/usr/bin/less"
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/4.4.3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_DEBUG="0"
PORTAGE_DEPCACHEDIR="/var/cache/edb/dep"
PORTAGE_ELOG_CLASSES="warn error log"
PORTAGE_GID="250"
PORTAGE_INST_GID="0"
PORTAGE_INST_UID="0"
PORTAGE_PYM_PATH="/usr/lib/portage/pym"
PORTAGE_TMPDIR="/var/tmp"
PORTAGE_TMPFS="/dev/shm"
PORTAGE_VERBOSE="1"
PORTAGE_WORKDIR_MODE="0700"
PORTDIR="/usr/portage"
PROFILE_ONLY_VARIABLES="ARCH ELIBC KERNEL USERLAND"
PYTHONDONTWRITEBYTECODE="1"
ROOT="/"
ROOTPATH="/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/4.4.3"
RPMDIR="/usr/portage/rpm"
SHELL="/bin/bash"
TERM="xterm-color"
TMP="/tmp/.private/root"
TMPDIR="/tmp/.private/root"
USE="X acl adns aio amrnb amrwb ao audit avahi bash-completion berkdb bzip2 caps cdb chroot cli common-lisp cracklib crypt curl cxx dbi dbus dri encode erandom expat faac faad fam ffmpeg fftw flac fontconfig fontforge gd gdbm gif glibc-omitfp gmp gnutls guile hpn iconv icu id3tag idea idn ieee1394 iodbc ipv6 ithreads java jpeg jpeg2k kerberos ladspa lame ldap libedit libffi libsamplerate libwww linuxthreads-tls lm_sensors lzma lzo mad mailwrapper mbox mmap mmx modules mp3 mudflap mysql ncurses network network-cron nls nptl nptlonly odbc ogg openmp pam pcre perl php png policykit pppd pwdb python readline reflection rle rpc samba sasl schroedinger session slang slp sndfile snmp speex spell sql sqlite sqlite3 sse sse2 ssl startup-notification sysfs syslog tcpd theora threads truetype unicode urandom userlocales vim-syntax vorbis wavpack x264 x86 xattr xcb xinetd xml xorg xpm xvid zlib" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LINGUAS="en_GB en" USERLAND="GNU" VIDEO_CARDS="vesa" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" 
USER="root"
USERLAND="GNU"
USE_EXPAND="ALSA_CARDS ALSA_PCM_PLUGINS APACHE2_MODULES APACHE2_MPMS CAMERAS CROSSCOMPILE_OPTS DVB_CARDS ELIBC FCDSL_CARDS FOO2ZJS_DEVICES FRITZCAPI_CARDS INPUT_DEVICES KERNEL LCD_DEVICES LINGUAS LIRC_DEVICES MISDN_CARDS NETBEANS_MODULES NGINX_MODULES_HTTP NGINX_MODULES_MAIL QEMU_SOFTMMU_TARGETS QEMU_USER_TARGETS RUBY_TARGETS SANE_BACKENDS USERLAND VIDEO_CARDS XFCE_PLUGINS XTABLES_ADDONS"
USE_EXPAND_HIDDEN="CROSSCOMPILE_OPTS ELIBC KERNEL USERLAND"
USE_ORDER="env:pkg:conf:defaults:pkginternal:env.d"
VIDEO_CARDS="vesa"
XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
XZ_OPT="--memory=max"
_="/usr/bin/emerge"
Comment 29 Stuart Shelton 2010-09-11 12:50:41 UTC
Rebuilt with -O2 and reduced CFLAGS in place of -Os, and the problem remains the same:


100911 13:48:38 - mysqld got signal 11 ;
This could be because you hit a bug. It is also possible that this binary
or one of the libraries it was linked against is corrupt, improperly built,
or misconfigured. This error can also be caused by malfunctioning hardware.
We will try our best to scrape up some info that will hopefully help diagnose
the problem, but since we have already crashed, something is definitely wrong
and this may fail.

key_buffer_size=16777216
read_buffer_size=262144
max_used_connections=0
max_threads=151
threads_connected=0
It is possible that mysqld could use up to 
key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 133324 K
bytes of memory
Hope that's ok; if not, decrease some variables in the equation.

thd: 0x92dbe90
Attempting backtrace. You can use the following information to find out
where mysqld died. If you see no messages after this, something went
terribly wrong...
stack_bottom = 0xbf914f74 thread_stack 0x30000
/usr/sbin/mysqld(my_print_stacktrace+0x22) [0x845b539]
/usr/sbin/mysqld(handle_segfault+0x476) [0x81b9cc7]
[0xb787c400]
/usr/sbin/mysqld(open_table_def(THD*, st_table_share*, unsigned int)+0xf83) [0x8208239]
/usr/sbin/mysqld(get_table_share(THD*, TABLE_LIST*, char*, unsigned int, unsigned int, int*)+0x1c8) [0x81fb4f3]
/usr/sbin/mysqld() [0x81fc99e]
/usr/sbin/mysqld(open_table(THD*, TABLE_LIST*, st_mem_root*, bool*, unsigned int)+0x716) [0x81ffd6a]
/usr/sbin/mysqld(open_tables(THD*, TABLE_LIST**, unsigned int*, unsigned int)+0x505) [0x820090e]
/usr/sbin/mysqld(open_and_lock_tables_derived(THD*, TABLE_LIST*, bool)+0x66) [0x8200b89]
/usr/sbin/mysqld(plugin_init(int*, char**, int)+0x5a5) [0x830b384]
/usr/sbin/mysqld() [0x81ba41c]
/usr/sbin/mysqld(main+0x191) [0x81bcb39]
/lib/libc.so.6(__libc_start_main+0xfe) [0xb7434bee]
Trying to get some variables.
Some pointers may be invalid and cause the dump to abort...
thd->query at (nil) is an invalid pointer
thd->thread_id=0
thd->killed=NOT_KILLED
The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains
information that should help you find out what is causing the crash.
Comment 30 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-09-11 18:13:18 UTC
Now this news with omit-frame-pointer is weird, because it's actually redundant in many of your CFLAGS. It's actually implied by many of the optimization levels.
Per the GCC manpage:
`-fomit-frame-pointer'
     ...
     Enabled at levels `-O', `-O2', `-O3', `-Os'.

So if simply removing omit-frame-pointer worked for Simon in comment 21 and Marc in comment 24, I'm inclined to think that something is getting mis-compiled on 32-bit x86 (because nobody has reported problems on amd64 or any other platform).

Can we try to vary something else, like maybe using GCC4.3.x (since 4.4.4-r1 produced the problem too).
Comment 31 Simon Koch 2010-09-11 20:32:31 UTC
(In reply to comment #30)
> Now this news with omit-frame-pointer is weird, because it's actually redundant
> in many of your CFLAGS. It's actually implied by many of the optimization
> levels.
> Per the GCC manpage:
> `-fomit-frame-pointer'
>      ...
>      Enabled at levels `-O', `-O2', `-O3', `-Os'.

In the -O section of that manpage:
"-O also turns on -fomit-frame-pointer on machines where doing so does not interfere with debugging."

So on x86, it's not included in the standard optimizations but it is on amd64.
Comment 32 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-09-11 21:18:36 UTC
Stuart Shelton:
what's the smallest set of CFLAGS you can reproduce with?
including:
CFLAGS="-O2 -march=pentium4 -pipe"
CXXFLAGS="-O2 -march=pentium4 -pipe"
please.
Comment 33 Simon Koch 2010-09-12 05:43:25 UTC
With gcc 4.3.4 and CFLAGS="-march=pentium3 -O3 -fomit-frame-pointer -pipe", the error does not appear.
Comment 34 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-09-12 08:46:48 UTC
Ok, so this is looking like a GCC regression :-(

Simon, you never posted your emerge --info, so is this correct?
C(XX)FLAGS="-march=pentium3 -O3 -fomit-frame-pointer -pipe"
GCC4.3.4 - works
GCC4.4.? - fails
Comment 35 Ondrej Zary 2010-09-12 08:59:57 UTC
Maybe it's this (old) bug? http://gcc.gnu.org/bugzilla/show_bug.cgi?id=38562
Comment 36 Simon Koch 2010-09-12 10:19:37 UTC
(In reply to comment #34)
> Ok, so this is looking like a GCC regression :-(
> 
> Simon, you never posted your emerge --info, so is this correct?
> C(XX)FLAGS="-march=pentium3 -O3 -fomit-frame-pointer -pipe"
> GCC4.3.4 - works
> GCC4.4.? - fails
> 

That's right.  And just to be thorough:

Portage 2.1.8.3 (default/linux/x86/10.0/desktop, gcc-4.4.3, glibc-2.11.2-r0, 2.6.32.9 i686)
=================================================================
System uname: Linux-2.6.32.9-i686-Pentium_III_-Coppermine-with-gentoo-1.12.13
Timestamp of tree: Sat, 04 Sep 2010 08:00:01 +0000
distcc 3.1 i686-pc-linux-gnu [disabled]
ccache version 2.4 [enabled]
app-shells/bash:     4.0_p37
dev-lang/python:     2.6.5-r3, 3.1.2-r4
dev-util/ccache:     2.4-r7
dev-util/cmake:      2.8.1-r2
sys-apps/baselayout: 1.12.13
sys-apps/sandbox:    1.6-r2
sys-devel/autoconf:  2.65
sys-devel/automake:  1.9.6-r3, 1.10.3, 1.11.1
sys-devel/binutils:  2.20.1-r1
sys-devel/gcc:       4.3.4, 4.4.3-r2
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6b
sys-devel/make:      3.81-r2
virtual/os-headers:  2.6.30-r1
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="* -@EULA"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=pentium3 -O3 -fomit-frame-pointer -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=pentium3 -O3 -fomit-frame-pointer -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests ccache distlocks fixpackages news parallel-fetch protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch"
GENTOO_MIRRORS="http://gentoo.chem.wisc.edu/gentoo/"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://192.168.1.2/gentoo-portage"
USE="a52 aac acl acpi alsa apache2 berkdb bluetooth branding bzip2 cairo cdparanoia cdr cli consolekit cracklib crypt curl cxx dbus dri dts dvb dvdr emboss encode exif firefox flac fortran gd gdbm gif hal iconv ieee1394 imlib ipv6 ivtv jpeg lcms libnotify lirc mad mikmod mmx mng modules mp3 mp4 mpeg mudflap mysql mysqli ncurses net nls nptl nptlonly ogg oggvorbis openmp openntpd pam pango pcre pdf perl png posix ppds pppd python qt qt3support qt4 readline reflection sasl session spell spl sse ssl startup-notification svg sysfs syslog tcpd tiff truetype unicode usb utf8 vorbis x264 x86 xcb xml xorg xulrunner xvid zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIRC_DEVICES="serial" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="mach64" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" 
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Comment 37 Stuart Shelton 2010-09-12 12:09:48 UTC
(In reply to comment #32)
> Stuart Shelton:
> what's the smallest set of CFLAGS you can reproduce with?
> including:
> CFLAGS="-O2 -march=pentium4 -pipe"
> CXXFLAGS="-O2 -march=pentium4 -pipe"
> please.
> 

Hi Robin,

Yup - tried that, same breakage.

I also have another server, a Mini-ITX machine with a VIA Nehemiah/C7 processor - this doesn't run MySQL directly but runs applications which themselves invoke mysqld to maintain their own private database files, and these applications are seeing behaviour more similar to what this bug opened with: MySQL doesn't crash immediately on startup, but segfaults as soon as a query is executed.

On the P4 machine, -fomit-frame-pointer allowed meaningful stack-traces, but didn't solve the problem.  I'm still rebuilding without -fofp on the C7-based machine... it may be some time ;)
Comment 38 Stuart Shelton 2010-09-12 12:12:43 UTC
(In reply to comment #37)
> (In reply to comment #32)
> > what's the smallest set of CFLAGS you can reproduce with?
> > including:
> > CFLAGS="-O2 -march=pentium4 -pipe"
> > CXXFLAGS="-O2 -march=pentium4 -pipe"
> > please.

Sorry - to be clear, when I said "reduced CFLAGS" in Comment #29, I should have said "minimal CFLAGS": I used those flags above to get the output reported in that comment.
Comment 39 Stuart Shelton 2010-09-12 12:16:34 UTC
Oh, and one final thing: the MiniITX/C7 machine which also experiences the problem is still running GCC 4.3.4:

$ gcc --version
gcc (Gentoo 4.3.4 p1.1, pie-10.1.5) 4.3.4

The P4 server is up to date, though:

$ gcc --version
gcc (Gentoo 4.4.3-r2 p1.2) 4.4.3
Comment 40 Christian Burger 2010-10-10 13:18:58 UTC
Created attachment 250109 [details]
mysqld.err excerpt
Comment 41 Christian Burger 2010-10-10 13:19:34 UTC
Created attachment 250111 [details]
emerge --info from defective system
Comment 42 Christian Burger 2010-10-10 13:20:04 UTC
Created attachment 250113 [details]
paludis --info from working system
Comment 43 Christian Burger 2010-10-10 13:21:26 UTC
Had the same problem here: Access with MySQL user 'root' works, but access with MySQL users having only table rights fails. As soon as I give a normal user global 'select'-rights (like 'root') the select-query works.

I have attached the error message from mysqld.err (not really helpful) and my 'emerge --info' output. The active gcc version was 4.4.3-r2.

I did the same update to mysql-5.1.50-r1 on a hardened system ('paludis --info' attached) where the bug did not manifest, which may be attributed to gcc 4.4.4-r2 being used, or maybe it was the hardened patch.

To fix the problem on the defective system I

1. added 'debug' to the MySQL USE-flags,
2. set '-O' and removed '-fomit-frame-pointer' in CFLAGS via /etc/portage/env/dev-db/mysql and
3. changed active gcc to version 4.3.2-r2.

I am pretty sure that this is not a problem with a library that needs to be rebuilt, because I ran revdep-rebuild twice before resorting to the mentioned fix.
Comment 44 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-10-28 20:33:06 UTC
Fixed already in dupe bug.

*** This bug has been marked as a duplicate of bug 335185 ***
Comment 45 Simon Koch 2010-10-28 21:09:52 UTC
(In reply to comment #44)
> Fixed already in dupe bug.
> 
> *** This bug has been marked as a duplicate of bug 335185 ***
> 

The fix from that bug applies only to gcc 4.5.  We're seeing the same breakage with gcc 4.4 here.
Comment 46 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-10-28 21:26:43 UTC
toolchain:
this looks identical to bug 335185, but it's gcc4.4, please advise if I should block omit-frame-pointer entirely?
Comment 47 SpanKY gentoo-dev 2010-10-28 21:44:48 UTC
as stated in the other bug, you should never be filtering flags unconditionally.  they need to be tied to specific gcc versions and bug #s.
Comment 48 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-10-29 05:39:51 UTC
We've got reports on the following now:
pentium3: 4.4 and newer is broken.
C7: 4.3.3 and newer are broken.

4.3.2 seems to work everywhere.

That's a large swath of the versions.
Comment 49 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2010-11-02 20:29:21 UTC
Ok, now blocked for GCC >= 4.3.3 on x86 only.
Comment 50 Ondrej Zary 2010-11-16 22:51:04 UTC
mysql-5.1.51 now works fine with gcc-4.4.4-r2