MySQL example (taken from MySQL documentation on www.mysql.com) compiles fine, but fails during runtime if compiled with gcc-4.4.1. It works fine with gcc prior to 4.4. I'm using AMD64 mashine and latest version of MySQL (result is the same with prior MySQL versions): $ emerge --info Portage 2.1.6.13 (default/linux/amd64/2008.0/desktop, gcc-4.3.4, glibc-2.10.1-r0, 2.6.31-rc6-git4 x86_64) ================================================================= System uname: Linux-2.6.31-rc6-git4-x86_64-Intel-R-_Core-TM-2_Duo_CPU_E4600_@_2.40GHz-with-gentoo-2.0.1 Timestamp of tree: Mon, 07 Sep 2009 01:15:02 +0000 ccache version 2.4 [enabled] app-shells/bash: 4.0_p28 dev-java/java-config: 2.1.9 dev-lang/python: 2.6.2-r1, 3.1.1 dev-util/ccache: 2.4-r8 dev-util/cmake: 2.6.4-r2 sys-apps/baselayout: 2.0.1 sys-apps/openrc: 0.4.3-r3 sys-apps/sandbox: 2.1 sys-devel/autoconf: 2.13, 2.63-r1 sys-devel/automake: 1.4_p6, 1.5, 1.9.6-r2, 1.10.2, 1.11 sys-devel/binutils: 2.19.1-r1 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.6a virtual/os-headers: 2.6.30-r1 ACCEPT_KEYWORDS="amd64 ~amd64" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=nocona" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/eselect/postgresql /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d" CXXFLAGS="-O2 -pipe -march=nocona" DISTDIR="/usr/portage/distfiles" FEATURES="ccache distlocks fixpackages parallel-fetch protect-owned sandbox sfperms splitdebug strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://ftp.swin.edu.au/gentoo ftp://ftp.swin.edu.au/gentoo ftp://mirror.pacific.net.au/linux/Gentoo http://gentoo.arcticnetwork.ca/source/ http://gentoo.osuosl.org/" LANG="C" LDFLAGS="-Wl,-O1" LINGUAS="en ru" MAKEOPTS="-j3" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/portage/local/my" SYNC="rsync://rsync.au.gentoo.org/gentoo-portage" USE="7zip X X509 a52 aac accessibility acl acpi alsa amd amd64 aspell autoipd avahi bash-completion beagle berkdb branding bzip2 cairo cardbus cdr cdrkit cdrom cli consolekit cpudetection cpufreq cracklib crypt css cups curl dbus devhelp divx dns dri dts dv dvd dvdarchive dvdchapjump dvdnav dvdr dvi dxr2 embedded emboss encode esd fam fame fat ffmpeg filepicker firefox firefox3 flac foomaticdb freetds ftp fuse gdbm gif gimp git glitz gmail gnome gpm gstreamer gtk gzip h323 hal hpn iconv ieee1394 imap ipv6 irc isdnlog java javascript jpeg jpeg2k lame ldap libnotify lm_sensors lzo mad matroska mdnsresponder-compat mikmod mime mjpeg mmap mmx mmxext mng mp3 mp4 mp4live mpeg mpeg2 mpi mplayer mudflap multilib mysql nautilus ncurses nfs nls nntp nptl nptlonly nsplugin ntfs obex odbc ogg opengl openmp pam pango pcmcia pcre pdf perl pipechan png postgres ppds pppd python qt3 qt3-static qt3support qt4 quicktime rar readline realmedia reflection reiser4 reiserfs rpm rtc samba scanner sdl session sip slang smp sockets socks5 sourceview spell spl sql sqlite sse sse2 ssl ssse3 startup-notification stream subtitles subversion svg sysfs tcpd tga theora thesaurus threads threadsafe thunar tiff transcode trayicon truetype type1 unicode usb v4l v4l2 valgrind vnc vorbis wav wavpack webdav webkit wma wmf wmp wxwindows x264 xanim xcb xcomposite xfs xine xinetd xml xmlreader xmlwriter xorg xosd xpm xprint xslt xulrunner xv xvid xvmc zip zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw copy dshare dsnoop extplug file hooks ladspa lfloat linear meter mulaw multi null rate route share shm asym dmix empty iec958 ioplug mmap_emul plug softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" APACHE2_MPMS="peruser" ELIBC="glibc" INPUT_DEVICES="keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en ru" USERLAND="GNU" VIDEO_CARDS="nvidia" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS The C++ source file and CMake project files are attached. Reproducible: Always Steps to Reproduce: 1. Create mysql database 'demo' 2. In the directory with provided two files: cmake . && make 3. ./bind_test Actual Results: Sizeof bind[] 336 prepare, INSERT successful total parameters in INSERT: 3 mysql_stmt_execute(), 1 failed Incorrect arguments to mysql_stmt_execute Expected Results: Sizeof bind[] 336 prepare, INSERT successful total parameters in INSERT: 3 total affected rows(insert 1): 1 total affected rows(insert 2): 1
Created attachment 203334 [details] C++ source code
Created attachment 203335 [details] CMake project file
As this may be a gcc bug, a few more details are required. At very least, see what gcc command triggers this bug. On x86, I couldn't reproduce this bug, neither by: g++ -o bind_test bind_test.cpp -lmysqlclient_r nor by: g++ -O3 -o bind_test bind_test.cpp -lmysqlclient_r Even trying your cmake project didn't help.
Some more information: Compiling bind_test.cpp with different gcc options doesn't make any difference, but recompiling mysql does. I've just ran into hal/glib problem - hal stops working if glib is compiled with gcc-4.4.1 with -O3 option. Recompiling glib with the same gcc-4.4.1 and -O2 option does the trick. I'll try recompiling mysql with -O2 tomorrow, and get back to you.
The problem is twofold actually. Due to some unsafe/buggy coding, MySQL requires -fno-strict-aliasing to compile w/ gcc 4.4. The mysql eclass only sets this for the c++ part (CXXFLAGS) but forgets about the c part (CFLAGS) and thus libmysqlclient (being pure C) gets miscompiled and all sorts of bad things happen... non-functional variable binding being one of them. Until the eclass is fixed, just appened -fno-strict-aliasing to your mysql CFLAGS... and only for that purpose. Hope that helps.
Since this bug will result in some kind of database corruption (wrong data and/or no data inserted), it should be marked critical and fixed asap.
Some more details. In some of my experiments with simple insert statements using two parameters, up to seven inserts worked fine (generated no error) but the inserted data was '0' for strings and 0 for integers regardless of parameter values. Recompiling with gcc option -O2 instead of -O3 didn't help.
Alexey, there is no need for testing any more. :-) Please have a look at my comments above from yesterday. Just to further clearify for you: -fstrict-aliasing is enabled at -O2 and above. So if you compile with -O1 or below, it should basically be the same as compiling with -fno-strict-aliasing and >= -O2. I am still somewhat astounded this fix didn't make it into the eclass along with a empty revbump for the mysql package since it's a serious bug which can lead to silently corrupted databases... even though it only affects gcc 4.4. users.
I committed this to the eclass: + # bug #283926, with GCC4.4, this is required to get correct behavior. + append-flags -fno-strict-aliasing However, now both of you get to go and trace what needs to be fixed in the MySQL source so we don't need the flag. You got pretty far already in the bind variables. Tracing those should be less than 500 lines of source.
(In reply to comment #8) > I am still somewhat astounded this fix didn't make it into the eclass along > with a empty revbump for the mysql package since it's a serious bug which can > lead to silently corrupted databases... even though it only affects gcc 4.4. > users. The bug has only been open for 24 hours. Why are you astounded? I don't spend every waking moment on Gentoo, as much as I would like to, I cannot afford to.
Robin, sorry, I did not want to criticize your personally or anything. I work on open source projects too and I know we are all doing this voluntarily in our free time and usually there is not much reward in doing this... quite the contrary actually, when users start to complain and get impatient. That said, MySQL is an important piece of software and any bug which can and will lead to silently corrupted databases, is serious. I know the bug has only been open for 24 hours but that bug shouldn't have been there in the first place. I know Gentoo lacks the resources to test everything w/ every supported compiler... and that's actually part of the problem. To make a long story short: Thanks for reacting swiftly. I was just worried that this serious bug would stay open for days or weeks to come like it has happened with other bugs in different areas of Gentoo in the past. May I suggest that you do an empty revbump for the MySQL packages to force people to recompile? Otherwise every one running a gcc 4.4 compiled mysql package, will continue to do so without knowing what has hit 'em until it's too late.
(In reply to comment #11) > That said, MySQL is an important piece of software and any bug which can and > will lead to silently corrupted databases, is serious. I know the bug has only > been open for 24 hours but that bug shouldn't have been there in the first > place. I know Gentoo lacks the resources to test everything w/ every supported > compiler... and that's actually part of the problem. Please submit a patch that merges Alexey's testcase into mysql's internal testsuite. I ran the testsuite when I bumped 5.0.84 a few days ago, but didn't notice any problems. (The ebuild has instructions for how to run the testsuite). > To make a long story short: Thanks for reacting swiftly. I was just worried > that this serious bug would stay open for days or weeks to come like it has > happened with other bugs in different areas of Gentoo in the past. For a bug to be fixed promptly: 1. It has to be noticed by the maintainer. 2. It has to affect them personally, or strongly prove itself. 3. Includes a fix either trivial to implement, or as a well-formed patch. For #1 above, I hadn't gotten to my bugs yet today. This was in the first pass of them. For #2 it doesn't effect me at the moment, but Alexey provided an absolutely excellent testcase, with sufficent information to say that it was a real problem (and not due to some configuration of his system, which accounts for probably 10-15% of the "bugs" reported in bugzilla). For #3: append-flags and a revbump were trivial to do. > May I suggest that you do an empty revbump for the MySQL packages to force > people to recompile? Otherwise every one running a gcc 4.4 compiled mysql > package, will continue to do so without knowing what has hit 'em until it's > too late. I did this about 5 minutes after the eclass commit (I had to wait for it to compile so I could run Alexey's testcase against it), and the commit logs and CIA.vc confirm that.
Robin, unfortunately your revbump caused other trouble. Since the minor version changed, not all patches get applied to the source due to the index file with mysql-extras which strictly limits some patches to 5.00.84.00. This also causes a failure to occur: bug #284078.
Robin, one more thing: I was considering going over the code to see if I could safely fix the aliasing issues. Before I had a look over at OpenSuse and Fedora how they deal with the issue and they too append -fno-strict-aliasing. Also, I read in some recent bugs on MySQL's bug tracking system that gcc 4.4 is not officially supported yet. Once they do, there are only two options for them: by default use -fno-strict-aliasing or fix their code once and for all. I don't think we should try to fix this ourselves because we obviously don't know the code as well as the devs do for this beast and it's better to be on the safe side by appending a flag instead of using a patch which might miss a spot or two or cause other issues which could again lead to corruption. That's just my 2 cents on this. I hope you agree. Thanks again for your time and work on this and all of Gentoo. It's greatly appreciated!
In that case, why on x86 it did work, even with -O3 ? Or is that a case of 64bit only problem, where strict aliasing breaks due to sizeof(int)!=sizeof(void *) ? And on that note, during mysqld start, I'm seeing a funny warning: option 'max_join_size': unsigned value 18446744073709551615 adjusted to 4294967295 That's FFFFFFFFFFFFFFFF and FFFFFFFF respectively. I don't see that value in config file, so it looks like upstream doing not quite sane things (well, something that does work, but makes you worried).
(In reply to comment #15) > In that case, why on x86 it did work, > even with -O3 ? > Or is that a case of 64bit only problem, > where strict aliasing breaks due to > sizeof(int)!=sizeof(void *) ? I've just tested it on x86 (32bit) machine with the same exact result. So, we are talking about problem confirmed on at least x86 and amd64 archs.
Interesting... g++ -O2 -Wall -o bind_test bind_test.cpp -lmysqlclient_r bind_test.cpp: In function ‘int main()’: bind_test.cpp:24: warning: unused variable ‘mysql2’ ./bind_test Sizeof bind[] 180 prepare, INSERT successful total parameters in INSERT: 3 total affected rows(insert 1): 1 total affected rows(insert 2): 1 Done with gcc 4.4.1 and mysql 5.0.84 (why do I get get feeling it will be just like that firefox/sqlite crash that I've never seen). And on a semi-related note, as I said in an other mysql bug awhile ago, mysql.eclass should notice that eautoreconf has been recursive for quite awhile, so it's run twice for innobase subdir (unless it's needed for an older version of mysql).
(In reply to comment #13) > Robin, unfortunately your revbump caused other trouble. Since the minor version > changed, not all patches get applied to the source due to the index file with > mysql-extras which strictly limits some patches to 5.00.84.00. This also causes > a failure to occur: bug #284078. Typo there, fixed that now.
(In reply to comment #14) > Robin, one more thing: I was considering going over the code to see if I could > safely fix the aliasing issues. Before I had a look over at OpenSuse and Fedora > how they deal with the issue and they too append -fno-strict-aliasing. Also, I > read in some recent bugs on MySQL's bug tracking system that gcc 4.4 is not > officially supported yet. Once they do, there are only two options for them: by > default use -fno-strict-aliasing or fix their code once and for all. MySQL's official support comes in two flavours: - If you pay, you need to run their binaries if you want full scale support. - For the open source users, you come after paying customers, unless your needs happen to align very well with one of said customers. > I don't think we should try to fix this ourselves because we obviously don't > know the code as well as the devs do for this beast and it's better to be on > the safe side by appending a flag instead of using a patch which might miss a > spot or two or cause other issues which could again lead to corruption. If it weren't for us going and tracing the issues, and fixing them, they'd NEVER get fixed. Over the years, at least 8 patches from Gentoo have made it to upstream MySQL. GCC and hardened fixups account for most of these, but there have also been bugs, security issues, and build system cleanups.
Rafał/Alexey: Write a patch that merges the testcase in first, and then we can start to trace where it needs to be fixed (Since we can run the testcase in an automated fashion, we can isolate a lot).
No response from users. 5.1.x series shouldn't have these problems either now.
