Problem: rc-script does not recognizes well if mysqld started or not. When i turn on my computer from time to time i see an error that mysqld does not started. If i want to run manually /etc/init.d/mysql script it shows the same error as while booting. If I use command: # ps | grep mysqld i got one row with mysql started. I have to kill that process and try again /etc/init.d/mysql. Now it starts ok and all dependencies (like f.e. mydns) are starting well. Steps to reproduce: 1. reboot your computer 2. if mysql start ok reboot once more (goto 1) 3. if mysql reports that it not started (goto 4) 4. login to shell and check output of: ps | grep mysqld 5. if there is one line with mysqld use: killall -9 mysqld 6. # /etc/init.d/mysql start 7. mysql should report normally that it started. How it should be to be ok? The problem happens in previous versions of mysql to 5.0.17 (the newest was not tested yet, but as i think its not an mysql issue) Mysql should start everytime when i turn on my computer so programs depending on it can also start while boot.
It may be a timing issue, the startup script abort to listen for a started mysqld too soon, this may happen with big tables, expecially innodb. try to put the following in your "/etc/conf.d/mysql" config file: --------- DEBUG=4 STOPTIMEOUT=60 mysql_slot_0=( "startup_timeout=60" ) --------- This will increase the start/stop timings, making the script much more verbose. Hopefully showing if there are errors too (but the error log is'nt perfect at the moment). You could also cut and paste on the command line the exact command used to start mysqld. Just prepend it with a "env -i " to clear the environment, # env -i mysqld --... As a last note, not as g.developer but as a somewhat db admin, never ever kill a database server with a "-9" signal, always use "-15" and be patient. The only case InMyHumbleKnowledge where a -15 can fail to stop is a broken disk or big IO problems. "-9" tell a process to shutdown _immediately_ so without clearing caches, writing to disk the last data, all things that a db don't like very much. BTW you're right, this should not be a mysql issue.
I've added requested lines to /etc/conf.d/mysql and rebooted few times Behaviour of rc-script is a bit weird. One time it failed without any couting and after when i wanted manually start mysql counted till 60 and crashed I got to kill mysqld process to get it working. but counter went to 16. One time it started with time counter 3 env -i mysqld -- ... was very very slow but mysql started then. Please notice that i did have small databases which all does not take more that 10MB of disc space.
(In reply to comment #2) > I've added requested lines to /etc/conf.d/mysql and rebooted few times > > Behaviour of rc-script is a bit weird. > One time it failed without any couting And without messages? > and after when i wanted manually start mysql counted till 60 and crashed Ok, this is normal if there is already an instance running > I got to kill mysqld process to get it working. but counter went to 16. > One time it started with time counter 3 Instead this is strange, with only few MB of data it should be up in 0-2 seconds on modern hardware (unless the cpu it's already at 99%) > > env -i mysqld -- ... was very very slow but mysql started then. idem > > Please notice that i did have small databases which all does not take more that > 10MB of disc space. > what about moving temporary the data outside of /var/lib/mysql (assumed your database are located there) and try to recreate from scratch the default DBs ? the command to issue is: "emerge -a --config mysql" . If it start fine the may be a problem of corrupted indexes. In this case a "REPAIR TABLE `...` " may suffice If it is still slow/buggy, try the latest 5.0.18, and/or attach my.cnf and emerge --info
The message while failure is ######### starting mysqld with: /usr/sbin/mysqld --basedir=/usr --datadir=/var/lib/mysql --pid-file=/var/run/mysqld/mysqld.pid --socket=/var/run/mysqld/mysqld.sock MySQL NOT started (1) ######### But for last few bootups i got counter at number 3 and mysql starts ok. I've checked if any table is broken but all in all databases are ok. I'll try to move data outside the /var/log/mysql later because now i can't turn off mysql (working hours). Here is my my.cnf: ######### # /etc/mysql/my.cnf: The global mysql configuration file. # $Header: /var/cvsroot/gentoo-x86/dev-db/mysql/files/my.cnf-4.1,v 1.2 2005/07/2 6 17:14:23 vivo Exp $ # The following options will be passed to all MySQL clients [client] #password = your_password port = 3306 socket = /var/run/mysqld/mysqld.sock [mysql] character-sets-dir=utf8 default-character-set=utf8 [mysqladmin] character-sets-dir=utf8 default-character-set=utf8 [mysqlcheck] character-sets-dir=utf8 default-character-set=utf8 [mysqldump] character-sets-dir=utf8 default-character-set=utf8 [mysqlimport] character-sets-dir=utf8 default-character-set=utf8 [mysqlshow] character-sets-dir=utf8 default-character-set=utf8 [myisamchk] character-sets-dir=utf8 [myisampack] character-sets-dir=utf8 # use [safe_mysqld] with mysql-3 [mysqld_safe] err-log = /var/log/mysql/mysql.err # add a section [mysqld-4.1] or [mysqld-5.0] for specific configurations. [mysqld] character-set-server = utf8 default-character-set = utf8 user = mysql port = 3306 socket = /var/run/mysqld/mysqld.sock pid-file = /var/run/mysqld/mysqld.pid log-error = /var/log/mysql/mysqld.err basedir = /usr datadir = /var/lib/mysql skip-locking #skip-innodb key_buffer = 16M max_allowed_packet = 1M table_cache = 64 sort_buffer_size = 512K net_buffer_length = 8K read_buffer_size = 256K read_rnd_buffer_size = 512K myisam_sort_buffer_size = 8M language = /usr/share/mysql/english #security: #using "localhost" in connects use sockets by default skip-networking #bind-address = 127.0.0.1 #log-bin server-id = 1 # Point the following paths to different dedicated disks tmpdir = /tmp/ #log-update = /path-to-dedicated-directory/hostname # you need debug use flag enabled to use this ones. # if needed uncomment them, start the server and issue # #tail -f /tmp/mysqld.sql /tmp/mysqld.trace # this will show you *exactly* what's appening in your server ;) #log = /tmp/mysqld.sql #gdb #debug = d:t:i:o,/tmp/mysqld.trace #one-thread # Uncomment the following if you are using BDB tables #bdb_cache_size = 4M #bdb_max_lock = 10000 # The following is the InnoDB configuration # if you wish to disable innodb instead # uncomment just the next line #skip-innodb # # the rest of the innodb config follows: # don't eat too much memory, we're trying to be safe on 64Mb boxes. # you might want to bump this up a bit on boxes with more RAM innodb_buffer_pool_size = 16M # this is the default, increase if you have lots of tables innodb_additional_mem_pool_size = 2M # # i'd like to use /var/lib/mysql/innodb, but that is seen as a database :-( # and upstream wants things to be under /var/lib/mysql/, so that's the route # we have to take for the moment #innodb_data_home_dir = /var/lib/mysql/ #innodb_log_arch_dir = /var/lib/mysql/ #innodb_log_group_home_dir = /var/lib/mysql/ # you may wish to change this size to be more suitable for your system # the max is there to avoid run-away growth on your machine innodb_data_file_path = ibdata1:10M:autoextend # we keep this at around 25% of of innodb_buffer_pool_size # sensible values range from 1MB to (1/innodb_log_files_in_group*innodb_buffer_pool_size) innodb_log_file_size = 5M # this is the default, increase if you have very large transactions. innodb_log_buffer_size = 8M # this is the default, and won't hurt you. # you shouldn't need to tweak it. set-variable = innodb_log_files_in_group=2 # see the innodb config docs, the other options are not always safe innodb_flush_log_at_trx_commit = 1 innodb_lock_wait_timeout = 50 [mysqldump] quick max_allowed_packet = 16M [mysql] # Remove the next comment character if you are not familiar with SQL #safe-updates [isamchk] key_buffer = 20M sort_buffer_size = 20M read_buffer = 2M write_buffer = 2M [myisamchk] key_buffer = 20M sort_buffer_size = 20M read_buffer = 2M write_buffer = 2M [mysqlhotcopy] interactive-timeout #vim: set tabstop=4 : #vim: set shiftwidth=4 : ##################### And my emerge info output: ################ Portage 2.0.53 (default-linux/x86/2005.1, gcc-4.0.2, glibc-2.3.6-r1, 2.6.14-suspend2-r9 i686) ================================================================= System uname: 2.6.14-suspend2-r9 i686 Pentium III (Coppermine) Gentoo Base System version 1.12.0_pre12 distcc 2.18.3 i686-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled] ccache version 2.4 [enabled] dev-lang/python: 2.4.2 sys-apps/sandbox: 1.2.17 sys-devel/autoconf: 2.13, 2.59-r7 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r1 sys-devel/binutils: 2.16.1-r1 sys-devel/libtool: 1.5.22 virtual/os-headers: 2.6.11-r3 ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-O2 -march=pentium3 -pipe -fomit-frame-pointer -fforce-addr" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/bind /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/init.d /etc/initng /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -march=pentium3 -pipe -fomit-frame-pointer -fforce-addr" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig ccache distclean distlocks sandbox sfperm sfperms strict userpriv usersandbox" GENTOO_MIRRORS="http://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo/ http://ftp.uni-erlangen.de/pub/mirrors/gentoo http://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/ http://gentoo.ITDNet.net/gentoo" LANG="pl_PL" LC_ALL="pl_PL" LDFLAGS="-s" LINGUAS="pl" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/home/nysander/gentoo_svn" SYNC="rsync://ftp.belnet.be/gentoo-portage/" USE="x86 X aalib acpi alsa apache2 audiofile avi berkdb bitmap-fonts bzip2 bzlib cdb crypt cups curl dba dbase divx4linux dlz eds emboss encode esd exif expat fam foomaticdb fortran ftp gd gdbm gif gimpprint glut gmp gpm gstreamer gtk gtk2 iconv idn imagemagick imlib irda java jpeg lcms ldap libg++ libwww local mad mhash mikmod mime mmx mmx2 mng motif mozsvg mp3 mpeg mpm-worker mysql ncurses nls ogg oggvorbis openal opengl openssh pam pcntl pcre pdflib perl php pic png posix python qt quicktime readline sdl session simplexml slang sndfile soap sockets speex spell spl sqlite sse sse2 ssl svg svga tcltk tcpd threads tiff tokenizer truetype truetype-fonts type1-fonts udev unicode usb utf8 vhosts vorbis wmf xchattext xine xml xml2 xmlrpc xpm xsl xv xvid zlib linguas_pl userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET ############### If you need more information please ask, because I don't have idea what could be also helpful in resolving this problem.
Upgrade to mysql 5.0.18 and behaviour the same as it was. Fails to start with no error as stated before.
another user reported me this problem with sys-apps/baselayout-1.12.0_pre12 , the prob. disappeared with sys-apps/baselayout-1.12.0_pre13-r1 please test this
Already got requested baselayout version --------------------- emerge -ptv baselayout These are the packages that I would merge, in reverse order: Calculating dependencies ...done! [ebuild R ] sys-apps/baselayout-1.12.0_pre13-r1 -bootstrap -build -static +unicode 0 kB ---------------------- Also could add that lines requested in comment #1 are not taken into account while booting. So mysql does not count until it start. If i then killall -15 mysqld and /etc/init.d/mysql start counting and it at maximum to 5 to start but the average is 2
I still hope it's a bug in my start script, so please could you try to download wget "http://www.gentoo.org/cgi-bin/viewcvs.cgi/dev-db/mysql/files/mysql-slot.rc6-r2?rev=1.2" -O "/etc/init.d/mysql" chmod +x /etc/init.d/mysql ? This script is a simplified version of -r1, after the decision to follow the net.* rc script example (for the upcoming slotted mysql). If it's a bug of the script may be it has been removed with the removed with the 40 removed lines (there was some trick there). --- /etc/conf.d/mysql DEBUG=4 mysql_slot_0=( "startup_timeout=15" "stop_timeout=60" ) --- /etc/conf.d/mysql Another thing, I've supposed you use innodb tables, if not you could safely uncomment the "skip-innodb" option in my.cnf please report success or not whenever you can restart your server
Created attachment 76796 [details] 6 bootups output data and checks done after every bootup Attachment contains data from 6 bootups with output of init script and done checks after booting system. Not including as a comment as it is very long file (plain text) My conclusions are (and/or): 1. While stopping mysqld initscript do not check the state of mysqld.sock 2. While stopping mysqld initscript do not wait for proper closing mysqld Have no idea about results of BOOTUP #3 and #5 you have to interpret them on yourself
this outputs are without set stop-timeout. I'll check (as it is now 3:32 am) later today how the script is running with this variable set.
Ok, time for http://www.gentoo.org/cgi-bin/viewcvs.cgi/dev-db/mysql/files/mysql-slot.rc6-r2?rev=1.3 there was a race condition in the start code, where both mysqld and start-stop-daemon were managing the pid file, this did result in a empty pid file sometimes. (In reply to comment #9) > Created an attachment (id=76796) [edit] > 6 bootups output data and checks done after every bootup > > Attachment contains data from 6 bootups with output of init script and done > checks after booting system. Not including as a comment as it is very long file > (plain text) good job > > My conclusions are (and/or): > 1. While stopping mysqld initscript do not check the state of mysqld.sock This is intended, mysqld.sock is the deleted before starting the server shutdown, immediately after receiving the kill -15 signal > 2. While stopping mysqld initscript do not wait for proper closing mysqld start-stop-daemon --stop should stop the server alone, the while ... done loop after should never be executed. But it wait for a proper closing mysql, cheking the result of "kill" program.
I'll do requested tests today but have to ask what should i have in /etc/conf.d/mysql for this tests: Actual: --------- DEBUG=4 STOPTIMEOUT=60 mysql_slot_0=( "startup_timeout=60" ) --------- or this one: --- DEBUG=4 mysql_slot_0=( "startup_timeout=15" "stop_timeout=60" ) ---
The second one: -------- DEBUG=4 mysql_slot_0=( "startup_timeout=15" "stop_timeout=60" ) -------- an updated conf.d/mysql example could be found at http://www.gentoo.org/cgi-bin/viewcvs.cgi/dev-db/mysql/files/mysql-slot.conf.d-r2?rev=1.2 or in "${PORTDIR}/dev-db/mysql/files/mysql-slot.conf.d-r2"
Created attachment 76849 [details] next 6 bootups ... procedure the same This results you have to check on yourself as i have no idea what happens now. Problem was not yet resolved.
I used this file http://www.gentoo.org/cgi-bin/viewcvs.cgi/dev-db/mysql/files/mysql-slot.rc6-r2?rev=1.3 with that configuration: -------- DEBUG=4 mysql_slot_0=( "startup_timeout=15" "stop_timeout=60" ) -------- Maybe it could help as I haven't said it before and you didn't asked for it but i got this: RC_PARALLEL_STARTUP="yes" in my /etc/conf.d/rc
This issue may have been fixed by baselayout-1.12.0_pre14
I do not see such baselayout on http://packages.gentoo.org
It takes about 2 hours from the time a developer commits a package to get to portage mirrors and packages.gentoo.org
Upgraded to baselayout-1.12.0_pre14. Then rebooted 6 times with configuration of /etc/conf.d/mysql as below: ---------------- DEBUG=4 mysql_slot_0=( "startup_timeout=15" "stop_timeout=60" ) ---------------- Counter was counting from 149 to 130 - 120 and everytime i got successful start of mysql Then tried this configuration and booted next 6 times ---------------- DEBUG=4 #mysql_slot_0=( # "startup_timeout=15" # "stop_timeout=60" #) ---------------- This time counter was counting from 49 to 32 - 23 And this time also all mysql starts were successful So in my opinion together we got resolved this bug. Thank you guys.
...
... closing, thanks for the patience
I cant say if it was set to crash on this or it is a bug. I got my laptop hang on mplayer and i had to hard reset it. and in startup i got this: * Starting service mysql dbg: >>> start() dbg: >>> get_slot_config() dbg: myservice mysql dbg: srv dbg: srv_slot 0 dbg: srv_num dbg: tmp_eval mysql_slot_[@] dbg: conf_d_parameters dbg: MY_SUFFIX dbg: chroot dir => dbg: niceness => dbg: basedir => /usr dbg: datadir => /var/lib/mysql dbg: pidfile => /var/run/mysqld/mysqld.pid dbg: socket => /var/run/mysqld/mysqld.sock dbg: Unamanged => dbg: <<< get_slot_config() ok "/var/run/mysqld/mysqld.pid" and/or "/var/run/mysqld/mysqld.sock" are still present. Please check that no server is running and remove them. * FAILED to start service mysql! So am i obliged do check it myself and do what script said or should initscript do those checks on itself?
(In reply to comment #22) > "/var/run/mysqld/mysqld.pid" and/or "/var/run/mysqld/mysqld.sock" are still > present. > Please check that no server is running and remove them. > * FAILED to start service mysql! > So am i obliged do check it myself and do what script said or should initscript > do those checks on itself? The rc script you have installed are the next generation one ;-) , this is the new (intended) behaviour. But I'm still uncertain on what will be done in the final revision, there are pros and cons. If mysql (like every database) is not cleanly closed (i.e. for a crash or kill -9) then it's good advice know that and check the data for integrity. Probably I will deal to document this and make it more clear, for the moment do exactly that: - check your data for (rare with mysql) corruption - rm -f /var/run/mysqld/mysqld.pid /var/run/mysqld/mysqld.sock - /etc/init.d/mysql start P.S. now should be safe to comment again the #DEBUG=4 in /etc/conf.d/mysql
