mariadb-10.5.13 crashes on startup with: ``` 2022-01-25 8:23:17 0 [Warning] mysqld: Couldn't allocate 130023424 bytes (Large/HugeTLB memory page size 2097152); errno 12; continuing to smaller size 2022-01-25 8:23:17 0 [Note] InnoDB: Uses event mutexes 2022-01-25 8:23:17 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 2022-01-25 8:23:17 0 [Note] InnoDB: Number of pools: 1 2022-01-25 8:23:17 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions 2022-01-25 8:23:17 0 [Note] mysqld: O_TMPFILE is not supported on /tmp/ (disabling future attempts) 2022-01-25 8:23:17 0 [Note] InnoDB: Using Linux native AIO 2022-01-25 8:23:17 0 [Note] InnoDB: Initializing buffer pool, total size = 6442450944, chunk size = 134217728 2022-01-25 8:23:17 0 [Note] InnoDB: Completed initialization of buffer pool 2022-01-25 8:23:17 0 [ERROR] InnoDB: corrupted TRX_NO 100020021bc6989 2022-01-25 8:23:17 0 [Note] InnoDB: Retry with innodb_force_recovery=5 2022-01-25 8:23:17 0 [ERROR] InnoDB: Plugin initialization aborted with error Data structure corruption 2022-01-25 8:23:17 0 [Note] InnoDB: Starting shutdown... 2022-01-25 8:23:18 0 [ERROR] Plugin 'InnoDB' init function returned error. 2022-01-25 8:23:18 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 2022-01-25 8:23:18 0 [Note] Plugin 'FEEDBACK' is disabled. 2022-01-25 8:23:18 0 [ERROR] Unknown/unsupported storage engine: InnoDB 2022-01-25 8:23:18 0 [ERROR] Aborting ``` … whilst reverting to mariadb-10.5.10-r2 works correctly, and all checks pass successfully. It looks as if a number of people are reporting the same issue on FreeBSD, although their patch doesn't apply to the 10.5.13 sources: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=257728 … this report suggests that mariadb up to 10.5.11 is okay, but 10.5.12 & 10.5.13 suffer from a bug causing this behaviour.
``` $ emerge --info Portage 3.0.28 (python 3.9.9-final-0, default/linux/amd64/17.1/no-multilib, gcc-11.2.0, glibc-2.33-r7, 5.16.0-gentoo x86_64) ================================================================= System uname: Linux-5.16.0-gentoo-x86_64-Intel-R-_Xeon-R-_CPU_E3-1240L_v5_@_2.10GHz-with-glibc2.33 KiB Mem: 66574456 total, 2225556 free KiB Swap: 25165808 total, 21809944 free Timestamp of repository gentoo: Thu, 20 Jan 2022 13:24:29 +0000 sh bash 5.1_p8 app-misc/pax-utils: 1.3.3::gentoo app-shells/bash: 5.1_p8::gentoo dev-lang/perl: 5.34.0-r6::gentoo dev-lang/python: 3.9.9-r1::gentoo sys-apps/baselayout: 2.7-r3::gentoo sys-apps/openrc: 0.44.10::gentoo sys-apps/sandbox: 2.25::gentoo sys-devel/gcc: 11.2.0::gentoo sys-devel/gcc-config: 2.5-r1::gentoo sys-libs/glibc: 2.33-r7::gentoo Repositories: gentoo location: /var/db/repo/gentoo sync-type: rsync sync-uri: rsync://rsync.europe.gentoo.org/gentoo-portage priority: -1000 eclass-overrides: srcshelton sync-rsync-verify-max-age: 24 sync-rsync-verify-metamanifest: no sync-rsync-extra-opts: sync-rsync-verify-jobs: 1 container location: /var/db/repo/container masters: gentoo ABI="amd64" ABI_X86="64" ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="*" ACCEPT_PROPERTIES="*" ACCEPT_RESTRICT="*" ADA_TARGET="gnat_2020" ARCH="amd64" AUTOCLEAN="yes" BINPKG_COMPRESS="bzip2" BOOTSTRAP_USE="unicode internal-glib pkg-config split-usr xml python_targets_python3_9 multilib" BROOT="" CALLIGRA_FEATURES="karbon sheets words" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=native" CFLAGS_amd64="-m64" CFLAGS_x32="-mx32" CFLAGS_x86="-m32" CHOST="x86_64-pc-linux-gnu" CHOST_amd64="x86_64-pc-linux-gnu" CHOST_x32="x86_64-pc-linux-gnux32" CHOST_x86="i686-pc-linux-gnu" CLEAN_DELAY="5" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" COLLISION_IGNORE="/boot/dtbs/* /lib/modules/*" COMMON_FLAGS="-O2 -pipe" CONFIG_PROTECT="/etc /etc/env.d /etc/hosts /lib/mdev /lib/rcscripts/addons /usr/share/gnupg/qualified.txt /var/bind /var/lib/unifi/data /var/www/localhost/htdocs/phpBB /var/www/localhost/htdocs/roundcube/config /var/www/localhost/htdocs/wordpress" CONFIG_PROTECT_MASK="/etc/X11/xkb /etc/ca-certificates.conf /etc/env.d /etc/fonts /etc/gconf /etc/gentoo-release /etc/php/apache2-php7.4/ext-active/ /etc/php/cgi-php7.4/ext-active/ /etc/php/cli-php7.4/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CPU_FLAGS_X86="mmx mmxext sse sse2 aes avx avx2 f16c fma3 pclmul popcnt rdrand sse3 sse4_1 sse4_2 ssse3" CXXFLAGS="-O2 -pipe -march=native" DEFAULT_ABI="amd64" DISTDIR="/var/cache/portage/dist" EDITOR="/usr/bin/vim" ELIBC="glibc" EMERGE_DEFAULT_OPTS="--verbose" EMERGE_WARNING_DELAY="10" ENV_UNSET="CARGO_HOME DBUS_SESSION_BUS_ADDRESS DISPLAY GOBIN GOPATH PERL5LIB PERL5OPT PERLPREFIX PERL_CORE PERL_MB_OPT PERL_MM_OPT XAUTHORITY XDG_CACHE_HOME XDG_CONFIG_HOME XDG_DATA_HOME XDG_RUNTIME_DIR" EPREFIX="" EROOT="/" ESYSROOT="/" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-docompress binpkg-dostrip binpkg-logs buildpkg clean-logs collision-protect compress-build-logs compressdebug config-protect-if-modified distlocks ebuild-locks fail-clean fakeroot fixlafiles ipc-sandbox merge-sync metadata-transfer multilib-strict network-sandbox news parallel-fetch pid-sandbox preserve-libs protect-owned qa-unresolved-soname-deps sandbox sfperms split-elog split-log splitdebug strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GCC_SPECS="" GENTOO_MIRRORS="http://distfiles.gentoo.org" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="" HOME="/home/stuart" INFOPATH="/usr/share/gcc-data/x86_64-pc-linux-gnu/11.2.0/info:/usr/share/info" INPUT_DEVICES="libinput" IUSE_IMPLICIT="abi_x86_64 prefix prefix-guest prefix-stack" KERNEL="linux" L10N="en en_GB" LANG="en_GB.UTF-8" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LC_MESSAGES="C" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LDFLAGS_amd64="-m elf_x86_64" LDFLAGS_x32="-m elf32_x86_64" LDFLAGS_x86="-m elf_i386" LESS="-R -M --shift 5" LESSOPEN="|lesspipe %s" LIBDIR_amd64="lib64" LIBDIR_x32="libx32" LIBDIR_x86="lib" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="BPF X86" LUA_SINGLE_TARGET="luajit" LUA_TARGETS="lua5-1" MAIL="/var/mail/stuart" MAKEOPTS="-j9" MANOPT="-a" MANPAGER="manpager" MANPATH="/usr/share/gcc-data/x86_64-pc-linux-gnu/11.2.0/man:/usr/lib64/php7.4/man/:/usr/local/share/man:/usr/share/man" MICROCODE_SIGNATURES="-S" MOTD_SHOWN="pam" MULTILIB_ABIS="amd64" MULTILIB_STRICT_DENY="64-bit.*shared object" MULTILIB_STRICT_DIRS="/lib32 /lib /usr/lib32 /usr/lib /usr/kde/*/lib32 /usr/kde/*/lib /usr/qt/*/lib32 /usr/qt/*/lib /usr/X11R6/lib32 /usr/X11R6/lib" MULTILIB_STRICT_EXEMPT="(perl5|gcc|binutils|eclipse-3|debug|portage|udev|systemd|clang|python-exec|llvm)" OFFICE_IMPLEMENTATION="libreoffice" PAGER="/usr/bin/less" PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin" PEAR_CACHEDIR="/var/cache/pear" PEAR_DOWNLOADDIR="/var/tmp/pear" PEAR_TEMPDIR="/var/tmp" PHP_TARGETS="php7-3 php7-4" PKGDIR="/space/cache/portage/pkg" PORTAGE_ARCHLIST="alpha amd64 amd64-linux arm arm-linux arm64 arm64-linux arm64-macos hppa ia64 m68k mips ppc ppc-macos ppc64 ppc64-linux riscv s390 sparc sparc-solaris sparc64-solaris x64-cygwin x64-macos x64-solaris x64-winnt x86 x86-linux x86-solaris x86-winnt" PORTAGE_BIN_PATH="/usr/lib/portage/python3.9" PORTAGE_COMPRESS_EXCLUDE_SUFFIXES="css gif htm[l]? jp[e]?g js pdf png" PORTAGE_CONFIGROOT="/" PORTAGE_DEBUG="0" PORTAGE_DEPCACHEDIR="/var/cache/edb/dep" PORTAGE_ELOG_CLASSES="log warn error" PORTAGE_ELOG_MAILFROM="portage@rufus.miltonroad.net" PORTAGE_ELOG_MAILSUBJECT="[portage] ebuild log for ${PACKAGE} on ${HOST}" PORTAGE_ELOG_MAILURI="stuart@shelton.me smtp.miltonroad.net" PORTAGE_ELOG_SYSTEM="echo save mail_summary" PORTAGE_FETCH_CHECKSUM_TRY_MIRRORS="5" PORTAGE_FETCH_RESUME_MIN_SIZE="350K" PORTAGE_GID="250" PORTAGE_GPG_SIGNING_COMMAND="gpg --sign --digest-algo SHA256 --clearsign --yes --default-key "${PORTAGE_GPG_KEY}" --homedir "${PORTAGE_GPG_DIR}" "${FILE}"" PORTAGE_INST_GID="0" PORTAGE_INST_UID="0" PORTAGE_INTERNAL_CALLER="1" PORTAGE_LOGDIR_CLEAN="find "${PORTAGE_LOGDIR}" -type f ! -name "summary.log*" -mtime +7 -delete" PORTAGE_NICENESS="3" PORTAGE_OVERRIDE_EPREFIX="" PORTAGE_PYM_PATH="/usr/lib/python3.9/site-packages" PORTAGE_PYTHONPATH="/usr/lib/python3.9/site-packages" PORTAGE_RSYNC_RETRIES="-1" PORTAGE_SYNC_STALE="30" PORTAGE_TMPDIR="/storage/system/var/tmp" PORTAGE_TMPFS="/dev/shm" PORTAGE_VERBOSE="1" PORTAGE_WORKDIR_MODE="0750" PORTAGE_XATTR_EXCLUDE="btrfs.* security.evm security.ima security.selinux system.nfs4_acl user.apache_handler user.Beagle.* user.dublincore.* user.mime_encoding user.xdg.*" PORT_LOGDIR="/var/log/portage" POSTGRES_TARGETS="postgres12 postgres13" PROFILE_ONLY_VARIABLES="ARCH ELIBC IUSE_IMPLICIT KERNEL USERLAND USE_EXPAND_IMPLICIT USE_EXPAND_UNPREFIXED USE_EXPAND_VALUES_ARCH USE_EXPAND_VALUES_ELIBC USE_EXPAND_VALUES_KERNEL USE_EXPAND_VALUES_USERLAND" PWD="/opt/containers/docker-gentoo-build" PYTHONDONTWRITEBYTECODE="1" PYTHON_SINGLE_TARGET="python3_9" PYTHON_TARGETS="python3_9" ROOT="/" ROOTPATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin" RPMDIR="/var/cache/portage/rpm" RUBY_TARGETS="ruby26 ruby27" SHELL="/bin/bash" SHLVL="1" SSH_CLIENT="2001:8b0:166:1:44ea:952a:a488:ffbd 50608 22" SSH_CONNECTION="2001:8b0:166:1:44ea:952a:a488:ffbd 50608 2001:8b0:166:1:529a:4cff:fe82:44a5 22" SSH_TTY="/dev/pts/0" SYMLINK_LIB="no" SYSROOT="/" TERM="screen-256color" TWISTED_DISABLE_WRITING_OF_PLUGIN_CACHE="1" UNINSTALL_IGNORE="/var/lib /var/www /usr/lib/debug" USE="acl acpi adns aio amd64 ao asm audit bash-completion bzip2 caps chroot cli crypt curl cxx dri encode expat fam ffmpeg fftw filecaps flac ftp gd gdbm gif gmp iconv icu id3tag idn inotify iodbc ipv6 ithreads javascript jemalloc jpeg jpeg2k json ladspa lame libatomic libedit libffi libglvnd libidn2 libsamplerate libtirpc libwww lighttpd lz4 lzma lzo mad mariadb mbox mdev memcached mmap mp3 mpfr mysql mysqli ncurses network nfs nls nntp nptl odbc ogg openssl pam pcre perl php png python readline rpc sctp seccomp sep-usr server slp smtp sndfile snmp speex spell split-usr sql sqlite ssl svg syslog theora threads tiff tls-heartbeat unicode unwind urandom uuid vim-syntax vorbis wavpack x264 xattr xinetd xml xvid xxhash yaml zlib zstd" ABI_X86="64" ADA_TARGET="gnat_2020" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="karbon sheets words" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2 aes avx avx2 f16c fma3 pclmul popcnt rdrand sse3 sse4_1 sse4_2 ssse3" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock greis isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="libinput" KERNEL="linux" L10N="en en_GB" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LLVM_TARGETS="BPF X86" LUA_SINGLE_TARGET="luajit" LUA_TARGETS="lua5-1" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php7-3 php7-4" POSTGRES_TARGETS="postgres12 postgres13" PYTHON_SINGLE_TARGET="python3_9" PYTHON_TARGETS="python3_9" RUBY_TARGETS="ruby26 ruby27" USERLAND="GNU" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy matrox v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" USER="stuart" USERLAND="GNU" USE_EXPAND="ABI_MIPS ABI_PPC ABI_S390 ABI_X86 ADA_TARGET ALSA_CARDS APACHE2_MODULES APACHE2_MPMS CALLIGRA_FEATURES CAMERAS COLLECTD_PLUGINS CPU_FLAGS_ARM CPU_FLAGS_PPC CPU_FLAGS_X86 CURL_SSL ELIBC FFTOOLS GPSD_PROTOCOLS GRUB_PLATFORMS INPUT_DEVICES KERNEL L10N LCD_DEVICES LIBREOFFICE_EXTENSIONS LLVM_TARGETS LUA_SINGLE_TARGET LUA_TARGETS MONKEYD_PLUGINS NGINX_MODULES_HTTP NGINX_MODULES_MAIL NGINX_MODULES_STREAM OFED_DRIVERS OFFICE_IMPLEMENTATION OPENMPI_FABRICS OPENMPI_OFED_FEATURES OPENMPI_RM PHP_TARGETS POSTGRES_TARGETS PYTHON_SINGLE_TARGET PYTHON_TARGETS QEMU_SOFTMMU_TARGETS QEMU_USER_TARGETS ROS_MESSAGES RUBY_TARGETS SANE_BACKENDS USERLAND UWSGI_PLUGINS VIDEO_CARDS VOICEMAIL_STORAGE XTABLES_ADDONS" USE_EXPAND_HIDDEN="ABI_MIPS ABI_PPC ABI_S390 ABI_X86 CPU_FLAGS_ARM CPU_FLAGS_PPC ELIBC KERNEL USERLAND" USE_EXPAND_IMPLICIT="ARCH ELIBC KERNEL USERLAND" USE_EXPAND_UNPREFIXED="ARCH" USE_EXPAND_VALUES_ARCH="alpha amd64 amd64-linux arm arm64 arm64-macos hppa ia64 m68k mips ppc ppc64 ppc64-linux ppc-macos riscv s390 sparc sparc64-solaris sparc-solaris x64-cygwin x64-macos x64-solaris x64-winnt x86 x86-linux x86-solaris x86-winnt" USE_EXPAND_VALUES_ELIBC="bionic Cygwin Darwin glibc mingw musl SunOS Winnt" USE_EXPAND_VALUES_KERNEL="Darwin linux SunOS Winnt" USE_EXPAND_VALUES_USERLAND="BSD GNU" USE_ORDER="env:pkg:conf:defaults:pkginternal:features:repo:env.d" VIDEO_CARDS="amdgpu fbdev intel nouveau radeon radeonsi vesa dummy matrox v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq proto steal rawnat logmark ipmark dhcpmac delude chaos account" gl_cv_type_time_t_bits_macro="no" ```
I have the same problem with the same version upgrade. My logs: 2022-02-14 23:51:50 0 [Note] InnoDB: Uses event mutexes 2022-02-14 23:51:50 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 2022-02-14 23:51:50 0 [Note] InnoDB: Number of pools: 1 2022-02-14 23:51:50 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions 2022-02-14 23:51:50 0 [Note] InnoDB: Using Linux native AIO 2022-02-14 23:51:50 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728 2022-02-14 23:51:50 0 [Note] InnoDB: Completed initialization of buffer pool 2022-02-14 23:51:50 0 [ERROR] InnoDB: corrupted TRX_NO 3666343539ae22e2 2022-02-14 23:51:50 0 [Note] InnoDB: Retry with innodb_force_recovery=5 2022-02-14 23:51:50 0 [ERROR] InnoDB: Plugin initialization aborted with error Data structure corruption 2022-02-14 23:51:50 0 [Note] InnoDB: Starting shutdown... 2022-02-14 23:51:50 0 [ERROR] Plugin 'InnoDB' init function returned error. 2022-02-14 23:51:50 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 2022-02-14 23:51:50 0 [Note] Plugin 'FEEDBACK' is disabled. 2022-02-14 23:51:50 0 [ERROR] Unknown/unsupported storage engine: InnoDB 2022-02-14 23:51:50 0 [ERROR] Aborting Reverting to mariadb-10.5.10-r2 works for me as well, but obviously staying there is not ideal. This seems very similar to the bug discussed here: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996028 But that's regarding a 10.3 upgrade. Since MariaDB supports multiple versions concurrently, still may be related, but none of the suggestions I could find worked for me. I've not tried the nuke everything and restore from backup option yet; hoping it doesn't come to that.
With some luck, this problem is fixed in the release from this week which should land shortly in Gentoo. Another workaround is creating a full backup (i.e. mysqldump --all-databases...), drop everything, stop mariadb, cleanup ib* files in your datadir, restart mariadb, upgrade, restore the dumps.
Two further updates on this: 1. If you create a backup using --insert-ignore and --replace, then `mysqldump` happily outputs 'REPLACE INTO' statements which mariadb can’t currently handle; 2. Restoring a full backup from 10.5.10-r2 to a fresh, empty 10.5.13 instance results in: ERROR 14 (HY000) at line 9374: Can’t change size of file (Errcode: -1073741376 “Internal error < 0 (Not system error)”) … this is not a happy release :(
Same error with mariadb-10.6.5-r1. I’m also finding that backups from 10.5.10-r2 aren’t restoring cleanly either.
(In reply to Thomas Deutschmann from comment #3) > With some luck, this problem is fixed in the release from this week which > should land shortly in Gentoo. > > Another workaround is creating a full backup (i.e. mysqldump > --all-databases...), drop everything, stop mariadb, cleanup ib* files in > your datadir, restart mariadb, upgrade, restore the dumps. ping
