MySQL has a plugin interface that can be used to add new storage engines. Building them out of tree may work but most engines are using mysql internal headers so building out of tree is not possible. so i have modified the mysql eclass to support external engines and plug them into the mysql build. To make this possible the external engines must be repacked (yes this a big downside) to follow a specific folderstucture the modified eclass can depend on. The eclass is just a show howto, not ready for primetime (but is already working on my production systems). a list of already included engines: pbxt-1.0.11-2-pre-ga transactional storage engine pbms-0.5.12-beta Scalable BLOB Streaming engine revision-0.2-alpha Versiond tuples storage engine soliddb-0081 solidDB Storage Engine innodb-1.0.6 innodb from www.innodb.com xtradb-1.0.6-10.1 Percona XtraDB enhanced version of the InnoDB storage engine, filesystem-0.1.1 A MySQL storage engine for arbitrary filesystem files sphinx-0.9.9 SQL full-text search engine spider-2.20 Transactional cluster storage engine oqgraph-0.7 a computation engine allowing hierarchies and more complex graph structures to be handled in a relational fashion more to come: q4m infobright awss3 memchached http vp qot .... changes that are on the todo list is figure out how to find out if a use flag is set: for ex. +pbxt -> build static, pbxt (not set) -> build dynamic, -pbxt -> disabled wrap all extraengines into the useflag extraengines thx Mario Reproducible: Always Steps to Reproduce:
Created attachment 233479 [details, diff] patch against the eclass in the mysql-overlay this is the patch against the eclass in the mysql-overlay
forgot something also extend the test suite to support the external storage engines to load the dynamic plugins
Overall I like the idea, but it needs a lot of polish. Couple of problems with this: 1. Some distributions of MySQL may bundle one or more of these engines already (eg XtraDB+PBXT in MariaDB). 2. Are you making those tarballs at http://ftp.disconnected-by-peer.at/mysql/ yourself? We need to endeavour to use upstream tarballs. 3. innodb-1.0.6 innodb from www.innodb.com should be named innodb_plugin, and must update the existing innodb_plugin if present. 4. Ability to run engine-specific src_configure, src_compile, src_install, pkg_config blocks. 5. The blocks like pbxt_available && use pbxt are critical for all of your plugin engines, as they might not be compatible with some release, and if you just have plain 'use pbxt' you'll get QA warnings.
One other thing that was being discussed, was making the external engines installable as separate packages, which I think might work much better.
> Couple of problems with this: > 1. Some distributions of MySQL may bundle one or more of these engines already > (eg XtraDB+PBXT in MariaDB). thats not a problem as it also checks if it is mariadb > 2. Are you making those tarballs at http://ftp.disconnected-by-peer.at/mysql/ > yourself? We need to endeavour to use upstream tarballs. yes these packages are selfemade as alredy described why in the bug desciption. > 3. innodb-1.0.6 innodb from www.innodb.com should be named innodb_plugin, and > must update the existing innodb_plugin if present. this is intentionally not named innodb_plugin as some other engines have hardcoced include paths > 4. Ability to run engine-specific src_configure, src_compile, src_install, > pkg_config blocks. thats not required as the plugins are changed to be configured in main configure process > 5. The blocks like pbxt_available && use pbxt are critical for all of your > plugin engines, as they might not be compatible with some release, and if you > just have plain 'use pbxt' you'll get QA warnings. > the pbxt blocks are not used they are removed
(In reply to comment #4) > One other thing that was being discussed, was making the external engines > installable as separate packages, which I think might work much better. > i alredy tried that and it did not work as most engines are using mysql internal headers
Re PBXT, this change of yours is what I meant: - if pbxt_available && use pbxt ; then + if use pbxt ; then
most changes to the upstream packages are makefile.am changes to add proper include paths remove all the autotools generetad files and change the name of the package to fit the required name and layout used by the eclass changes to the Makefile.am: ;include the proper name for shared or static (yes they differ sometimes, correct the includepatch from external to internal build, add some gcc-4.4 patches this modifications can later be moved to the gentoo mysql patches the only thing that still have to be done is the change of the package layout for ex. (recreating from memory) original pbxt package pbxt-1.0.6.. test src makefile.am plug.in new layout pbxt-gentoo-1.0.6.. storage src test Makefile.am plug.in setup.sh README.pbxt so the new package layout reflects the mysql package layout the setup.sh script is run to link the tests into the main test, apply test patches have not included the readme install at the moment the test for the external engines are failing when my laptop returnes from service this week i will create a svn for the packages with all the patches that i am applying to the original upstream packages
(In reply to comment #7) > Re PBXT, this change of yours is what I meant: > - if pbxt_available && use pbxt ; then > + if use pbxt ; then > yes this is a bug
(In reply to comment #4) > One other thing that was being discussed, was making the external engines > installable as separate packages, which I think might work much better. > I've tried to do this as well and got stonewalled by emerge to figure out and unpack the currently installed (MySQL|MariaDB). This is especially true if the tarball doesn't exist on the system. If we could agree on a framework of unpacking the current base, then I can hack something together to make separate packages.
(In reply to comment #10) > I've tried to do this as well and got stonewalled by emerge to figure out and > unpack the currently installed (MySQL|MariaDB). This is especially true if the > tarball doesn't exist on the system. > > If we could agree on a framework of unpacking the current base, then I can hack > something together to make separate packages. > as already described most engines depend on internel headers we can't depend on thats the reason i plug in the storage engines in the main build i had to do external builds for a propritary storage engine, most problems where created by the internal header changes so making the engins work with all mysql >=5.1 would be a horrible job (for ex. change of mysql_priv.h DEFINATIONS moved to my_proto.h (if i remember correctly) from 5.1.43 to 5.1.44) its enough to do the job for a specific mysql version thats my point of view (for now)
Please respin for latest eclasses & reopen.
