MySQL to PostgreSQL real-time replication. https://pgchameleon.org/ https://github.com/the4thdoctor/pg_chameleon Reproducible: Always
Do you want to maintain this package in Gentoo? I'm sure we can help with the ebuild.
I don't mind giving it a go, but I don't know the first thing about packaging python apps and/or how their dependencies work. I know this uses pip install and virtual env, but I know how that'd work in an ebuild. Are there are good how-tos/guides on this and I'll give it a go.
https://wiki.gentoo.org/wiki/Basic_guide_to_write_Gentoo_Ebuilds https://devmanual.gentoo.org/ In a local repository let's create an ebuild pg_chameleon-2.0.12.ebuild: {code} # Copyright 2020 Gentoo Authors # Distributed under the terms of the GNU General Public License v2 EAPI=7 PYTHON_COMPAT=( python3_{6..9} ) inherit distutils-r1 DESCRIPTION="MySQL to PostgreSQL replica system" HOMEPAGE="https://pgchameleon.org/" SRC_URI="mirror://pypi/${PN:0:1}/${PN}/${P}.tar.gz" LICENSE="BSD-2" SLOT="0" KEYWORDS="~amd64 ~x86" IUSE="" DEPEND="" RDEPEND="${DEPEND}" BDEPEND="" {code} This can be placed in a local overlay so that we can install it with emerge: https://wiki.gentoo.org/wiki/Handbook:AMD64/Portage/CustomTree#Defining_a_custom_repository https://wiki.gentoo.org/wiki/Custom_repository Now the package can be installed (emerge -av dev-db/pg_chameleon) but it won't run as it misses the dependencies: {code} # chameleon Traceback (most recent call last): File "/usr/lib/python-exec/python3.8/chameleon", line 5, in <module> exec(compile(open(__file__).read(), __file__, 'exec')) File "/usr/lib/python-exec/python3.8/chameleon.py", line 5, in <module> from pg_chameleon import replica_engine File "/usr/lib/python3.8/site-packages/pg_chameleon/__init__.py", line 2, in <module> from .lib.mysql_lib import * File "/usr/lib/python3.8/site-packages/pg_chameleon/lib/mysql_lib.py", line 4, in <module> import pymysql ModuleNotFoundError: No module named 'pymysql' {code} Let's look what we need: https://github.com/the4thdoctor/pg_chameleon/blob/master/setup.py#L53 PyMySQL>=0.7.11 argparse>=1.2.1 mysql-replication>=0.15 psycopg2-binary>=2.8.3 PyYAML>=5.1.2 tabulate>=0.8.1 daemonize>=2.4.7 rollbar>=0.13.17 Some are already present in Gentoo, some we'll need to add before we can actually use chameleon. Going forward, let's add pymysql to chameleon dependencies: {code} RDEPEND="${DEPEND} >=dev-python/pymysql-0.7.11[${PYTHON_USEDEP}]" {code} Now if we re-install pg_chameleon, it will also install pymysql and when we run it we get a different error: {code} # chameleon Traceback (most recent call last): File "/usr/lib/python-exec/python3.8/chameleon", line 5, in <module> exec(compile(open(__file__).read(), __file__, 'exec')) File "/usr/lib/python-exec/python3.8/chameleon.py", line 5, in <module> from pg_chameleon import replica_engine File "/usr/lib/python3.8/site-packages/pg_chameleon/__init__.py", line 2, in <module> from .lib.mysql_lib import * File "/usr/lib/python3.8/site-packages/pg_chameleon/lib/mysql_lib.py", line 7, in <module> from pymysqlreplication import BinLogStreamReader ModuleNotFoundError: No module named 'pymysqlreplication' {code} Meaning it already found pymysql but it needs the other dependencies too. Let's create mysql-replication-0.21.ebuild: {code} # Copyright 1999-2020 Gentoo Authors # Distributed under the terms of the GNU General Public License v2 EAPI=7 PYTHON_COMPAT=( python3_{6..9} ) inherit distutils-r1 DESCRIPTION="Pure Python Implementation of MySQL replication protocol build on top of PyMYSQL" HOMEPAGE="https://github.com/noplay/python-mysql-replication" SRC_URI="mirror://pypi/${PN:0:1}/${PN}/${P}.tar.gz" LICENSE="Apache-2.0" SLOT="0" KEYWORDS="~amd64 ~x86" IUSE="" {code} Now let's add it as a dependency to pg_chameleon along with psycopg: {code} RDEPEND="${DEPEND} >=dev-python/mysql-replication-0.15[${PYTHON_USEDEP}] >=dev-python/psycopg-2.8.3[${PYTHON_USEDEP}] >=dev-python/pymysql-0.7.11[${PYTHON_USEDEP}]" {code} Now it fails on other missing dependencies. To sum it up, trial and error reveals we need: 1) add ebuilds for dev-db/pg_chameleon, dev-python/mysql-replication and dev-python/rollbar 2) extend PYTHON_COMPAT in existing dev-python/daemonize 3) enable tests if possible In the end the command shows help: {code} # chameleon usage: chameleon [-h] [--config CONFIG] [--schema SCHEMA] [--source SOURCE] [--tables TABLES] [--logid LOGID] [--debug] [--version] [--rollbar-level ROLLBAR_LEVEL] [--full] command chameleon: error: the following arguments are required: command {code} But I have no idea if it works :) Please give it a try and let me know if it works for you.
Created attachment 644556 [details] daemonize-2.5.0.ebuild
Created attachment 644558 [details] mysql-replication-0.21.ebuild
Created attachment 644560 [details] pg_chameleon-2.0.12.ebuild
Created attachment 644562 [details] rollbar-0.15.0.ebuild
Damn... that looks pretty straight forward... might become a python convert after all. Thanks for the guide, I'll give it a go first chance I get!
At first glance, that works a treat. Thanks. I'll give a more thorough test on Monday when I'm back at work.
This all works great. What's best next steps given this has a dependency on a different pull request?
I think it's enough to mention the dependency in the PR. Or you can make that commit yourself in the PR adding pg_chameleon and I'll close my PR. Maybe best to ask proxy-maint on irc.
