Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 540322 - sys-cluster/gearmand-1.1.12 version bump
Summary: sys-cluster/gearmand-1.1.12 version bump
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 2 votes (vote)
Assignee: No maintainer - Look at https://wiki.gentoo.org/wiki/Project:Proxy_Maintainers if you want to take care of it
URL:
Whiteboard: Pending removal: 2018-04-13
Keywords: PMASKED
: 471426 (view as bug list)
Depends on:
Blocks:
 
Reported: 2015-02-17 00:23 UTC by Pavel Denisov
Modified: 2018-04-29 18:06 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
gearmand-1.1.12.ebuild (gearmand-1.1.12.ebuild,2.35 KB, text/plain)
2015-02-17 00:23 UTC, Pavel Denisov
Details
gearmand.init.d (gearmand.init.d,3.12 KB, text/plain)
2015-02-17 00:24 UTC, Pavel Denisov
Details
gearmand-1.1.16.ebuild (gearmand-1.1.16.ebuild,2.27 KB, text/plain)
2017-06-17 05:18 UTC, Alex Barker
Details
gearmand-1.1.16.ebuild (gearmand-1.1.16.ebuild,2.26 KB, text/plain)
2017-07-04 20:40 UTC, Alex Barker
Details
newest gearmand (gearmand-1.1.18.ebuild,2.22 KB, text/plain)
2018-04-10 08:44 UTC, manwe
Details
gearmand-1.1.18.ebuild (gearmand-1.1.18.ebuild,2.22 KB, text/plain)
2018-04-10 08:45 UTC, manwe
Details
pecl-gearman-2.0.3.ebuild (pecl-gearman-2.0.3.ebuild,691 bytes, text/plain)
2018-04-10 08:46 UTC, manwe
Details
gearmand.init.d (gearmand.init.d,2.98 KB, text/x-dsrc)
2018-04-10 08:51 UTC, manwe
Details
gearmand.conf.d (gearmand.conf.d,1.51 KB, text/x-dsrc)
2018-04-10 08:51 UTC, manwe
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Pavel Denisov 2015-02-17 00:23:28 UTC
Created attachment 396644 [details]
gearmand-1.1.12.ebuild

There is similar bug #471426, but its settings look incorrect and it is outdated a bit.
Comment 1 Pavel Denisov 2015-02-17 00:24:18 UTC
Created attachment 396646 [details]
gearmand.init.d
Comment 2 Jeroen Roovers (RETIRED) gentoo-dev 2015-02-17 07:45:18 UTC
Comment on attachment 396644 [details]
gearmand-1.1.12.ebuild

--- gearmand-0.34-r1.ebuild
+++ gearmand-1.1.12.ebuild
@@ -8,7 +8,7 @@

 DESCRIPTION="Generic framework to farm out work to other machines"
 HOMEPAGE="http://www.gearman.org/"
-SRC_URI="http://launchpad.net/gearmand/trunk/${PV}/+download/${P}.tar.gz"
+SRC_URI="https://launchpad.net/gearmand/1.2/${PV}/+download/${P}.tar.gz"

 LICENSE="MIT"
 SLOT="0"
@@ -72,7 +72,7 @@
 src_install() {
        autotools-utils_src_install

-       newinitd "${FILESDIR}"/gearmand.init.d.2 gearmand
+       newinitd "${FILESDIR}"/gearmand.init.d gearmand
        newconfd "${FILESDIR}"/gearmand.conf.d gearmand
 }
Comment 3 Sergey Popov gentoo-dev 2015-02-19 11:45:47 UTC
*** Bug 471426 has been marked as a duplicate of this bug. ***
Comment 4 Alex Barker 2017-06-17 03:32:16 UTC
Any updates on this?
Comment 5 Alex Barker 2017-06-17 05:18:46 UTC
Created attachment 476722 [details]
gearmand-1.1.16.ebuild

I have created a new 1.1.16 ebuild that moves to EAPI=6.  It should be noted that this will completely break pecl-gearman, which is pretty much abandoned at this point as PHP56 approaches EOL.  There are a number of github alternatives that have heavy development and work with php7+ so I will be looking into that next.

--- /usr/portage/sys-cluster/gearmand/gearmand-0.34-r1.ebuild   2017-02-28 11:50:50.000000000 -0800
+++ /usr/local/portage/sys-cluster/gearmand/gearmand-1.1.16.ebuild      2017-06-16 22:08:22.946821901 -0700
@@ -1,25 +1,22 @@
-# Copyright 1999-2016 Gentoo Foundation
+# Copyright 1999-2017 Gentoo Foundation
 # Distributed under the terms of the GNU General Public License v2
 
-EAPI=5
+EAPI=6
 
-AUTOTOOLS_AUTORECONF=1
-
-inherit autotools-utils eutils flag-o-matic libtool user
+inherit eutils flag-o-matic libtool user
 
 DESCRIPTION="Generic framework to farm out work to other machines"
 HOMEPAGE="http://www.gearman.org/"
-SRC_URI="https://launchpad.net/gearmand/trunk/${PV}/+download/${P}.tar.gz"
+SRC_URI="https://github.com/gearman/gearmand/releases/download/${PV}/${P}.tar.gz"
 
 LICENSE="MIT"
 SLOT="0"
 KEYWORDS="~amd64 ~x86"
-IUSE="debug tcmalloc +memcache sqlite tokyocabinet postgres"
+IUSE="debug +memcache sqlite tokyocabinet postgres"
 
 RDEPEND="dev-libs/libevent
        >=dev-libs/boost-1.39:=[threads(+)]
        || ( >=sys-apps/util-linux-2.16 <sys-libs/e2fsprogs-libs-1.41.8 )
-       tcmalloc? ( dev-util/google-perftools )
        memcache? ( >=dev-libs/libmemcached-0.47 )
        sqlite? ( dev-db/sqlite:3 )
        tokyocabinet? ( dev-db/tokyocabinet )
@@ -33,26 +30,11 @@
 }
 
 src_prepare() {
-       # fixes bug 574558, which is due to an outdated bundled boost.m4
-       rm m4/boost.m4 || die
-       sed -i -e 's/AM_INIT_AUTOMAKE.*//g' m4/pandora_canonical.m4 || die
-       epatch -p1 "${FILESDIR}/${P}-stdbool-h.patch"
-       autotools-utils_src_prepare
+       epatch -p1 "${FILESDIR}/${PN}-stdbool-h.patch"
+       eapply_user
 }
 
 src_configure() {
-       local myeconfargs=(
-               $(use_enable memcache libmemcached)
-               $(use_enable tcmalloc)
-               $(use_enable tokyocabinet libtokyocabinet)
-               $(use_with postgres postgresql)
-               $(use_with sqlite sqlite3)
-               --disable-mtmalloc
-               --disable-static
-       )
-
-       # Don't ever use --enable-assert since configure.ac is broken, and
-       # only does --disable-assert correctly.
        if use debug; then
                # Since --with-debug would turn off optimisations as well as
                # enabling debug, we just enable debug through the
@@ -63,27 +45,37 @@
        # Explicitly enable c++11 mode
        append-cxxflags -std=c++11
 
-       autotools-utils_src_configure
+       # Don't ever use --enable-assert since configure.ac is broken, and
+       # only does --disable-assert correctly.
+       econf \
+               $(use_enable memcache libmemcached) \
+               $(use_enable tokyocabinet libtokyocabinet) \
+               $(use_with postgres postgresql) \
+               $(use_with sqlite sqlite3) \
+               --disable-static
 }
 
-src_test() {
+#src_test() {
        # Since libtool is stupid and doesn't discard /usr/lib64 from the
        # load path, we'd end up testing against the installed copy of
        # gearmand (bad).
        #
        # We thus cheat and "fix" the scripts by hand.
-       sed -i -e '/LD_LIBRARY_PATH=/s|/usr/lib64:||' "${BUILD_DIR}"/tests/*_test \
-               || die "test fixing failed"
+#      sed -i -e '/LD_LIBRARY_PATH=/s|/usr/lib64:||' "${BUILD_DIR}"/tests/*_test \
+#              || die "test fixing failed"
 
-       autotools-utils_src_test
-}
+#}
 
-DOCS=( README AUTHORS ChangeLog )
+DOCS=( AUTHORS ChangeLog )
 
 src_install() {
-       autotools-utils_src_install
+       if [[ -f Makefile ]] || [[ -f GNUmakefile ]] || [[ -f makefile ]] ; then
+               emake DESTDIR="${D}" install
+       fi
+
+       einstalldocs
 
-       newinitd "${FILESDIR}"/gearmand.init.d.2 gearmand
+       newinitd "${FILESDIR}"/gearmand.init.d gearmand
        newconfd "${FILESDIR}"/gearmand.conf.d gearmand
 }
Comment 6 Alex Barker 2017-07-04 20:40:57 UTC
Created attachment 480810 [details]
gearmand-1.1.16.ebuild

Made a few small tweaks to the ebuild.  The server appears to work but 1 test is  failing. 

unittest.application.gbd true --fubar                                   [ skipped ]
unittest.application.gbd true                                   [ skipped ]
unittest.application.true --fubar                                       [ skipped ]

libtest/unittest.cc:663: in application_doesnotexist_BINARY() pid(24685) Assertion 'Application::SUCCESS' != 'true_app.run(args)'
unittest.application.doesnotexist --fubar                                       [ failed ]
unittest.application.echo fubar                                 0:000000126 [ ok ]


On a side note, I am having issues with pecl-gearman adding servers and functions to libgearman.  I am not sure what is causing this or if it is a problem with pecl or the server.  The php side of the errors doesn't provide much info, and the server never logs an issue in debug mode.

PHP Warning:  GearmanWorker::addServers(): (null) in ...
PHP Warning:  GearmanWorker::addFunction(): Unable to add function to Gearman Worker: (null) GEARMAN_INVALID_ARGUMENT in ...
Comment 7 Alex Barker 2017-07-06 16:29:59 UTC
(In reply to Alex Barker from comment #6)
> Created attachment 480810 [details]
> gearmand-1.1.16.ebuild
> 
> PHP Warning:  GearmanWorker::addServers(): (null) in ...
> PHP Warning:  GearmanWorker::addFunction(): Unable to add function to
> Gearman Worker: (null) GEARMAN_INVALID_ARGUMENT in ...

The ebuild is working correctly.  The issue I experienced is a bug with the interaction of pthreads and gearman worker.  It seems to soil the pants only if the GearmanWorker class is a member variable of a class that extends Stackable!  I.E. `
```
class MyWorker extends Stackable {
    /** @var GearmanWorker $gearman */
    protected $gearman;

    public function __construct()
    {
        $this->gearman = new GearmanWorker();

        // Poops pants here.
        $this->gearman->addServer("127.0.0.1", 4730);
    }

    ...
}
```
Comment 8 Pacho Ramos gentoo-dev 2017-11-09 16:00:54 UTC
Does anyone want to proxy maintain it?
https://wiki.gentoo.org/wiki/Project:Proxy_Maintainers
Comment 9 Alex Barker 2017-11-14 04:57:28 UTC
So I updated the ebuild and did extensive testing and I concluded that gearmand, at it's current version, is not stable with any version of pecl-gearman.  Apparently 0.33 was the last stable daemon.  I am really not happy with the quality of this queue so I have moved to beanstalkd.

I may proxy maintain if I come back to gearmand with a pure php library.
Comment 10 Alex Barker 2017-11-14 04:58:06 UTC
If you send me emails about this package, I'll proxy maintain ;)
Comment 11 Pacho Ramos gentoo-dev 2017-11-18 16:47:30 UTC
OK, I will CC proxy-maint team then :)

Thanks for volunteering
Comment 12 Pacho Ramos gentoo-dev 2018-02-05 16:17:31 UTC
Either this finally gets proxy-maintained or will be removed because current versions in the tree are severely broken
Comment 13 Alex Barker 2018-04-03 06:30:23 UTC
As it currently stands, pecl-gearman is still broken and should be removed on the scheduled date.  If pecl gearman sees a version update for 7.2+ I will revisit this package, else EOL with 5.4.
Comment 14 Alex Barker 2018-04-03 06:31:34 UTC
Errr, s/5.4/5.6
Comment 15 manwe 2018-04-10 08:44:18 UTC
There's pecl-gearman in this repo https://github.com/wcgallego/pecl-gearman that works fine with PHP 7.0 up to 7.2. I'm using it on my production servers togerther with gearmand 1.1.x. Unfortunately PHP 5.6 is not supported.

I can also help with maintaining, together with Alex.
Comment 16 manwe 2018-04-10 08:44:59 UTC
Created attachment 526990 [details]
newest gearmand
Comment 17 manwe 2018-04-10 08:45:54 UTC
Created attachment 526992 [details]
gearmand-1.1.18.ebuild

Ups, wrong file name
Comment 18 manwe 2018-04-10 08:46:24 UTC
Created attachment 526994 [details]
pecl-gearman-2.0.3.ebuild
Comment 19 manwe 2018-04-10 08:51:41 UTC
Created attachment 526996 [details]
gearmand.init.d
Comment 20 manwe 2018-04-10 08:51:57 UTC
Created attachment 526998 [details]
gearmand.conf.d
Comment 21 Alex Barker 2018-04-10 16:41:49 UTC
Hi manwe,

The major issue I was having was that the Gearman extension (1.1.2) with the gearman daemon 1.1.6 was unstable.  It worked ok for a small number of jobs with low concurrency, however, if as the number of jobs rose with a reasonable amount of concurrency gearmand would deadlock and stop delivering & accepting jobs.

The github repo (hjr3/pecl-gearman) for the original 1.x version has been archived and has not seen any development for 4 years. The 2.x development line was taken from a fork (wcgallego/pecl-gearman), and that has not seen development in nearly a year.  IIRC this project has also lost access to the pecl-gearman page and can no longer update it.

I strongly suspect that the deadlocking issue is related to the daemon and not the client code.  Many have reported that the last stable release of the daemon was somewhere in the 0.3x range which is quite old at this point.  After messing around with this packages for a month or so, I decided that it simply wasn't worth fixing as many other queues are available that are significantly more reliable and have client API's that are better suited for parallel processing. If you are looking for replacements, Beanstalkd has served me very well and is pecl-pthread compatible.
Comment 22 manwe 2018-04-10 19:36:18 UTC
I didn't get any problems with stability, but I never have more than few thousand jobs on one gearmand. So far I'm stuck with it because too many systems in my company depends on it. So I'll keep making my own ebuild ;)
Comment 23 manwe 2018-04-11 07:45:02 UTC
One more thing Alex, I don't see ebuild for pheanstalk (PHP client for beanstalkd) in neither main portage tree or any of the layouts. DO you have ebuild for it?
Comment 24 Alex Barker 2018-04-11 19:47:05 UTC
(In reply to manwe from comment #23)
> One more thing Alex, I don't see ebuild for pheanstalk (PHP client for
> beanstalkd) in neither main portage tree or any of the layouts. DO you have
> ebuild for it?

It is a pure php package available via composer. IMHO, Only pecl extensions should have ebuilds. If you are not using composer, find a new job. If you are the person that made that decision for your organization, do us all a favor and stop working in the industry.

As for gearmand, I was getting a lot of issues when I had 1 daemon running with around 10K-100k jobs, and a 4-8 long running workers processing thouse jobs on the same box.  I would run for a while and then just stop, and the gearman server would not longer respond, accept jobs, etc. Based on my recent testing, I would not move to 7.x with this queue and hence my beanstalkd direction.  With that said, I have seen it used in production, very recently in fact, but that doesn't give me much confidence knowing full well how fortune 500 companies like Equifax and so many others run their technology departments.  You could probably get away with using it for slow, low volume, workloads but I would still be worried about hanging and losing jobs.
Comment 25 Pacho Ramos gentoo-dev 2018-04-29 18:06:59 UTC
removed