Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 69553 - media-gfx/drqueue (New package)
Summary: media-gfx/drqueue (New package)
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All All
: High enhancement (vote)
Assignee: Sebastian Pipping
URL: http://www.drqueue.org/
Whiteboard:
Keywords: EBUILD
Depends on: 302717
Blocks:
  Show dependency tree
 
Reported: 2004-10-30 14:35 UTC by Jorge Daza
Modified: 2010-01-29 15:53 UTC (History)
8 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
drqueue-0.61.3.ebuild (drqueue-0.61.3.ebuild,978 bytes, text/plain)
2006-01-30 02:05 UTC, Daniel Black (RETIRED)
Details
ebuild for drqueue 0.64.1 (drqueue-0.64.1.ebuild,976 bytes, application/octet-stream)
2006-10-05 11:17 UTC, Hanni Ali
Details
drqueue-0.64.1.ebuild (drqueue-0.64.1.ebuild,1.11 KB, text/plain)
2006-10-06 03:09 UTC, Hanni Ali
Details
drqueue-0.64.1.ebuild (drqueue-0.64.1.ebuild,1.13 KB, text/plain)
2006-10-06 03:53 UTC, Hanni Ali
Details
drqueue-0.64.1.ebuild (drqueue-0.64.1.ebuild,2.39 KB, text/plain)
2006-10-08 06:02 UTC, Hanni Ali
Details
conf/init (files.tar.bz2,909 bytes, application/octet-stream)
2006-10-08 06:03 UTC, Hanni Ali
Details
files/conf-drqmd (conf-drqmd,525 bytes, text/plain)
2006-10-08 06:29 UTC, Hanni Ali
Details
files/conf-drqsd (conf-drqsd,519 bytes, text/plain)
2006-10-08 06:29 UTC, Hanni Ali
Details
files/init-drqmd (init-drqmd,517 bytes, text/plain)
2006-10-08 06:30 UTC, Hanni Ali
Details
files/init-drqsd (init-drqsd,509 bytes, text/plain)
2006-10-08 06:30 UTC, Hanni Ali
Details
DrQueue 0.64.2 propose ebuild (drqueue-0.64.2.ebuild,2.66 KB, text/plain)
2007-02-05 09:47 UTC, Hanni Ali
Details
conf for master (conf-drqmd-0.64.2,653 bytes, text/plain)
2007-02-05 09:47 UTC, Hanni Ali
Details
conf for slave (conf-drqsd-0.64.2,647 bytes, text/plain)
2007-02-05 09:47 UTC, Hanni Ali
Details
init for master (init-drqmd-0.64.2,511 bytes, text/plain)
2007-02-05 09:48 UTC, Hanni Ali
Details
files/init-drqsd-0.64.2 (init-drqsd-0.64.2,503 bytes, text/plain)
2007-02-05 09:49 UTC, Hanni Ali
Details
files/env (env,73 bytes, text/plain)
2007-02-05 09:49 UTC, Hanni Ali
Details
Mips patch tested with nekoware Irix software (gentoo.patch,14.90 KB, patch)
2007-04-10 20:25 UTC, Jorge Daza
Details | Diff
compile warnings and errors with python-mips patch (drqueue-compile.txt,13.77 KB, text/plain)
2007-04-11 08:40 UTC, kaaZoo
Details
Mips patch without the PATH_MAX hack (gentoo.patch,14.62 KB, patch)
2007-04-11 08:54 UTC, Jorge Daza
Details | Diff
mips compile output #2 (drqueue-compile_2.txt,8.25 KB, text/plain)
2007-04-11 11:15 UTC, kaaZoo
Details
slave output on linux/mips (drqueue-patch-slave.txt,3.00 KB, text/plain)
2007-05-22 13:22 UTC, kaaZoo
Details
patch for linux/mips (drqueue-patch-slave.patch,1.36 KB, patch)
2007-05-22 13:27 UTC, kaaZoo
Details | Diff
files/drqueue-0.64.3_mips_linux.patch (drqueue-0.64.3_mips_linux.patch,1.46 KB, patch)
2007-08-04 13:31 UTC, kaaZoo
Details | Diff
files/conf-drqmd-0.64.3 (conf-drqmd-0.64.3,708 bytes, text/plain)
2007-08-04 13:32 UTC, kaaZoo
Details
files/conf-drqsd-0.64.3 (conf-drqsd-0.64.3,702 bytes, text/plain)
2007-08-04 13:32 UTC, kaaZoo
Details
files/init-drqmd-0.64.3 (init-drqmd-0.64.3,527 bytes, text/plain)
2007-08-04 13:33 UTC, kaaZoo
Details
files/init-drqsd-0.64.3 (init-drqsd-0.64.3,518 bytes, text/plain)
2007-08-04 13:34 UTC, kaaZoo
Details
drqueue-0.64.3.ebuild (drqueue-0.64.3.ebuild,3.30 KB, text/plain)
2007-08-04 13:35 UTC, kaaZoo
Details
drqueue-0.64.3.ebuild (drqueue-0.64.3.ebuild,3.33 KB, text/plain)
2007-08-14 09:03 UTC, Hanni Ali
Details
drqueue-0.64.3.ebuild (drqueue-0.64.3.ebuild,3.12 KB, text/plain)
2007-08-15 20:39 UTC, Hanni Ali
Details
SConstruct patch for ruby use flag (SConstruct.patch,522 bytes, patch)
2007-08-15 20:39 UTC, Hanni Ali
Details | Diff
files/init-drqmd-0.64.3 (init-drqmd-0.64.3,557 bytes, text/plain)
2007-08-15 20:40 UTC, Hanni Ali
Details
files/init-drqsd-0.64.3 (init-drqsd-0.64.3,548 bytes, text/plain)
2007-08-15 20:41 UTC, Hanni Ali
Details
drqueue-0.64.3.ebuild (drqueue-0.64.3.ebuild,3.25 KB, text/plain)
2007-08-16 11:12 UTC, kaaZoo
Details
files/init-drqsd-0.64.3 (init-drqsd-0.64.3,547 bytes, text/plain)
2007-08-16 11:14 UTC, kaaZoo
Details
files/conf-drqsd-0.64.3 (conf-drqsd-0.64.3,762 bytes, text/plain)
2007-09-10 16:47 UTC, kaaZoo
Details
files/conf-drqsd-0.64.3 (conf-drqsd-0.64.3,818 bytes, text/plain)
2007-09-10 16:55 UTC, kaaZoo
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jorge Daza 2004-10-30 14:35:50 UTC
Hello, I'm the developer of DrQueue (http://www.drqueue.org/). I have been asked several times if there was a gentoo "package" for DrQueue. I looked and I couldn't find it so I think it could be a good idea to include it in your distribution.

DrQueue is a distributed render queueing system, used to manage big (or small) render farms running Linux,Mac OS X, Irix or FreeBSD (up to now). It can use any renderer that can be called from the command line, but includes script generators for Maya, Pixie, Blender and Bmrt. It will include more in the future. Well, I hope I'm writing this in the right place.

Reproducible: Always
Steps to Reproduce:
1.
2.
3.




Please contact me if you need any help. I'll gladly do it.
Comment 1 Gregg Casillo 2005-04-08 01:17:58 UTC
I'm not an ebuild guru, but I would like to see this find its way into portage.
Comment 2 Daniel Black (RETIRED) gentoo-dev 2005-05-28 06:19:41 UTC
I've half started one. 
Comment 3 Henti Smith 2005-07-06 07:08:48 UTC
anything I can do to help with this ebuild ? 
Comment 4 Jorge Daza 2005-07-09 20:07:00 UTC
Come on, somebody, do it. :)
Comment 5 Henti Smith 2006-01-30 01:04:46 UTC
Daniel .. 

can you please post what you have done so far. 

thanks .. 

H 
Comment 6 Daniel Black (RETIRED) gentoo-dev 2006-01-30 02:05:57 UTC
Created attachment 78495 [details]
drqueue-0.61.3.ebuild

Hope this helps

I got a bit distracted. Thanks for volunteering to help.
Comment 7 Henti Smith 2006-02-08 04:36:03 UTC
Ok .. started working on this ... 

sorry so late .. didn't get notification of your post .. 

First thing ...where do we wanna install this ... My thoughts are /opt/drqueue as there are some dirs that will not fit into the /usr directory .. 

the linux install dirs looks like this : 

Linux_install:
  install -d -m 0777 $(INSTROOT)/tmp
  install -d -m 0777 $(INSTROOT)/logs
  install -d -m 0755 $(INSTROOT)/bin
  install -d -m 0755 $(INSTROOT)/etc
  install -d -m 0777 $(INSTROOT)/db
  install -d -m 0777 $(INSTROOT)/contrib

I think we should patch the Makefile to make $INSTROOT /opt/drqueue

thereby negating any need to patch the install structure. 

suggestions ? 
Comment 8 Henti Smith 2006-02-08 04:39:32 UTC
(In reply to comment #7)

> I think we should patch the Makefile to make $INSTROOT /opt/drqueue

Sorry .. just went through some gentoo docs and /opt is reserved for binary installations. 

I'll patch the make file to make everything fit where they should go .. 

H 
Comment 9 Hanni Ali 2006-10-05 11:17:15 UTC
Created attachment 98869 [details]
ebuild for drqueue 0.64.1
Comment 10 Hanni Ali 2006-10-05 11:22:14 UTC
(In reply to comment #9)
> Created an attachment (id=98869) [edit]
> ebuild for drqueue 0.64.1
> 

I have update the ebuild I will check it on a cluster either later this evening or tomorrow evening. I have installed the program into /usr/share/drqueue as the directory structure appears to be integral to the binaries then a DRQUEUE_ROOT is exported and the executables appear to be happy.

I think it's worth sorting this program out for Gentoo it's mature and powerful and there is no real alternative in the tree for distributed rendering across multiple OS/Programs, PVM-POV is too old...

Hanni
Comment 11 Hanni Ali 2006-10-06 03:09:01 UTC
Created attachment 98917 [details]
drqueue-0.64.1.ebuild

Checked the install on a cluster and noticed it was using tcsh not bash I try to keep clusters light and fast hence the build failed because i didn't have tcsh installed.

Rather than put this as a dependancy I just change it, if it is felt tcsh would be more appropriate for drqueue user I suggest we add a use flag.

Additionally sorted out where it is installing everything, it seems to work ok, but I'll check it with a render.

Hanni
Comment 12 Luca Barbato gentoo-dev 2006-10-06 03:34:49 UTC
sorry for the lack of comments, thank you for your effort.

Now something:

DEPEND="X? ( >=x11-libs/gtk+-2 )"

RDEPEND=""

it should RDEPEND on gtk too or it statically link to it?
it just need system provided facilities?

	make INSTROOT=${D}/usr install || die

emake -j1

pkg_postinst() {
	einfo "You must add to bashrc or run:"
	einfo "export DRQUEUE_ROOT=/usr"
	einfo "export DRQUEUE_MASTER=hostname"
	einfo "where hostname is your master's hostname."
}

what about an env.d or a conf.d file?

the user/group creation is really necessary on pkg_setup?
Comment 13 Hanni Ali 2006-10-06 03:53:52 UTC
Created attachment 98923 [details]
drqueue-0.64.1.ebuild

(In reply to comment #12)
> it should RDEPEND on gtk too or it statically link to it?
> it just need system provided facilities?

It depends on it for run time only, if use is X we builds the gui part as well which has the x deps. In the compile part I've split it so for headless servers - common in clustering - we don't pull in gtk.

> emake -j1

are you suggesting we only have one thread for compile???

> what about an env.d or a conf.d file?

dunno still investigationg the program once I've got it working I may rethink the way we install it, but there is no configure for the build so not sure how easy it is. I haven't looked at the code I just want to get it working first and try it out.
 
> the user/group creation is really necessary on pkg_setup?

this was put in by the pearson who first made the ebuild and I think it is useful, it alearted me to the fact that he had uses tcsh and on further investigation I found most of the scripts generated use tcsh hence I've added it as a dependency in yet another ebuild revision.

Hanni

P.S. thanks for joining in I appreciate it, plus kudos to you I found you howto and work on the Cell very helpful although I've not got a sys sim fully functional yet...
Comment 14 Luca Barbato gentoo-dev 2006-10-06 04:24:45 UTC
> 
> It depends on it for run time only, if use is X we builds the gui part as well
> which has the x deps. In the compile part I've split it so for headless servers
> - common in clustering - we don't pull in gtk.

so don't empty rdepend, otherwise gtk+ could be pulled out by depclean...
(RDEPEND things that you need to run it, DEPEND things you need to build it)

> 
> > emake -j1
> 
> are you suggesting we only have one thread for compile???

use emake -j1 install instead of make install.

> 
> > the user/group creation is really necessary on pkg_setup?
> 

move it in pkg_postinst.

> P.S. thanks for joining in I appreciate it, plus kudos to you I found you howto
> and work on the Cell very helpful although I've not got a sys sim fully
> functional yet...

Hm, contact me on irc so we could sort the Cell problem...
> 

Comment 15 Hanni Ali 2006-10-06 04:41:32 UTC
(In reply to comment #14)
> so don't empty rdepend, otherwise gtk+ could be pulled out by depclean...
> (RDEPEND things that you need to run it, DEPEND things you need to build it)

ok, seems sensible, thanks for explaining RDEPEND never really clicked
 
> > > emake -j1
> > 
> > are you suggesting we only have one thread for compile???
> 
> use emake -j1 install instead of make install.

why? I'm not saying no just I don't yet know why you would need/want this

> move it in pkg_postinst.

will do, but is there a reason it is better here?

> Hm, contact me on irc so we could sort the Cell problem...

will do when i get back to that, also trying to sort out the globus toolkit ebuild, trying to prioratise.

I'm also sorting the etc and trying to get it such that the package is vaguely ready to run, has anyone tried installing drqueue on Gentoo, I'm still trying to work out how all the exacutables work by trawling the website...

Hanni
Comment 16 Jorge Daza 2006-10-06 22:31:17 UTC
Just wanted to point out that gtk is only needed for drqman (the gui) but not for slave or master.

The slave would be the program running on every node of the farm, and probably the most common install. It is a console based program, not needing gtk at all.

drqman on the other hand could be used in computers that neither run the slave nor the master. Users that want to check the queue or send new jobs. That's what needs gtk.

Just to let you know.

In you need any help, please do not hesitate in mailing me. Also, tell me about any suggestion or change to the code that could ease the solution.
Comment 17 Luca Barbato gentoo-dev 2006-10-07 01:50:13 UTC
(In reply to comment #16)
> Just wanted to point out that gtk is only needed for drqman (the gui) but not
> for slave or master.

Ok

> 
> The slave would be the program running on every node of the farm, and probably
> the most common install. It is a console based program, not needing gtk at all.
> 

I'd have split tarballs between master and server then.

> drqman on the other hand could be used in computers that neither run the slave
> nor the master. Users that want to check the queue or send new jobs. That's
> what needs gtk.

Ok

> 
> In you need any help, please do not hesitate in mailing me. Also, tell me about
> any suggestion or change to the code that could ease the solution.
> 

If you could release drqman as stand alone package probably would be nicer, but isn't necessary.
Are you sure drqueue just need what is provided by system?

Beside that I guess I could start testing it a bit once I got more time...
Comment 18 Hanni Ali 2006-10-08 06:02:43 UTC
Created attachment 99108 [details]
drqueue-0.64.1.ebuild

Ok, I think this is a fairly complete ebuild now. I've modified it heavily and now create two daemons in init, drqmd (master) and drqsd (slave), these can now be added to rc-update for deafult and use config files in conf to define how they run.

The previous modifications have been revised, it is essential for the compile to * create the drqueue group and hence this is moved back to setup
* there are two compiles, with X and without
* the daemons also use a pid file to manage start/stopping

the monitor works but some package configuration outside  of the ebuild is required

I will add a files attachment in a second with the init's and conf's

Additionally there are some python bindings which can be activated with the python flag they provide a web front end, this is built using scons and adds a dependency on swig and python. However this is not yet flawless when I try to build it there is a ACCESS DENIED error caused by the final line in the SConstruct file:

drqueue = env.SharedLibrary ("drqueue", drqueue_all)

Jorge how key is this (my python understanding is rudimentary) and lu_zero, can you think how we can install the python bindings with emerge or should it be done outside of emerge I'm not so familiar with the rules.

I will try it out now on my cluster and try some renders but I'm fairly confident this is all ok now

Hanni
Comment 19 Hanni Ali 2006-10-08 06:03:35 UTC
Created attachment 99109 [details]
conf/init
Comment 20 Luca Barbato gentoo-dev 2006-10-08 06:11:14 UTC
You may check the python eclass.

Jorge, how many beers I should offer you to make you reconsider scons as build system?

Hanni please attach files as separate plain text ones please.
Comment 21 Hanni Ali 2006-10-08 06:29:35 UTC
Created attachment 99112 [details]
files/conf-drqmd
Comment 22 Hanni Ali 2006-10-08 06:29:56 UTC
Created attachment 99113 [details]
files/conf-drqsd
Comment 23 Hanni Ali 2006-10-08 06:30:15 UTC
Created attachment 99114 [details]
files/init-drqmd
Comment 24 Hanni Ali 2006-10-08 06:30:37 UTC
Created attachment 99115 [details]
files/init-drqsd
Comment 25 Luca Barbato gentoo-dev 2006-10-08 06:38:13 UTC
Jorge: could be possible have more specific names for the applications? master&slave don't sound good...

Hanni: the conf.d files are wrong you shouldn't put variables that won't change (e.g the executable name and pitfile) and you shouldn't manipolate explicitly env.
Comment 26 Hanni Ali 2006-10-08 06:54:38 UTC
(In reply to comment #25)
> Jorge: could be possible have more specific names for the applications?
> master&slave don't sound good...

that's why I made the init names drqmd and drqsd, they could also go to background automatically that would be sensible seen as they are daemons. Unless you want to log to the screen?

> Hanni: the conf.d files are wrong you shouldn't put variables that won't change
> (e.g the executable name and pitfile) and you shouldn't manipolate explicitly
> env.

I used distcc as a template it had many variables which I didn't think changed but I'm not sure and I couldn't get the environment variables to work any other way I was at it for hours but it just wouldn't work so I'm at a loss how to write them in any other way.I had the env in env.d the variables in conf and exported in the terminal to no avail then I found another conf file that had it done how I did it which worked...

where is some info on how to use the python eclass

Hanni
Comment 27 Hanni Ali 2007-02-05 09:47:02 UTC
Created attachment 109210 [details]
DrQueue 0.64.2 propose ebuild

Updated ebuild for the latest version, additionally, python now works as does DrKeewee the web front end. Basically production ready now. Only additional point is that a dependency should probably be added for CGI, but there are so many CGI implementations I can't decide if we should foce a particular one or just let the individual choose. Updated conf/init to follow.

Hanni
Comment 28 Hanni Ali 2007-02-05 09:47:39 UTC
Created attachment 109211 [details]
conf for master
Comment 29 Hanni Ali 2007-02-05 09:47:56 UTC
Created attachment 109212 [details]
conf for slave
Comment 30 Hanni Ali 2007-02-05 09:48:26 UTC
Created attachment 109214 [details]
init for master
Comment 31 Hanni Ali 2007-02-05 09:49:02 UTC
Created attachment 109216 [details]
files/init-drqsd-0.64.2
Comment 32 Hanni Ali 2007-02-05 09:49:30 UTC
Created attachment 109218 [details]
files/env
Comment 33 kaaZoo 2007-04-10 09:36:29 UTC
thank you for providing an ebuild. i'm currently setting up a renderfarm and really need this.
Comment 34 kaaZoo 2007-04-10 10:19:21 UTC
i'm getting the following error when trying to install drqueue-0.64.2:

compiling python bindings

---------------------------------------------------------------------------
This script requires setuptools version 0.6c3 to run (even to display
help).  I will attempt to download it for you (from
http://cheeseshop.python.org/packages/2.4/s/setuptools/), but
you may need to enable firewall access for this script first.
I will start the download in 15 seconds.

(Note: if this machine does not have network access, please obtain the file

   http://cheeseshop.python.org/packages/2.4/s/setuptools/setuptools-0.6c3-py2.4.egg

and place it in this directory before rerunning this script.)
---------------------------------------------------------------------------
Downloading http://cheeseshop.python.org/packages/2.4/s/setuptools/setuptools-0.6c3-py2.4.egg
Machine not listed: mips64

!!! ERROR: sys-cluster/drqueue-0.64.2 failed.
Call stack:
  ebuild.sh, line 1614:   Called dyn_compile
  ebuild.sh, line 971:   Called qa_call 'src_compile'
  environment, line 3337:   Called src_compile
  drqueue-0.64.2.ebuild, line 45:   Called distutils_src_compile
  distutils.eclass, line 38:   Called die

!!! compilation failed
!!! If you need support, post the topmost build error, and the call stack if relevant.
!!! A complete build log is located at '/var/tmp/portage/sys-cluster/drqueue-0.64.2/temp/build.log'.

!!! This ebuild is from an overlay: '/raid5/portage-overlay'



here is the output of "emerge --info":

Portage 2.1.2.2 (default-linux/mips/2006.1/ip27/o32/nptl, gcc-4.1.1, glibc-2.3.6-r4, 2.6.17.10-mipsgit-20060618-ip27r10k+ mips64)
=================================================================
System uname: 2.6.17.10-mipsgit-20060618-ip27r10k+ mips64 R10000 V2.6  FPU V0.0
Gentoo Base System version 1.12.1
Timestamp of tree: Wed, 04 Apr 2007 19:30:01 +0000
distcc 2.18.3 mips-unknown-linux-gnu (protocols 1 and 2) (default port 3632) [enabled]
dev-lang/python:     2.4.3-r1
dev-python/pycrypto: 2.0.1-r5
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.61
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.3.13-r3
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.14.4
ACCEPT_KEYWORDS="mips"
AUTOCLEAN="yes"
CBUILD="mips-unknown-linux-gnu"
CFLAGS="-O2 -march=mips4 -pipe -fomit-frame-pointer -ftracer -fforce-addr"
CHOST="mips-unknown-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/php/apache1-php5/ext-active/ /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-O2 -march=mips4 -pipe -fomit-frame-pointer -ftracer -fforce-addr"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig ccache distcc distlocks metadata-transfer sfperms strict"
GENTOO_MIRRORS="ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/ ftp://ftp.mneisen.org/gentoo ftp://ftp.wh2.tu-dresden.de/pub/mirrors/gentoo"
MAKEOPTS="-j7"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/raid5/portage-overlay"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="berkdb bitmap-fonts cli cracklib fortran gdbm gpm iconv ip27 isdnlog libwww midi mips nls nptl nptlonly pam pcre perl pppd python readline reflection sdl session spl ssl tcpd truetype-fonts type1-fonts xorg" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="dummy fbdev impact newport v4l"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

Comment 35 Hanni Ali 2007-04-10 10:39:14 UTC
> http://cheeseshop.python.org/packages/2.4/s/setuptools/setuptools-0.6c3-py2.4.egg
> Machine not listed: mips64

It looks like you are testing on a mips machine, I don't think the python setup-tools supports mips.

There should be a new version available soon  which will be using autotools so this should resolve this current issue your having.

We would be very interested in any feedback ou have when you get DrQueue running on mips over at http://drqueue.org feel free to let us know how you manage in the forums there.

Hanni

Comment 36 kaaZoo 2007-04-10 12:10:17 UTC
i'll give you feedback on this after some testing is done.

now i installed it without the python use-flag and it went fine.

but it would be nice to be abe to use the webservice in the future because my master node also runs a webserver which could serve those rendering statistics and so on.
Comment 37 kaaZoo 2007-04-10 13:12:16 UTC
there seems to be a bug in the path


renderfarm drqueue # drqman 
/usr/bin/drqman: line 8: /bin/shlib: No such file or directory


shlib is not in /bin but in /usr/bin


Comment 38 kaaZoo 2007-04-10 13:19:43 UTC
ok, the env vars where not set in my shell.

but i think the binary wasn't built anyway because i didn't use the X useflag
Comment 39 Jorge Daza 2007-04-10 15:34:48 UTC
Hello,

I'm working on a patch right now and testing on a 32 R12000 cpus Origin 3000 family laptop.

uname -a = IRIX64 xxxxx 6.5 xxxxxxx IP35

I'll post here the patch asap, no worries. Stay cool. :)

Jorge
Comment 40 Jorge Daza 2007-04-10 20:25:49 UTC
Created attachment 115937 [details, diff]
Mips patch tested with nekoware Irix software

Trying to address the setup.py python problems mentioned.
Comment 41 Jorge Daza 2007-04-10 21:07:18 UTC
(In reply to comment #40)
> Created an attachment (id=115937) [edit]
> Mips patch tested with nekoware Irix software
> Trying to address the setup.py python problems mentioned.

Feedback appreciated.
Comment 42 kaaZoo 2007-04-11 08:40:53 UTC
Created attachment 115952 [details]
compile warnings and errors with python-mips patch

I used your patch in the ebuild but i get a lot of warnings and the build stops with an error. Please have a look at the compile output.

Greetings, kaazoo
Comment 43 Jorge Daza 2007-04-11 08:48:42 UTC
(In reply to comment #42)
> Created an attachment (id=115952) [edit]
> compile warnings and errors with python-mips patch
> I used your patch in the ebuild but i get a lot of warnings and the build stops
> with an error. Please have a look at the compile output.
> Greetings, kaazoo

Yes, that PATH_MAX was one of my concerns. I'll send a new one.
Comment 44 Jorge Daza 2007-04-11 08:54:54 UTC
Created attachment 115953 [details, diff]
Mips patch without the PATH_MAX hack

Same as before without the redefinition of PATH_MAX
Comment 45 kaaZoo 2007-04-11 11:15:59 UTC
Created attachment 115959 [details]
mips compile output #2

Still a lot of warnings and errors...
Comment 46 Jorge Daza 2007-04-12 18:11:16 UTC
(In reply to comment #45)
> Created an attachment (id=115959) [edit]
> mips compile output #2
> Still a lot of warnings and errors...

Sorry, couple of details that might help debugging:
* are you using gcc or MIPSpro ? Try c99.
* also try removing line 32 ('#include "libdrqueue.h"') from file libdrqueue/job.h

I'm doing the tests with MIPSpro c99 version 7.4.4m and there are lots of warnings due to the unsafe type conversions to handle with 64 and 32 bit processors.

In any case, those makefile versions have been superseeded by the autoolized versions available on the repo.
Comment 47 kaaZoo 2007-04-12 23:27:26 UTC
(In reply to comment #46)
> Sorry, couple of details that might help debugging:
> * are you using gcc or MIPSpro ? Try c99.

i'm using gcc-4.1.1
see the output of "emerge --info" in comment #34



Comment 48 kaaZoo 2007-05-15 14:26:01 UTC
i installed setuptools and scons and was playing around with the manually downloaded source.
scons (i'm not experienced with it) was not able to build the source, giving the following error:

renderfarm drqueue-0.64.2c3 # scons
scons: Reading SConscript files ...
Architecture: mips64
NameError: name 'bitsFlag' is not defined:
  File "/home/kaazoo/drqueue-0.64.2c3/SConstruct", line 33:
    env.Append (CCFLAGS = [bitsFlag,])



then i tried the source from svn.
when i execute "python setup.py install" it seems to work.
i had to set
DRQUEUE_MASTER=127.0.0.1
DRQUEUE_ROOT=/usr/share/drqueue
PYTHON_EGG_CACHE=/tmp

DrKeewee was giving me an error because of permissions on the cache so i executed "chmod -R 777 drqueue-0.65.0b2_r1879-py2.4-linux-mips64.egg-tmp/".


now DrKeewee seems to work or a least there are no errors anymore.

one problem remains: i don't get any computers listed allthough i have started one master and one slave.



Comment 49 kaaZoo 2007-05-15 15:07:10 UTC
ok, the problem is that the slave is starting properly:

octane2 drqueue # slave.Linux.mips64 -l 3 -o
Logging level set to: Debug
Logging on screen.
Could not open config file: '/etc/drqueue/slave.conf'
Could not open config file: '/usr/etc/slave.conf'
Tue May 15 17:05:00 2007 (PID:22687) : |  Debug | -> MSG: get_shared_memory_slave(): using file '/usr/bin/slave' as key for shared resources.
Tue May 15 17:05:00 2007 (PID:22687) : |  Debug |  : -> MSG: get_semaphores_slave(): using file '/usr/bin/slave' as key for shared resources.
Tue May 15 17:05:00 2007 (PID:22687) : |   Info | | : -> MSG: Starting...
ERROR: Proc type not found on /proc/cpuinfo
Tue May 15 17:05:00 2007 (PID:22687) : |   Info | Computer: name:'octane2' id:'0' : -> MSG: Cleaning...

Comment 50 kaaZoo 2007-05-22 13:22:26 UTC
Created attachment 119998 [details]
slave output on linux/mips

i added some code to libdrqueue/computer_info.linux.h in order to support linux on mips. maybe it's a little bit ugly but it seems that the slave has no more worries...
Comment 51 kaaZoo 2007-05-22 13:27:15 UTC
Created attachment 119999 [details, diff]
patch for linux/mips
Comment 52 kaaZoo 2007-08-04 13:24:15 UTC
ok folks, i created a new ebuild for stable version 0.64.3. there are also some new patch, init and conf files.

i tested it on x86 (slave) and mips (server).

problems killing the daemons with the init scripts remain. start-stop-daemon kills a process called 'master' or 'slave' but not the other architecture specific processes like 'master.Linux.i686' or 'slave.Linux.mips64'.


greetings, kaazoo
Comment 53 kaaZoo 2007-08-04 13:31:11 UTC
Created attachment 126853 [details, diff]
files/drqueue-0.64.3_mips_linux.patch
Comment 54 kaaZoo 2007-08-04 13:32:02 UTC
Created attachment 126854 [details]
files/conf-drqmd-0.64.3
Comment 55 kaaZoo 2007-08-04 13:32:39 UTC
Created attachment 126855 [details]
files/conf-drqsd-0.64.3
Comment 56 kaaZoo 2007-08-04 13:33:29 UTC
Created attachment 126856 [details]
files/init-drqmd-0.64.3
Comment 57 kaaZoo 2007-08-04 13:34:25 UTC
Created attachment 126857 [details]
files/init-drqsd-0.64.3
Comment 58 kaaZoo 2007-08-04 13:35:07 UTC
Created attachment 126859 [details]
drqueue-0.64.3.ebuild
Comment 59 Hanni Ali 2007-08-14 08:37:50 UTC
Comment on attachment 109210 [details]
DrQueue 0.64.2 propose ebuild

newer more reliable version available
Comment 60 Hanni Ali 2007-08-14 09:03:54 UTC
Created attachment 128026 [details]
drqueue-0.64.3.ebuild

I have now tested this on x86 master/slaves with an amd64 machine in there as well the ebuild just adds ~amd64 for which it compile fine, it also seems to communicate fine between archs.

Thanks for doing the newer ebuild Kazoo I couldn't seem to get scons to do what I wanted and what you have done is so simple i feel a bit embarrassed.

I have also tested the python bindings specifically for DrKeewee which seem to be working fine. This required an update to the setuptools dependency.

DrQueue has been working well for me across a few versions now. How do we get this into the main tree?

Hanni
Comment 61 kaaZoo 2007-08-14 14:14:10 UTC
I also would like to see DrQueue in the main tree and even wouldn't mind to be the maintainer for it.
But there are some problems left. First, as I mentioned in comment #52 there are still problems with stopping the master/slave services. The subprocesses don't quit when the main process quits. I think a 'killall slave.Linux.i686' in the init script should do it. But this has to be done architecture specific ([slave,master].Linux.[mips64,i686,amd64]). Just have a look at your processes when the master or slave is running.
The second thing is that all old lines in the ebuild which I commented out should be removed in order to get a clean ebuild.
Thirdly, I'm currently working on Ruby bindings for DrQueue (which are included in 0.64.3 for the first time). So there should be a dependency to ruby (and also to swig).

I will try to build such an ebuild in the next days. But you can also do it first, if you like.

Greetings, kaazoo.
Comment 62 Hanni Ali 2007-08-15 13:41:59 UTC
> But there are some problems left. First, as I mentioned in comment #52 there
> are still problems with stopping the master/slave services. The subprocesses
> don't quit when the main process quits. I think a 'killall slave.Linux.i686' in
> the init script should do it. But this has to be done architecture specific
> ([slave,master].Linux.[mips64,i686,amd64]). Just have a look at your processes
> when the master or slave is running.

Ok I have added a line:

 pkill -u root,drqueue master

to the stop() part of the scripts to take care of the child processes.

I will clean up the ebuild and upload a better one shortly, either today or tomorrow.

There is already a dependency on swig and I will add ruby, then you will just need to test the ruby stuff and we should be good to go.

Hanni

Comment 63 kaaZoo 2007-08-15 16:25:43 UTC
As a small note, this is the way the Ruby bindings are built:

cd /ruby
ruby extconf.rb
make
make install

I tested compiling the Ruby bindings on Linux once and noticed that gcc complains about libdrqueue.a not being built with '-fPIC'. I had to change the SConstruct file (see http://drqueue.org/cwebsite/drqueue_community/comments.php?DiscussionID=432&page=1#Item_0).

Comment 64 Hanni Ali 2007-08-15 20:39:09 UTC
Created attachment 128202 [details]
drqueue-0.64.3.ebuild

Ok I have update all the respective files, however I am having a bit of trouble with your ruby stuff. I seem to be getting Access Denied errors, even though it should be installing with the specified prefix...

Have a go and let me know if you know how to fix it I will also have a go.
Comment 65 Hanni Ali 2007-08-15 20:39:54 UTC
Created attachment 128204 [details, diff]
SConstruct patch for ruby use flag
Comment 66 Hanni Ali 2007-08-15 20:40:36 UTC
Created attachment 128205 [details]
files/init-drqmd-0.64.3

added ability to kill slave.master*
Comment 67 Hanni Ali 2007-08-15 20:41:40 UTC
Created attachment 128207 [details]
files/init-drqsd-0.64.3

added ability to kill master.*
Comment 68 kaaZoo 2007-08-16 11:12:03 UTC
Created attachment 128264 [details]
drqueue-0.64.3.ebuild

New ebuild checked with repoman and tested on amd64 and mips.
I had to take out ~mips keyword because setuptools are not available for mips, which means that the python use flag will not work on mips.
Comment 69 kaaZoo 2007-08-16 11:14:38 UTC
Created attachment 128267 [details]
files/init-drqsd-0.64.3

In this init file the slave instead of the master process should be killed.
Comment 70 Hanni Ali 2007-08-16 11:21:24 UTC
Comment on attachment 128207 [details]
files/init-drqsd-0.64.3

Correct, well spotted I don't know why I missed that.
Comment 71 Hanni Ali 2007-09-03 09:55:09 UTC
Just for anyone who wishes to use this ebuild I'm sticking a quick Gentoo specific How-TO on the WIKI here's the link:

http://gentoo-wiki.com/HOWTO_DrQueue%2C_Installation_and_Setup_on_Gentoo
Comment 72 kaaZoo 2007-09-10 16:47:48 UTC
Created attachment 130515 [details]
files/conf-drqsd-0.64.3

you can now set the pools in the slave configuration.

greetings, kaazoo
Comment 73 kaaZoo 2007-09-10 16:55:35 UTC
Created attachment 130517 [details]
files/conf-drqsd-0.64.3

sorry, there was something missing. now it should work.
Comment 74 Hanni Ali 2007-09-10 17:24:08 UTC
(In reply to comment #73)
> Created an attachment (id=130517) [edit]
> files/conf-drqsd-0.64.3
> 
> sorry, there was something missing. now it should work.
> 
Nice addition thanks.

ATM there is a bug in the ebuild which doesn't create a db folder on /usr/db
I can't seem to get DrQueue to start without using this directory as such I am thinking it may be simpler to change the variable DRQUEUE_ROOT="/usr/share/drqueue" and put a symlink to /usr/bin which will still be where all the binaries are installed.

What do you think of this kaaZoo?

Plus have you had a look at the HowTo on Gentoo wiki I need to add some trouble shooting help, but I'd appreciate any additions you think would be useful, someone else has also suggested I add some examples which I will do and duplicate on the official drqueue.org wiki

Hanni
Comment 75 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2009-01-15 17:59:08 UTC
(this is an automated message based on filtering criteria that matched this bug)
'EBUILD' is in the KEYWORDS which should mean that there is a ebuild attached 
to this bug.
This bug is assigned to maintainer-wanted which means that it is not in the 
main tree.

Hello, The Gentoo Team would like to firstly thank you for your ebuild 
submission. We also apologize for not being able to accommodate you in a timely
manner. There are simply too many new packages.

Allow me to use this opportunity to introduce you to Gentoo Sunrise. The 
sunrise overlay[1] is a overlay for Gentoo which we allow trusted users to 
commit to and all users can have ebuilds reviewed by Gentoo devs for entry 
into the overlay. So, the sunrise team is suggesting that you look into this 
and submit your ebuild to the overlay where even *you* can commit to. =)

Because this is a mass message, we are also asking you to be patient with us. 
We anticipate a large number of requests in a short time. 

Thanks,
On behalf of the Gentoo Sunrise Team,
Jeremy.

[1]: http://www.gentoo.org/proj/en/sunrise/
[2]: http://overlays.gentoo.org/proj/sunrise/wiki/SunriseFaq
Comment 76 Sebastian Pipping gentoo-dev 2009-12-31 22:32:32 UTC
Added to the "sping" overlay.
http://git.goodpoint.de/?p=overlay-sping.git;a=tree;f=media-gfx/drqueue

Quality level is somewhere near "beta" I would say, patches welcome.
Comment 77 Sebastian Pipping gentoo-dev 2010-01-29 15:53:12 UTC
0.64.3-r1 just hit the Gentoo tree.  Please open new bugs on anything that doesn't work about it.  Have fun.