Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 622492 - dev-util/buildbot-worker PYTHON_TARGETS and initd
Summary: dev-util/buildbot-worker PYTHON_TARGETS and initd
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Markos Chandras (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-06-23 00:30 UTC by Daniel M. Weeks
Modified: 2018-03-13 23:57 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel M. Weeks 2017-06-23 00:30:02 UTC
If system python is not one in buildbot-worker PYTHON_TARGETS initd script fails to start buildbot-worker. For example: selecting python3.4 as the system default causes the following:

Unhandled Error
Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/twisted/application/app.py", line 662, in run
    runApp(config)
  File "/usr/lib64/python3.4/site-packages/twisted/scripts/twistd.py", line 25, in runApp
    _SomeApplicationRunner(config).run()
  File "/usr/lib64/python3.4/site-packages/twisted/application/app.py", line 380, in run
    self.application = self.createOrGetApplication()
  File "/usr/lib64/python3.4/site-packages/twisted/application/app.py", line 445, in createOrGetApplication
    application = getApplication(self.config, passphrase)
--- <exception caught here> ---
  File "/usr/lib64/python3.4/site-packages/twisted/application/app.py", line 456, in getApplication
    application = service.loadApplication(filename, style, passphrase)
  File "/usr/lib64/python3.4/site-packages/twisted/application/service.py", line 412, in loadApplication
    application = sob.loadValueFromFile(filename, 'application')
  File "/usr/lib64/python3.4/site-packages/twisted/persisted/sob.py", line 177, in loadValueFromFile
    eval(codeObj, d, d)
  File "/var/lib/buildbot_worker/buildbot.tac", line 5, in <module>
    from buildbot_worker.bot import Worker
builtins.ImportError: No module named 'buildbot_worker'


Failed to load application: No module named 'buildbot_worker'

I believe a straightforward remedy is adding "--env EPYTHON=python2.7" to the start-stop-daemon options in initd. However, I think to be more robust this would need to be done dynamically by the ebuild depending on PYTHON_TARGET.

Reproducible: Always
Comment 1 Daniel M. Weeks 2017-06-23 00:39:35 UTC
Additional information: This is the same problem as 591150 (for dev-util/buildbot). That was resolved by forcing python2 in initd, however, the buildbot ebuilds recently gained a python3.5 capability. In some ways this is a regression on 591150 since PYTHON_TARGETS="python3_5" for dev-util/buildbot will break initd.
Comment 2 Jonas Stein gentoo-dev 2017-06-24 11:16:37 UTC
Thank you for the report. Please create one ticket per package.
Comment 3 Brian Dolbec (RETIRED) gentoo-dev 2017-06-24 15:46:12 UTC
Well, thank you for the reminder.  I will update the init script to not force python2 only.  But I will only do this after I am able to do some testing on python3.  I do know that buildbot-worker had full python3 support before master, but there were compatibility issues with the master on py2 and the workers on py3.

Python3 is still considered experimental in upstream buildbot.  But it is ready for some testing.  The latest 0.9.8 release along with twisted-17.5 release has numerous python3 fixes.  If you wish, go ahead and change the init script to not force python2.  Please report any problems/patches to fix those issues upstream.  I know they would be very appreciative of those.  

But for production, it is still recommended to stay with python2.

P.S. I will be adding python3.6 to compat as well once all deps have been updated for that target.
Comment 4 Brian Dolbec (RETIRED) gentoo-dev 2018-03-13 23:57:16 UTC
Fixed in 0.9.12 releases and newer.

All older releases are now cleaned from the tree.
The desired run time python version is configured in the conf.d file.

Please be careful to have the worker's python version match the buildbot-master.

ie: do not run one on python2 and the other on python3.  This is a known failure.
It may be possible to use different versions of python3 between the master and the worker.  But you should test your build scripts work correctly.