Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 474968 - app-admin/webapp-config fails when sys-apps/portage has not been installed for a supported python version
Summary: app-admin/webapp-config fails when sys-apps/portage has not been installed fo...
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Development (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Web Application Packages Maintainers
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-06-27 11:11 UTC by Bruno
Modified: 2023-01-28 20:30 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info (django-emerge-info.txt,14.01 KB, text/plain)
2013-06-27 11:12 UTC, Bruno
Details
Ebuild patch for webapp-config (webapp-config.patch,2.32 KB, patch)
2013-07-03 15:02 UTC, Mike Gilbert
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Bruno 2013-06-27 11:11:15 UTC
I'm trying to install django-1.4.5 on a system with portage.

For the system I'm using PYTHON_TARGETS="python3_2" (e.g. no python-2.7) but for django and its dependencies that are python-2.7-only I have set PYTHON_TARGETS="python2_7".

When trying to emerge django I get a python traceback.

Reproducible: Always

Actual Results:  
[ebuild   R    ] sys-apps/portage-2.1.12.2  USE="(ipc) (xattr) -build -doc -epydoc (-pypy2_0) -python2 -python3 (-selinux)" LINGUAS="-pl -ru" PYTHON_TARGETS="python3_2 (-pypy1_9) (-pypy2_0) -python2_6 -python2_7 -python3_1 (-python3_3) (-python3_4)" 0 kB

[ebuild  N     ] dev-python/django-1.4.5  USE="mysql vhosts -doc -postgres -sqlite {-test}" PYTHON_TARGETS="python2_7 -python2_5 -python2_6" 0 kB

>>> Emerging (1 of 4) dev-python/django-1.4.5
 * Django-1.4.5.tar.gz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                                                                                  [ ok ]
Traceback (most recent call last):
  File "/usr/sbin/webapp-config-2.7", line 27, in <module>
    from WebappConfig.config import Config
  File "/usr/libx32/python2.7/site-packages/WebappConfig/config.py", line 29, in <module>
    import WebappConfig.server
  File "/usr/libx32/python2.7/site-packages/WebappConfig/server.py", line 25, in <module>
    from WebappConfig.worker       import WebappRemove, WebappAdd
  File "/usr/libx32/python2.7/site-packages/WebappConfig/worker.py", line 29, in <module>
    import WebappConfig.wrapper as wrapper
  File "/usr/libx32/python2.7/site-packages/WebappConfig/wrapper.py", line 36, in <module>
    import sys, portage, os, types
ImportError: No module named portage


Expected Results:  
As package manager and package are unrelated their respective python version needs should not clash and install should succeed.
Comment 1 Bruno 2013-06-27 11:12:59 UTC
Created attachment 352078 [details]
emerge --info
Comment 2 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2013-06-30 08:14:48 UTC
They are related, because of the web application configuration.
Comment 3 Arfrever Frehtes Taifersar Arahesis 2013-06-30 08:29:09 UTC
app-admin/webapp-config should depend on sys-apps/portage with appropriate USE flags enabled.
Comment 4 Bruno 2013-06-30 10:04:51 UTC
I would hope webapps-related tasks would depend/use Portage features using Portage's Python target(s) and be unrelated to ebuild's possible python dependency.
That way a different package manager that's no based on python and properly implements webapps would not cause non-python web-packages to need python.
Comment 5 Mike Gilbert gentoo-dev 2013-07-01 20:08:56 UTC
Copying webapp-config maintainers.
Comment 6 Anthony Basile gentoo-dev 2013-07-01 23:53:45 UTC
> Expected Results:  
> As package manager and package are unrelated their respective python version
> needs should not clash and install should succeed.

I don't full understand the situation: you have PYTHON_TARGETS="python3_2" in your make.conf but you are installing a version of django which has RESTRICT_PYTHON_ABIS="3.*" and PYTHON_DEPEND="2".  Isn't this like saying I only want python 3.2 on my system but I'm going to install a package which requires python 2.x?

Notwithstanding that, I get your point that the python-dep needs of the package manager, the package and webapp-config are independant.  We can get that working but I'm stuck on the original contradiction.
Comment 7 Bruno 2013-07-02 16:23:33 UTC
(In reply to Anthony Basile from comment #6)
> > Expected Results:  
> > As package manager and package are unrelated their respective python version
> > needs should not clash and install should succeed.
> 
> I don't full understand the situation: you have PYTHON_TARGETS="python3_2"
> in your make.conf but you are installing a version of django which has
> RESTRICT_PYTHON_ABIS="3.*" and PYTHON_DEPEND="2".  Isn't this like saying I
> only want python 3.2 on my system but I'm going to install a package which
> requires python 2.x?

Initially the system just had python3_2 for portage and python3_2 being the only one in PYTHON_TARGETS.

Then I did want to install django (with imaging and python-mysql - didn't check their exact package names) which unfortunately are still python2_7-only.

The base idea is to be able to install django via binpkg to a chroot without pulling more than needed.
I have already too much in /etc/portage/profile/package.provided and I don't want to add even more to get unneeded multi-python deps sorted out.

Changing the whole system to python2_7-only is hard as some portage tools (revdep-rebuild) moved to requiring python3_2.
Comment 8 Anthony Basile gentoo-dev 2013-07-03 12:42:35 UTC
(In reply to Bruno from comment #7)
> (In reply to Anthony Basile from comment #6)
> > > Expected Results:  
> > > As package manager and package are unrelated their respective python version
> > > needs should not clash and install should succeed.
> > 
> The base idea is to be able to install django via binpkg to a chroot without
> pulling more than needed.

Got it.

(In reply to Arfrever Frehtes Taifersar Arahesis from comment #3)
> app-admin/webapp-config should depend on sys-apps/portage with appropriate
> USE flags enabled.

I'm not sure what you are suggesting here.  But I think if we push out another release of webapp-config that is both python2/3 we should be good.  I think we are there, but I wanted to clean up some bugs first.  Admittedly webapp-config has been low on my priority.
Comment 9 Mike Gilbert gentoo-dev 2013-07-03 15:02:07 UTC
Created attachment 352534 [details, diff]
Ebuild patch for webapp-config

I would recommend migrating the ebuild to distutils-r1 and adding a "portage" use flag to enforce python_targets on sys-apps/portage.

See patch.
Comment 10 Mike Gilbert gentoo-dev 2013-07-03 15:06:47 UTC
As another fallback option, you could override sys.path in WebappConfig/wrapper.py. Something like this should work:

sys.path.append("/usr/lib/portage/pym")

This problem was probably introduced when we removed a patch from the dev-lang/python ebuild which hardcoded /usr/lib/portage/pym in sys.path at a global level.
Comment 11 Mike Gilbert gentoo-dev 2013-07-03 15:10:39 UTC
Reassigning and updating summary.
Comment 12 Anthony Basile gentoo-dev 2013-07-05 00:35:01 UTC
(In reply to Mike Gilbert from comment #9)
> Created attachment 352534 [details, diff] [details, diff]
> Ebuild patch for webapp-config
> 
> I would recommend migrating the ebuild to distutils-r1 and adding a
> "portage" use flag to enforce python_targets on sys-apps/portage.
> 
> See patch.

I like this solution, it is the most natural one.  Its in webapp-config-1.50.19-r1 which is now on the tree.

@Bruno, can you please test this and see if it fixes your problem?  If so I'll close this.


(In reply to Mike Gilbert from comment #10)
> As another fallback option, you could override sys.path in
> WebappConfig/wrapper.py. Something like this should work:
> 
> sys.path.append("/usr/lib/portage/pym")
> 
> This problem was probably introduced when we removed a patch from the
> dev-lang/python ebuild which hardcoded /usr/lib/portage/pym in sys.path at a
> global level.

I don't like this.  It hard codes a lib path and is not multilib friendly.  Let's go with the solution which properly reflects the dependencies.

Thanks Mike!
Comment 13 Mike Gilbert gentoo-dev 2013-07-05 01:28:38 UTC
(In reply to Anthony Basile from comment #12)
> I don't like this.  It hard codes a lib path and is not multilib friendly. 

Portage is always installed in /usr/lib/portage. multilib is not a factor.
Comment 14 Bruno 2013-07-05 13:13:11 UTC
(In reply to Anthony Basile from comment #12)
> (In reply to Mike Gilbert from comment #9)
> > Created attachment 352534 [details, diff] [details, diff] [details, diff]
> > Ebuild patch for webapp-config
> > 
> > I would recommend migrating the ebuild to distutils-r1 and adding a
> > "portage" use flag to enforce python_targets on sys-apps/portage.
> > 
> > See patch.
> 
> I like this solution, it is the most natural one.  Its in
> webapp-config-1.50.19-r1 which is now on the tree.
> 
> @Bruno, can you please test this and see if it fixes your problem?  If so
> I'll close this.

If I say:
[ebuild   R   ~] app-admin/webapp-config-1.50.19-r1  USE="-portage" PYTHON_TARGETS="python2_7 (-pypy1_9) (-pypy2_0) -python2_6" 0 kB
[ebuild   R    ] sys-apps/portage-2.1.12.2  USE="(ipc) (xattr) -build -doc -epydoc (-pypy2_0) -python2 -python3 (-selinux)" LINGUAS="-pl -ru" PYTHON_TARGETS="python3_2 (-pypy1_9) (-pypy2_0) -python2_6 -python2_7 -python3_1 (-python3_3) (-python3_4)" 0 kB
[ebuild  N     ] dev-python/django-1.4.5  USE="mysql vhosts -doc -postgres -sqlite {-test}" PYTHON_TARGETS="python2_7 -python2_5 -python2_6" 0 kB

I get:

>>> Emerging (1 of 1) dev-python/django-1.4.5
 * Django-1.4.5.tar.gz SHA256 SHA512 WHIRLPOOL size ;-) ...                                                                                                                                                                  [ ok ]
* Fatal error: Portage libraries not found, quitting:
* Fatal error: No module named portage
* Fatal error(s) - aborting
 * ERROR: dev-python/django-1.4.5 failed (setup phase):
 *   Could not read settings from webapp-config!
 * 
 * Call stack:
 *       ebuild.sh, line  93:  Called pkg_setup
 *       ebuild.sh, line 324:  Called webapp_pkg_setup
 *   webapp.eclass, line 392:  Called webapp_read_config
 *   webapp.eclass, line  60:  Called die
 * The specific snippet of code:
 *              ENVVAR=$(${WEBAPP_CONFIG} --query ${PN} ${PVR}) || die "Could not read settings from webapp-config!"
 * 
 * If you need support, post the output of `emerge --info '=dev-python/django-1.4.5'`,
 * the complete build log and the output of `emerge -pqv '=dev-python/django-1.4.5'`.
 * The complete build log is located at '/var/tmp/portage/dev-python/django-1.4.5/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/dev-python/django-1.4.5/temp/die.env'.
 * Working directory: '/usr/lib/portage/pym'
 * S: '/var/tmp/portage/dev-python/django-1.4.5/work/Django-1.4.5'


That might match the description of "portage" use flag for webapp-config but
is still broken.
Either portage use-flag must prevent webapp-config from using portage when unset
or the dep must be permanent (I prefer the first option).


Probably part of the issue is that webapp-config is limited to python2_X targets
while it should be synchronised to portage's python targets (dep and support)
if it plans to make use of portage.


When I enable portage use-flag on webapp-config the dep wants portage's targets to be adjusted.
With that one adjusted, emerge succeeds.
Comment 15 Mike Gilbert gentoo-dev 2013-07-05 15:33:16 UTC
(In reply to Bruno from comment #14)

Portage is an option runtime dep. It is not possible to perfectly implement dependencies at build time.

I don't think any sane person would expect webapp-config to work when configured with USE="-portage" and package_manager="portage" in the config file.
Comment 16 Anthony Basile gentoo-dev 2013-07-05 18:12:07 UTC
(In reply to Mike Gilbert from comment #13)
> (In reply to Anthony Basile from comment #12)
> > I don't like this.  It hard codes a lib path and is not multilib friendly. 
> 
> Portage is always installed in /usr/lib/portage. multilib is not a factor.

Yeah I noticed this after I posted.  Its off topic for this bug, but is that correct?

(In reply to Mike Gilbert from comment #15)
> (In reply to Bruno from comment #14)
>
> I don't think any sane person would expect webapp-config to work when
> configured with USE="-portage" and package_manager="portage" in the config
> file.

WebappConfig/wrapper.py imports portage when pm == portage around line 48, so it can't run without it.

Bruno, can I better understand your chroot situation.  Where is django, webapp-config and portage install with respect to your chroot?  I'm guessing that django and webapp-config are installed inside the chroot, but that you don't have portage installed there?
Comment 17 Bruno 2013-07-24 06:36:12 UTC
(In reply to Anthony Basile from comment #16)
> Bruno, can I better understand your chroot situation.  Where is django,
> webapp-config and portage install with respect to your chroot?  I'm guessing
> that django and webapp-config are installed inside the chroot, but that you
> don't have portage installed there?

I have portage, webapp-config and django installed on the build system, but only django installed in the chroot (emerged from binary packages built on build system with ROOT=/chroot and PORTAGE_CONFIGROOT=/chroot).

On python side, I have both python-2.7 and python-3.2 on build side but only python-2.7 on chroot side (for django which is python-2.7-only)