Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 454752 - >=app-portage/gentoolkit-0.3.0 eclean-dist and eclean-pkg links broken by python-exec
Summary: >=app-portage/gentoolkit-0.3.0 eclean-dist and eclean-pkg links broken by pyt...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Third-Party Tools (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage Tools Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-01-31 01:21 UTC by Brian Dolbec
Modified: 2013-09-05 17:26 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Brian Dolbec (RETIRED) gentoo-dev 2013-01-31 01:21:39 UTC
This is similar to bug 445664, but also includes the failure of python-exec to pass on argv[0]

eg:

big_daddy brian # eclean-dist
/usr/bin/eclean-dist: no supported Python implementation variant found!
big_daddy brian # eclean-pkg
/usr/bin/eclean-pkg: no supported Python implementation variant found!
big_daddy brian # ls -l /usr/bin/eclean*
lrwxrwxrwx 1 root root   11 Jan 27 22:52 /usr/bin/eclean -> python-exec
lrwxrwxrwx 1 root root    6 Jan 27 22:52 /usr/bin/eclean-dist -> eclean
lrwxrwxrwx 1 root root   11 Oct 23 22:19 /usr/bin/eclean-kernel -> python-exec
-rwxr-xr-x 1 root root  635 Oct 23 22:19 /usr/bin/eclean-kernel-python2.7
-rwxr-xr-x 1 root root  635 Oct 23 22:19 /usr/bin/eclean-kernel-python3.2
lrwxrwxrwx 1 root root    6 Jan 27 22:52 /usr/bin/eclean-pkg -> eclean
-rwxr-xr-x 1 root root 1073 Jan 27 22:52 /usr/bin/eclean-python2.6
-rwxr-xr-x 1 root root 1073 Jan 27 22:52 /usr/bin/eclean-python2.7
-rwxr-xr-x 1 root root 1073 Jan 27 22:52 /usr/bin/eclean-python3.2
big_daddy brian # 

This symlink feature of using the name it was started with has been part of eclean since the start and worked previously using the python and distutils eclasses.

Reproducible: Always
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-01 12:10:39 UTC
This is not really a failure but a removal of an ugly hack from the past. To be honest, this one is a mess to maintain. We basically rename the scripts, so we shouldn't replace argv[0] with something fake. Especially that it is not an issue specific to Python, yet it is hacked like this only in Gentoo Python patch sets.

The major issue I see is that we can end up with two kinds of scripts:

- 'main' scripts which are properly versioned,

- symlinks which are not properly versioned.

And this is a mess. To fix it, we'd have to version symlinks.

If we version symlinks, then direct call to 'eclean-dist-fooX.Y' will no longer work as expected (because once again argv[0] will be different). And that starts another kind of mess.

To be honest, I'd just drop the symlinks and use proper, safe wrappers. Something that is completely expectable and doesn't start to work differently when user renames or symlinks it under another name.
Comment 2 Brian Dolbec (RETIRED) gentoo-dev 2013-02-16 16:51:37 UTC
I agree the old python eclass was a mess of ugly hacks.

I can see that that python-exec is limited in it's ability to adapt.  To me it looks like this is a failing of the way python-exec is designed.  Not that I may be able to do anything about it.

For the record, those eclean-dist and eclean-pkg synmlinks were part of eclean's install long before the mess that became the python eclass.
Comment 3 Mike Gilbert gentoo-dev 2013-02-16 17:11:48 UTC
(In reply to comment #2)
> I can see that that python-exec is limited in it's ability to adapt.  To me
> it looks like this is a failing of the way python-exec is designed.  Not
> that I may be able to do anything about it.
> 

Yes, it is a flaw in python-exec -- but one that might not be worth fixing. It seems to be very rare for a python package to depend on the value of sys.argv[0]. Personally, I don't think it is too much effort to make this work, but good luck changing mgorny's mind.

If you really don't want to change gentoolkit, here are a couple of alternatives:

You could eliminate the python-exec stuff and install gentoolkit scripts with #!/usr/bin/python shebangs. This may cause interesting failures if the user has an unsupported version of python selected, however.

Or you could install your own wrapper to deal with Gentoo's multi-python environment.
Comment 4 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-16 18:04:43 UTC
(In reply to comment #2)
> I agree the old python eclass was a mess of ugly hacks.
> 
> I can see that that python-exec is limited in it's ability to adapt.  To me
> it looks like this is a failing of the way python-exec is designed.  Not
> that I may be able to do anything about it.
> 
> For the record, those eclean-dist and eclean-pkg synmlinks were part of
> eclean's install long before the mess that became the python eclass.

Well, the more important mess here is the patches that we apply to Python sources to make those hacks work. One of the goals of python-exec was to try to get rid of reliance on those patches while converting ebuilds to -r1 which would hopefully allow us to use upstream-compliant Python at some point.

While I understand the reasons for argv[0] passing, I believe Python in Gentoo is not the place to start working of it. This is an issue common to all uses of shebang and if we really want to change that, I think we should try to do it in a way consistent to all scripting languages. Even better, in a way which could leave Gentoo and go upstream.

And while I understand you wanting to keep things as they are, I don't think installing additional wrappers would cause problems to anyone. I'm thinking of scripts like:

  #!/usr/bin/eclean dist
  import os, sys
  os.execv('/usr/bin/eclean', sys.argv)

(written from memory, untested)

so that the script would launch 'eclean' directly when invoked via shebang, and exec to it when invoked via Python interpreter. Of course, you could go step further and just maintain three frontends, or deprecate them in favor of one variant.
Comment 5 Brian Dolbec (RETIRED) gentoo-dev 2013-02-16 19:05:15 UTC
I understand the ways to "work around " the python-exec limitation.  I also am all for cleaning up python patching.

But aside from this one breakage for eclean.  This very same problem prevents all users from symlinking any python script, even those without the need of checking it's argv[0] value.

I just did some testing, Brian Harring's python-shebang does not suffer from python-execs symlink limitation.  It also looks like it would be an overall cleaner solution.
Comment 6 Mike Gilbert gentoo-dev 2013-02-16 19:25:11 UTC
We are still waiting for Brian to actually implement the eclass changes for that. If someone else wants to take a crack at it, I'm happy to review it.
Comment 7 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-02-16 22:37:34 UTC
(In reply to comment #5)
> I understand the ways to "work around " the python-exec limitation.  I also
> am all for cleaning up python patching.
> 
> But aside from this one breakage for eclean.  This very same problem
> prevents all users from symlinking any python script, even those without the
> need of checking it's argv[0] value.

The symlink problem is a separate bug to solve, and I will fix it if I don't forget about it again. In the meantime, please focus on the argv[0] issue here.
Comment 8 Paul Varner (RETIRED) gentoo-dev 2013-09-03 20:56:18 UTC
One of us needs to step up and come up with a fix for the symlink and the argv[0] issue.  

I really need to release gentoolkit-0.3.0.8.

If I use the new python eclasses, the symlinks break and a lot of user's scripts will break.

The other alternative is to release it using the deprecated python eclasses and ignore the warnings from repoman.
Comment 9 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-09-03 21:02:55 UTC
I've suggested possible solution on the ml but didn't get a single reply.

Alternative is to give proper 'eclean-dist' and 'eclean-pkg' wrappers that simply exec 'eclean', or the other way around. Without playing dirty with argv[0].
Comment 10 Mike Gilbert gentoo-dev 2013-09-03 23:00:00 UTC
(In reply to Michał Górny from comment #9)
> I've suggested possible solution on the ml but didn't get a single reply.

Sorry, I must have missed it, or had no useful feedback. Link?
Comment 11 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2013-09-04 07:23:07 UTC
http://thread.gmane.org/gmane.linux.gentoo.python/541
Comment 12 Paul Varner (RETIRED) gentoo-dev 2013-09-04 22:02:47 UTC
For now, I've worked around it by making eclean-dist and eclean-pkg copies of eclean and making the argv[0] test less strict.

I went with this solution, since it was the least intrusive change to the code that worked.  We can revisit it later, if necessary.

As far as the proposal goes for using directories instead of versioned names, it sounds okay to me, but I would like to play with a working implementation.
Comment 13 Paul Varner (RETIRED) gentoo-dev 2013-09-05 17:26:26 UTC
The workaround is in gentoolkit-0.3.0.8, we can revisit things as the new eclasess evolve.