Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 506158

Summary: sys-apps/portage: please (finally) start using EAPI=5 and python-r1
Product: Portage Development Reporter: Michał Górny <mgorny>
Component: UnclassifiedAssignee: Portage team <dev-portage>
Status: RESOLVED FIXED    
Severity: normal CC: hasufell, pacho, rodolphe.rocca
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
See Also: https://bugs.gentoo.org/show_bug.cgi?id=506328
https://bugs.gentoo.org/show_bug.cgi?id=506546
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 484436, 488776, 520770    
Attachments: portage-2.2.12.ebuild.diff

Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-03-29 14:35:38 UTC
The portage ebuild has large parts of python-r1 inlined which is nowhere close to neat, plus it uses custom USE flags to control Python implementations.

Considering that profiles are EAPI=5 now, please finally switch *new* portage ebuilds to use EAPI=5 and python-r1. Thanks.
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-04-06 16:40:22 UTC
After my failed first attempt, a few notes to consider:

1. it would be really useful to get rid of all that pseudo-compatibility non-sense with prepending /usr/lib/portage everywhere and just letting portage use proper sys.path (proper impls are enforced by PYTHON_TARGETS anyway),

2. setup.py would be a blessing. Right now, the build system is split in half between ugly Makefile and even uglier code in the ebuild. Proper setup.py would handle choosing proper directories, setting shebangs and other stuff that we need to hack manually now.

3. and yes, use python-exec for Python executables.
Comment 2 Julian Ospald 2014-07-19 19:37:43 UTC
I'd say inlining python-r1 stuff like that is a serious QA violation.

Please talk to the python team before comitting such... things.
Comment 3 Julian Ospald 2014-08-24 15:06:10 UTC
Created attachment 383518 [details, diff]
portage-2.2.12.ebuild.diff

This is not a clean solution, but still better than the current situation which breaks various ebuilds.

I'll commit this in a few days because of maintainer timeout, unless someone has a better patch.
Comment 4 Brian Dolbec (RETIRED) gentoo-dev 2014-08-24 15:49:29 UTC
1) There is an active portage team.

2) A proper solution is being worked on and is currently being reviewed, it involves a new setup.py which does the installation like normal python pkgs.

3) If you want a patch applied to anything portage, please submit the patches to the gentoo-portage-dev mail list for review and consideration.

4) Unless you are part of the QA team, there is no "maintainer timeout" on an active project like portage.  By the way there are 2 active QA team members on the portage team as well.


Submitting a patch with a few days deadline is now considered a "maintainer timeout"?  REALLY!
Comment 5 Julian Ospald 2014-08-24 15:57:12 UTC
(In reply to Brian Dolbec from comment #4)
>
> 2) A proper solution is being worked on and is currently being reviewed, it
> involves a new setup.py which does the installation like normal python pkgs.
> 

Then I expect an answer on this bug report. How could anyone know otherwise?
Comment 6 Brian Dolbec (RETIRED) gentoo-dev 2014-08-24 16:14:04 UTC
The patch submital thread:
http://blog.gmane.org/gmane.linux.gentoo.portage.devel

oh, and mgorny is now part of the portage team as well as on the python team.
Comment 7 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-08-24 20:41:27 UTC
I already have a better build system :). It's just awaiting review, or rather till I lose my patience and fork portage.
Comment 8 Julian Ospald 2014-08-24 21:15:10 UTC
Whatever you do, do it quick. This already affects stable arch and I am losing patience too since the bug is almost half a year old.

Users are complaining on mailing lists.
Comment 9 Brian Dolbec (RETIRED) gentoo-dev 2014-08-24 22:00:38 UTC
The changes can not be applied just yet.  So far testing with catalyst fails.  Catalyst needs at least one update to work with it correctly, possibly more.

It'll be applied as soon as everything is tested working correctly.  Then it will take some time in the tree before being stabilized.

Not everything can be done INSTANTLY!
Comment 10 Julian Ospald 2014-08-24 22:49:52 UTC
(In reply to Brian Dolbec from comment #9)
> The changes can not be applied just yet.  So far testing with catalyst
> fails.  Catalyst needs at least one update to work with it correctly,
> possibly more.
> 

You mean catalyst with the above patch fails?

> 
> Not everything can be done INSTANTLY!

That's not the point.

If the proper solution takes longer, then we have to go with an intermediate solution first (running the above patch since months, but I don't know about catalyst).

We have users with unresolvable blockers on stable arch. That's not really a "normal" severity thing, nor is it easy to explain to them how to fix it.
Comment 11 Joakim Tjernlund 2014-09-01 07:29:32 UTC
Could the new portage ebuild also add epatch_user?
Comment 12 Brian Dolbec (RETIRED) gentoo-dev 2014-09-01 15:43:52 UTC
(In reply to Joakim Tjernlund from comment #11)
> Could the new portage ebuild also add epatch_user?

No, our portage team decision on that has been that epatch_user be allowed on portage-9999, but not regular releases.

two main reasons:

1) portage-9999 users normally know more what they are doing, plus it allows easier testing of proposed patches.  These patches would be applied to current portage code.  Not possibly older release versions of the code which could have completely different results or consequences.

2) We want patches submitted for review and possible inclusion.  We also don't want to have bugzilla flooded with bugs which are due to bad or unknown user applied patches to portage which we have no way of detecting from emerge --info

In summary if a user wants to patch portage releases, then they must be able to make patched versions of the release ebuilds, etc...  That requires more system knowledge and is less likely to be playing havoc with their system and then blame us or some innocent pkg.
Comment 13 Joakim Tjernlund 2014-09-01 18:59:37 UTC
(In reply to Brian Dolbec from comment #12)
> (In reply to Joakim Tjernlund from comment #11)
> > Could the new portage ebuild also add epatch_user?
> 
> No, our portage team decision on that has been that epatch_user be allowed
> on portage-9999, but not regular releases.
> 
> two main reasons:
> 
> 1) portage-9999 users normally know more what they are doing, plus it allows
> easier testing of proposed patches.  These patches would be applied to
> current portage code.  Not possibly older release versions of the code which
> could have completely different results or consequences.
> 
> 2) We want patches submitted for review and possible inclusion.  We also
> don't want to have bugzilla flooded with bugs which are due to bad or
> unknown user applied patches to portage which we have no way of detecting
> from emerge --info

Why would 2) prevent possible inclusion? Considering how long time it can take to get a patch applied it would make perfekt sense to allow epatch_user

Perhaps emerge --info could be taught to report if there were any user patches applied?
Comment 14 Brian Dolbec (RETIRED) gentoo-dev 2014-09-01 19:17:56 UTC
adding code to portage to detect and report user patches would be difficult and produce further patches to remove it.  It is not so much the patch originators that we are concerned about.

What happens if a submitted patch is rejected for good reasons, but the patch is floating around the web and users are blindly adding it because someone said it fixed 'blah'.  Then that patch affected other code which in turn...

It simply makes too many unknowns and bug fixing even harder.
If someone needs a custom patch for a group of servers, they know how to do it for a release and maintain their patch between release versions.  These users are not the problem or the concern.  Adding epatch_user for them would make it easier, but..

Epatch_user will circumvent the knowledge barrier making it too easy for anyone to apply patches willy-nilly to a key utility that can affect their entire system adversely.
Comment 15 Julian Ospald 2014-09-01 19:57:12 UTC
Not including epatch_user has never stopped gentoo users from hacking on packages. So while I don't see the point brian is trying to get at, I don't see a reason to strongly debate over it. Just hack the ebuild. You can even use phase hooks to let pkg_setup die when it's not from your own overlay, so you don't have to mask the in-tree ebuilds.
Comment 16 Thomas Deutschmann (RETIRED) gentoo-dev 2014-09-03 17:37:17 UTC
I requested epatch_user support in the past (bug 491256).

(In reply to Brian Dolbec from comment #12)
> [...]
> 
> In summary if a user wants to patch portage releases, then they must be able
> to make patched versions of the release ebuilds, etc...  That requires more
> system knowledge and is less likely to be playing havoc with their system
> and then blame us or some innocent pkg.

From developer perspective I can understand all your arguments. But I disagree with you:

1) Why is portage special? In other words: Aren't your arguments valid for any other ebuild? So when we would follow your argumentation, we would have to ban epatch_user in general... not?

2) It is nice to see that -9999 supports epatch_user. But if you are using a stable version and just want to patch something in that version you don't want to deal will all the other changes from -9999 your are not familiar with.

3) Your main argument seems to be that you don't want to waste your time with broken portage versions due to local patches from inexperienced users.
You admitted in comment 14 that the ban of epatch_user will hurt people who know what they are doing. Be honest, you only hurting people. There are multiple ways to apply custom patches. You cannot solve your initial problem with banning epatch_user. You are only forcing people to use dirty tricks like the bashrc way [1].

So for me all your arguments aren't valid at the end.


[1] http://wiki.gentoo.org/wiki//etc/portage/patches
Comment 17 Julian Ospald 2014-09-03 17:54:39 UTC
(In reply to Thomas D. from comment #16)

User patching is NOT an EAPI/PM feature, so it's a maintainer decision until that day. Also, it is unrelated to this bug report.
Comment 18 Zac Medico gentoo-dev 2014-09-21 02:11:10 UTC
This is fixed in 2.2.13:

https://github.com/gentoo/portage/commit/0cc4c1ac21a2ea94cfb1f6ff4b461a9e349d47df