| Summary: | GLEP 27 needs to be implemented (Portage Management of UIDs/GIDs) | ||
|---|---|---|---|
| Product: | Gentoo Hosted Projects | Reporter: | SpanKY <vapier> |
| Component: | PMS/EAPI | Assignee: | PMS/EAPI <pms> |
| Status: | RESOLVED WONTFIX | ||
| Severity: | major | CC: | alchark, alexander, alexxy, aross, base-system, bertrand, billie, bugs+gentoo, bugzilla-gentoo, c.affolter, chrb, darkside, donald.r.gray, dragonheart, drobbins, dschridde+gentoobugs, esigra, gengor, gentoo_bugs, gurligebis, jaco, Jimmy.Jazz, joakim.tjernlund, kfm, leho, linuxblues, mail, marcelocoelho, max.gentoo.bugzilla, mgorny, narancs, nolan, pacho, pentek.imre, plaes, radek, rafallalik, releng, rentorbuy, roman.zilka, staff, steeeeeveee, tcunha, Tiger683, ufs, vax456, wolf31o2, wolf31o2, yuanyelele, yuval |
| Priority: | High | Keywords: | Inclusion |
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | All | ||
| URL: | https://www.gentoo.org/glep/glep-0027.html | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Bug Depends on: | 541406 | ||
| Bug Blocks: | 137867, 138388, 174380, 284718, 395961, 549360 | ||
|
Description
SpanKY
2004-06-07 19:43:26 UTC
Someone needs to write the DTDs or whatever... Create the parser and dev-portage cna handle the rest. *** Bug 58741 has been marked as a duplicate of this bug. *** *** Bug 78741 has been marked as a duplicate of this bug. *** *** Bug 42320 has been marked as a duplicate of this bug. *** *** Bug 107514 has been marked as a duplicate of this bug. *** *** Bug 134081 has been marked as a duplicate of this bug. *** *** Bug 135915 has been marked as a duplicate of this bug. *** *** Bug 113298 has been marked as a duplicate of this bug. *** *** Bug 113298 has been marked as a duplicate of this bug. *** *** Bug 142637 has been marked as a duplicate of this bug. *** *** Bug 149157 has been marked as a duplicate of this bug. *** *** Bug 149996 has been marked as a duplicate of this bug. *** *** Bug 169345 has been marked as a duplicate of this bug. *** *** Bug 170197 has been marked as a duplicate of this bug. *** *** Bug 171378 has been marked as a duplicate of this bug. *** *** Bug 184981 has been marked as a duplicate of this bug. *** *** Bug 180797 has been marked as a duplicate of this bug. *** [11:32pm] <mkelly32> also, if anyone wants to take on my glep-27 stuff (really, just needs to be plugged into portage) [11:32pm] <mkelly32> the source is around [11:33pm] <grobian> glep27 is? [11:34pm] <mkelly32> ENEWUSERS and ENEWGROUPS variables, keeping info for users and groups added by enewuser/enewgroup in one place [11:34pm] <mkelly32> it was my summer of code project a year ago, never got it pushed into portage [11:34pm] <grobian> ohw yeah [11:34pm] <grobian> I recall that [11:34pm] <mkelly32> but i have pretty flowcharts and such showing how to do it [11:36pm] <mkelly32> basically, it would be really easy to do if portage had phase hooks, or something similar, but a patch i wrote for that was rejected at the time [11:37pm] <mkelly32> for someone willing to add code for it directly into portage, the flowcharts and such at http://www.pioto.org/~pioto/creandus/API.html#public-api-usage-flowchart will be useful [11:38pm] <mkelly32> and code is at svn://svn.pioto.org/creandus [11:38pm] <mkelly32> with some samples of stuff from the tree side in the overlay/ dir [11:40pm] <jakub> is that something for Bug 53269 ? [11:40pm] <jeeves> jakub: https://bugs.gentoo.org/53269 maj, P2, All, vapier@gentoo.org->dev-portage@gentoo.org, NEW, pending, GLEP 27 needs to be implemented (Portage Management of UIDs/GIDs) [11:41pm] <mkelly32> most likely [11:41pm] <mkelly32> i don't think i have an active bugzilla account anymore [11:41pm] <mkelly32> so would you mind adding what i said in a comment or something? *** Bug 207596 has been marked as a duplicate of this bug. *** *** Bug 219225 has been marked as a duplicate of this bug. *** What about a quick fix? Give the user a bash-friendly script to install the necessary things that a certain package needs after the installation. cp /etc/passwd $ROOT/etc/passwd Maybe there is another and probably easier way of managing groups and
users in gentoo instead of creandus. I don't know the approach of
creandus and this is just an idea which came into my mind.
What about modifying enewuser and enewgroup in eutils.eclass to manage a
few files which store information about groups and users.
/etc/portage/groups/{system,pm,local}
/etc/portage/users/{system,pm,local}
Where system is for the default set of groups/users coming with the
releases. pm for the ones needed by installed ebuilds and managed by
the package manager and local for software installed without the control
of the package manager controlled by the user.
Of course the names and locations are just an example. Maybe system
should go into the profiles and pm somewhere to /etc.
When software is installed by an ebuild, the pm file is updated with
the package which needs a group/user. If it is the first ebuild which
needs a particular group/user etc/{passwd, shadow, group} are updated
accordingly.
Every time an ebuild is unmerged which needs a group/user the pm file is
checked if there is still a need for the group/user. When there is no
entry for the group/user left, the group/user is removed from
/etc/{passwd, shadow, group}.
Of course the other files are taken into account for every operation.
The file could simply look like this:
Sample for users/pm:
Ebuild: User:
cat1-pkg1-ver1 user1; user4
cat1-pkg1-ver2 user1
cat1-pkg2-ver1 user2
cat2-pkg1-ver1 user3
or
User: Ebuild:
user1 cat1-pkg1-ver1, cat1-pkg1-ver2
user2 cat1-pkg2-ver1
user3 cat2-pkg1-ver1
user4 cat1-pkg1-ver1
cat/pkg-ver has to be stored in the file because of packages installed
more then one time because they are slotted.
To get the initial database every ebuild which invokes enewuser and
enewgroup needs a revision bump? There are also ebuilds which are using
this functions not directly but via eclass like the the ebuilds using
the games.eclass
Any comments on this? Am I missing something important why this is not
easy to implement? I am just trying to help and as can not help by
coding I try to help with ideas. Maybe it is a good approach maybe it is
crap, but I think it is at least worth to mention it.
Regards,
Daniel
Is there a reason for UID and GID to be hardcoded into ebuilds? Because if not, that would make the implementation of GLEP 27 a lot easier: just get the first system UID/GID available. The Linux Standard Base has something to say about it: http://refspecs.linux-foundation.org/LSB_3.2.0/LSB-Core-generic/LSB-Core-generic/uidrange.html So system accounts should be dynamically assigned. On the other hand, this only applies to Linux. Don't know about BSD or other systems. (In reply to comment #24) > Is there a reason for UID and GID to be hardcoded into ebuilds? Because if not, > that would make the implementation of GLEP 27 a lot easier: just get the first > system UID/GID available. > The Linux Standard Base has something to say about it: > http://refspecs.linux-foundation.org/LSB_3.2.0/LSB-Core-generic/LSB-Core-generic/uidrange.html > So system accounts should be dynamically assigned. On the other hand, this only > applies to Linux. Don't know about BSD or other systems. > Some of us have more than 1 system and would prefer that the apache/mail/$OTHER_CRITICAL user be the same UID on every system we have. -Alec (In reply to comment #25) > (In reply to comment #24) > > Is there a reason for UID and GID to be hardcoded into ebuilds? Because if not, > > that would make the implementation of GLEP 27 a lot easier: just get the first > > system UID/GID available. > > The Linux Standard Base has something to say about it: > > http://refspecs.linux-foundation.org/LSB_3.2.0/LSB-Core-generic/LSB-Core-generic/uidrange.html > > So system accounts should be dynamically assigned. On the other hand, this only > > applies to Linux. Don't know about BSD or other systems. > > > > Some of us have more than 1 system and would prefer that the > apache/mail/$OTHER_CRITICAL user be the same UID on every system we have. > > -Alec > But that should be managed by servies like nis or ldap, not the package manager. please read the glep entirely. your concern about static vs dynamic vs user policy is already fully accounted for. First, I'd like to say that I'm not trying to be a smartass and I'm no Gentoo Developer. And I don't have any knowledge of which is the process to aprove a GLEP. I don't even know if this is the right place to write! I've never done this before. I'm just another Gentoo user who happens to like this distro a lot and wants to make it as better as possible. That's why I'm voicing my opinion, which is just that: an opinion. So plese, don't get mad if I have some doubts on this GLEP. (In reply to comment #27) > please read the glep entirely. your concern about static vs dynamic vs user > policy is already fully accounted for. > Yes I did, but still have some concerns about it. Forcing that behaviour upon portage introduces a bunch of problems. -First, this makes the implementation of portage more difficult without an actual need. That was the point of my first message. -Second, it ignores LSB and I think that's serious. -As far as I cant tell, GLEP 27 is specifying the solution to a problem solved and implemented long ago by the internal user management tools of each distro/system (useradd in Linux, I bet BSD has something similar). Implementing it again adds a burden to developers, (type the code, test and mantain it) and users. For instance: I want to change the way UID/GID are assigned on my system. On any linux distro, I just have to edit /etc/login.defs. In Gentoo+GLEP27, I'd have to edit /etc/login.defs AND portage's config files. This puts Gentoo on a worse situation than other OS because makes everything more complicated. To sum up: keeping track of the users/groups created by portage is a great idea, but enforcing it's assignment isn't. That should be done by the corresponding system tools, and enewuser/group already use them (they may need to be fine tuned, though). On the other hand, you could enforce the UID/GID assignment for ALL the ebuilds if it's so important. Just create and maintain a list of all Gentoo services and their corresponding xID, which means more work and would make some options unadjustable for the user (in fact, some already are). But I still think that's a better option than mixing both policies. Again, this is just my point of view on the policy specified in GLEP 27, which I hope is not set in stone. I want to avoid a long conversation on this bug. If you have a problem with a glep, talk to mike or provide a diff. If you and mike fundamentally disagree, write a competing GLEP and get it approved and then implemented. No one has implemented glep 27, so at its core what it says is not really useful (since none of it applies in real life) This bug is about implementing said GLEP, not about the glep itself. There are a number of places you can have that conversation (email with mike, the gentoo-dev list, a separate bug marked as a blocker of this bug, and so forth.) And yeah I'm being a bit pedantic, sorry for that :/ (and to be clear, I don't care one way or another how this gets done, or even if it gets done) -Alec *** Bug 266246 has been marked as a duplicate of this bug. *** *** Bug 253754 has been marked as a duplicate of this bug. *** *** Bug 344543 has been marked as a duplicate of this bug. *** Latest LSB (4.1) spec section about users and groups. http://refspecs.linux-foundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/tocusersgroups.html I'm adding a comment in this bug as *all* stage building is failing now that bl2/openrc were marked stable since baselayout2 dropped the man user / group definition (bug 113298). This has been a source of conflict between base-system and releng for years now. I don't want to argue about what is the correct or best solution for this issue, but if we want to have stages building, we need to find a solution *asap*. As far as I can see there are at least 2 available solutions: 1. hardcode man user/group on baselayout2 2. add the shadow dependency to man 3. implement glep27 and have the PM take care of users/groups What should we use? If there are other solutions please point them out. (In reply to comment #33) > I don't want to argue about what is the correct or best solution for this > issue, but if we want to have stages building, we need to find a solution > *asap*. > As far as I can see there are at least 2 available solutions: > 1. hardcode man user/group on baselayout2 > 2. add the shadow dependency to man > 3. implement glep27 and have the PM take care of users/groups > What should we use? If there are other solutions please point them out. All, this really needs to be fixed. As Jorge said, we currently are unable to build stages at all, so we need to come with something *ASAP*. Solutions 1 and 2 listed above are just bandaids until we get glep 27 implementedI think, but we need to use one of them, get glep 27 implemented, or come up with another solution *ASAP*. imo, implementing (1) or (2) will simply make it so no one cares about (3) anymore. the only reason people are talking about (3) again is because things broke (like we said it was going to *years* ago). (In reply to comment #35) > imo, implementing (1) or (2) will simply make it so no one cares about (3) > anymore. the only reason people are talking about (3) again is because things > broke (like we said it was going to *years* ago). Mike / Zac, is there anything done about GLEP27 that could be used to get it finally implemented? Could Mike (pioto) work[1] be used? Does anyone have any suggestions / pointers about how to go forward? [1] - http://trac.pioto.org/creandus A few more questions: * should baselayout use GLEP27 to create the required and optional users / groups defined in http://refspecs.linux-foundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/usernames.html or a subset from this list? * do we want to "honor" that list of user / groups or a subset, but instead hard code them in the /etc files? * is there a different list or some other users / groups that should be defined in the "base system"? Using GLEP27 or by hard coding them into files? * should the implementation of GLEP27 integrate or take into account centralized solutions for users and groups like nis / ldap / databases? * looking at the creandus api [1], it seems it doesn't allow to pass any parameters besides the user and group names, such as the desired uid / gid and group membership, It also seems it doesn't start by checking if a user / group is already defined in the database and that it needs to know about slots to remove users / groups created by a previous version of the package. Is this API acceptable / desirable or does it need to be extended? [1] - http://creandus.pioto.org/doc/API.html It's come to my attention that exheres uses something similar to GLEP 37 virtual meta-packages to manage users and groups: http://www.exherbo.org/docs/exheres-for-smarties.html#repository_metadata One thing that is interesting about this approach is that it allows DEPEND and/or RDEPEND to be used to indicate which users/group meta-packages are required at build-time and/or run-time, respectively. Also, you can imagine that emerge --depclean would automatically remove any user/group meta-packages that no longer have reverse dependencies. However, when these user/group meta-packages are uninstalled, you'd want them to leave behind "orphan" database entries that would be required to implement GLEP 27's users-update interface. Zac, find me on irc some time later this week and we can talk about these issues and how to resolve them. (I want to make sure I understand the full scope of the problem first before I start hacking.) I want to get a good system in place to address this, and have some ideas. I have created a wiki page with a description of the implementation I would like to see, and have been chatting a little bit with Zac about this on IRC: http://www.funtoo.org/wiki/User_and_Group_Management I'll help with coding as well as testing the implementation in Funtoo Linux. The proposal takes ispiration from GLEP 27, some of the good ideas from Exheres, and tries to leverage cascading profiles as much as possible. Feel free to leave comments on the "Talk" page of the wiki if you see anything you feel compelled to comment on. I'm a bit sceptical about reusing the *DEPEND variables for this purpose. - Do we know how many third-party tools such a change would break? - Using package names would impose restrictions on allowed user names. For example, "joe-42" is a valid user name but not a valid package name. And maybe some future systems will allow utf-8 in user and group names? - Past experience with the old-style virtual category (that we just got rid of) doesn't seem to favour such special-purpose categories. Why can't we stick with EUSERS and EGROUPS? The PM could still use the same algorithm for these variables that it uses for resolution of package dependencies. (In reply to comment #41) > Why can't we stick with EUSERS and EGROUPS? The PM could still use the same > algorithm for these variables that it uses for resolution of package > dependencies. That seems reasonable to me, assuming that there's no need to separate build-time and run-time users/groups. Plus, you could always add syntax extensions to support things like build-time and run-time annotations in future EAPIs. In order to implement GLEP 27's users-update interface, we'll need a package-manager indendent database to represent users/groups that have become orphans since the packages that once referenced them have been removed by tools like emerge --depclean. This database will need a specification, similar in some ways to the package-manager independent /var/lib/gentoo/news/ database that is used to keep track of read and unread GLEP 42 news items. Related to the idea of run-time/build-time separation, we may also want include the idea of cross-compiling build-host/target separation in our model. Obviously, if we need to create a bunch of users and groups just for the purpose of cross-compiling, we want to be able to purge those from the build-host when they become orphaned. For the Exheres implementation we had to add in a special "never depclean me" behaviour for installed accounts. Things get icky if you remove a uid which still has files owned by it somewhere on the system. Ulrich, to answer your questions: 1. EAPIs exist to address backwards compatibility issues, and any change in Portage has the possibility of breaking a 3rd-party tool. Portage progress can't be beholden to the unquantified fear of breaking random 3rd-party (unsupported) tools. 2. Using the existing DEPEND and RDEPEND is more elegant and intuitive than adding yet another set of Portage variables. 3. Adding EUSERS and EGROUPS will expand the size of cached metadata for a feature that relatively few ebuilds actually use. 4. Exheres is using identical syntax in ebuilds - so there are potential benefits to using the same syntax. 5. Re: UTF-8 - non-issue, just use UTF-8. And regarding invalid names - Portage would need to allow user/* and group/* and not apply PVR validation to the stuff after the /. So not a real issue. I just think that Exheres has a good foundational design here that makes sense and the right thing to do is to go with it. It is a logical design decision and is much cleaner than the original GLEP 27 proposal. It will help interoperability within the larger Gentoo community. If the design sucked, I wouldn't support it, but it doesn't, and will make our lives easier in the future. *** Bug 166937 has been marked as a duplicate of this bug. *** so i'm apparently running into this trying to build templates for my vservers with ROOT= and noticing enewuser/group don't add daemon accounts. there are no error or failure messages either, if nothing else so far, perhaps a warning message could be patched in about this issue when ROOT is used? since it's 5 months from last comment here, i guess this message serves as a ping too, is anyone hacking on this at all? in the meanwhile, anyone happen to have any patch they're running just for the use case of regular passwd/group setup? i'm running portage-2.1.10.11, perhaps there's some hold-me-over functionality in the newer development portage versions? thank you for your attention and pardon the OTish nature. (In reply to comment #47) > in the meanwhile, anyone happen to have any patch they're running just for the > use case of regular passwd/group setup? The only recent development that I'm aware of is that enewuser, enewgroup, and their dependencies have been moved from eutils.eclass to user.eclass, as discussed here: http://archives.gentoo.org/gentoo-dev/msg_404359a3054a3bcea5aa84fc90b647ce.xml pardon me and please let me know if this is not directly relevant to this bug, but browsing shadow's trunk tonight in search of a ROOT solution hack revealed that shadow-4.1.4.5 release is going to have --root option for i think all relevant utilities, i.e. useradd and friends [1]. if this is used in user.eclass, doesn't it mitigate at least some amount of the pain? [1]: http://anonscm.debian.org/viewvc/pkg-shadow/upstream/trunk/NEWS?view=markup once it actually gets released and in the tree and stabilized, we can talk about extending user.eclass *** Bug 413223 has been marked as a duplicate of this bug. *** *** Bug 454034 has been marked as a duplicate of this bug. *** *** Bug 487060 has been marked as a duplicate of this bug. *** *** Bug 479384 has been marked as a duplicate of this bug. *** *** Bug 491868 has been marked as a duplicate of this bug. *** *** Bug 481704 has been marked as a duplicate of this bug. *** *** Bug 506410 has been marked as a duplicate of this bug. *** *** Bug 517712 has been marked as a duplicate of this bug. *** (In reply to Leho Kraav (:macmaN @lkraav) from comment #49) > pardon me and please let me know if this is not directly relevant to this > bug, but browsing shadow's trunk tonight in search of a ROOT solution hack > revealed that shadow-4.1.4.5 release is going to have --root option for i > think all relevant utilities, i.e. useradd and friends [1]. if this is used > in user.eclass, doesn't it mitigate at least some amount of the pain? > > [1]: > http://anonscm.debian.org/viewvc/pkg-shadow/upstream/trunk/NEWS?view=markup I added a patch against user.eclass which did just that to bug 479384. That bug was closed as a duplicate of this one so I thought I would mention it here so it doesn't just get lost. We use it successfully to automatically build our Xen VMs so it has had some testing and seems to work. (In reply to Max Hacking from comment #59) > (In reply to Leho Kraav (:macmaN @lkraav) from comment #49) > > pardon me and please let me know if this is not directly relevant to this > > bug, but browsing shadow's trunk tonight in search of a ROOT solution hack > > revealed that shadow-4.1.4.5 release is going to have --root option for i > > think all relevant utilities, i.e. useradd and friends [1]. if this is used > > in user.eclass, doesn't it mitigate at least some amount of the pain? > > > > [1]: > > http://anonscm.debian.org/viewvc/pkg-shadow/upstream/trunk/NEWS?view=markup > > I added a patch against user.eclass which did just that to bug 479384. > > That bug was closed as a duplicate of this one so I thought I would mention > it here so it doesn't just get lost. We use it successfully to > automatically build our Xen VMs so it has had some testing and seems to work. I brifly tried this --root option and it didn't work. I think shadow actually does a chroot(2) and that only works when: 1) you root 2) the arch in your ROOT is that same as your HOST I think we nedd a --prefix ROOT option instead which will prefix all /etc/x accesses with your ROOT (In reply to Joakim Tjernlund from comment #60) > (In reply to Max Hacking from comment #59) > > (In reply to Leho Kraav (:macmaN @lkraav) from comment #49) > > > pardon me and please let me know if this is not directly relevant to this > > > bug, but browsing shadow's trunk tonight in search of a ROOT solution hack > > > revealed that shadow-4.1.4.5 release is going to have --root option for i > > > think all relevant utilities, i.e. useradd and friends [1]. if this is used > > > in user.eclass, doesn't it mitigate at least some amount of the pain? > > > > > > [1]: > > > http://anonscm.debian.org/viewvc/pkg-shadow/upstream/trunk/NEWS?view=markup > > > > I added a patch against user.eclass which did just that to bug 479384. > > > > That bug was closed as a duplicate of this one so I thought I would mention > > it here so it doesn't just get lost. We use it successfully to > > automatically build our Xen VMs so it has had some testing and seems to work. > > I brifly tried this --root option and it didn't work. I think shadow actually > does a chroot(2) and that only works when: > 1) you root > 2) the arch in your ROOT is that same as your HOST > > I think we nedd a --prefix ROOT option instead which will prefix all > /etc/x accesses with your ROOT For got to mention bug https://bugs.gentoo.org/show_bug.cgi?id=517712 . Using: proot -0 --bind=${ROOT}/etc:/etc /usr/sbin/groupadd -r ${opts} "${egroup}" || die instead of --root one can make it work for x-compile and non root too. proot is howerver only available for x86/amd64 GLEP 27 is Deferred now, so this isn't something we expect to be implemented anytime soon. Or without another review of the GLEP. |
