Steps to reproduce: 1) Merge a base system using crossdev 2) Merge openssh 3) Find no sshd in /usr/$CTARGET/{passwd,shadow,group} It appears that enewuser does not respect $ROOT (or whatever is needed, I have no idea), which makes cross-building gentoo a pretty useless work.
Confirmed. At least on Linux, useradd does not utilize -R switch
Unfortunately it's really not as simple as it first appears. As I noted in the README of my cross-boss project: "The various tools such as useradd do have a -R option to specify a root directory but this performs an actual chroot, making it useless for non-native environments. Even if this somehow worked or if it were run through QEMU, it would still not be sufficient because Portage needs to know about these users and groups from the perspective of the build system. I believe what is needed is some way to intelligently sync the accounts between / and ROOT. If a user or group already exists in / then use the same ID in ROOT. If it doesn't already exist then create it in / first, ensuring that the new ID doesn't clash with one already in ROOT. If there is an unresolvable ID clash then error out."
(In reply to James Le Cuirot from comment #2) Seems to be a sensible approach – it is basically what I am doing currently manually, using sdiff and the like.
*** Bug 581198 has been marked as a duplicate of this bug. ***
Hi, I'm currently working on that subject. In my case the target is x86 too, so I do not have the --root chroot problem. I'm really surprised that nothing exist already to do that. About portage, I've checked the user.eclass file, and it needs some modifications: - replace getent with an equivalent. The usage done by enewuser and enewgroup is simple: just check that the given user and group exist. An simple equivalent is grep "^${key}" ${EROOT}/etc/group (or passwd) - replace all calls to useradd and groupadd to add "--root ${EROOT}". Now for cross compilation, except for the password generation (which is not a problem since an ebuild only creates system users I think), we can create simple equivalent to append a line in $EROOT/etc/{group,passwd,shadow,gshadow} I'm trying to test that, but I'm totally puzzled by some strange behavior: whatever the modification I'm doing inside /usr/portage/eclass/user.eclass, it's as if my modifications are ignored when emerging some a package (I can see the traces from the ebuild when calling enewgroup...) I also created an overlay, but the same. I'm most likely missing something (a cache? A security feature?) Any idea?
(In reply to fariouche from comment #5) > - replace getent with an equivalent. The usage done by enewuser and > enewgroup is simple: just check that the given user and group exist. An > simple equivalent is grep "^${key}" ${EROOT}/etc/group (or passwd) How does that interact with e.g. pam_ldap? Will you add a warning, if that is used in /etc/pam.d? Are there other things that `getent` does, that cannot be mapped directly to /etc/passwd and similar?
getent can do many other things, I agree, but for the usage done in enewgroup and enewuser, it is rather basic: just query the users/group. If I'm not mistaken, enewgroup/enewuser are only called from ebuilds --> does it make sense for a ebuild to create/query any other kind of group/users than local system ones? Plus, my idea was to only workaround the getent/useradd/groupadd when EROOT is not "/" to avoid breaking any normal gentoo users. I believe that EROOT gentoo "users" are only building an embedded systems, isn't it?
(In reply to fariouche from comment #7) > getent can do many other things, I agree, but for the usage done in > enewgroup and enewuser, it is rather basic: just query the users/group. > If I'm not mistaken, enewgroup/enewuser are only called from ebuilds --> > does it make sense for a ebuild to create/query any other kind of > group/users than local system ones? > > Plus, my idea was to only workaround the getent/useradd/groupadd when EROOT > is not "/" to avoid breaking any normal gentoo users. > I believe that EROOT gentoo "users" are only building an embedded systems, > isn't it? My intention was to have any differences in behaviour clearly documented. Your current proposal (special case EROOT!=/, only) seems fine to me. As an implementation detail: Maybe you can extract the choice of which tool to call into a new euseradd function, which then dispatches to Darwin, Linux, EROOT!=/, ...-specific code. I would suggest to add a GLEP-42 news entry for the EROOT behaviour, though, to prevent any surprises on Gentoo user's side.
ok, we are in line. For now I'm stuck: I cannot test my modification. It's as if my modifications are ignored. I'm sure that my overlay is working (if a move away the /usr/portage/eclass/user.eclass file, portage does not complain. But it does if I remove my overlay) I also tried to modify directly the main user.eclass file, but same behavior!
(In reply to fariouche from comment #9) Please seek help outside of this bug report. For instance, you could ask in #gentoo-dev-help on Freenode, or ask for help on the gentoo-user mailing list.
Hi, I do see 2 ways of solving this point: 1- patching the shadow package to add a --prefix argument. I've done a quick proof of concept, and it seems to work. But no NIS and no LDAP when checking if the user/group exist. 2- Suggested by someone in the shadow upstream team: Do an overlayfs of a readonly "/", copy the target's group/shadow/passwd etc file, call adduser/group --root to chroot, and then retrieve the resulting files. My concern about this point is that it may not be allowed by the sandboxing/security features of portage? Plus the fact that it requires overlayfs in the kernel...
ok, I've solved my eclass modification issue. Now that I've done some tests, I better understand what should be done. My proposal is as follows: 1- if ROOT or EROOT are not specified, the behavior will not be changed (no possible regression) 2- enewgroup, enewuser and egetent will be modified to create a user and group and verify the user/group existence inside the ROOT/EROOT folder 3- For NIS/LDAP, they can be checked depending on the selected solution. In that case, the host configuration files will be used. 4- For PAM authentication, the host pam module will be used. 5- the ROOT/EROOT folder may not be based on shadow utilities. (busybox for example) 6- EROOT may be of any architecture (so we cannot use the --root option of groupadd to chroot) 7- no foreign dependency like a mandatory kernel option (overlayfs) or a new package. As far as I can see, enewgroup and enewuser can only create system group/user (the -r option is hardcoded), so the point 3 above is irrelevant. I'm exploring 2 solutions: a/ modify shadow to add --prefix option. This will fullfil all the points above and is very easy to integrate in portage. the only exception is getent, that is part of the glibc. b/ manually add new internal portage functions to do that. No NIS/LDAP/PAM verifications, just basic system account creation for linux (group,passwd,shadow,gshadow) For now, I'm discussing point a/ since this may not benefit only to gentoo. If I cannot find an agreement with them, I will go the gentoo specific modification.
Hi, I've push my modifications to the upstream shadow repo. Basically, I've added a --prefix option to user{add,mod,del} and group{add,mod,del} Now, for the gentoo part, I do have a working solution. A new user.eclass file with modified enewuser,enewgroup and egetent that all supports ${ROOT} option via --prefix in shadow utilities. For now I've only added this option for linux. However, I've encountered some unexpected issues: some ebuilds are using direct calls to chown and fowners. Both are not compatible with ${ROOT}... To solve this, I've created 2 new calls in user.eclass: echown and efowners. The only thing the new functions are doing is to get the uid/gid from the correct passwd/group files from ${ROOT} using the modified egetent function and pass that to the native chown/fowners... For example, in sys-power/nut we can find: chown nut:nut ${ROOT}/var/lib/nut This should be changed to echown nut nut ${ROOT}/var/lib/nut Same to fowners. If the modification is not done, either the ebuild will fail because the nut user does not exists in the host, or the incorrect uid will be user in ${ROOT} I'm uploading the patches to ebuilds for lighttpd and nut, plus my patch for user.eclass for review in this bug... we have time until upstream shadow team reviews and commits my modifications (at least). Side note: it's a bit complicated to know when to add ${ROOT} and when not in a ebuild... For example, chown needs ${ROOT} but fowners must not!...
Created attachment 434468 [details, diff] modified user.eclass for ${ROOT} proper handling
Created attachment 434472 [details, diff] modified nut.ebuild for support of ${ROOT} user/group creation
Created attachment 434474 [details, diff] modified lighttpd.ebuild for support of ${ROOT} user/group creation
Created attachment 434740 [details, diff] new user.eclass patch better handling of egetent when given an uid/gid better argument parsing for echaonw and efowners
HI, I've just finished to rebuild a full cross compiled target system using my patches, and everything is fine. I just got an issue with fowners when installing a package previously installed with invalid user/groups. I removed the package from the target and re-installer it and it was ok. Tested also with binary packages compiled previously in my host. Just a though: may I can add a fallback mechanism if useradd/groupadd do not have the --prefix option (maybe for gentoo systems that do not use shadow? I do not know if such linux systems are possible)
*** Bug 604136 has been marked as a duplicate of this bug. ***
*** Bug 681972 has been marked as a duplicate of this bug. ***
*** Bug 637584 has been marked as a duplicate of this bug. ***
(In reply to fariouche from comment #18) > Just a though: may I can add a fallback mechanism if useradd/groupadd do not > have the --prefix option (maybe for gentoo systems that do not use shadow? I > do not know if such linux systems are possible) Please note that --prefix was added by fariouche upstream in https://github.com/shadow-maint/shadow/commit/65b4f587037d480f8eca61f98f4b832b44b04128 (shadow 4.6 which is the oldest version available in portage at the moment) user.eclass takes care of ROOT for home directory and shell handling, however this is completely ignored for user management since (for Linux) `getent' and `id' do not support root or prefix and also might get data from LDAP/SQL/.. If a system relies on nss for user management, why would user.eclass uses shadow to create user and groups ? I feel mixing using glibc (getent), coreutils (id) and shadow (useradd, usermod, groupadd) seems inappropriate, however shadow does not currently offer tools to query data as used by user.eclass
*** Bug 728386 has been marked as a duplicate of this bug. ***
This is still a problem, and in fact, I think bug #734002 may be a duplicate of this issue. acct-user.eclass needs to be adjusted as well.
+1 for integrating this change, current years-old state breaks every system built with emerge --root
Created attachment 688161 [details, diff] 2021-02-23 up-to-date patchset Here is an up-to-date version of the patchset. I adapted most of fariouche's changes (except some that did not seem necessary) and added a few more because of the separation of user.eclass into {user, user-info, acct-user and acct-group}.eclass This is a quick patch so I'm not sure how good it is. Especially adding shadow support to egetent to check for account lock. There should be no changes if you use ROOT==/ (except for shadow)
Created attachment 688164 [details, diff] 2021-02-23 up-to-date patchset #2 Uploaded wrong file
Created attachment 688167 [details, diff] 2021-02-23 up-to-date patchset #3 I found a lone chown and a lone fowners in user.eclass and acct-user.eclass so I added back fariouche's echown fowners and fixed it
Created attachment 688170 [details, diff] 2021-02-23 up-to-date patchset
Created attachment 688293 [details, diff] 2021-02-23 up-to-date patchset apologies for all the attachments, I updated the regex for egetgroups
commit 01eb3f7d7e1d897aefeab941326670e74512a0cb Author: Jérémy Connat <morderca@morderca.net> AuthorDate: Sun Jan 24 21:02:23 2021 +0100 Commit: Sam James <sam@gentoo.org> CommitDate: Sat Apr 23 02:26:42 2022 +0100 user.eclass: Fixing user/group creation when using different ROOT When creating a user for another environement, user is created on the HOST and not the ROOT dir. Adding "-R <CHROOT_DIR>" for all user* / group* commands fix the issue. Signed-off-by: Jérémy Connat <morderca@morderca.net> Signed-off-by: Sam James <sam@gentoo.org>
*** Bug 654270 has been marked as a duplicate of this bug. ***
