Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 541406 - user.eclass: does not respect ROOT
Summary: user.eclass: does not respect ROOT
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Eclasses (show other bugs)
Hardware: All Linux
: Normal normal with 1 vote (vote)
Assignee: Gentoo's Team for Core System packages
URL:
Whiteboard:
Keywords: PATCH, PullRequest
: 581198 604136 637584 681972 728386 (view as bug list)
Depends on:
Blocks: glep27
  Show dependency tree
 
Reported: 2015-02-26 11:21 UTC by Alexey Kharlamov
Modified: 2021-03-30 11:56 UTC (History)
22 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
modified user.eclass for ${ROOT} proper handling (user.eclass.patch,5.53 KB, patch)
2016-05-16 20:04 UTC, fariouche
Details | Diff
modified nut.ebuild for support of ${ROOT} user/group creation (sys-power-nut.patch,1.76 KB, patch)
2016-05-16 20:06 UTC, fariouche
Details | Diff
modified lighttpd.ebuild for support of ${ROOT} user/group creation (lighttpd.patch,630 bytes, patch)
2016-05-16 20:06 UTC, fariouche
Details | Diff
new user.eclass patch (user.eclass.patch,6.30 KB, patch)
2016-05-19 20:03 UTC, fariouche
Details | Diff
2021-02-23 up-to-date patchset (user.eclass.patch,6.30 KB, patch)
2021-02-24 01:01 UTC, dkjii
Details | Diff
2021-02-23 up-to-date patchset #2 (user-root.patch,5.67 KB, patch)
2021-02-24 01:08 UTC, dkjii
Details | Diff
2021-02-23 up-to-date patchset #3 (user-root-v2.patch,8.20 KB, patch)
2021-02-24 01:27 UTC, dkjii
Details | Diff
2021-02-23 up-to-date patchset (user-root-v2.patch,8.21 KB, patch)
2021-02-24 01:37 UTC, dkjii
Details | Diff
2021-02-23 up-to-date patchset (user-root-v2.patch,8.22 KB, patch)
2021-02-25 02:52 UTC, dkjii
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alexey Kharlamov 2015-02-26 11:21:20 UTC
Steps to reproduce:

1) Merge a base system using crossdev

2) Merge openssh

3) Find no sshd in /usr/$CTARGET/{passwd,shadow,group}

It appears that enewuser does not respect $ROOT (or whatever is needed, I have no idea), which makes cross-building gentoo a pretty useless work.
Comment 1 Sergey Popov gentoo-dev 2015-02-26 11:24:09 UTC
Confirmed.

At least on Linux, useradd does not utilize -R switch
Comment 2 James Le Cuirot gentoo-dev 2015-02-26 11:34:06 UTC
Unfortunately it's really not as simple as it first appears. As I noted in the README of my cross-boss project:

"The various tools such as useradd do have a -R option to specify a root directory but this performs an actual chroot, making it useless for non-native environments. Even if this somehow worked or if it were run through QEMU, it would still not be sufficient because Portage needs to know about these users and groups from the perspective of the build system.

I believe what is needed is some way to intelligently sync the accounts between / and ROOT. If a user or group already exists in / then use the same ID in ROOT. If it doesn't already exist then create it in / first, ensuring that the new ID doesn't clash with one already in ROOT. If there is an unresolvable ID clash then error out."
Comment 3 Dennis Schridde 2015-02-26 12:42:08 UTC
(In reply to James Le Cuirot from comment #2)

Seems to be a sensible approach – it is basically what I am doing currently manually, using sdiff and the like.
Comment 4 Mike Gilbert gentoo-dev 2016-04-25 19:55:20 UTC
*** Bug 581198 has been marked as a duplicate of this bug. ***
Comment 5 fariouche 2016-04-26 19:50:02 UTC
Hi,

I'm currently working on that subject. In my case the target is x86 too, so I do not have the --root chroot problem.

I'm really surprised that nothing exist already to do that.

About portage, I've checked the user.eclass file, and it needs some modifications:
- replace getent with an equivalent. The usage done by enewuser and enewgroup is simple: just check that the given user and group exist. An simple equivalent is grep "^${key}" ${EROOT}/etc/group (or passwd)
- replace all calls to useradd and groupadd to add "--root ${EROOT}".

Now for cross compilation, except for the password generation (which is not a problem since an ebuild only creates system users I think), we can create simple equivalent to append a line in $EROOT/etc/{group,passwd,shadow,gshadow}

I'm trying to test that, but I'm totally puzzled by some strange behavior: whatever the modification I'm doing inside /usr/portage/eclass/user.eclass, it's as if my modifications are ignored when emerging some a package (I can see the traces from the ebuild when calling enewgroup...)
I also created an overlay, but the same.
I'm most likely missing something (a cache? A security feature?)
Any idea?
Comment 6 Dennis Schridde 2016-04-27 12:35:51 UTC
(In reply to fariouche from comment #5)
> - replace getent with an equivalent. The usage done by enewuser and
> enewgroup is simple: just check that the given user and group exist. An
> simple equivalent is grep "^${key}" ${EROOT}/etc/group (or passwd)

How does that interact with e.g. pam_ldap? Will you add a warning, if that is used in /etc/pam.d? Are there other things that `getent` does, that cannot be mapped directly to /etc/passwd and similar?
Comment 7 fariouche 2016-04-27 18:46:36 UTC
getent can do many other things, I agree, but for the usage done in enewgroup and enewuser, it is rather basic: just query the users/group.
If I'm not mistaken, enewgroup/enewuser are only called from ebuilds --> does it make sense for a ebuild to create/query any other kind of group/users than local system ones?

Plus, my idea was to only workaround the getent/useradd/groupadd when EROOT is not "/" to avoid breaking any normal gentoo users.
I believe that EROOT gentoo "users" are only building an embedded systems, isn't it?
Comment 8 Dennis Schridde 2016-04-29 10:01:24 UTC
(In reply to fariouche from comment #7)
> getent can do many other things, I agree, but for the usage done in
> enewgroup and enewuser, it is rather basic: just query the users/group.
> If I'm not mistaken, enewgroup/enewuser are only called from ebuilds -->
> does it make sense for a ebuild to create/query any other kind of
> group/users than local system ones?
> 
> Plus, my idea was to only workaround the getent/useradd/groupadd when EROOT
> is not "/" to avoid breaking any normal gentoo users.
> I believe that EROOT gentoo "users" are only building an embedded systems,
> isn't it?

My intention was to have any differences in behaviour clearly documented. Your current proposal (special case EROOT!=/, only) seems fine to me. As an implementation detail: Maybe you can extract the choice of which tool to call into a new euseradd function, which then dispatches to Darwin, Linux, EROOT!=/, ...-specific code.

I would suggest to add a GLEP-42 news entry for the EROOT behaviour, though, to prevent any surprises on Gentoo user's side.
Comment 9 fariouche 2016-04-29 19:01:02 UTC
ok, we are in line.
For now I'm stuck: I cannot test my modification. It's as if my modifications are ignored.
I'm sure that my overlay is working (if a move away the /usr/portage/eclass/user.eclass file, portage does not complain. But it does if I remove my overlay)
I also tried to modify directly the main user.eclass file, but same behavior!
Comment 10 Mike Gilbert gentoo-dev 2016-04-29 20:31:39 UTC
(In reply to fariouche from comment #9)

Please seek help outside of this bug report. For instance, you could ask in #gentoo-dev-help on Freenode, or ask for help on the gentoo-user mailing list.
Comment 11 fariouche 2016-05-02 18:36:49 UTC
Hi,

I do see 2 ways of solving this point:
1- patching the shadow package to add a --prefix argument. I've done a quick proof of concept, and it seems to work. But no NIS and no LDAP when checking if the user/group exist.
2- Suggested by someone in the shadow upstream team: Do an overlayfs of a readonly "/", copy the target's group/shadow/passwd etc file, call adduser/group --root to chroot, and then retrieve the resulting files.
My concern about this point is that it may not be allowed by the sandboxing/security features of portage? Plus the fact that it requires overlayfs in the kernel...
Comment 12 fariouche 2016-05-05 12:42:11 UTC
ok, I've solved my eclass modification issue. Now that I've done some tests, I better understand what should be done.

My proposal is as follows:
1- if ROOT or EROOT are not specified, the behavior will not be changed (no possible regression)
2- enewgroup, enewuser and egetent will be modified to create a user and group and verify the user/group existence inside the ROOT/EROOT folder
3- For NIS/LDAP, they can be checked depending on the selected solution. In that case, the host configuration files will be used.
4- For PAM authentication, the host pam module will be used.
5- the ROOT/EROOT folder may not be based on shadow utilities. (busybox for example)
6- EROOT may be of any architecture (so we cannot use the --root option of groupadd to chroot)
7- no foreign dependency like a mandatory kernel option (overlayfs) or a new package.

As far as I can see, enewgroup and enewuser can only create system group/user (the -r option is hardcoded), so the point 3 above is irrelevant.

I'm exploring 2 solutions:
a/ modify shadow to add --prefix option. This will fullfil all the points above and is very easy to integrate in portage. the only exception is getent, that is part of the glibc.
b/ manually add new internal portage functions to do that. No NIS/LDAP/PAM verifications, just basic system  account creation for linux (group,passwd,shadow,gshadow)


For now, I'm discussing point a/ since this may not benefit only to gentoo.
If I cannot find an agreement with them, I will go the gentoo specific modification.
Comment 13 fariouche 2016-05-16 20:04:01 UTC
Hi,

I've push my modifications to the upstream shadow repo.
Basically, I've added a --prefix option to user{add,mod,del} and group{add,mod,del}

Now, for the gentoo part, I do have a working solution.
A new user.eclass file with modified enewuser,enewgroup and egetent that all supports ${ROOT} option via --prefix in shadow utilities.
For now I've only added this option for linux.

However, I've encountered some unexpected issues: some ebuilds are using direct calls to chown and fowners. Both are not compatible with ${ROOT}...

To solve this, I've created 2 new calls in user.eclass: echown and efowners.
The only thing the new functions are doing is to get the uid/gid from the correct passwd/group files from ${ROOT} using the modified egetent function and pass that to the native chown/fowners...

For example, in sys-power/nut we can find:
chown nut:nut ${ROOT}/var/lib/nut

This should be changed to
echown nut nut ${ROOT}/var/lib/nut

Same to fowners.
If the modification is not done, either the ebuild will fail because the nut user does not exists in the host, or the incorrect uid will be user in ${ROOT}



I'm uploading the patches to ebuilds for lighttpd and nut, plus my patch for user.eclass for review in this bug... we have time until upstream shadow team reviews and commits my modifications (at least).

Side note: it's a bit complicated to know when to add ${ROOT} and when not in a ebuild... For example, chown needs ${ROOT} but fowners must not!...
Comment 14 fariouche 2016-05-16 20:04:50 UTC
Created attachment 434468 [details, diff]
modified user.eclass for ${ROOT} proper handling
Comment 15 fariouche 2016-05-16 20:06:05 UTC
Created attachment 434472 [details, diff]
modified nut.ebuild for support of ${ROOT} user/group creation
Comment 16 fariouche 2016-05-16 20:06:29 UTC
Created attachment 434474 [details, diff]
modified lighttpd.ebuild for support of ${ROOT} user/group creation
Comment 17 fariouche 2016-05-19 20:03:13 UTC
Created attachment 434740 [details, diff]
new user.eclass patch

better handling of egetent when given an uid/gid
better argument parsing for echaonw and efowners
Comment 18 fariouche 2016-05-19 20:09:45 UTC
HI,

I've just finished to rebuild a full cross compiled target system using my patches, and everything is fine.

I just got an issue with fowners when installing a package previously installed with invalid user/groups.
I removed the package from the target and re-installer it and it was ok.

Tested also with binary packages compiled previously in my host.

Just a though: may I can add a fallback mechanism if useradd/groupadd do not have the --prefix option (maybe for gentoo systems that do not use shadow? I do not know if such linux systems are possible)
Comment 19 Coacher 2016-12-30 13:22:05 UTC
*** Bug 604136 has been marked as a duplicate of this bug. ***
Comment 20 James Le Cuirot gentoo-dev 2019-03-28 22:37:46 UTC
*** Bug 681972 has been marked as a duplicate of this bug. ***
Comment 21 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2019-05-29 17:44:31 UTC
*** Bug 637584 has been marked as a duplicate of this bug. ***
Comment 22 Bertrand Jacquin 2019-09-27 23:21:28 UTC
(In reply to fariouche from comment #18)

> Just a though: may I can add a fallback mechanism if useradd/groupadd do not
> have the --prefix option (maybe for gentoo systems that do not use shadow? I
> do not know if such linux systems are possible)

Please note that --prefix was added by fariouche upstream in https://github.com/shadow-maint/shadow/commit/65b4f587037d480f8eca61f98f4b832b44b04128 (shadow 4.6 which is the oldest version available in portage at the moment)

user.eclass takes care of ROOT for home directory and shell handling, however this is completely ignored for user management since (for Linux) `getent' and `id' do not support root or prefix and also might get data from LDAP/SQL/..

If a system relies on nss for user management, why would user.eclass uses shadow to create user and groups ?

I feel mixing using glibc (getent), coreutils (id) and shadow (useradd, usermod, groupadd) seems inappropriate, however shadow does not currently offer tools to query data as used by user.eclass
Comment 23 Mike Gilbert gentoo-dev 2020-06-16 13:58:50 UTC
*** Bug 728386 has been marked as a duplicate of this bug. ***
Comment 24 Hypoon 2020-08-18 05:43:51 UTC
This is still a problem, and in fact, I think bug #734002 may be a duplicate of this issue. acct-user.eclass needs to be adjusted as well.
Comment 25 Mikaël Cluseau 2020-09-10 07:27:02 UTC
+1 for integrating this change, current years-old state breaks every system built with emerge --root
Comment 26 dkjii 2021-02-24 01:01:26 UTC
Created attachment 688161 [details, diff]
2021-02-23 up-to-date patchset

Here is an up-to-date version of the patchset. I adapted most of fariouche's changes (except some that did not seem necessary) and added a few more because of the separation of user.eclass into {user, user-info, acct-user and acct-group}.eclass

This is a quick patch so I'm not sure how good it is. Especially adding shadow support to egetent to check for account lock.

There should be no changes if you use ROOT==/ (except for shadow)
Comment 27 dkjii 2021-02-24 01:08:38 UTC
Created attachment 688164 [details, diff]
2021-02-23 up-to-date patchset #2

Uploaded wrong file
Comment 28 dkjii 2021-02-24 01:27:02 UTC
Created attachment 688167 [details, diff]
2021-02-23 up-to-date patchset #3

I found a lone chown and a lone fowners in user.eclass and acct-user.eclass so I added back fariouche's echown fowners and fixed it
Comment 29 dkjii 2021-02-24 01:37:34 UTC
Created attachment 688170 [details, diff]
2021-02-23 up-to-date patchset
Comment 30 dkjii 2021-02-25 02:52:56 UTC
Created attachment 688293 [details, diff]
2021-02-23 up-to-date patchset

apologies for all the attachments, I updated the regex for egetgroups