| Summary: | Gentoo Prefix bootstrap stage 3 fails with error "Cannot chown a lockfile" | ||
|---|---|---|---|
| Product: | Gentoo/Alt | Reporter: | Yanbin Zhang <yanbin.zhang> |
| Component: | Prefix Support | Assignee: | Gentoo Prefix <prefix> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | cuihao.leo, dev-portage, sam, simonmysun |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | AMD64 | ||
| OS: | Linux | ||
| See Also: | https://bugs.gentoo.org/show_bug.cgi?id=831075 | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
*** Bug 767841 has been marked as a duplicate of this bug. *** Hi, Is there any updates on this thread? Or is there a workaround? Thx (In reply to Maoyin Sun from comment #2) > Hi, > > Is there any updates on this thread? Or is there a workaround? > > Thx I'm also waiting for a fix or workaround. I'm sure Benda is looking at it but it looks like it takes some time to figure out. I thought Portage skipped chowns when we know we're unprivileged anyway. The warning you get when running emerge -u @world is weird too. Maybe I'm misunderstanding which Portage RAP vs non-RAP uses... Could you please check the directory permission bits of EPREFIX/.. ? Portage secretly checks that directory. The central cause is at https://gitweb.gentoo.org/proj/portage.git/tree/lib/portage/data.py#n95, which I reproduce here: # If the current user is not root, but has write access to the # EROOT directory (not due to the 0002 bit), then use "unprivileged" # mode which sets secpass = 2 and uses the UID and GID of the EROOT # directory to generate default PORTAGE_INST_GID, PORTAGE_INST_UID, # PORTAGE_USERNAME, and PORTAGE_GRPNAME settings. def _unprivileged_mode(eroot, eroot_st): return os.getuid() != 0 and os.access(eroot, os.W_OK) and \ not eroot_st.st_mode & 0o0002 If the permission bit of EPREFIX directory is set to 775, then the above condition fails. My previous comment was wrong, the permission bit that counts is EPREFIX, not EPREFIX/.. Please check is this is the cause. I checked mine. My EPREFIX=/data/rpo_yazn/gentoo. Here are the permission bits: drwxr-sr-x 28 yazn g_hreseng 0 Jan 26 15:43 rpo_yazn drwxr-sr-x 11 yazn g_hreseng 0 Jan 15 08:56 gentoo (In reply to Benda Xu from comment #6) > The central cause is at > https://gitweb.gentoo.org/proj/portage.git/tree/lib/portage/data.py#n95, > which I reproduce here: > > # If the current user is not root, but has write access to the > # EROOT directory (not due to the 0002 bit), then use "unprivileged" > # mode which sets secpass = 2 and uses the UID and GID of the EROOT > # directory to generate default PORTAGE_INST_GID, PORTAGE_INST_UID, > # PORTAGE_USERNAME, and PORTAGE_GRPNAME settings. > def _unprivileged_mode(eroot, eroot_st): > return os.getuid() != 0 and os.access(eroot, os.W_OK) and \ > not eroot_st.st_mode & 0o0002 > > If the permission bit of EPREFIX directory is set to 775, then the above > condition fails. My previous comment was wrong, the permission bit that > counts is EPREFIX, not EPREFIX/.. > > Please check is this is the cause. Hi Benda, I don't see why this is the cause. The function _unprevileged_mode returns True in my case. Yanbin (In reply to Yanbin Zhang from comment #8) > > Hi Benda, > I don't see why this is the cause. The function _unprevileged_mode returns > True in my case. OK, at least this cause is excluded in your case. This bug is hard to reproduce. You have a precious sample to chase it down. Please carry on bug hunting, by understanding the privilege logic in portage source code. Ccing portage team for hints. (In reply to Yanbin Zhang from comment #0) > === Sync completed for gentoo > !!! Directory initialization failed: '/data/rpo_yazn/gentoo/var/lib/portage' > !!! chown('/data/rpo_yazn/gentoo/var/lib/portage', -1, 0) > !!! Directory initialization failed: '/data/rpo_yazn/gentoo/var/cache/edb' > !!! chown('/data/rpo_yazn/gentoo/var/cache/edb', -1, 0) The 0 gid shown here is inappropriate. It shows that portage_gid was 0 in the _init_dirs method where the message comes from. So, something has gone wrong in the evaluation of portage_gid. The first thing you check is that a simple command like this gives a correct result: python -c 'import portage; print(portage.portage_gid)' It that gives a correct result, then we need to dig deeper to see what's happening to corrupt the result in the other case. Since portage_gid is lazy, things like changes to import relationships between modules could affect its behavior. Also note that if the PORTAGE_GRPNAME variable is set then that will change your portage_gid. (In reply to Zac Medico from comment #11) > Also note that if the PORTAGE_GRPNAME variable is set then that will change > your portage_gid. Indeed, the variable can be used to override portage's (wrong) guess. Hi, I've done some tests and here's what I found out: When the permission bit of $EPREFIX is 777, the bootstrap process first had problems chowining linux header files. After retrying (with nothing changed) it finally reports 'portage' user or group missing and that it cannot chown the lockfile, same as described by Yabin Zhang in comment #0. I did so twice and the result was the same. When the permission bit of $EPREFIX is 755, everything goes well. However 'portage' user or group does not appear in $EPREFIX/etc/passwd or $EPREFIX/etc/group. > python -c 'import portage; print(portage.portage_gid)' mine after successful install returns the group id of my current user. I don't know how to import portage in the failed one... I hope the information will help. (In reply to Maoyin Sun from comment #13) > Hi, I've done some tests and here's what I found out: > > When the permission bit of $EPREFIX is 777, the bootstrap process first had > problems chowining linux header files. After retrying (with nothing changed) > it finally reports 'portage' user or group missing and that it cannot chown > the lockfile, same as described by Yabin Zhang in comment #0. I did so twice > and the result was the same. Acknowledged. > When the permission bit of $EPREFIX is 755, everything goes well. However > 'portage' user or group does not appear in $EPREFIX/etc/passwd or > $EPREFIX/etc/group. > > python -c 'import portage; print(portage.portage_gid)' > > mine after successful install returns the group id of my current user. I > don't know how to import portage in the failed one... That's expected. portage is run by an unprivileged user having no permission to create `portage` user/group. (In reply to Benda Xu from comment #14) > (In reply to Maoyin Sun from comment #13) > > Hi, I've done some tests and here's what I found out: > > > > When the permission bit of $EPREFIX is 777, the bootstrap process first had > > problems chowining linux header files. After retrying (with nothing changed) > > it finally reports 'portage' user or group missing and that it cannot chown > > the lockfile, same as described by Yabin Zhang in comment #0. I did so twice > > and the result was the same. > > Acknowledged. > > > When the permission bit of $EPREFIX is 755, everything goes well. However > > 'portage' user or group does not appear in $EPREFIX/etc/passwd or > > $EPREFIX/etc/group. > > > > python -c 'import portage; print(portage.portage_gid)' > > > > mine after successful install returns the group id of my current user. I > > don't know how to import portage in the failed one... > > That's expected. portage is run by an unprivileged user having no > permission to create `portage` user/group. In conclusion, Maoyin's bug is circumvented by changing permission bits of EPREFIX directory. But we don't have a clue of Yanbin's case. It is common for HPC environments to set umask other than 022. We should explore better triggers for portage unprivileged mode or at least detect this in bootstrap-prefix.sh to inform users. (In reply to Yanbin Zhang from comment #7) > I checked mine. My EPREFIX=/data/rpo_yazn/gentoo. Here are the permission > bits: > drwxr-sr-x 28 yazn g_hreseng 0 Jan 26 15:43 rpo_yazn > drwxr-sr-x 11 yazn g_hreseng 0 Jan 15 08:56 gentoo The sticky bit on group is suspicious. Is `g_hreseng` your primary group? (In reply to Benda Xu from comment #15) > It is common for HPC environments to set umask other than 022. We should > explore better triggers for portage unprivileged mode or at least detect > this in bootstrap-prefix.sh to inform users. Though I haven't understand why the permission bit matters here, I think a helpful message in bootstrap-prefix.sh or in the install guide page would be great. If it's a rhel7 and kernel 3.10.0 related problem, I'm happy to assist. I am having the same issue as Yanbin, also on Redhat 7 with kernel 3.10.0-1160.31.1.el7.x86_64. Any updates on this? I've tried setting PORTAGE_INST_UID, PORTAGE_INST_GID, PORTAGE_USERNAME and PORTAGE_GRPNAME to the values of my user and group, but this seems to have no effect. Also I've tried added my user to $EPREFIX/etc/passwd (as suggested in bug #777303) but this has no effect either. * Bootstrapping Gentoo prefixed portage installation using * host: x86_64-pc-linux-gnu * prefix: /scratch/mharkonen3/gentoo * ready to bootstrap stage3_log * Triggering Linux RAP bootstrap * running emerge -u system setlocale: unsupported locale setting portage: 'portage' user or group missing. For the defaults, line 1 goes into passwd, and 2 into group. portage:x:250:250:portage:/var/tmp/portage:/bin/false portage::250:portage *** WARNING *** For security reasons, only system administrators should be *** WARNING *** allowed in the portage group. Untrusted users or processes *** WARNING *** can potentially exploit the portage group for attacks such as *** WARNING *** local privilege escalation. !!! Directory initialization failed: '/scratch/mharkonen3/gentoo/var/lib/portage' !!! chown('/scratch/mharkonen3/gentoo/var/lib/portage', -1, 0) !!! Directory initialization failed: '/scratch/mharkonen3/gentoo/var/cache/edb' !!! chown('/scratch/mharkonen3/gentoo/var/cache/edb', -1, 0) setlocale: unsupported locale setting [Errno 1] Operation not permitted: b'/scratch/mharkonen3/gentoo/var/lib/gentoo/news/.news-gentoo.unread.portage_lockfile': chown('/scratch/mharkonen3/gentoo/var/lib/gentoo/news/.news-gentoo.unread.portage_lockfile', -1, 0) Cannot chown a lockfile: '/scratch/mharkonen3/gentoo/var/lib/gentoo/news/.news-gentoo.unread.portage_lockfile' Group IDs of current user: 2626 106004 106005 106006 106008 106028 106058 106381 106478 108033 200004 200005 200007 200008 200013 200017 200018 200023 200024 200067 200069 200134 200135 200950 202766 202767 202768 202769 202771 202772 202773 202774 202775 203711 203712 203713 203719 203720 203813 204186 204187 204938 204994 204995 205002 205003 205004 205005 205009 205010 205011 205014 205018 205019 205020 205024 205027 205029 205031 205034 205036 205039 205042 205043 205044 205047 205048 205050 205053 205056 205060 205061 205063 205070 205071 205072 205073 205074 205077 205079 205080 205084 205087 205090 205091 205094 205095 205098 205103 205108 205109 205110 205111 205114 205115 205123 205136 205160 205161 205163 205167 205168 205171 205172 205173 205174 205175 205176 205177 205315 205333 205495 205504 206017 206598 206880 207316 207317 207318 214656 214734 215104 215810 216129 216330 216365 216814 217058 217059 217062 217103 217181 217185 217186 217205 217251 221132 222924 223931 223944 223954 223957 224063 224064 224096 224762 226778 226787 227309 231307 233157 258439 258472 265281 279170 298306 300639 300658 304655 304661 310270 315176 315376 315742 315743 316004 318705 318706 318707 318708 318711 318712 318715 318716 318717 318718 318720 318721 318769 319135 319136 319640 319641 319642 319643 340391 340849 346695 347595 347879 347888 347917 367378 367400 368119 368958 369458 416699 These are the packages that would be merged, in order: Calculating dependencies [Errno 1] Operation not permitted: b'/scratch/mharkonen3/gentoo/var/db/.pkg.portage_lockfile': chown('/scratch/mharkonen3/gentoo/var/db/.pkg.portage_lockfile', -1, 0) Cannot chown a lockfile: '/scratch/mharkonen3/gentoo/var/db/.pkg.portage_lockfile' Group IDs of current user: 2626 106004 106005 106006 106008 106028 106058 106381 106478 108033 200004 200005 200007 200008 200013 200017 200018 200023 200024 200067 200069 200134 200135 200950 202766 202767 202768 202769 202771 202772 202773 202774 202775 203711 203712 203713 203719 203720 203813 204186 204187 204938 204994 204995 205002 205003 205004 205005 205009 205010 205011 205014 205018 205019 205020 205024 205027 205029 205031 205034 205036 205039 205042 205043 205044 205047 205048 205050 205053 205056 205060 205061 205063 205070 205071 205072 205073 205074 205077 205079 205080 205084 205087 205090 205091 205094 205095 205098 205103 205108 205109 205110 205111 205114 205115 205123 205136 205160 205161 205163 205167 205168 205171 205172 205173 205174 205175 205176 205177 205315 205333 205495 205504 206017 206598 206880 207316 207317 207318 214656 214734 215104 215810 216129 216330 216365 216814 217058 217059 217062 217103 217181 217185 217186 217205 217251 221132 222924 223931 223944 223954 223957 224063 224064 224096 224762 226778 226787 227309 231307 233157 258439 258472 265281 279170 298306 300639 300658 304655 304661 310270 315176 315376 315742 315743 316004 318705 318706 318707 318708 318711 318712 318715 318716 318717 318718 318720 318721 318769 319135 319136 319640 319641 319642 319643 340391 340849 346695 347595 347879 347888 347917 367378 367400 368119 368958 369458 416699 ... done! [ebuild N ] acct-group/man-0-r1::gentoo 0 KiB [ebuild N ] app-misc/editor-wrapper-4-r1::gentoo 0 KiB [ebuild N ] app-text/manpager-1::gentoo 0 KiB [ebuild N ] sys-apps/which-2.21::gentoo 146 KiB [ebuild N ] app-text/sgml-common-0.6.3-r7::gentoo 126 KiB [ebuild N ] acct-user/man-1-r1::gentoo 0 KiB [ebuild N ] dev-libs/libpipeline-1.5.3::gentoo USE="-static-libs -test" 972 KiB [ebuild N ] sys-process/procps-3.3.17-r1:0/8::gentoo USE="kill ncurses nls (split-usr) (unicode) -elogind -modern-top (-selinux) -static-libs (-systemd) -test" 985 KiB [ebuild N ] sys-apps/net-tools-2.10::gentoo USE="arp hostname ipv6 nls -nis -plipconfig (-selinux) -slattach -static" 225 KiB [ebuild N ] net-libs/libtirpc-1.3.2:0/3::gentoo USE="ipv6 (split-usr) -kerberos -static-libs" 510 KiB [ebuild N ] net-libs/libnsl-1.3.0-r2:0/2::gentoo USE="-static-libs" 314 KiB [ebuild N ] dev-libs/libpcre2-10.37-r2:0/3::gentoo USE="bzip2 jit pcre16 readline recursion-limit (split-usr) unicode zlib -libedit -pcre32 -static-libs" 1691 KiB [ebuild N ] dev-libs/libpcre-8.45:3::gentoo USE="bzip2 cxx jit readline recursion-limit (split-usr) (unicode) zlib -libedit -pcre16 -pcre32 -static-libs" 1542 KiB [ebuild N ] sys-apps/less-581.2::gentoo USE="pcre unicode" 342 KiB [ebuild N ] virtual/pager-0::gentoo 0 KiB [ebuild N ] dev-libs/libxslt-1.1.34-r1::gentoo USE="-crypt -debug -examples -static-libs -verify-sig" 3470 KiB [ebuild UD ] sys-apps/grep-3.5::gentoo [3.7::gentoo] USE="nls* pcre* -static" 1550 KiB [ebuild N ] app-editors/nano-5.6.1-r2::gentoo USE="ncurses nls spell (split-usr) (unicode) -debug -justify -magic -minimal -static" 2889 KiB [ebuild N ] sys-apps/acl-2.3.1::gentoo USE="nls (split-usr) -static-libs" 507 KiB [ebuild N ] sys-apps/diffutils-3.7-r1::gentoo USE="nls -static" 1415 KiB [ebuild N ] app-text/opensp-1.5.2-r3::gentoo USE="nls -doc -static-libs -test" 1486 KiB [ebuild N ] virtual/editor-0-r3::gentoo 0 KiB [ebuild N ] virtual/libc-1-r1::gentoo 0 KiB [ebuild UD ] sys-apps/util-linux-2.36.2-r1::gentoo [2.37.2::gentoo] USE="cramfs logger ncurses nls* readline* (split-usr) suid (unicode) -audit -build -caps -cryptsetup -fdformat -hardlink -kill -magic (-pam) -python (-selinux) -slang -static-libs -su (-systemd) -test -tty-helpers -udev" PYTHON_TARGETS="python3_9 -python3_8" 5223 KiB [ebuild N ] app-text/build-docbook-catalog-1.21::gentoo 5 KiB [ebuild N ] app-text/docbook-xsl-stylesheets-1.79.1-r2::gentoo USE="-ruby" 21454 KiB [ebuild N ] app-text/docbook-xml-dtd-4.1.2-r7:4.1.2::gentoo 74 KiB [ebuild N ] app-text/docbook-xml-dtd-4.2-r3:4.2::gentoo 77 KiB [ebuild N ] app-text/docbook-xml-dtd-4.5-r2:4.5::gentoo 97 KiB [ebuild N ] net-misc/wget-1.21.1::gentoo USE="ipv6 nls pcre (ssl) zlib -cookie_check -debug -gnutls -idn -metalink -ntlm -static -test -uuid" 4771 KiB [ebuild N ] dev-util/meson-format-array-0::gentoo PYTHON_TARGETS="python3_9 -python3_8" 0 KiB [ebuild N ] virtual/package-manager-1::gentoo 0 KiB [ebuild N ] virtual/perl-ExtUtils-MakeMaker-7.620.0::gentoo 0 KiB [ebuild N ] virtual/perl-File-Spec-3.800.0::gentoo 0 KiB [ebuild N ] dev-perl/Text-CharWidth-0.40.0-r1::gentoo 9 KiB [ebuild N ] sys-apps/texinfo-6.7::gentoo USE="nls standalone -static" 8817 KiB [ebuild N ] virtual/perl-Carp-1.520.0-r1::gentoo 0 KiB [ebuild N ] virtual/perl-Exporter-5.760.0::gentoo 0 KiB [ebuild N ] perl-core/Scalar-List-Utils-1.560.0::gentoo 170 KiB [ebuild N ] virtual/perl-ExtUtils-CBuilder-0.280.236::gentoo 0 KiB [ebuild N ] virtual/perl-ExtUtils-Install-2.200.0::gentoo 0 KiB [ebuild N ] virtual/perl-ExtUtils-Manifest-1.730.0::gentoo 0 KiB [ebuild N ] virtual/perl-ExtUtils-ParseXS-3.430.0::gentoo 0 KiB [ebuild N ] virtual/perl-Getopt-Long-2.520.0::gentoo 0 KiB [ebuild N ] virtual/perl-Module-Metadata-1.0.37-r1::gentoo 0 KiB [ebuild N ] virtual/perl-Perl-OSType-1.10.0-r5::gentoo 0 KiB [ebuild N ] virtual/perl-Text-ParseWords-3.300.0-r8::gentoo 0 KiB [ebuild N ] virtual/perl-podlators-4.140.0-r2::gentoo 0 KiB [ebuild N ] virtual/perl-version-0.992.800::gentoo 0 KiB [ebuild N ] virtual/perl-CPAN-Meta-YAML-0.18.0-r7::gentoo 0 KiB [ebuild N ] virtual/perl-JSON-PP-4.60.0::gentoo 0 KiB [ebuild N ] virtual/perl-Parse-CPAN-Meta-2.150.10-r5::gentoo 0 KiB [ebuild N ] sys-apps/groff-1.22.4::gentoo USE="-X -examples (-uchardet)" 4041 KiB [ebuild N ] dev-perl/Locale-gettext-1.70.0::gentoo 9 KiB [ebuild N ] dev-perl/TermReadKey-2.370.0::gentoo USE="-examples" 84 KiB [ebuild N ] dev-perl/Text-WrapI18N-0.60.0-r1::gentoo 4 KiB [ebuild N ] dev-perl/Pod-Parser-1.630.0-r1::gentoo USE="-test" 78 KiB [ebuild N ] virtual/perl-Scalar-List-Utils-1.560.0::gentoo 0 KiB [ebuild N ] virtual/perl-CPAN-Meta-2.150.10-r5::gentoo 0 KiB [ebuild N ] dev-perl/Module-Build-0.423.100::gentoo USE="-test" 297 KiB [ebuild N ] dev-perl/YAML-Tiny-1.730.0::gentoo USE="-minimal -test" 72 KiB [ebuild N ] dev-perl/MIME-Charset-1.12.2::gentoo L10N="-ja -zh" 55 KiB [ebuild N ] sys-libs/gdbm-1.20:0/6::gentoo USE="berkdb nls readline -static-libs" 965 KiB [ebuild N ] sys-apps/kmod-29::gentoo USE="lzma (tools) zlib -debug -doc -pkcs7 -python -static-libs -zstd" PYTHON_TARGETS="python3_9 -python3_8" 548 KiB [ebuild N ] sys-process/psmisc-23.4-r1::gentoo USE="ipv6 nls -X (-selinux)" 362 KiB [ebuild N ] dev-perl/SGMLSpm-1.1-r2::gentoo USE="-test" 112 KiB [ebuild N ] dev-perl/Unicode-LineBreak-2019.1.0::gentoo 305 KiB [ebuild N ] dev-util/re2c-2.1.1-r1::gentoo USE="-debug" 1310 KiB [ebuild N ] app-text/po4a-0.57-r1::gentoo USE="-test" 3008 KiB [ebuild N ] dev-util/ninja-1.10.1::gentoo USE="-doc -emacs -test -vim-syntax" 207 KiB [ebuild N ] dev-util/meson-0.58.2-r1::gentoo USE="(-test)" PYTHON_TARGETS="python3_9 (-python3_10) -python3_8" 1855 KiB [ebuild N ] sys-apps/systemd-tmpfiles-246::gentoo USE="(-selinux) -test" 9311 KiB [ebuild N ] virtual/tmpfiles-0-r1::gentoo 0 KiB [ebuild N ] sys-apps/man-db-2.9.4-r1::gentoo USE="gdbm manpager nls seccomp zlib -berkdb (-selinux) -static-libs" 1865 KiB [ebuild N ] virtual/man-0-r4::gentoo 0 KiB Total: 75 packages (2 downgrades, 73 new), Size of downloads: 83332 KiB Traceback (most recent call last): File "/scratch/mharkonen3/gentoo/usr/lib/python3.9/site-packages/portage/util/__init__.py", line 1071, in apply_permissions os.chown(filename, uid, gid) File "/scratch/mharkonen3/gentoo/usr/lib/python3.9/site-packages/portage/__init__.py", line 231, in __call__ rval = self._func(*wrapped_args, **wrapped_kwargs) PermissionError: [Errno 1] Operation not permitted: b'/scratch/mharkonen3/gentoo/var/lib/portage' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/scratch/mharkonen3/gentoo/usr/lib/python-exec/python3.9/emerge", line 51, in <module> retval = emerge_main() File "/scratch/mharkonen3/gentoo/usr/lib/python3.9/site-packages/_emerge/main.py", line 1319, in emerge_main return run_action(emerge_config) File "/scratch/mharkonen3/gentoo/usr/lib/python3.9/site-packages/_emerge/actions.py", line 3392, in run_action retval = action_build(emerge_config, spinner=spinner) File "/scratch/mharkonen3/gentoo/usr/lib/python3.9/site-packages/_emerge/actions.py", line 549, in action_build mydepgraph.saveNomergeFavorites() File "/scratch/mharkonen3/gentoo/usr/lib/python3.9/site-packages/_emerge/depgraph.py", line 9280, in saveNomergeFavorites world_set.lock() File "/scratch/mharkonen3/gentoo/usr/lib/python3.9/site-packages/portage/_sets/files.py", line 231, in lock self._pkgset.lock() File "/scratch/mharkonen3/gentoo/usr/lib/python3.9/site-packages/portage/_sets/files.py", line 295, in lock self._ensure_dirs() File "/scratch/mharkonen3/gentoo/usr/lib/python3.9/site-packages/portage/_sets/files.py", line 290, in _ensure_dirs ensure_dirs(os.path.dirname(self._filename), gid=portage_gid, mode=0o2750, mask=0o2) File "/scratch/mharkonen3/gentoo/usr/lib/python3.9/site-packages/portage/util/__init__.py", line 1414, in ensure_dirs perms_modified = apply_permissions(dir_path, **kwargs) File "/scratch/mharkonen3/gentoo/usr/lib/python3.9/site-packages/portage/util/__init__.py", line 1078, in apply_permissions raise OperationNotPermitted(func_call) portage.exception.OperationNotPermitted: chown('/scratch/mharkonen3/gentoo/var/lib/portage', -1, 0) Hmmmm, I was already afraid of this to happen. Running /scratch/mharkonen3/gentoo/bin/bash ./bootstrap-prefix.sh "/scratch/mharkonen3/gentoo" stage3 somewhere failed :( Details might be found in the build log: (no build logs found?!?) I have no clue, really. Please find friendly folks in #gentoo-prefix on irc.gentoo.org, gentoo-alt@lists.gentoo.org mailing list, or file a bug at bugs.gentoo.org under Gentoo/Alt, Prefix Support. This is most inconvenient, and it crushed my ego. Sorry, I give up. Should you want to give it a try, there is /scratch/mharkonen3/gentoo/stage3.log I am hitting this too. This happens because my user is handled by Kerberos, and I am bootstrapping the prefix as myself. In particular, it seems that there is a keyerror in the lines below
elif k in ('portage_gid', 'portage_uid'):
#Discover the uid and gid of the portage user/group
keyerror = False
try:
portage_uid = pwd.getpwnam(_get_global('_portage_username')).pw_uid
except KeyError:
keyerror = True
portage_uid = 0
try:
portage_gid = grp.getgrnam(_get_global('_portage_grpname')).gr_gid
except KeyError:
keyerror = True
portage_gid = 0
in _get_global() in data.py when the user is not local. Then portage_uid and portage_gid get set to 0 by the fallback code. My suggestion would be to use the value of unprivileged and change the fallbacks to set uid/gid to os.getuid() and os.getgid() if unprivileged = True, because that would give the correct result if the user running portage is the owner of $EPREFIX directory.
*** Bug 777303 has been marked as a duplicate of this bug. *** (In reply to Guilherme Amadio from comment #19) > in _get_global() in data.py when the user is not local. Then portage_uid and > portage_gid get set to 0 by the fallback code. My suggestion would be to use > the value of unprivileged and change the fallbacks to set uid/gid to > os.getuid() and os.getgid() if unprivileged = True, because that would give > the correct result if the user running portage is the owner of $EPREFIX > directory. Normally, the unprivileged code sets PORTAGE_USERNAME and PORTAGE_GRPNAME from the uid/gid of EPREFIX if EPREFIX is writable by the current user. For example, if I create an new EPREFIX owned by me then everything follows from that: > $ whoami > zmedico > $ id -g > 1000 > $ id -u > 1000 > $ mkdir /tmp/eprefix > $ EPREFIX=/tmp/eprefix portageq envvar PORTAGE_USERNAME > zmedico > $ EPREFIX=/tmp/eprefix portageq envvar PORTAGE_GRPNAME > zmedico > $ EPREFIX=/tmp/eprefix python -c 'import portage; print(portage.data.portage_uid)' > 1000 > $ EPREFIX=/tmp/eprefix python -c 'import portage; print(portage.data.portage_gid)' > 1000 (In reply to Guilherme Amadio from comment #19) > I am hitting this too. This happens because my user is handled by Kerberos, > and I am bootstrapping the prefix as myself. In particular, it seems that > there is a keyerror in the lines below I think part of the struggle is that these should not keyerror. > > elif k in ('portage_gid', 'portage_uid'): > > #Discover the uid and gid of the portage user/group > keyerror = False > try: > portage_uid = > pwd.getpwnam(_get_global('_portage_username')).pw_uid > except KeyError: > keyerror = True > portage_uid = 0 > > try: > portage_gid = > grp.getgrnam(_get_global('_portage_grpname')).gr_gid > except KeyError: > keyerror = True > portage_gid = 0 > > > in _get_global() in data.py when the user is not local. Then portage_uid and > portage_gid get set to 0 by the fallback code. My suggestion would be to use > the value of unprivileged and change the fallbacks to set uid/gid to > os.getuid() and os.getgid() if unprivileged = True, because that would give > the correct result if the user running portage is the owner of $EPREFIX > directory. I'm not sure this is safe to change in every scenario though? I'd rather work out why you can't getpwnam() your own name on your system. Bug in NSS? As I've explained over IRC, the problem only happens when bootstrapping a RAP prefix (a.k.a. prefix-standalone), where glibc is part of the prefix itself and does not know about Kerberos users, but only those from /etc/passwd. The current bootstrap process either copies or links passwd from /etc, but that is not enough when the user is on Kerberos. See link below. https://gitweb.gentoo.org/repo/proj/prefix.git/tree/scripts/bootstrap-prefix.sh#n346 That's why in stage3 portage (and every other program as well) gets confused about identities. A workaround was pointed out in bug 777303, and I think that I will use that for the bootstrapping process. Any solution that lets portage figure out the uid/gid without using their names should work when the user is on Kerberos, but I understand if you want to keep the fallback setting to uid/gid=0 and not the uid/gid of the user running portage. The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=d39c38ea123c5eaea7f06c1cde0a47ec4d58807d commit d39c38ea123c5eaea7f06c1cde0a47ec4d58807d Author: Guilherme Amadio <amadio@gentoo.org> AuthorDate: 2021-10-28 11:22:42 +0000 Commit: Guilherme Amadio <amadio@gentoo.org> CommitDate: 2021-10-28 11:22:42 +0000 bootstrap-prefix.sh: fix RAP bootstrap with non-local user, bug 766417 When the bootstrapping user is handled by Kerberos and is not local to the host, the libc from prefix doesn't know how to convert usernames to user ids, which ends up confusing portage. Using getent from the host to add the bootstrapping user to ${EPREFIX}/etc/passwd works around this issue. Closes: https://bugs.gentoo.org/766417 Signed-off-by: Guilherme Amadio <amadio@gentoo.org> scripts/bootstrap-prefix.sh | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) In case of my user account provided by systemd-homed, grep falsely found my group in /etc/group even if it was not there and a symlink was installed, just because I was a member of a few other groups and obviously my group name was the same as my account name.
My solution was to further patch the bootstrap-prefix.sh script:
--- a/bootstrap-prefix.sh 2023-10-21 23:45:15.823968392 +0200
+++ b/bootstrap-prefix.sh 2023-11-03 23:12:10.075154234 +0100
@@ -314,7 +314,7 @@ bootstrap_setup() {
fi
fi
if [[ ! -f ${ROOT}/etc/group ]]; then
- if grep -q $(id -gn) /etc/group; then
+ if grep -q "^$(id -gn):" /etc/group; then
ln -sf {,"${ROOT}"}/etc/group
else
getent group > "${ROOT}"/etc/group
You may want to use the same approach for /etc/passwd to be on the safe side.
Moreover, both 'getent passwd' and 'getent group' may return accounts / groups from other providers, then adding some explicitly may result in duplicate records. That's usually harmless, and so far I haven't noticed any nasty side effects, but who knows. You may want to either limit base passwd / group data to /etc source by running 'getent passwd|group' with '-s files' option, or grep for user account / group in results of full getent.
Thanks,
Janusz
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=971f326949216a7edf673bf1b26c644ef20b072d commit 971f326949216a7edf673bf1b26c644ef20b072d Author: Janusz Krzysztofik <jmkrzyszt@gmail.com> AuthorDate: 2023-11-04 07:34:11 +0000 Commit: Fabian Groffen <grobian@gentoo.org> CommitDate: 2023-11-04 07:34:11 +0000 scripts/bootstrap-prefix: match user/group more precise Avoid over-matching on entries from /etc/{passwd,group}. Bug: https://bugs.gentoo.org/766417 Signed-off-by: Fabian Groffen <grobian@gentoo.org> scripts/bootstrap-prefix.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) |
I am installing Gentoo Prefix on my linux workstation (Redhat 7, kernel 3.10.0-1127.13.1.el7.x64_64) which I do not have root access. Stage 1 and Stage 2 completed and I got the following error at Stage 3. Looks like something’s not right with user/group. Any idea? I don't know if any other system info would be helpful for diagnosing. Please let me know. Thanks in advance. === Sync completed for gentoo !!! Directory initialization failed: '/data/rpo_yazn/gentoo/var/lib/portage' !!! chown('/data/rpo_yazn/gentoo/var/lib/portage', -1, 0) !!! Directory initialization failed: '/data/rpo_yazn/gentoo/var/cache/edb' !!! chown('/data/rpo_yazn/gentoo/var/cache/edb', -1, 0) * IMPORTANT: config file '/data/rpo_yazn/gentoo/etc/hosts' needs updating. [Errno 1] Operation not permitted: b'/data/rpo_yazn/gentoo/var/lib/gentoo/news/.news-gentoo.unread.portage_lockfile': chown('/data/rpo_yazn/gentoo/var/lib/gentoo/news/.new\ s-gentoo.unread.portage_lockfile', -1, 0) Cannot chown a lockfile: '/data/rpo_yazn/gentoo/var/lib/gentoo/news/.news-gentoo.unread.portage_lockfile' Group IDs of current user: 3073 3077 3239 3325 3430 3625 3661 65113 12041132 * See the CONFIGURATION FILES and CONFIGURATION FILES UPDATE TOOLS * sections of the emerge man page to learn how to update config files. Action: sync for repo: gentoo, returned code = 0 * running emerge -u system ^[]2;stage3: emerge -u system^Gsetlocale: unsupported locale setting portage: 'portage' user or group missing. For the defaults, line 1 goes into passwd, and 2 into group. portage:x:250:250:portage:/var/tmp/portage:/bin/false portage::250:portage *** WARNING *** For security reasons, only system administrators should be *** WARNING *** allowed in the portage group. Untrusted users or processes *** WARNING *** can potentially exploit the portage group for attacks such as *** WARNING *** local privilege escalation. !!! Directory initialization failed: '/data/rpo_yazn/gentoo/var/lib/portage' !!! chown('/data/rpo_yazn/gentoo/var/lib/portage', -1, 0) !!! Directory initialization failed: '/data/rpo_yazn/gentoo/var/cache/edb' !!! chown('/data/rpo_yazn/gentoo/var/cache/edb', -1, 0) setlocale: unsupported locale setting [Errno 1] Operation not permitted: b'/data/rpo_yazn/gentoo/var/lib/gentoo/news/.news-gentoo.unread.portage_lockfile': chown('/data/rpo_yazn/gentoo/var/lib/gentoo/news/.new\ s-gentoo.unread.portage_lockfile', -1, 0) Cannot chown a lockfile: '/data/rpo_yazn/gentoo/var/lib/gentoo/news/.news-gentoo.unread.portage_lockfile' Group IDs of current user: 3073 3077 3239 3325 3430 3625 3661 65113 12041132 These are the packages that would be merged, in order: Calculating dependencies [Errno 1] Operation not permitted: b'/data/rpo_yazn/gentoo/var/db/.pkg.portage_lockfile': chown('/data/rpo_yazn/gentoo/var/db/.pkg.portage_lockfi\ le', -1, 0) Cannot chown a lockfile: '/data/rpo_yazn/gentoo/var/db/.pkg.portage_lockfile' Group IDs of current user: 3073 3077 3239 3325 3430 3625 3661 65113 12041132 ... done! [ebuild N ] acct-group/sshd-0-r1::gentoo 0 KiB [ebuild N ] app-misc/editor-wrapper-4-r1::gentoo 0 KiB [ebuild N ] sys-apps/which-2.21::gentoo 146 KiB [ebuild N ] acct-user/sshd-0-r1::gentoo 0 KiB [ebuild N ] sys-apps/diffutils-3.7-r1::gentoo USE="nls -static" 0 KiB [ebuild N ] sys-apps/net-tools-2.10::gentoo USE="arp hostname ipv6 nls -nis -plipconfig (-selinux) -slattach -static" 225 KiB [ebuild N ] sys-process/psmisc-23.3-r1::gentoo USE="ipv6 nls -X (-selinux)" 305 KiB [ebuild N ] virtual/libc-1-r1::gentoo 0 KiB [ebuild N ] virtual/package-manager-1::gentoo 0 KiB [ebuild N ] dev-libs/libpcre2-10.36::gentoo USE="bzip2 jit readline recursion-limit (split-usr) unicode zlib -libedit -pcre16 -pcre32 -static-libs" 1684 KiB [ebuild N ] app-editors/nano-5.5::gentoo USE="ncurses nls spell (split-usr) (unicode) -debug -justify -magic -minimal -static" 2875 KiB [ebuild N ] net-misc/openssh-8.4_p1-r2::gentoo USE="pie scp ssl -X -X509 -audit -bindist -debug -hpn -kerberos -ldns -libedit -libressl -livecd (-pam) -sctp -securit\ y-key (-selinux) -static -test -xmss" 1702 KiB [ebuild N ] sys-apps/kmod-28::gentoo USE="lzma (tools) zlib -debug -doc -libressl -pkcs7 -python -static-libs -zstd" PYTHON_TARGETS="python3_8 (-python3_6) -python3_\ 7 -python3_9" 540 KiB [ebuild N ] sys-process/procps-3.3.16-r2:0/8::gentoo USE="kill ncurses nls (split-usr) (unicode) -elogind -modern-top (-selinux) -static-libs (-systemd) -test" 840 K\ iB [ebuild N ] sys-apps/less-563-r1::gentoo USE="pcre unicode" 328 KiB [ebuild N ] net-misc/wget-1.21.1::gentoo USE="ipv6 nls pcre (ssl) zlib -cookie_check -debug -gnutls -idn -libressl -metalink -ntlm -static -test -uuid" 4771 KiB [ebuild N ] virtual/editor-0-r3::gentoo 0 KiB [ebuild N ] virtual/ssh-0::gentoo USE="-minimal" 0 KiB [ebuild N ] virtual/pager-0::gentoo 0 KiB Total: 19 packages (19 new), Size of downloads: 13412 KiB Traceback (most recent call last): File "/data/rpo_yazn/gentoo/usr/lib/python3.8/site-packages/portage/util/__init__.py", line 1071, in apply_permissions os.chown(filename, uid, gid) File "/data/rpo_yazn/gentoo/usr/lib/python3.8/site-packages/portage/__init__.py", line 230, in __call__ rval = self._func(*wrapped_args, **wrapped_kwargs) PermissionError: [Errno 1] Operation not permitted: b'/data/rpo_yazn/gentoo/var/lib/portage' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/data/rpo_yazn/gentoo/usr/lib/python-exec/python3.8/emerge", line 51, in <module> retval = emerge_main() File "/data/rpo_yazn/gentoo/usr/lib/python3.8/site-packages/_emerge/main.py", line 1317, in emerge_main return run_action(emerge_config) File "/data/rpo_yazn/gentoo/usr/lib/python3.8/site-packages/_emerge/actions.py", line 3376, in run_action retval = action_build(emerge_config, spinner=spinner) File "/data/rpo_yazn/gentoo/usr/lib/python3.8/site-packages/_emerge/actions.py", line 548, in action_build mydepgraph.saveNomergeFavorites() File "/data/rpo_yazn/gentoo/usr/lib/python3.8/site-packages/_emerge/depgraph.py", line 9275, in saveNomergeFavorites world_set.lock() File "/data/rpo_yazn/gentoo/usr/lib/python3.8/site-packages/portage/_sets/files.py", line 231, in lock self._pkgset.lock() File "/data/rpo_yazn/gentoo/usr/lib/python3.8/site-packages/portage/_sets/files.py", line 295, in lock self._ensure_dirs() File "/data/rpo_yazn/gentoo/usr/lib/python3.8/site-packages/portage/_sets/files.py", line 290, in _ensure_dirs ensure_dirs(os.path.dirname(self._filename), gid=portage_gid, mode=0o2750, mask=0o2) File "/data/rpo_yazn/gentoo/usr/lib/python3.8/site-packages/portage/util/__init__.py", line 1414, in ensure_dirs perms_modified = apply_permissions(dir_path, **kwargs) File "/data/rpo_yazn/gentoo/usr/lib/python3.8/site-packages/portage/util/__init__.py", line 1078, in apply_permissions raise OperationNotPermitted(func_call) portage.exception.OperationNotPermitted: chown('/data/rpo_yazn/gentoo/var/lib/portage', -1, 0)