Bug 668158 – ownership of directories will get lost on first merge for newly created user/groups

• Product: Portage Development
• Status: CONFIRMED ---

[Description Thomas Deutschmann (RETIRED) 2018-10-09 12:13:56 UTC]

When you set permission/ownership for directories in $D (the image) for an
user/group which will be created during emerge, group ownership will be
lost when image will be merged to $ROOT.

This doesn't happen on second run (the directory which will be merged was
of course removed before) or when you use `ebuild` but do the merge in two
steps (i.e. `ebuild foo-1.ebuild clean install && ebuild foo-1.ebuild merge`).

I added some debug code to prove that the image was created like expected:

> --- a/app-metrics/mysqld_exporter/mysqld_exporter-0.10.0-r1.ebuild
> +++ b/app-metrics/mysqld_exporter/mysqld_exporter-0.10.0-r1.ebuild
> @@ -53,6 +53,11 @@ src_install() {
>         newins "${FILESDIR}"/${PN}.logrotate ${PN}
>  }
> 
> +pkg_preinst() {
> +       einfo "Debug: ls -ld ${ED%/}/var/lib/mysqld_exporter"
> +       ls -ld "${ED%/}"/var/lib/mysqld_exporter
> +}
> +
>  pkg_postinst() {
>         if [[ -z "${REPLACING_VERSIONS}" ]]; then
>                 # This is a new installation

Test run:
 
> vm-gentoo-x64 ~ # id mysqld_exporter
> id: ‘mysqld_exporter’: no such user
> vm-gentoo-x64 ~ # ls -ld /var/lib/mysqld_exporter
> ls: cannot access '/var/lib/mysqld_exporter': No such file or directory

Clean environment, the user and folder our test ebuild will create doesn't
exist yet.

> vm-gentoo-x64 ~ # emerge -1 app-metrics/mysqld_exporter
> 
> These are the packages that would be merged, in order:
> 
> Calculating dependencies... done!
> [ebuild   R    ] app-metrics/mysqld_exporter-0.10.0-r1::gentoo  0 KiB
> 
> Total: 1 package (1 reinstall), Size of downloads: 0 KiB
> 
> 
> >>> Verifying ebuild manifests
> 
> >>> Emerging (1 of 1) app-metrics/mysqld_exporter-0.10.0-r1::gentoo
>  * mysqld_exporter-0.10.0.tar.gz BLAKE2B SHA512 size ;-) ...                                                     [ ok ]
>  * Adding group 'mysqld_exporter' to your system ...
>  *  - Groupid: next available
>  * Adding user 'mysqld_exporter' to your system ...
>  *  - Userid: 149
>  *  - Shell: /sbin/nologin
>  *  - Home: /var/lib/mysqld_exporter
>  *  - Groups: mysqld_exporter
>  *  - GECOS: added by portage for mysqld_exporter
>  *  - Creating /var/lib/mysqld_exporter in /
> >>> Unpacking source...
> >>> Source unpacked in /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work
> >>> Preparing source in /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0 ...
> >>> Source prepared.
> >>> Configuring source in /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0 ...
> >>> Source configured.
> >>> Compiling source in /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0 ...
> /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0/src/github.com/prometheus/mysqld_exporter /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0
> Using config file: /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0/src/github.com/prometheus/mysqld_exporter/.promu.yml
>  >   mysqld_exporter
>  +  go build -o bin/mysqld_exporter -ldflags -X github.com/prometheus/mysqld_exporter/vendor/github.com/prometheus/common/version.Version=0.10.0 -X github.com/prometheus/mysqld_exporter/vendor/github.com/prometheus/common/version.Revision=8068006 -X github.com/prometheus/mysqld_exporter/vendor/github.com/prometheus/common/version.Branch=non-git -X github.com/prometheus/mysqld_exporter/vendor/github.com/prometheus/common/version.BuildUser=portage@vm-gentoo-x64 -X github.com/prometheus/mysqld_exporter/vendor/github.com/prometheus/common/version.BuildDate=20181009-11:51:40  -extldflags '-static' -a -tags netgo github.com/prometheus/mysqld_exporter
> /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0
> >>> Source compiled.
> >>> Test phase [not enabled]: app-metrics/mysqld_exporter-0.10.0-r1
> 
> >>> Install mysqld_exporter-0.10.0-r1 into /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/image/ category app-metrics
> /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0/src/github.com/prometheus/mysqld_exporter /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0
> /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0
> >>> Completed installing mysqld_exporter-0.10.0-r1 into /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/image/
> 
>  * Final size of build directory: 17656 KiB (17.2 MiB)
>  * Final size of installed tree:  10996 KiB (10.7 MiB)
> 
> strip: x86_64-pc-linux-gnu-strip --strip-unneeded -R .comment -R .GCC.command.line -R .note.gnu.gold-version
>    usr/bin/mysqld_exporter
> 
> >>> Installing (1 of 1) app-metrics/mysqld_exporter-0.10.0-r1::gentoo
>  * checking 9 files for package collisions
> >>> Merging app-metrics/mysqld_exporter-0.10.0-r1 to /
>  * Debug: ls -ld /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/image/var/lib/mysqld_exporter
> drwxrwx--- 2 mysqld_exporter mysqld_exporter 4096 Oct  9 13:51 /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/image/var/lib/mysqld_exporter
> --- /var/
> --- /var/lib/
> --- /var/lib/mysqld_exporter/
> >>> /var/lib/mysqld_exporter/.keep_app-metrics_mysqld_exporter-0
> --- /var/log/
> --- /var/log/mysqld_exporter/
> >>> /var/log/mysqld_exporter/.keep_app-metrics_mysqld_exporter-0
> --- /usr/
> --- /usr/bin/
> >>> /usr/bin/mysqld_exporter
> --- /usr/share/
> --- /usr/share/doc/
> --- /usr/share/doc/mysqld_exporter-0.10.0-r1/
> >>> /usr/share/doc/mysqld_exporter-0.10.0-r1/CONTRIBUTING.md.bz2
> >>> /usr/share/doc/mysqld_exporter-0.10.0-r1/CHANGELOG.md.bz2
> >>> /usr/share/doc/mysqld_exporter-0.10.0-r1/README.md.bz2
> --- /etc/
> --- /etc/init.d/
> >>> /etc/init.d/mysqld_exporter
> --- /etc/conf.d/
> >>> /etc/conf.d/mysqld_exporter
> --- /etc/logrotate.d/
> >>> /etc/logrotate.d/mysqld_exporter
> >>> Safely unmerging already-installed instance...
> No package files given... Grabbing a set.
> --- replaced obj /var/log/mysqld_exporter/.keep_app-metrics_mysqld_exporter-0
> --- replaced dir /var/log/mysqld_exporter
> --- replaced dir /var/log
> --- replaced obj /var/lib/mysqld_exporter/.keep_app-metrics_mysqld_exporter-0
> --- replaced dir /var/lib/mysqld_exporter
> --- replaced dir /var/lib
> --- replaced dir /var
> --- replaced obj /usr/share/doc/mysqld_exporter-0.10.0-r1/README.md.bz2
> --- replaced obj /usr/share/doc/mysqld_exporter-0.10.0-r1/CONTRIBUTING.md.bz2
> --- replaced obj /usr/share/doc/mysqld_exporter-0.10.0-r1/CHANGELOG.md.bz2
> --- replaced dir /usr/share/doc/mysqld_exporter-0.10.0-r1
> --- replaced dir /usr/share/doc
> --- replaced dir /usr/share
> --- replaced obj /usr/bin/mysqld_exporter
> --- replaced dir /usr/bin
> --- replaced dir /usr
> --- replaced obj /etc/logrotate.d/mysqld_exporter
> --- replaced dir /etc/logrotate.d
> --- replaced obj /etc/init.d/mysqld_exporter
> --- replaced dir /etc/init.d
> --- replaced obj /etc/conf.d/mysqld_exporter
> --- replaced dir /etc/conf.d
> --- replaced dir /etc
> >>> Regenerating /etc/ld.so.cache...
> >>> Original instance of package unmerged safely.
> >>> app-metrics/mysqld_exporter-0.10.0-r1 merged.
> >>> Auto-cleaning packages...
> 
> >>> No outdated packages were found on your system.
> 
>  * GNU info directory index is up-to-date.
> vm-gentoo-x64 ~ # ls -ld /var/lib/mysqld_exporter
> drwxr-xr-x 2 mysqld_exporter root 4096 Oct  9 13:51 /var/lib/mysqld_exporter

...and that's unexpected:

Like shown above, the ownership of "/var/lib/mysqld_exporter" was set to
mysqld_exporter:mysqld_exporter but when merged to $ROOT, group ownership
was lost.

Second run. First we verify that user/group exists and remove "/var/lib/mysqld_exporter"
to allow new permission/ownership when the new install will be merged
to $ROOT:

> vm-gentoo-x64 ~ # id mysqld_exporter
> uid=149(mysqld_exporter) gid=972(mysqld_exporter) groups=972(mysqld_exporter)
> vm-gentoo-x64 ~ # rm /var/lib/mysqld_exporter/ -rf
> vm-gentoo-x64 ~ # emerge -1 app-metrics/mysqld_exporter
> 
> These are the packages that would be merged, in order:
> 
> Calculating dependencies... done!
> [ebuild   R    ] app-metrics/mysqld_exporter-0.10.0-r1::gentoo  0 KiB
> 
> Total: 1 package (1 reinstall), Size of downloads: 0 KiB
> 
> 
> >>> Verifying ebuild manifests
> 
> >>> Emerging (1 of 1) app-metrics/mysqld_exporter-0.10.0-r1::gentoo
>  * mysqld_exporter-0.10.0.tar.gz BLAKE2B SHA512 size ;-) ...                                                     [ ok ]
> >>> Unpacking source...
> >>> Source unpacked in /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work
> >>> Preparing source in /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0 ...
> >>> Source prepared.
> >>> Configuring source in /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0 ...
> >>> Source configured.
> >>> Compiling source in /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0 ...
> /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0/src/github.com/prometheus/mysqld_exporter /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0
> Using config file: /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0/src/github.com/prometheus/mysqld_exporter/.promu.yml
>  >   mysqld_exporter
>  +  go build -o bin/mysqld_exporter -ldflags -X github.com/prometheus/mysqld_exporter/vendor/github.com/prometheus/common/version.Version=0.10.0 -X github.com/prometheus/mysqld_exporter/vendor/github.com/prometheus/common/version.Revision=8068006 -X github.com/prometheus/mysqld_exporter/vendor/github.com/prometheus/common/version.Branch=non-git -X github.com/prometheus/mysqld_exporter/vendor/github.com/prometheus/common/version.BuildUser=portage@vm-gentoo-x64 -X github.com/prometheus/mysqld_exporter/vendor/github.com/prometheus/common/version.BuildDate=20181009-11:52:18  -extldflags '-static' -a -tags netgo github.com/prometheus/mysqld_exporter
> /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0
> >>> Source compiled.
> >>> Test phase [not enabled]: app-metrics/mysqld_exporter-0.10.0-r1
> 
> >>> Install mysqld_exporter-0.10.0-r1 into /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/image/ category app-metrics
> /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0/src/github.com/prometheus/mysqld_exporter /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0
> /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/work/mysqld_exporter-0.10.0
> >>> Completed installing mysqld_exporter-0.10.0-r1 into /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/image/
> 
>  * Final size of build directory: 17656 KiB (17.2 MiB)
>  * Final size of installed tree:  10996 KiB (10.7 MiB)
> 
> strip: x86_64-pc-linux-gnu-strip --strip-unneeded -R .comment -R .GCC.command.line -R .note.gnu.gold-version
>    usr/bin/mysqld_exporter
> 
> >>> Installing (1 of 1) app-metrics/mysqld_exporter-0.10.0-r1::gentoo
>  * checking 9 files for package collisions
> >>> Merging app-metrics/mysqld_exporter-0.10.0-r1 to /
>  * Debug: ls -ld /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/image/var/lib/mysqld_exporter
> drwxrwx--- 2 mysqld_exporter mysqld_exporter 4096 Oct  9 13:52 /var/tmp/portage/app-metrics/mysqld_exporter-0.10.0-r1/image/var/lib/mysqld_exporter
> --- /var/
> --- /var/lib/
> >>> /var/lib/mysqld_exporter/
> >>> /var/lib/mysqld_exporter/.keep_app-metrics_mysqld_exporter-0
> --- /var/log/
> --- /var/log/mysqld_exporter/
> >>> /var/log/mysqld_exporter/.keep_app-metrics_mysqld_exporter-0
> --- /usr/
> --- /usr/bin/
> >>> /usr/bin/mysqld_exporter
> --- /usr/share/
> --- /usr/share/doc/
> --- /usr/share/doc/mysqld_exporter-0.10.0-r1/
> >>> /usr/share/doc/mysqld_exporter-0.10.0-r1/CONTRIBUTING.md.bz2
> >>> /usr/share/doc/mysqld_exporter-0.10.0-r1/CHANGELOG.md.bz2
> >>> /usr/share/doc/mysqld_exporter-0.10.0-r1/README.md.bz2
> --- /etc/
> --- /etc/init.d/
> >>> /etc/init.d/mysqld_exporter
> --- /etc/conf.d/
> >>> /etc/conf.d/mysqld_exporter
> --- /etc/logrotate.d/
> >>> /etc/logrotate.d/mysqld_exporter
> >>> Safely unmerging already-installed instance...
> No package files given... Grabbing a set.
> --- replaced obj /var/log/mysqld_exporter/.keep_app-metrics_mysqld_exporter-0
> --- replaced dir /var/log/mysqld_exporter
> --- replaced dir /var/log
> --- replaced obj /var/lib/mysqld_exporter/.keep_app-metrics_mysqld_exporter-0
> --- replaced dir /var/lib/mysqld_exporter
> --- replaced dir /var/lib
> --- replaced dir /var
> --- replaced obj /usr/share/doc/mysqld_exporter-0.10.0-r1/README.md.bz2
> --- replaced obj /usr/share/doc/mysqld_exporter-0.10.0-r1/CONTRIBUTING.md.bz2
> --- replaced obj /usr/share/doc/mysqld_exporter-0.10.0-r1/CHANGELOG.md.bz2
> --- replaced dir /usr/share/doc/mysqld_exporter-0.10.0-r1
> --- replaced dir /usr/share/doc
> --- replaced dir /usr/share
> --- replaced obj /usr/bin/mysqld_exporter
> --- replaced dir /usr/bin
> --- replaced dir /usr
> --- replaced obj /etc/logrotate.d/mysqld_exporter
> --- replaced dir /etc/logrotate.d
> --- replaced obj /etc/init.d/mysqld_exporter
> --- replaced dir /etc/init.d
> --- replaced obj /etc/conf.d/mysqld_exporter
> --- replaced dir /etc/conf.d
> --- replaced dir /etc
> >>> Regenerating /etc/ld.so.cache...
> >>> Original instance of package unmerged safely.
> >>> app-metrics/mysqld_exporter-0.10.0-r1 merged.
> >>> Auto-cleaning packages...
> 
> >>> No outdated packages were found on your system.
> 
>  * GNU info directory index is up-to-date.
> vm-gentoo-x64 ~ # ls -ld /var/lib/mysqld_exporter
> drwxrwx--- 2 mysqld_exporter mysqld_exporter 4096 Oct  9 13:52 /var/lib/mysqld_exporter

...and this time everything worked like expected.

Tested with latest portage git master.

[Comment 1 Zac Medico 2018-10-09 17:03:07 UTC]

Bug 396153 proposes a PMS extension that will allow users to persistently override directory permissions when necessary.

Also, I'm planning a FEATURES=overwrite-dir-perms setting, see bug 654138.

*** This bug has been marked as a duplicate of bug 141619 ***