Bugzilla Bug 141619

A sample ebuild stub to reproduce:

<snip>
# Copyright 1999-2006 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $

inherit eutils

DESCRIPTION="MIMEDefang is a mail filtering framework for sendmail"
HOMEPAGE="http://www.mimedefang.org/"
SRC_URI=""
LICENSE="GPL-2"
SLOT="0"
KEYWORDS="~x86"
DEPEND=""
RDEPEND="${DEPEND}"

pkg_setup() {
        enewgroup defang
        enewuser defang -1 -1 /var/spool/MIMEDefang defang
}

src_install() {
        keepdir /var/spool/{MIMEDefang,MD-Quarantine}
        chmod 0775 ${D}/var/spool/MIMEDefang
        chmod 0755 ${D}/var/spool/MD-Quarantine
        chown defang:defang ${D}/var/spool/{MIMEDefang,MD-Quarantine}
}
</snip>

1/ User doesn't exist + first emerge

# ls -ld /var/spool/{MIMEDefang,MD-Quarantine}
drwxr-xr-x 2 defang defang 96 2006-07-24 21:32 /var/spool/MD-Quarantine
drwxr-xr-x 2 defang root   96 2006-07-24 21:32 /var/spool/MIMEDefang

2/ User does exist + re-emerge

# ls -ld /var/spool/{MIMEDefang,MD-Quarantine}
drwxr-xr-x 2 defang defang 96 2006-07-24 21:33 /var/spool/MD-Quarantine
drwxr-xr-x 2 defang root   96 2006-07-24 21:33 /var/spool/MIMEDefang

Still broken.

3/ User does exist + unmerge and emerge

# ls -ld /var/spool/{MIMEDefang,MD-Quarantine}
drwxr-xr-x 2 defang defang 96 2006-07-24 21:34 /var/spool/MD-Quarantine
drwxrwxr-x 2 defang defang 96 2006-07-24 21:34 /var/spool/MIMEDefang

That's what it should be like for the first time!

4/ unmerge + userdel -f defang + emerge 

# ls -ld /var/spool/{MIMEDefang,MD-Quarantine}
drwxr-xr-x 2 defang defang 96 2006-07-24 21:33 /var/spool/MD-Quarantine
drwxr-xr-x 2 defang root   96 2006-07-24 21:33 /var/spool/MIMEDefang

Busted again.

Portage 2.1.1_pre3-r4 (default-linux/x86/2006.0, gcc-4.1.1, glibc-2.4-r3,
2.6.17-gentoo-r3 i686)
=================================================================
System uname: 2.6.17-gentoo-r3 i686 AMD Athlon(tm) XP 1600+
Gentoo Base System version 1.12.1
ccache version 2.4 [enabled]
app-admin/eselect-compiler: [Not Present]
dev-lang/python:     2.4.3-r1
dev-python/pycrypto: 2.0.1-r5
dev-util/ccache:     2.4-r2
dev-util/confcache:  0.4.2-r1
sys-apps/sandbox:    1.2.18.1
sys-devel/autoconf:  2.13, 2.60
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2
sys-devel/binutils:  2.16.1-r2, 2.16.93, 2.17, 2.17.50.0.2, 2.17.50.0.3
sys-devel/gcc-config: 1.3.13-r3
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.11-r5
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=athlon-xp -O2 -pipe -fomit-frame-pointer -fforce-addr
-ftree-vectorize"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config
/usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf
/etc/java-config/vms/ /etc/revdep-rebuild /etc/splash /etc/terminfo"
CXXFLAGS="-march=athlon-xp -O2 -pipe -fomit-frame-pointer -fforce-addr
-ftree-vectorize"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--alphabetical"
FEATURES="autoconfig ccache collision-protect confcache distlocks
metadata-transfer parallel-fetch sandbox sfperms splitdebug strict userfetch
userpriv usersandbox"
GENTOO_MIRRORS="ftp://ftp.sh.cvut.cz/MIRRORS/gentoo/gentoo
ftp://ftp.fi.muni.cz/pub/linux/gentoo/"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--sort-common"
LINGUAS="cs en"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_EXTRA_OPTS="--progress"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress
--force --whole-file --delete --delete-after --stats --timeout=180
--exclude='/distfiles' --exclude='/local' --exclude='/packages'"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage /mnt/nfs/overlays/vmware"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 3dnow 3dnowext 7zip X X509 a52 aac acl acpi alsa apm asf audiofile
bash-completion berkdb bluetooth bzip2 caps cddb cdparanoia cdr chroot cli
crypt cscope css cups curl curlwrappers dbx dga dlloader dri dts dv dvd dvdr
dvdread encode ethereal exif expat fam fbcon ffmpeg fftw firefox flac flash
flatfile foomaticdb gdbm gif glibc-omitfp glut gmp gpm gstreamer iconv icq idn
imagemagick imap imlib inifile ipv6 irda jack javascript jbig joystick jpeg
jpeg2k kdeenablefinal kdehiddenvisibility lcms libcaca libg++ libsamplerate
libwww lirc lm_sensors logrotate mad maildir matroska mikmod mime mmap mmx mng
mp3 mpeg musepack musicbrainz ncurses nls nodrm nptl nptlonly nsplugin nvidia
offensive ogg openal opengl pam pcre pdf perl png ppds python qt3 quicktime
readline real reflection samba sdl session sftplogging skey sndfile speex spell
spl sse ssl svg symlink tcpd theora threads tiff truetype udev unicode urandom
usb v4l v4l2 vcd vorbis win32codecs wmf x264 xine xinerama xinetd xml xml2
xmlrpc xorg xosd xpm xv xvid xvmc zlib elibc_glibc input_devices_joystick
input_devices_keyboard input_devices_mouse input_devices_evdev kernel_linux
linguas_cs linguas_en lirc_devices_cph06x userland_GNU video_cards_fbdev
video_cards_nv video_cards_nvidia video_cards_v4l video_cards_vesa
video_cards_vmware"
Unset:  CTARGET, INSTALL_MASK, LC_ALL

<genone> jakub: what are the perms in ${D}?
<genone> jakub: also when you say "first emerge", are you _sure_ the dirs
didn't exist previously?
<jakub> genone: it does... enewuser creates it
<genone> ignore enewuser
<jakub> well then yeah, I'm pretty damn sure it doesn't, I've re-checked about
10 times while pulling my hair here
<genone> "it" == ?
<jakub> well, the dir that should be 775 user:group and is 755 user:root
instead
<jakub> no way to do that unless the user and group already exist
<jakub> ie., exist _before_ emerge
<jakub> genone: in ${D} it's correct, but doesn't get merged w/ those perms
<genone> ok, so if I'm understanding this right useradd (called by enewuser)
creates the dir in ${ROOT} with wrong perms/ownership
<jakub> genone: there's no way to specify such thing in enewuser
<genone> and portage doesn't change ownership/perms of existing files in
${ROOT] at merge time
<jakub> yup
<genone> the second is by design
<jakub> genone: uhm?
<genone> uhm what?
<jakub> also, s/files/dirs/ - there are no files involved here
<genone> dirs are files
<jakub> well, by design doesn't seem to be very good here, because you get
broken ebuild
* genone is too lazy to type {dirs,files,fifos,devs,...}
<genone> jakub: you'd rather break peoples systems?
<jakub> shrug, broken how exactly? it permissions set in ebuild break the
system, they are not probably sane...
<jakub> whatever, any solution here?
<genone> postinst
<genone> or get spanky to fix enewuser to not create the dir in ${ROOT}
<jakub> chown it in postinst?
<jakub> well, then I don't exactly see the difference from changing the design,
can break the system equally
<genone> well, take it up on -dev if you want that to get changed
<jakub> heh, not sure if that's the best place :=) usually doesn't get very
productive
<jakub> so, how viable is to change the eclass?
<genone> ask Spanky

Uhm, reopening this and sending vapier's way... Two solutions I see here:

- let people specify the owner and permissions for homedir in enewuser call
- create the homedir in ${D} instead of ${ROOT}

(Or, obviously let portage change the perms on merge but that's not exactly
what portage folks like...)

(In reply to comment #3)
> (Or, obviously let portage change the perms on merge but that's not exactly
> what portage folks like...)

Ehm, I'm not attached to the behavior in one way or the other, but if this
should get changed it needs the support from the community, not just yours.

i see this as being a dupe of this thread:
http://article.gmane.org/gmane.linux.gentoo.portage.devel/2182

(In reply to comment #5)
> i see this as being a dupe of this thread:
> http://article.gmane.org/gmane.linux.gentoo.portage.devel/2182

Or bug 58611.

Did we ever get to a conclusion here?

(In reply to comment #7)
> Did we ever get to a conclusion here?

No. :( The worst thing is the inconsistent behaviour, it's confusing and
produces pretty inexplicable bugs.

*** Bug 172576 has been marked as a duplicate of this bug. ***

*** Bug 174916 has been marked as a duplicate of this bug. ***

(In reply to comment #10)
> *** Bug 174916 has been marked as a duplicate of this bug. ***

OK, the above mentioned bug was in the first place about a specific ebuild,
openmotif, and its possible need to fix the mistakes of a previous revision.
I guess this might already be fixed by postinst, without breaking much.

As this thing was duped here, I'll give you my thoughts on the general issue.
From the admin perspective I'd like to differentiate between directories owned
exclusively by some package, and directories created in image to represent
shared structures.
So if foo-0.ebuild decides to make directory /var/lib/foo group writable, there
might be a good reason for this, while doing the same for /var/lib is probably
a bad idea.

I would not want permissions for existing directories merged unconditionally,
because it could overwrite modifications the admin made on purpose. However, if
an ebuild chooses to change some permissions from the way they were before,
there should be a way to do so.
Something like this:
src_postinst {
  echmod 755 775 /var/lib/foo
}
to say that permissions on /var/lib/foo should be changed to 775 if they had
been 755 (the old default) before. This is still not perfect, because when the
admin chooses to revert these changes, his modifications get overwritten the
next time this package is upgraded. But to solve that issue, we would need some
information about the version the ebuild currently being emerged is going to
replace. Say "change mode if upgrading from Version <1.0" or something similar.

It might also be nice to carry some mode information from the install phase
into the merge or postinst phase. Then the install phase could say "let the
newly installed versions of /var/lib/foo [recursively] override those already
there", and there would be no need to specify the modes in the ebuild itself.
I guess this could best be solved by some postinst script that is part of the
package, created during install by some eclass function, will be merged with
the rest, then run automatically, and deleted afterwards.

I have no idea how all this might cooperate with portage, and if this would
work equally well for ownership. On the other hand, I would think this a
feasible approach for files as well, not only dirs. Maybe there should be some
FEATURE or other tool for ebuild writers that complains if a file in image does
not agree with wat is currently installed, so that forgotten changes will be
noticed.

Found two new instances today: /usr/include/gnome-xml from dev-libs/libxml and
/usr/include/lqt from media-libs/libquicktime had no execute bit set. Deleting
dirs and remerging ebuilds solved the issue. Don't know where this came from.

Could we ewarn about differences between present permissions and the
permissions from the ebuild? That would be one step towards solving the
problem, and should be better than what's there right now.

*** Bug 140250 has been marked as a duplicate of this bug. ***

*** Bug 182983 has been marked as a duplicate of this bug. ***

*** Bug 193728 has been marked as a duplicate of this bug. ***

*** Bug 199197 has been marked as a duplicate of this bug. ***

(In reply to comment #11)
> I would not want permissions for existing directories merged unconditionally,
> because it could overwrite modifications the admin made on purpose.

You already get these overwritten w/ tons of ebuilds b/c people keep working
around this portage bug via doing chmod/chown in unsandboxed phases. We really
should move somewhere with this as this feature serves no real purpose but
adding hacks to ebuilds.

I think the real solution here would be to store the permissions/ownership in
the installed package database and only preserve them if modified after
installation to satisfy both use cases. But I'd rather want to include that in
a general vdb redesign rather than adding yet another file (as extending
CONTENTS isn't an option)

Created an attachment (id=140656) [edit]
implement FEATURES=merge-dir-perms and enable in make.globals

With this patch, the average user who doesn't want to control directory
permissions themselves will have permissions merged from ${D}. Users who want
to manage directory permissions themselves can add FEATURES="-merge-dir-perms"
to /etc/make.conf.

I strongly dislike that hack, as it doesn't really help anyone, devs still have
to work around the issue in postinst for older versions and users without that
setting, and it really shouldn't be a optional thing anyway (and do I have to
repeat that I don't want to bloat FEATURES with cornercase stuff?)

(In reply to comment #18)
> I think the real solution here would be to store the permissions/ownership in
> the installed package database and only preserve them if modified after
> installation to satisfy both use cases.

What about multiple packages that install the same directory with different
permissions? Whose permissions win?

The vdb redesign would likely contain a central database about file objects, so
the last one would win. But that shouldn't happen, as two packages trying to
install the same dir with conflicting permissions would be detected with a
central db.

(In reply to comment #21)
> What about multiple packages that install the same directory with different
> permissions? Whose permissions win?

This is ordinary collision and should be reported and fixed by maintainers of
said packages.

From the user point of view I think portage should keep permissions developer
assigned (actually overrided default permissions), and there is no need to
completely disable permission control mechanism (like -merge-dir-perms do).
But user should have some way to override permissions. I do not know what is
the best way to go but at least two possible solutions exist:

1. Either some user defined permission db (may be file with "file perm"
entries and recursion mark for directories)
2. Or add portage FEATURE which keeps modified by user permissions and show a
warning that permissions were changed.

Variant 2. may be does not work right now as we have many packages which set
permissions in pkg_postinst and will warn even if user want such different
permissions. And yes central db simplify things and is necessary in any case.

(In reply to comment #23)
> (In reply to comment #21)
> > What about multiple packages that install the same directory with different
> > permissions? Whose permissions win?
> 
> This is ordinary collision and should be reported and fixed by maintainers of
> said packages.

Well, multiple packages installing the same dir isn't a collision, only the
different permissions would be and that's something we can't really detect atm
without reference data.

> From the user point of view I think portage should keep permissions developer
> assigned (actually overrided default permissions), and there is no need to
> completely disable permission control mechanism (like -merge-dir-perms do).
> But user should have some way to override permissions. I do not know what is
> the best way to go but at least two possible solutions exist:
> 
> 1. Either some user defined permission db (may be file with "file perm"
> entries and recursion mark for directories)
> 2. Or add portage FEATURE which keeps modified by user permissions and show a
> warning that permissions were changed.
> 
> Variant 2. may be does not work right now as we have many packages which set
> permissions in pkg_postinst and will warn even if user want such different
> permissions. And yes central db simplify things and is necessary in any case.

As said, IMO the best way is to track which permissions were set when a
directory was merged, and on upgrades only preserve the current permissions if
they differ from the saved state (keeping track of permissions has other
benefits as well). On top of that a config file to specify wanted permissions
could also be implemented, but I don't think that's really needed if we respect
manual modifications to ownership/permissions.